这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@kainino0x
Copy link
Contributor

@kainino0x kainino0x commented Apr 20, 2021

Split from #1581, because it's not immediately clear whether this overload is useful.

  • It is somewhat fragile, because videos can change resolution between frames, but application code could easily (accidentally) not handle this.
  • GPUExternalTexture is a more straightforward and less error-prone way of using video data on a per-frame basis, with the caveat that it has lifetime restrictions and can only be sampled from.

Preview | Diff

@shaoboyan
Copy link
Contributor

@kainino0x Thanks for raising this. One case I'd like to mention is the subrect copy(and copy several video onto the same texture). Without the HTMLVideoElement + CopyExternalImageToTexture, it will need to import to get the GPUExternalTexture first and did the follow-up copies. From impl view, it will in-out dawn several times with a potential copy overhead(e.g. fallback path for the import), sync issues.
And it might be hard to define ImageSource if we miss the HTMLVideoElement :-)

@kainino0x
Copy link
Contributor Author

Is there a specific use case for copying multiple videos into one texture? Especially since a lot of video sources have unpredictable resolutions, which makes them harder to fit into an array or atlas.

One example I can think of is loading all frames of a specific known video file into a texture (an atlas or array texture), e.g. for an animated or animatable texture on a 3D object. I'm not sure how common it would be to encode such textures as video files, though.

@kainino0x
Copy link
Contributor Author

Of course, this could still be done with GPUExternalTextures, but the app would have to write their own code to draw the video frame into the target texture using a render pass.

@shaoboyan
Copy link
Contributor

shaoboyan commented Apr 21, 2021

One case is a camera matrix(a lots of camera) which used to generate immersive scene. They will have different view of the scene and a single resource to store camera contents and possible depth contents based on the protocol(or some rules). The following up algorithms could use these infos to generate a living 3d video and "walking" in the scene.
More details here. You can find that the stiching video frame is from lots of camera.

@kainino0x kainino0x force-pushed the copyToTexture-video branch from 822497a to a4aaf1e Compare April 27, 2021 00:20
@kainino0x kainino0x marked this pull request as ready for review April 27, 2021 00:21
@kainino0x
Copy link
Contributor Author

The base change has been landed, and this one is rebased and ready for discussion.

@github-actions
Copy link
Contributor

Previews, as seen when this build job started (a4aaf1e):
WebGPU | IDL
WGSL
Explainer

@kainino0x kainino0x marked this pull request as draft April 28, 2021 01:14
@kainino0x
Copy link
Contributor Author

Resolution:

  • Omit copyExternalImageToTexture for HTMLVideoElement for now.
  • Open to adding it later
  • See what developer feedback is

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants