CopyImageBitmapToTexture is the API used by WebGPU to interact with some HTMLElements like <video>, <image> in current spec. It uses ImageBitmap as the snapshot.
During the implementation of this API in Chromium, I find it is hard to achieve the best performance when interacting ImageBitmap with WebGPU Texture provided by users.

Here are current status:

Generate ImageBitmap

An ImageBitmap is an opaque container of resources. It can be backed by GPU memory or non-GPU memory. Users have limited info of this container (only width, height) and limited method to config it (through ImageBitmapOptions).

Implicit steps in CopyImageBitmapToTexture

So when users invoke copyImageBitmapToTexture, they're in a great chance to hit:

• color formats are not compatible : ImageBitmap is a opaque container and users cannot know the accurate color formats. In WebGPU, we have a very strict rules in T2T copy. User can(and in a great chance to) provide a different format texture with ImageBitmap backed texture(if it is GPU backed). To resolve this, we need to start a rendering pipeline to do the conversion if dest format is renderable or read contents back to CPU and do conversion to cover more cases.
• transform : ImageBitmap has options imageOrientation = none, premultiplyAlpha = default and colorspaceConversion = default in beginning. The value default means Implementation-specific behavior .
  So we need to handle these transforms too. (For example, in Chromium, imageBitmap backed texture are Y flipped in implementation by default).
• browser implementation related issue : Except Implementation-specific behavior in transform part, there are other issues related with browser implementation. For example, SharedImage is the system used by chromium to share resources between contexts. In current chromium, ImageBitmap always backed by gl resource(or CPU buffer) and cannot be shared to dawn texture. We need an extra copy to handle it.

Thoughts

Based on above facts, when users upload content from some elements to WebGPU texture through generating ImageBitmap and copyImageBitmapToTexture, they have great chance to trigger several transformations(or copies). First, the browser will do a snapshot first to generate a ImageBitmap. Second if the backed resource cannot upload to provided WebGPU texture directly, another pipeline(or CPU-side conversion) is needed to do the conversion.

I have some thoughts on these steps and think maybe we can improve this through:

• Interact with Video and Image element directly: Current uploading model relies on snapshot step that generated ImageBitmap. In Chromium, these YUV format backed resources will do a YUV -> intermediate renderable format(e.g. RGBA8Unorm) conversion during snapshot and then follow CopyImageBitmapToTexture progress. If we have an API that can interact with such elements backed by multi-planars directly (e.g. CreateTextureFromVideo), we can fully control this path and have more opportunity to accelerate it. For example, we can do YUV to dest_texture conversion directly and bypass all intermediate steps.

• Add an option to ImageBitmap to notify it will be used by WebGPU : In Chromium, ImageBitmap that contains GPU resource is backed by non-sharable gl resource by default. And for perf consideration, this will still be the default path. But if ImageBitmap knows it will be used by WebGPU, it can be optimized. What's more, ImageBitmap defined lots of default options as "Implementation-specific behavior", so maybe this will help on more WebGPU specific optmization.

• Texture format compatibility : In current T2T copy rules in WebGPU, we don't have compatible format definition. So we need to start a pipeline to do some format conversion like RGBA8Unorm to RGBA8Uint. If we can use copy but not a pipeline, we may have perf improvement.
  I did some rough exploring about format compatiblility. In Vulkan and D3D12, there are rules to define compatibility. But metal is an open because it has strict rules for T2T copy and the possible compatible copy needs to leverage create texture view on a texture, which I have concern about the perf.