There are a lot of possible semantics for importing canvases as GPUExternalTextures. This issue was first raised by @shaoboyan here: #1415 (comment). Yesterday I was trying to come up with a proposal, so I decided to write out the options.

First, we should talk about what the use cases are. This was discussed on VF2F day 3. Cases I'm aware of:

• Have some UI element or text that changes often (~every frame) and needs to be integrated into a 3D scene. For example, XR UI rendered with 2d canvas, or an FPS counter.
  • Note, for UI/text that is rendered less frequently, it's probably better to copy into a non-external GPUTexture. One example is a label atlas for labels on a map/globe.
• Interop between WebGL and WebGPU, primarily(?) for partially-ported applications (e.g. deep image processing pipelines).
  • This can be done, with some overhead, with copyExternalImageToTexture.

The different canvas contexts behave in very different ways, and it's tricky to reconcile them. Here are some options, with caveats, which can be partially mixed and matched.

1. Error/no-op.
   • Make drawing to the imported canvas an error or no-op. Seems too intrusive on the 2d canvas and WebGL APIs.
2. Clearing.
   • For {2d, webgl with preserveDrawingBuffer: true} sources, importing clears the source drawing buffer.
   • 2a. Clear like clearRect or transferToImageBitmap.
     • For {webgl with preserveDrawingBuffer: false, webgpu} sources, I think this is the same thing that presenting does. (Takes the drawing buffer and leaves a new, blank one.)
   • Note: 2d canvas contexts do not normally have such a semantic. OffscreenCanvas.transferToImageBitmap does this, but AFAIK canvas elements can only do this with clearRect. Resizing a 2d canvas is not the same; it also clears persistent state like fillStyle.
     • 2b. Clear like canvas resize.
3. Copy-on-write.
   • For all canvas sources, importExternalTexture effectively makes a copy of the contents of the canvas. Implementations must implement copy-on-write to optimize.
   • Most obvious semantic for users. But harder for implementations.
   • "Write" is well-defined in WebGL, but I'm not sure about 2d canvas since it doesn't have an equivalent of preserveDrawingBuffer: false.
   • For a WebGPU source canvas, we avoided defining "write" by saying "present iff you called getCurrentTexture", so either browsers would have to define it, or the spec would have to use option 2 for WebGPU canvases specifically.
4. Restrict import canvas context types.
   • At least for now, disallow importing from any (or all!) types of input canvases. Users would have to use copyExternalImageToTexture instead for certain canvas types.
   • It seems like a no-brainer to restrict imports to, at most, only 2d/webgl/webgl2 canvases.
5. Restrict to OffscreenCanvas only. Requires browser to implement OffscreenCanvas.
   • Allow importing OffscreenCanvas only, not HTMLCanvasElement. Rely on the semantic of transferToImageBitmap to define what happens - conveniently, it is fundamentally similar, as ImageBitmaps are also immutable.
   • Most likely restrict to constructed OffscreenCanvases only; OffscreenCanvases from transferControlToOffscreen() are not important here.
   • Possible porting pain for apps previously that normally use HTMLCanvasElement for this.
6. ImageBitmap import instead of canvas import. Requires browser to implement OffscreenCanvas and transferToImageBitmap.
   • Disallow importing from all input canvases, but add importing from ImageBitmap, which users would use with the actual transferToImageBitmap() method. Could, but would not have to, consume (close) the imported ImageBitmap.
7. Make drawing to the imported canvas cause immediate destruction of the GPUExternalTexture.

Proposal

• 4: restrict to only 2d/webgl/webgl2
• plus
  • 5: constructed OffscreenCanvases only, rely on the semantic of transferToImageBitmap;
  • or: 2a, 2b, 3, 7