Previews, as seen when this build job started (ae7350e):
WebGPU | IDL
WGSL
Explainer

I just realized that https://github.com/gpuweb/cts/blob/main/src/webgpu/api/operation/buffers/map.spec.ts#L318 explicitly populates a buffer, maps the buffer for WRITE, then reads the data from the WRITE mapping.

This is pretty unfortunate because it means, on non-UMA systems, mapping for WRITE has to actually download the data from the device, just for it to be clobbered by javascript.

Can you just map any memory as long as that memory came from the same page?

In other words, mapAsync(WRITE, offset, size), when mode === WRITE, it doesn't matter what's in the arrayBuffer that will eventually be returned by calling getMappedRange as long as the data in that arrayBuffer comes from the page itself.

So, you can keep a cache of arrayBuffers behind the scenes and when mapAsync(mode === WRITE) is true, then, if you have an unused arrayBuffer that is large enough in your cache of arrayBuffers for mapping you just return that buffer. It's only got data the user put in it from previous mapAsync calls so it's their data.

Is that a solution or is it scary because it means potentially different behavior across implementations if the way they implement their cache is different?

Oh, maybe this is why this validation rule exists during buffer creation:

If descriptor.usage contains MAP_WRITE:
descriptor.usage contains no other flags except COPY_SRC.

I guess the idea is that, if you can guarantee that the only way data gets into the buffer is via buffer mapping (or writeBuffer()), then the web process can just create a shadow ArrayBuffer around, holding the contents that was last uploaded into the buffer.

That seems like a pretty high penalty, though, just to enforce portability between UMA and non-UMA devices. Zero-filling would also achieve that portability without the requirement that map_writable buffers can't be used for anything else.

Oh, maybe this is why this validation rule exists during buffer creation:

Exactly, and the intent is that a shadow copy is kept as you guessed. There is a single way for data to get into a MAP_WRITE buffer which is filling it with javascript (either after mappedAtCreation:true or mapAsync). WriteBuffer requires COPY_DST so it is not possible to use it on MAP_WRITE buffers.

If we were to fill the buffers with zeroes on mapping, then we would essentially perform half a copy more for uploads (memset 0 is equivalent memory-wise to half a memcpy). That copy would happen on the CPU in the Web process when buffers aren't triply-mapped (which should be common, triply mapping will stress the OS and is not available everywhere anyway), so it will be quite expensive.

Note that in the short proposal for UMA mapping, MAP_WRITE buffers can be used with any readonly usages, which is somewhat generous. We could also add support for MAP_READ|MAP_WRITE buffers that can support all usages on UMA, but when buffers aren't triply-mapped, then you'd have one extra copy (GPU->Web for mapping for writing, Web->GPU when mapping for reading).

Presumably, when people map, they want to overwrite data. Zeroing that data would let us not have to keep a shadow map around, but it would be kinda pessimal for large uploads like images or other large buffers, to zero them first, and then only after zeroing, overwrite (almost?) all of the buffer again with the data to upload.

If an author is fine with zero-filled data, I believe that's what mappedAtCreation:true does during createBuffer, so they might have the opportunity to choose that tradeoff, if using a new buffer object as a one-shot upload is an option.

We can also talk about behavior of getMappedRange if doing zero-filling there would be useful instead. Having to zero-fill the whole mapAsync range feels heavy.

For background for tomorrow's discussion: the original premise of this PR was that getMappedRange was zeroing instead of mapAsync - and that it should be moved to mapAsync. However, this was incorrect, as no zeroing occurs in either place. Mapping for WRITE gives you the current data in the buffer, which is usually expected to be kept in a shadow copy on non-triple-mapping systems.

So the tradeoff to discuss here, I think, is to add the cost of zeroing on all systems, and remove the shadow copy memory cost on non-triple-mapping systems.

Note that browsers could remove shadow copies in GPU-renderer process shmem if they have memory pressure, and recreate the shmem the next time the buffer is used. When that happens a memcpy is needed instead of memset 0, so it's more expensive.

We could also have a GPUMapMode.ZERO that says the mapping for writing should be filled with zeroes, and using it consistently makes the shadow copy not be created. (but that's kinda like a new feature, so polish-postv1?)

Sorry, I meant to write this last week but got sidetracked. This issue is incorrectly titled, and muddled in what it’s asking for.

Current state

Right now, 2 things are required of a conformant implementation:

1. If descriptor.usage contains MAP_WRITE: descriptor.usage contains no other flags except COPY_SRC.
2. The ArrayBuffer the JS receives for MAP_WRITE calls comes prepopulated with the contents of the buffer in it. So, if an application maps for writing, they can then (erroneously?) read the buffer they receive. This requirement is (ostensibly) for portability - the contents of the Array Buffer have to be well-defined.

On a discrete GPU, implementations would want to avoid downloading the data in the buffer to service MAP_WRITE calls, because the common case when an application is using MAP_WRITE correctly would be to never read the contents of the array buffer. So, the intention here is for the implementation to keep a CPU-side buffer alive that mirrors the contents of the GPU buffer. Because of (1) above, this ‘mirror’ is possible to maintain, because the only way changes can appear in the buffer is via MAP_WRITE calls. It’s a one-directional data flow.

So, the implementation of MAP_WRITE is intended to be:
a. Every MAP_WRITE-able buffer has a CPU-side shadow allocation, with (at least) the same lifetime as the buffer
b. MAP_WRITE delivers pointers to this shadow allocation to the JS
c. JS can read or write the shadow allocation
d. When unmapping, WebGPU performs a copy from the shadow allocation to the backing GPU buffer

Analysis

Both of the requirements listed above are unfortunate:
A. The fact that descriptor.usage must contain no other flags except COPY_SRC means that these mappable buffers are useless by themselves for any non-trivial use-case. Any application that wants to upload data from the CPU and then use it in a shader must do a double allocation - one for the MAP_WRITE-able buffer, and one for the buffer that will actually be used. The fact that the application is forced to do a double allocation means that they must pessimize; the WebGPU implementation can’t elide one of the buffers, because the source program explicitly tells us to create them.
B. But it’s worse than that, because the ArrayBuffer comes prepopulated with the contents of the buffer ahead of time, thereby requiring either i) an expensive download just in case the application might look at the contents of the buffer, or ii) a third buffer-sized memory allocation.

On mobile devices, it’s incredibly wasteful to allocate 3x as much memory as is necessary.

Proposal

It doesn’t have to be this way, though! The contents of the ArrayBuffer delivered to JS just has to be well-defined; it doesn't have to hold the contents of the buffer. So, an alternative is for the contents of these JS-exposed buffers to be zero-ed out.

The downside to this is that, yes, the mapping operation would have to zero-fill some memory (or run a system call to get pre-zero’ed memory). This can be somewhat mitigated by having the GPU zero-fill the memory, rather than the CPU literally call memset(). This requires that the zero-ing happens as part of the task scheduled by mapAsync(), rather than inside getMappedRange().

The upside, however, is a 33% (or possibly 66%) memory reduction for common usage patterns. By zero-ing out the data, we don’t need to have a shadow allocation; we just need to cause either the GPU or the CPU to zero-fill the mappable destination buffer directly. For a triply-mapped buffer (which we expect to be every MAP_WRITE-able buffer, at least on Apple Silicon, probably Intel devices, and maybe even on all devices), the ArrayBuffers exposed to JS can point directly into the real destination buffer. And for non-triply-mapped buffers, it’s still better than what we have today, because the shadow buffers could be temporary and transient, rather than being required to live for at least the lifetime of the GPUBuffer they’re shadowing.

But it gets better, because, if we don’t need to maintain shadow buffers, we can open up write access to MAP_WRITE-able buffers to shaders. We can remove the restriction that MAP_WRITE-able buffers cannot be used as storage buffers, etc. Concurrent accesses would be naturally forbidden using our existing buffer state tracking infrastructure - ownership of the contents of a buffer would be transferred from GPU to CPU via map()/unmap() calls, and access by the non-owning side can be statically detected and averted.

On a discrete GPU, an application might not want to use the MAP_WRITE-able buffer directly in their shaders. There are a few possible ways of allowing this to happen:

• Just adding a note in the spec saying “Performance of GPU operations on MAP_WRITEable buffers may be suboptimal
• By exposing a “you might want your shaders to operate on non-MAP_WRITE-able buffers for performance” bit
• By adding in a bake() call which doesn’t need to do anything on UMA but on discrete cards would move the data out of the CPU-visible memory region
• Maybe other options

If we did something like this, then instead of allocating 3x as much memory as is necessary, we’d only be allocating 1x as much memory as is necessary. And, not only that, but it becomes way easier for authors to write WebGPU applications - they don’t have to write code to make twice as many buffers as they need and shuffle data between them.

Very useful post, thanks. I'm confused about one thing. You say zeroing would benefit triple-mapping implementations because the clearing could be done on the GPU. But wouldn't keeping the original contents be even better? Then triple-mapping implementations don't have to clear at all.

That said, I think in theory, zeroing could still be better, if there is memory compression for zeroed pages. Then you could avoid flushing all those zeroes out to main memory (and loading them back in if they get read, regardless of whether by the GPU or the CPU). I have no idea whether anything like this is possible though.

Right, that’s a good point. On a UMA system, the triply-mapped buffer gets the data for free.

I’m imagining a world like this:

1. On a UMA device, zero-filling occurs to try to maintain portability with non-UMA systems. This is a pessimization.
2. Un-pessimizing would be part of a larger extension for the future, focused solely on UMA.
3. Even without the extension, discrete cards wouldn’t have to have the shadow copy
4. Authors wouldn’t be required to use staging resources, on any card

Pros:

1. Memory use on a discrete card could go down by 66% ({shadow buffer, staging buffer, useful buffer} => {useful buffer})
2. Memory use on a UMA card could go down by 50% ({staging buffer, useful buffer} => {useful buffer})
3. Authors don’t have to schedule copies from the staging buffer to the useful buffer

Cons:

1. On a triply mapped buffer, every MAP_WRITE requires a GPU blit (which isn’t necessary today)
2. On a non-triply mapped buffer, every MAP_WRITE requires either an mmap(MAP_ANONYMOUS) (which returns zero-filled memory), or a memset().

I think the pros outweigh the cons.

Ah, got it, what I got confused about was triple-mapping on non-UMA.

There are two aspects I see to the proposal:

1. allow usages other than COPY_SRC with MAP_WRITE
2. zero-fill the contents of the buffer in a task scheduled by mapAsync. Then, when you call getMappedRange(), you see zeros instead of the buffer contents

We don’t think we should pursue either of these changes and should keep the current spec as-is. We could improve memory usage and reduce copies in a UMA extension.

Part 1:

Allowing usages other than COPY_SRC with MAP_WRITE is just like issue #2388. While it would be great to reduce the number of copies on unified memory architectures, lifting this restriction should be done in a fuller proposal for a UMA extension. If done poorly, lifting the usage rules would make it easy for developers to use an extremely inefficient path on discrete GPUs.

Myles had some good ideas here - maybe a bake() step so that flushing updates is explicit. Regardless though - we need a fully fleshed out proposal for how this (or something else) will work that will allow developers to get good behavior on both UMA and discrete systems without performance pitfalls.

Part 2:

I’m not convinced by the purported memory savings of returning a zero-filled mapping. As discussed in todays's meeting, at least Safari/Chrome expect to be able to use “triple mapping” everywhere. So, there should never be a need for a CPU-side shadow allocation. On all platforms, a MAP_WRITE|COPY_SRC buffer is backed by a single buffer allocation that is both visible to the web page and accessible to the GPU.
Yes, without a UMA extension, you still need to copy from this staging buffer into the “useful buffer” but there is no third “shadow buffer”.

The remaining memory savings argued in Myles’ proposal come from the fact that since the contents of the mapping are zero-filled, the implementation is free to make their storage temporary and transient. While this is true, this is exactly the same memory savings the application would get if they explicitly destroyed their staging buffer and recreated it again when they needed to. It is better to give control of allocations to the application so they can manage it themselves.

The current design of buffer mapping is clear that device.createBuffer handles allocation, and buffer.mapAsync mediates access to the memory. If the implementation is freeing and allocating buffers under the hood, then mapAsync will need to handle out-of-memory errors as well. I don’t think it’s good to automatically manage this memory when there are existing mechanisms for the application to robustly do so.

So given we don’t see memory savings to zero-filling we don’t think it’s worth doing. It also has a few downsides.

• It complicates the error handling mechanisms without sufficient benefit
• Will still need to pay the cost of the memcpy/GPU-clear
• It makes it more difficult for an application to do sparse writes to the buffer. Mapping would clear out the entire range A-B-C, so the application needs to write the entire range again even if you only want to modify parts A and C

I do think zero-filling could be part of the future UMA extension - perhaps adding a GPUMapMode.ZERO. Zero-filling will be valuable in some situations on a discrete GPU. It would allow the implementation to avoid reading back data into the mappable staging buffer. This could happen in situations where the buffer has MAP_WRITE|STORAGE usage, and the GPU modifies the contents in a shader. In the current spec, mapping the buffer after storage writes means those writes should be visible in the mapping. There are options here, but I think they should be left to the UMA extension. Perhaps:

• Just pay the cost, with a note in the spec.
• Validate you use GPUMapMode.ZERO
• Have explicit methods to enqueue the readback if the mapping is GPU-writable. The readback does nothing on UMA. Validate that you have requested to read back all the ranges you try to map.
• Something else

I just want to clarify my understanding based on @austinEng 's comments above.

It sounds like a buffer created with

buf = device.createBuffer({usage: GPUBufferUsage.WRITE | GPUBufferUsage.COPY_SRC, size});

Is not intended to be a GPU buffer at all. It's entirely a single allocation, ideally that both the CPU and GPU process can see. Those 2 flags make it a CPU->GPU staging buffer and that's all it's useful for.

If an implementation can do some optimization where it's actually allocated in memory the GPU can see that's great, but without that optimization it's a single CPU ram buffer, ideally in shared memory between the CPU and GPU process. It's possibly not a Metal/Vulkan/D3D buffer at all.

Correct or am I mis-understanding?

Sorry if that last post didn't make any sense.

In my mind, given the restrictions of MAP_WRITE | COPY_SRC I thought one valid implementation would be something like

class GPUBufferImpl {}

class RealGPUBuffer : public GPUBufferImpl {
    APISpecificBuffer mNativeBuffer;
}

class CPUOnlyBuffer : public GPUBufferImpl {
    void* mSharedMemoryBytes;
}

class GPUBuffer {
  GPUBuffer(impl) : mImpl(impl) {}
  GPUBufferImpl mImpl;
}


class GPUDevice {
   createBuffer(desc) {
     if (desc.usage == (MAP_WRITE | COPY_SRC)) {
        return new GPUBuffer(new CPUOnlyBuffer());
     } else {
        return new GPUBuffer(new RealGPUBuffer());
     }
   }
}

And that a MAP_WRITE_ | COPY_SRC buffer is just an obfusticated way of accessing a chunk of shared memory between the page and the GPU process. Whether that chunk of memory also happens to represent a native GPU buffer is an implementation detail. Behind the scenes, device.copyBufferToBuffer can do whatever it needs to copy the data into some other buffer.

I guess this is wrong though.

I believe your understanding is correct. And, you're right that the implementation may implement this buffer as a CPU-only shared memory buffer. Whether or not it is GPU-accessible is an implementation detail. However, we expect to be able to share the shared memory directly with the GPU everywhere. This is what we've been referring to in this issue as "triple mapping".