We, Microsoft, have been working with Qualcomm to enable WebGPU by default in Chromium browsers. As part of the journey, we've discovered that the alpha to coverage samples mask test cannot pass on Qualcomm hardware. We'd like modifications to the WebGPU CTS and spec to accomodate.

Background: The test renders one pixel with 4x MSAA on, with the alpha-to-coverage option enabled. The test draws 4 samples inside the pixel, each centered around a sample and renders with different color (red, green, blue, yellow). It tests with alpha value from range { < 0, = 0, = 1/16, = 2/16, ..., = 15/16, = 1, > 1 }. As the alpha increases, more samples will be covered, and the number of bits in the alpha to coverage mask is non-decreasing. The test result for color output is 4 samples color value: sample1_r, sample1_g, sample1_b, sample1_a, sample2_r, sample2_g ... sample4_b, sample4_a. See sample_mask.spec.ts for test details.

The test is problematic for a couple of reasons.

First is the requirement, which the test itself says is "not included in the spec", that if one sample is included in the alpha to coverage mask, then any alpha value >= alpha_a must include this sample in the mask as well. This is not possible per the following explanation on how Qualcomm hardware works, as quoted with permission from them.

What QCOM does is look up a sample mask table indexed by the alpha value. In 4xMSAA case, there are 16 samples in a 4x4 pixel-tile.
Alpha = 0 => no sample covered
Alpha ~= 1/16 => one sample is picked for that 4x4 tile
Alpha ~= 2/16 => two samples are picked for that 4x4 tile
…
Alpha ~= 15/16 => 15 samples are picked for that 4x4 tile
Alpha = 1 => all samples are covered.
The alpha-to-coverage mask from the look-up-table is then AND-ed with the input mask for the final sample mask.

We can guarantee that
-- for the same location, when alpha is larger, the sample has more chance to be visible.
-- If a primitive covers the all samples in a pixel, when alpha is larger, the pixel has more visible samples.
We cannot guarantee that at different locations, larger alpha value => more visible samples.

Secondly, since final sample mask = input sample mask AND alpha-to-coverage mask and the test only checks one pixel at a time, you can't really draw conclusions since the final value of the pixel depends on where it happens to land in Qualcomm's mask.

The DirectX spec allows this in the Alpha-to-Coverage section, specifically, the following text.

As alpha goes from 0 to 1, the resulting coverages should generally increase monotonically, however hardware may or may not perform area dithering to provide some better quantization of alpha values at the cost of spatial resolution and noise. An alpha value of NaN results in a no coverage (zero) mask.

Note how intentionally loose the language is, including the mention of area dithering. Qualcomm is following this rule over a 4x4 pixels area, which is an acceptable interpretation of "area dithering" in the spec text above.

Recommandation: For the test, modify the algorithm such that the test triangles cover 4x4 pixels, then check the covered samples. If we find that another IHV dithers over an even larger area, we can adjust, but that seems unlikely.

For the spec, update the last bullet in the Alpha to Coverage section to be the following:

• If alpha is greater than some other alpha1 for some area of pixels, then the produced sample masks for that area of pixels collectively have at least as many bits set to 1 as the mask for alpha1. The size of the area of pixels over which this property is guaranteed is device dependent - it could be 1 pixel or could be some larger region like 4x4 pixels. This doesn't mean alpha must be constant across pixels, rather that the monotonically increasing property is only measurable across an arbitrary sized area.