We are piping video frames into neural networks for post-processing as part of our playback pipeline. These networks run on the luma channel, so we don't want (or need) RGB frames. After the frame is processed by the network we then convert the frame to RGB using the modified Luma plane and existing Chroma planes. Today what that means in practice is that there is a YUV->RGB conversion that happens in a shader that runs prior to our network, and so we end up doing an RGB->YUV conversion in the input layer of our neural network.

If we get the texture and some information about which YUV format the texture is in that's enough for us to work off of. We can do the RGB conversion after the network for display to the user.