Fix remote edited shader reflection lookup

[samtou-arm]

Description

When editing a shader during remote replay, subsequent edits, views, or debugging sessions could show the original shader source instead of the edited source.
This pull request tracks the mapping between original and replacement resource IDs and serialises it alongside the pipeline state.
The client then uses the replacement IDs when fetching shader reflections from the remote, while preserving the original IDs in the pipeline state.

This change has been tested on the following environments using the demo app and the *_Simple_Triangle sample:

• Linux client + Windows remote server: Vulkan, D3D11, D3D12 and OpenGL.
• Windows client + Linux remote server: Vulkan.

[baldurk]

I don't think this is the best way to go about solving this.

Either the replay proxy should itself track replacements on the local side (which doesn't require anything being serialised - as it already knows all off the replacements that are made and removed) and handle applying that transparently without any changes on the remote side, or else the remote should serialise sideband data specifying exactly what shaders to look up for each stage and those exact shaders should be queried.

This is an odd middle-ground where the remote serialises back data that is then possibly used which makes it a little harder to follow the logic than normal. The reason this happens at all is because we can't serialise the pointers across natively and they need to be re-allocated on the local side.

[samtou-arm]

Thanks very much for your comment.

I'd like to go with the approach of tracking everything on the local side. I am just confused about a small thing might be preventing it from working trivially.

Either the replay proxy should itself track replacements on the local side (which doesn't require anything being serialised - as it already knows all off the replacements that are made and removed) and handle applying that transparently without any changes on the remote side

From my understanding, for Vulkan, when replacing a resource it creates a new pipeline on the remote as well and this pipeline has to be known from the host otherwise we can't use it to query the right replacement shader (because of specialisation constants?).

So I might need a small change on the remote side as well to retrieve that pipeline ID.

Is my understanding correct? This doesn't seem to be the case for any other graphics API (D3D11, D3D12 and OpenGL) as for those we seem to query the replacement shader regardless of the pipeline.

[baldurk]

That's a good point... that does make it rather messy to handle this with the proxy tracking replacements blindly since it won't know about that. I'd rather not have further code in the proxy that is worrying about the details of pipelines vs not and I don't want to have proxy-only queries to the driver. The dumber the proxy is the less likely we'll hit bugs like this one that only show up in remote-replay cases.

I think it's best then to take the other approach then and make it explicit. Instead of reconstructing IDs on the local side and trying to replicate the right GetShader parameters we should just ask the remote to give us the pointed object directly without caring where/why it came from - serialise the pointer as a uint64_t and use that as cache key, and have a proxy-local packet which serialises the ShaderReflection requested by pointer value. Effectively a "read this pointer" RPC that requires no extra backend code. The only reason we didn't do this entirely self-contains in the serialiser code is because shader reflection objects are large and need to persist so deallocation is non-local.

All we lose doing that is any sharing between 'real' calls to GetShader and those that happen automatically in the pipeline since it won't be able to tell the difference, but it means the proxy has less special-case code to go wrong and really is just proxying data.

This doesn't seem to be the case for any other graphics API (D3D11, D3D12 and OpenGL) as for those we seem to query the replacement shader regardless of the pipeline.

As a note, D3D11 and OpenGL indeed don't have pipelines but D3D12 does. Currently the pipeline doesn't affect the shader reflection on D3D12 but it should still be passed along otherwise we'd have a latent bug sitting in case the D3D12 driver is changed in future. Doing things as above would sort this by treating all backends the same way without worrying about whether the pipeline is relevant or not.

[samtou-arm]

Thanks for the detailed explanation.

I have now updated the branch with the latest changes that should follow the second approach you suggested.

I created another cache map keyed by a uint64_t which is essentially the value of the pointer on the remote. When it comes to retrieve the reflection, we first look up in the cache and if we don't have any local copy of the ShaderReflection object on the host, we serialise it with a proxy-local packet once and store a newly created local copy on the host.

I suppose a bit a specialisation is still needed in the replay_proxy when it comes to serialise the pipeline states as I need to retrieve and store all the ShaderReflections for all the shader stages.

There was also a special case for D3D11 inputAssembly, which needed special handling from my understanding.

Feel free to tell me if I got something wrong in my implementation of your approach, regarding the cleanup of the cache or the special case for D3D11 or anything!