Fix the remote build of CA derivations

Make sure that the derivation we send to the remote builder is exactly
the one that we want to build locally so that the output ids are exactly
the same

Fix #4845