Deep fakes work by using a single encoder but then having a different decoder for different people.
But I wondered what if the encoder encodes say "closed eyes" of person A as the same code for "closed mouth" of person B. i.e. the codes could use the same codewords for different aspects of person A and person B. i.e. person A and person B could use the same codewords to descibe each of them except the codewords don't mean the same thing.
Then when you do a deep fake on person A with closed eyes it emerges as person B with a closed mouth.
How does one combat this effect. Or is does it just work and no-one knows why?
