I am having trouble understanding positional encoding. Say after the wor2vec or some encoding algo we get the tensor $[0.7, 0.4, 0.2]$ for the second position. Now the final input into the model would add a positional encoding, making it $[0.7 + \sin(1.0), 0.4 + \cos(1.0), 0.2 + \sin(1.0)]$ right? Here's the question:
How can the model know if it's word tensor $[0.7, 0.4, 0.2]$ with positional encoding $[\sin(1.0), \cos(1.0), \sin(1.0)]$, or if it is word tensor $[0.7 + \sin(1.0) - \sin(0.0), 0.4 + \cos(1.0) - \cos(0.0), 0.2 + \sin(1.0) - \sin(0.0)]$ with positional encoding $[\sin(0.0), \cos(0.0), \sin(0.0)]$? They both have the same numbers.
