I don't get how the training of the RPN works. From the forward propagation, I have $W \times H \times k$ outputs from the RPN.
How is the training data labeled such that I can use the loss function and update the weights through bach propagation? Is the training data labeled in the same shape of the output, as there are $W \times H \times k$ anchor boxes and we use the loss function directly or what?
