Is there a way to further fine tune image alignment homography after using Spatial Transformer Networks?

Question

I'm using a Spatial Transformer Networks (STN) to align an image by optimization using L1 loss between prediction and target, they works great for large mismatch, but not so much for small misalignment. For example, in this image below you could see it warped them to close as the target, but some lines still out of its place (the green pointing arrow).

I'm trying further fine tune them, is there a technique/networks that I could use that I might not be aware of that could be used in-place or extended from STN?

Answer 1

Depending on the accuracy of your model this might be normal and you could use more traditional algorithms like ecc to refine the homography.