So I was able to do this by adding a another region keyboard (using english UK)
I used xev
to get the keycode
KeyRelease event, serial 37, synthetic NO, window 0x2e00001,
root 0x5c5, subw 0x0, time 790965, (1386,793), root:(1394,865),
state 0x10, keycode 44 (keysym 0x6a, j), same_screen YES,
XLookupString gives 1 bytes: (6a) "j"
XFilterEvent returns: False
KeyRelease event, serial 37, synthetic NO, window 0x3000001,
root 0x5c5, subw 0x0, time 2590604, (72,-17), root:(80,55),
state 0x2010, keycode 31 (keysym 0x69, i), same_screen YES,
XLookupString gives 1 bytes: (69) "i"
XFilterEvent returns: False
KeyRelease event, serial 37, synthetic NO, window 0x3000001,
root 0x5c5, subw 0x0, time 2612248, (947,850), root:(955,922),
state 0x2010, keycode 45 (keysym 0x6b, k), same_screen YES,
XLookupString gives 1 bytes: (6b) "k"
XFilterEvent returns: False
KeyPress event, serial 37, synthetic NO, window 0x3000001,
root 0x5c5, subw 0x0, time 2612767, (947,850), root:(955,922),
state 0x2010, keycode 46 (keysym 0x6c, l), same_screen YES,
XLookupString gives 1 bytes: (6c) "l"
XmbLookupString gives 1 bytes: (6c) "l"
XFilterEvent returns: False
Then I used xmodmap to bind the keys
xmodmap -e "keycode 31 = i I KP_Up i"
xmodmap -e "keycode 45 = k K KP_Down k"
xmodmap -e "keycode 46 = l L KP_Right l"
xmodmap -e "keycode 44 = j J KP_Left j"
If you want to reset the keybindings you can use
setxkbmap -layout us
To make this load everytime I open the computer, I've added it to ~/.bashrc