Something like RANCID might help you out here. Plus, if you don't do config management it would take care of that for you as well.
RANCID goes out and logs into all of your devices [launched via cron] grabs config/inventory, places it in a source control system and emails you config/inventory diffs. (This would catch any hardware moving around)
Using RANCID for inventory tracking isnt all that pretty though. I just grep what I am looking for from the config/inventory archive.
rancid@network configs$ grep XENPAK *
6509-1.stackexchange.com:!PID: XENPAK-10GB-SR
6509-1.stackexchange.com:!PID: XENPAK-10GB-SR
6509-1.stackexchange.com:!PID: XENPAK-10GB-LR
6509-1.stackexchange.com:!PID: XENPAK-10GB-SR
6509-1.stackexchange.com:!PID: XENPAK-10GB-SR
6509-1.stackexchange.com:!PID: XENPAK-10GB-SR
6509-1.stackexchange.com:!PID: XENPAK-10GB-LR
6509-1.stackexchange.com:!PID: XENPAK-10GB-SR
core-2.stackexchange.com:!PID: XENPAK-10GB-LR VID: V02 , SN: XXXXXXXXXXX
core-2.stackexchange.com:!PID: XENPAK-10GB-LR VID: V02 , SN: XXXXXXXXXXX
core-2.stackexchange.com:!PID: XENPAK-10GB-LR VID: V02 , SN: XXXXXXXXXXX
core-2.stackexchange.com:!PID: XENPAK-10GB-LR VID: V02 , SN: XXXXXXXXXXX
rancid@network configs$
Since this is all kept in a revision control system you can look back to see when a module was added or removed.