One strategy that might work would be to mount (with the --bind option) a common folder to the chroot's package cache. This cache is located at /var/cache/apt/archives/
Using this method would make it so that any command run by apt will operate with the common local archive.
Examples
Having two chroot's share a common apt archive
sudo mount --bind ~/builds/cache <chroot location>/var/cache/apt/archives
sudo mount --bind ~/builds/cache <another chroot location>/var/cache/apt/archives
That would make ~/builds/cache and both chroot's apt archive operating against the same directory.
Havind two chroot's share the local install's archive
sudo mount --bind /var/cache/apt/archives <chroot location>/var/cache/apt/archives
sudo mount --bind /var/cache/apt/archives <another chroot location>/var/cache/apt/archives
Similar to the first one, but now the main install and both chroots will use the same apt archive, ie, apt in all three will look in the same place for downloaded .deb files.
(I thought packages don't directly care about the ubuntu release, just the versions of associated packages. So you may not need to worry about separate folders for each release, and you could even possibly use the local install's package archive.)
Cleaning up
Lastly, and importantly, in the section about cleaning up the chroot in preparation for the final build, you don't want to run apt clean. Since apt is now using the same folder across all chroots setup this way, apt clean will essentially clear the archive for all chroots at once. Instead, within the chroot, run:
umount /var/cache/apt/archives
This will effectively "apt clean" the archive from just the chroot where the command is run, while leaving the common folder untouched.
Bind mounts manual info
The following is a straight copy from the man mount page. Please take a look at it for full explanation of mount --bind, and possibly to see other ways of using bind.
The bind mounts.
Since Linux 2.4.0 it is possible to remount part of the file
hierarchy somewhere else. The call is:
mount --bind olddir newdir
or by using this fstab entry:
/olddir /newdir none bind
After this call the same contents are accessible in two places.
One can also remount a single file (on a single file). It's
also possible to use the bind mount to create a mountpoint from
a regular directory, for example:
mount --bind foo foo
The bind mount call attaches only (part of) a single filesystem,
not possible submounts. The entire file hierarchy including
submounts is attached a second place by using:
mount --rbind olddir newdir
Note that the filesystem mount options will remain the same as
those on the original mount point, and cannot be changed by
passing the -o option along with --bind/--rbind. The mount
options can be changed by a separate remount command, for exam‐
ple:
mount --bind olddir newdir
mount -o remount,ro newdir
Note that the behavior of the remount operation depends on the
/etc/mtab file. The first command stores the 'bind' flag in the
/etc/mtab file and the second command reads the flag from the
file. If you have a system without the /etc/mtab file or if you
explicitly define source and target for the remount command
(then mount(8) does not read /etc/mtab), then you have to use
the bind flag (or option) for the remount command too. For
example:
mount --bind olddir newdir
mount -o remount,ro,bind olddir newdir
Note that remount,ro,bind will create a read-only mountpoint
(VFS entry), but the original filesystem superblock will still
be writable, meaning that the olddir will be writable, but the
newdir will be read-only.
