Delay when using modprobe v4l2loopback with ffmpeg

Question

I will state, that for the first maybe 2/3 mins of having the camera started, it appears that the audio/video is in sync. Thats to say, there is no delay on the video. Then I hear a "click" on the camera as the shutter closes and then starts the delays.

sudo modprobe v4l2loopback && gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 0 -f v4l2 /dev/video2

It appears that the key part is with the speed=1x... if the speed is > 1 then we are having fun... however the longer the stream is open, the slower it gets it appears.

    ffmpeg version N-49161-g50e194e6e1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
      built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
      configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
      libavutil      56. 30.100 / 56. 30.100
      libavcodec     58. 53.101 / 58. 53.101
      libavformat    58. 28.101 / 58. 28.101
      libavdevice    58.  7.100 / 58.  7.100
      libavfilter     7. 55.100 /  7. 55.100
      libswscale      5.  4.101 /  5.  4.101
      libswresample   3.  4.100 /  3.  4.100
      libpostproc    55.  4.100 / 55.  4.100
    Capturing preview frames as movie to 'stdout'. Press Ctrl-C to abort.
    [mjpeg @ 0x57bc940] Format mjpeg detected only with low score of 25, misdetection possible!
    Input #0, mjpeg, from 'pipe:':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 640x426, 25 tbr, 1200k tbn, 25 tbc
    Stream mapping:
      Stream #0:0 -> #0:0 (mjpeg (native) -> rawvideo (native))
    [swscaler @ 0x57e9040] deprecated pixel format used, make sure you did set range correctly
    Output #0, video4linux2,v4l2, to '/dev/video2':
      Metadata:
        encoder         : Lavf58.28.101
        Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x426, q=2-31, 81792 kb/s, 25 fps, 25 tbn, 25 tbc
        Metadata:
          encoder         : Lavc58.53.101 rawvideo
    ^CCtrl-C pressed ... Exiting.ze=N/A time=00:01:18.04 bitrate=N/A speed=0.973x    

I am uncertain how, or if at all this is fixable?