Sounds like you didn't configure x11vnc as a system service, so that it's automatically started when the system boots. I just did the same thing you're trying to do, and it works. Execute the following and it should resolve your problem.
Step 1: Set a password specific to your username that you'll use to login to x11vnc from a remote system.
x11vnc -storepasswd
Enter a password and store the file to: /home/USERNAME/.vnc/passwd
Note: make sure to replace USERNAME with your actual username.
Step 2: Configure the 'x11vnc.service' file so that vnc will be automatically started when the system boots up. You can use any text editor, but you'll need sudo access to edit the file. I'm using nano in the example here.
sudo nano /lib/systemd/system/x11vnc.service
Copy and paste the following lines into nano.
[Unit]
Description=Start x11vnc at startup.
After=multi-user.target
[Service]
Type=simple
ExecStart=/usr/bin/x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/USERNAME/.vnc/passwd -rfbport 5900 -shared
[Install]
WantedBy=multi-user.target
Exit and save the file by pressing 'Ctrl-X', then type 'Y' and press 'Enter'. Again, make sure you replace USERNAME with your actual username.
Step 3: Reload the services so your computer knows about the x11vnc service that you just configured in step 2 above. And enable the x11vnc service.
sudo systemctl daemon-reload
sudo systemctl enable x11vnc.service
Step 4: Start the x11vnc service.
sudo systemctl start x11vnc
Step 5: Celebrate! :-)
Conclusion
Following these steps should startup x11vnc and configure your system so that systemd, the system and service manager, will start x11vnc automatically every time the computer boots up.
References: I derived this answer from Ubuntu's documentation on VNC/Servers. Specifically, I referenced the sections titled "x11vnc" and "Have x11vnc start automatically via systemd in any environment (Vivid+)" .The second section I reference pertains to Ubuntu Vivid Vervet 15.04. If you're running a different version of a Debian linux distro, you'll need to reference the correct section from Ubuntu's VNC/Server page.