The clipboard itself is a feature of X11, so you will not be able to use it without having any X11 server instance running anywhere. More about that below.
However, xclip
/xsel
don't have the X11 server as dependency, they only need some client libraries installed. You should install xsel
though if you want to have as few packages as possible, as it has significantly less dependencies than xclip
(compare the output of apt depends --recurse --important xsel
and apt depends --recurse --important xclip
).
Back to the X11 server. You said you don't want to install a full graphical X11 server stack on your headless Ubuntu server, which is not necessary. That requires you to have an X11 server running on your local client instead though, if you want to use its features like the clipboard.
On a Linux desktop client this is no problem, as you already have X11 running there usually, but on Windows you don't. Luckily there seems to be a small X11 server version for Windows too, called Xming.
Once you have that running and configured, you should be able to forward your local X11 session via SSH (using PuTTY) to your headless server. There you can then use e.g. xsel
to interact with it and access your local clipboard.
More details about how to set up Xming, PuTTY and the SSH server for X11 forwarding can be found in these posts:
screen
andtmux
have copy-paste buffers: Does non-X Linux have a clipboard? – dessert Jan 21 '19 at 13:45