0

I have an Ubuntu 20.04 Linux setup, with xfce, xorg and xrdp installed. My objective is to run an application inside a graphics environment, do some clicks, monitor its performance, and them close it after some time.

I am able to do it using an RDP session, by running "app.sh" in the terminal on the remote machine. But I would like this process to be more automated, and the app to start using remote ssh, whithout having to do the RDP session. Unfortunately I am not familiar enough with linux to know where to start.

What have I attempted so far:

  1. Just running "app.sh" from a remote ssh results in "Could not detect graphics environment" error.
  2. Using "xinit app.sh 'displayconfig'" command. The app seems to run but the app ui does not seem to initialize on the remote machine (or at least I cannot detect it)

For my purposes, I do not care whether or not a display session or rdp session is active (please correct me if I got the terminology wrong). My only objective is to start the app on the remote machine, send a click event to some (X,Y) coords, and then close the app. I do not want to actually see the actual UI.

Edit 1: Expected behavior (for clarity):

  1. I connect to the remote machine though ssh
  2. I run some command to start the app on the remote server
  3. The app starts on the remote server (with the ui and everithing)
  4. I dont see the UI on the local machine, but it is running on remote
  5. I send a click event to some (X,Y) coords (I have a python script for that)
  6. I monitor the performance of the app (the cloud setup does that for me)
  7. I close the app after some time
  8. I close the ssh connection

May be relevant: My app is a Java Swing based app, and I am using JVMTI to detect the UI

Edit 2: I will try to explain my setup a bit more, as the answers provided make the wrong assumptions:

I have 2 linux servers: server A and server B. server A acts as a repository (no graphics or anything) where different versions of the same app are uploaded. Once a new version for my app is uploaded, Server A uploaded that version to Server B, and deletes the old version (this part I have figured out). Now server B has some graphics installed. I need Server A to send some command to Server B to start the app as if it was a user (human) using the GUI. Now server A does not have any display, so I cannot forward the gui anywhere. So far, I have an 'app.sh' file with which managed to start the app, using RDP (from a windows machine) to connect. This, unfortunately, is not an option for production. What I need is some way to start a graphics environment on server B from server A and run the app there, or just have some 'always on' session which has a graphics environment on server B, and somehow start my app from server A with that session. I do not actually need the visual feed of the gui, as I dont need to see it, nor will I have any physical display for it, it can be purely virtual/simulated on server B, or it can be missing at all if there is a way to "click a button" without it.

Screenshots (redacted for confidentiality):

  1. App running with RDP session What you can see: Open RDP session, App running in background (behind terminal, not as background process), Terminal 2 started the app by calling 'app.sh', Terminal 1 sent a 'list ui' command to the app. All works fine

  2. SSH connection, attempting same commands as previously (remember, no PHISICAL display / no desktop screen available in production

What can you see: an open SSH connection, attempting to call same 'app.sh' as before.

More info: running 'echo $XDG_CURRENT_DESKTOP':

  1. in RDP session: > 'XFCE'
  2. in SSH session: > ''

Would something like this be possible? Please dont hesitate to request additional info

Petru Tanas
  • 111
  • Is there a reason why you're using RDP? Couldn't you just ssh into the server while having an Xserver running on your local machine and then just start programs on the server, with them appearing locally on your machine? Have you tried this before going down the RDP path? – Ray Aug 22 '22 at 13:37
  • @Ray I userd RDP just to test the setup, as I am familiar with it. As of local Xserver, yes, I have managed to do that, but this is the point, I dont want to see the ui locally. It can appear on the remote machine if it needs to, I just dont want to see it. I want a script which will start "in the background" in some sense, without opening any UI on my local machine and then write the result to a file. If the UI is actually running on the remote machine thats fine. I hope I am not to confusing :D – Petru Tanas Aug 22 '22 at 13:45
  • Doesn't ssh -X yourhost ./app.sh (or wherever it is) work for you? It should display the application's gui on your local sccreen. – ubfan1 Aug 22 '22 at 16:05
  • No, it does not, as I do not actually want to see the gui locally. It may be there, on the remote machine, but I not want the visual feed. – Petru Tanas Aug 22 '22 at 16:07
  • 1
    Sorry, but what you want is a bit confusing. If there is no GUI that appears, how do you interact with it? I mean, how will you "click" on it if you can't see it appear on your screen? What you're asking for sounds like it's a problem only the developer of that app can solve. Some programs have various arguments, and if you don't include them, it loads up the GUI. You basically need a program that has both a GUI and a command-line interface together in the same executable. No RDP or ssh with an XServer can solve that problem for you... – Ray Aug 22 '22 at 16:22
  • An image program like gimp might allow you to "crop" an image by loading up a GUI and you draw a box around what you want to crop it. The Imagemagick family of programs allow you to crop an image (for example) via the command-line. Here, this is two separate programs, but I can imagine one program being able to do this. But that's up to the developer to make such a program. That sounds like what you're looking for? – Ray Aug 22 '22 at 16:26
  • @Ray I understand this is confusing, and I will try my best to clarify the problem. The problem is, in production, there is no PHYSICAL display/monitor for anyone to see the GUI. the gui SHOULD be there, precisely for the reason that it needs to be a gui interaction. But the 'video feed' simply put, I do not care about, it can be streamed to a file, into nothing, or not streamed at all. – Petru Tanas Aug 22 '22 at 16:30
  • So I guess the question can be rephrased as 'can I run a GUI app remotely without a PHYSICAL display' (no actual screen for anyone to see) – Petru Tanas Aug 22 '22 at 16:32
  • The answer is you can -- of course you can -- but whoever wrote the app is the one that has to allow it. Is this you / your colleagues? It's not up to the user to solve the problem you describe; it's the developer's problem to make a GUI-based program that also accepts command-line options. – Ray Aug 22 '22 at 16:38
  • Yes, it is our app. And we can both start the app from terminal and pass it commands from command-line. the issue is that it only works with RDP, while being presented on my physical display. Once I close the rdp connection, and attempt to run the same commands from ssh, I get "Could not detect graphics environment". I will add some screenshots, maybe it will be more helpful – Petru Tanas Aug 22 '22 at 16:46
  • Hmmmmm, perhaps it is a "problem with your app". It sounds like your app is looking for a graphics environment. Ignoring RDP for the moment, as others have replied to you, you can ssh into the server and run myapp &. It would start up and run in the background. If I change the display to the server's screen instead of my own, it would then pop up there instead of my screen. But if you get errors from this step, then this means the app seems to want/need it to be displayed. Not something to do with Ubuntu (as far as I can tell). – Ray Aug 22 '22 at 17:04
  • @Ray Thats the thing, I dont know how to change the display to the servers's screen instead of my own, nor I know how to keep servers's screen running once I exit rdp (or is it always running by default?). Solving this is exactly what I need :) . Sorry if my explaining is bad – Petru Tanas Aug 22 '22 at 17:15
  • Let us continue this discussion in chat. – Ray Aug 23 '22 at 09:05

2 Answers2

1

Start ssh with: ssh -Y user_id@host.com and provide login credentials. Start X11 software with it's command, for example xclock (one might use & to preserve prompt). To start program in Windows 10/11 environment most recommendable is WSL2 with it's GUI support. Other option for Windows is Xming (https://sourceforge.net/projects/xming/). On Linux systems just ssh with -Y and start program. One has to enable X11 forwarding in target machine (where you start the program) in the /etc/ssh/sshd_config file by removing # in X11Forwarding yes (if no change to yes). Good practice is also enable forwarding in client (/etc/ssh/ssh_config) ForwardX11 yes. Remember to restart ssh by sudo systemctl restart ssh.

Sami Hulkko
  • 93
  • xclock returns "Error: Can't open display: ". xstart returns "(EE) no screens found(EE)", with "(EE) VESA(0): Cannot read int vect" in the logs – Petru Tanas Aug 22 '22 at 18:00
  • startx seems to work with sudo, and starts the xorg process, but does not set any DISPLAY variable still. xclock still gives the same error with sudo. – Petru Tanas Aug 22 '22 at 18:07
  • so: export DISPLAY= and start.sh. Do you have X11 running? 1. startx 2. export DISPLAY 3. start script. Display info is in your /etc/X11 folder configuration files. Export DISPLAY= can take ip_of_the_connected_host or ip_of_server_you_connected:<display_number>. Display number in /etc/X11 conf files (don't remeber where there...). I would install VNC server and login (over SSH tunnel) with it into target, start the script and close vnc connection. That requires running X11 server though. – Sami Hulkko Aug 22 '22 at 22:18
0

From your comment,

I dont want to see the ui locally. It can appear on the remote machine if it needs to, I just dont want to see it. I want a script which will start "in the background" in some sense, without opening any UI on my local machine and then write the result to a file. If the UI is actually running on the remote machine thats fine.

It sounds like you want command-line control over software running on a remote machine, which has been used exclusively with GUI until this point, so that you can automate some task with that software. You want to execute commands on a remote machine, and receive the output.

It's impossible to know what the CLI commands are for that software without knowing what the software is, but the general structure would be this:

ssh user@remotemachine [command {flags, objects}] > /file/path/to/output/file/output.extension

For example, if I wanted to see the fstab contents on the remote computer at the IP address 123.45.67.890, then I would do the following:

ssh root@123.45.67.890 cat /etc/fstab > /home/me/fstab.txt

For your case, you will need to find the command-line commands for the software you wish to use.

Isaiah Davis
  • 11
  • This is not quite my question. Sorry If It was not quite clear enough. I have an 'app.sh' file which is starting the app from console, and I am perfectly able to do that while being connected in an RDP session, which initiates the xfce graphics environment. The problem is that I dont want to connect to the RDP session, as I dont want to actually see the UI. But then I run 'app.sh' from a remote ssh, i get 'Could not detect graphics environment' error – Petru Tanas Aug 22 '22 at 15:43
  • The reason you are receiving the "Could not detect graphics environment" error is because when you connect from your local machine to your remote machine via SSH, then tell your remote machine to run "app.sh", the remote machine tries to launch "app.sh" inside your user's session, which is active through SSH, and does not have a graphical environment configured/enabled. As a result you receive "Could not detect graphics environment", because there is no graphical environment to which app.sh can connect. – Isaiah Davis Aug 22 '22 at 15:51
  • What is app.sh doing, and what is your goal here? If you want the GUI to open on a remote machine, as if someone physically connected to the machine with a mouse, keyboard, and monitor, then why? Do you want it to be ready for the next person who logs into the physical machine? If so, why not have it launch at start, or at boot, or at login? Does the GUI provide text output to the console you need on your local machine? If so, then surely there is a CLI-only method to interface with the software, which you can execute remotely from your local machine. – Isaiah Davis Aug 22 '22 at 15:55
  • Yes. That seems reasonable. Hence the question, is there a way to provide this graphics environment? or maybe just have it "always running" and just point to it somehow? The RPD session seems to do this, as it runs xfce, but once I close the RDP, xfce also stops. And I dont actually want RDP because I dont need the visual feed, i just used it for testing. Or maybe just to have a different user session which is 'always on' which has a graphics environment, and I somehow start the app inside that session? – Petru Tanas Aug 22 '22 at 15:59
  • To the question of why, I need to monitor how the app behaves when some GUI events happen. Simply put, I want to see "what happens when a button is clicked". Now I am able to detect the effect of the app working, but it has to come from the button being clicked, as if a user clicked it. Now I dont actually need to see the button being clicked, but it as to be an GUI click – Petru Tanas Aug 22 '22 at 16:04
  • @PetruTanas the problem with that is that you don't seem to understand how X11 works on Linux. Apps that need a GUI act as an "X client", requesting the services of an "X server" to display the app. If an app can only run as an X client, you need an X server already running in order to service the app with a display and input devices, etc. Otherwise, the app tries to use the services of the X server and doesn't find one, so it errors out. You can possibly run an X server with a dummy display (may or may not require a physical device connected with HDMI) to service the app... – Esther Aug 22 '22 at 18:11
  • @PetruTanas ... but with no X server running, no app that requires the services of an X server will be able to run. The problem is that in order to run an X server, you need an actual display, or a piece of software or hardware that "spoofs" being an actual display to the X server. For this you may be able to use a "dummy" driver or device. – Esther Aug 22 '22 at 18:12
  • @Esther Yes, I guess a 'dummy' display for the x server would be exactly what I need – Petru Tanas Aug 22 '22 at 18:50
  • @PetruTanas https://askubuntu.com/questions/453109/add-fake-display-when-no-monitor-is-plugged-in, but be sure to read all the comments on whichever solution you choose, since most have important information added as comments later. – Esther Aug 22 '22 at 18:52