Laptop as KVM with X11 and SSH
TL;DR
There is a detailed explanation below, but this is the short version of the story:
You can use SSH with X11 forwarding to launch a desktop environment
on a remote host, and display this desktop environment on an X11
server running on your local host. You can run multiple X11 servers
for each remote host (if you have enough memory) and attach each one
to a different virtual terminal, and switch between them using the
"Ctrl-Alt-F[1-9]" keys. This allows you to use the laptop
as a keyboard-video-mouse (KVM) console, or you could also call it a
graphical "dumb terminal", for the remote host.
The advantage of this approach is that you do not need to install remote desktop services on all of your servers. Most Linux and Unix systems have SSH with X11 forwarding pre-installed. Most Linux/Unix systems have Xorg pre-installed or is easily installed from their default software repositories. It works, is secure, and uses software that is likely already installed.
Prerequisites:
I assume you have installed a graphical desktop environment such as Gnome, KDE, or Xfce (DE) on the remote host (e.g. a server) computer. In this article, I will use Xfce and the "
xfce4-session" command to launch the DE on the remote host.I assume you can login to the remote host with the "
ssh -X" command and run GUI apps like Firefox on the remote host but have their windows displayed on your local host (e.g. your laptop).I assume your local host (laptop) has an X11 server such as Xorg installed onto it, and that it is setup to switch between virtual terminals (VTs) using "
Ctrl-Alt-F1" through "Ctrl-Alt-F7", and that you might already be running a DE on one of these virtual terminals (usually the 7th one).I assume you already have your SSH public keys setup for remote login on the remote host (server) without a password using "
ssh-copy-id", and that on the local host (laptop) you can run a SSH key server like "ssh-agent" and load it with your SSH private keys using "ssh-add" on a VT.I assume you have already somehow enabled the SSH option "
ForwardX11 yes" in the SSH configuration for your remote login, and that you can run a command like "ssh -X remote-host firefox" and have the remote host (server) run Firefox but display its window on your local host (laptop).
Run a remote DE session displayed in a local X11 server:
On your local host (laptop), switch to any virtual terminal (VT) not currently being used by an X11 or Wayland server, for example, you may switch to "
/dev/tt1" by pressing "Ctrl-Alt-F1". Now login with your usual user name and password.On this VT, prepare your shell for no-password remote login. Run "
eval $(ssh-agent)" to setup the SSH password service for this login, and "ssh-add" to unlock your master password.On this VT, launch an Xorg server with "
xinit" but tell it to run the DE command, in this case "xfce4-session", on the remote host (server) using SSH with X11 forwarding enabled:xinit /usr/bin/ssh -vX remote-host xfc4-session -- :15 vt1
The symbol "
:15" is an arbitrary choice by me, it tells the new Xorg server instance to identify itself as ":15". Any integer will do, but it is probably best to avoid 0 through 7.The symbol "
vt1" tells the new Xorg server instance to commandeer the virtual terminal attached to "/dev/tty1", which is the VT we are currently using. Make sure you specify the correct virtual terminal.Of course, "
remote-host" should be replaced with the address of the remote host (server) into which you want to login, and the "xfce4-session" command should be the command to launch whatever DE is available on the remote host.Be aware that the Xinit command looks for full paths like "
/usr/bin/ssh" instead of just "ssh" to differentiate between commands and arguments.
Streaming audio on the remote host over to the local host is beyond the scope of this article, but can be accomplished with audio servers like PulseAudio or PipeWire.
Detailed explanation
A keyboard-video-mouse (KVM) console is useful when you have several ordinarily "headless" computers. Headless computers are usually computers that perform a task for a long time, like servers, or perhaps embedded computers like Raspberry Pis. Usually, it is a waste of time, space, and money to have a keyboard and monitor for each of these, so you have one KVM console that you can attach to any server or embedded computer only when it is necessary, like performing maintenance.
If you have used Linux you might know that even with a graphical
login, you can (on most all distros) press "Ctrl-Alt-F1"
to switch over to a "virtual terminal" (VTs) connected directly to
a /dev/tty device and control the computer using a
command line, rather than a GUI. It is one of these virtual terminals
that you would also see if you installed Linux without Xorg or Walyand
— a so-called "headless" setup. Some people (often Arch or
Gentoo users) prefer to not install a session manager at all, they
rather login on the VT and then launch the desktop environment (DE)
like Gnome, Xfce, or KDE Plasma using the "startx"
command.
Most Linux distros also configure a number of VTs you can switch
between freely. On Debian-based distros these virtual terminals are
each on successive F-keys from "Ctrl-Alt-F1" to
"Ctrl-Alt-F7" where the GUI (the DE) usually runs on an
Xorg or Wayland server that commandeers the 7th virtual terminal. If
you like to hack on software like Xfce or KDE Plasma, it is often
helpful to run a second Xorg or Wayland server on one of these other
VTs and launch the DE there. If your code is buggy and it crashes, you
can switch back to another VT running a more stable DE. (See also the
"chvt" command which switch the VT that you are currently
seeing programmatically.)
I only recently realized that I could launch a DE on a remote
computer using SSH X11 forwarding, and have it display on one of
these other VTs on my laptop, making my laptop work effectively as a
keyboard-video-mouse (KVM) console for my various Linux computers. I
could have each computer attach to each VT and switch between them
using Ctrl-Alt-F(n). The incantation to do this is
relatively simple.
If you are able to run the command "ssh -X remote-host
/some/command" where "/some/command" would
ordinarily launch a GUI app (take Firefox, for example), you can run
Firefox on the remote host and have it displayed in a window on your
local computer (laptop).
But why be content with just Firefox? Why not instead launch
the entire desktop environment instead and
get ALL of the apps! For example you could launch the Xfce DE
using the "xfce4-session" command...
But...
don't try this in an already running DE,
because it will open several windows on top of your current DE, such
as the panel and the desktop itself, and also could screw around with
your themes and cause other issues. This is not a problem, however, if
you launch a second Xorg server on a separate VT, and then run
"ssh -X".
So finally, here is how to do it:
First I setup the remote machine for login without a password
(using "ssh-copy-id"), and I test it out to make sure I
can run "ssh -X remote-host firefox" successfully without
entering a password.
Then I press "Ctrl-Alt-F1" to switch to another VT,
and I login to the new VT with my usual user account and password. I
launch a new "ssh-agent" for this VT and unlock my master
password with "ssh-add". Then I enter these commands:
xinit /usr/bin/ssh -vX remote-host /usr/bin/xfc4-session
-- :15 vt1
The "xinit" starts a new Xorg server on the current
VT, but instead of running a local DE, it runs
"/usr/bin/xfce4-session" on the remote host via ssh
-X. SSH will forward all X11 commands on the remote host over
to my laptop which then displays them for me, and also sends X11
keyboard and mouse events over SSH back to the remote host. The
symbols "-- :15 vt1" tells the Xorg server to identify
itself as ":15" and attach to virtual terminal 1 (which I switched to
with "Ctrl-Alt-F1"). The number 15 is an arbitrary choice
I made, you can use any integer, but 1 through 7 are usually
reserved. For other logins I could change these symbols to "--
:16 vt2", and "-- :17 vt3", and so on.
Also note that usually after the double-dash "--" and
before the ":17 vt1" symbols you should ordinarily
specify the full path to the X11 server executable, for example
"/usr/bin/xorg". However, if you do not specify this,
Xinit has its own means of finding an X11 server to run. But here
Xinit does use the full path (as symbol starting with a slash or
dot-slash) to determine that ":15" is the server ID and
not the server executable. But a more proper invocation of the command
would be:
xinit /usr/bin/ssh -vX remote-host /usr/bin/xfc4-session -- /usr/bin/xorg :15 vt1
Conclusion:
It works nicely, except for a few details like audio, which there is a way to make work using some PulseAudio or PipeWire magical incantation that I haven't figured out yet. Also, I have no idea how well this would work for apps based on Wayland, or whether Wayland has a wire protocol that works over SSH similarly to X11. But for now, I am satisfied with this solution of using my laptop to switch between a few Linux machines, and I am quite happy that this solution uses only software that is already installed by default on all of my Linux installations (Xorg and SSH), and it isn't necessary to install anything like a special remote desktop server on all of my remote computers.
I should mention that there are remote desktop services that use SSH and X11 forwarding like in this approach that I just described in this article. I recommend you investigate using Xpra. Xpra will take care of audio, can launch remote desktop sessions in a window in an existing desktop environment (no need to use VTs), and also offers useful features like the ability to detach and re-attach to running sessions.
Happy hacking!