You might have seen the self healing reverse SSH setup with systemd that helps you get easy SSH access to devices that are behind complicated NAT systems (or in the hands on non-technical people you support). This can also come in quite handy when you have a reMarkable tablet. Since you can SSH into it with root access, we can also map our SSH port onto another server. The systemd unit is shown below:

[Unit]
Description=Reverse SSH connection
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/ssh -i /home/root/.ssh/id_rsa -g -N -T -o "ExitOnForwardFailure yes" -R 22221:localhost:22  hole@YOURSERVER.TLD
Restart=always
RestartSec=30s

[Install]
WantedBy=default.target

Let’s pull that apart. The Unit part described when the unit should be run and its description. We need a network connection to SSH, so we run after we have that.

But the interesting part is the [Service], it diverges lightly from the version in the article I stole the idea from, this is because the SSH server on the remarkable is not OpenSSH but Dropbear. This SSH server is much smaller, ideal for embedded devices but it also has fewer functionalities: no ServerAliveInterval and no extra verbose mode. The argument we give tho ssh are:

  • -i /home/root/.ssh/id_rsa specifying the identity file the reMarkable uses to identify itself to the remote server,
  • -g permits the server the reMarkable is connecting to initiate connections back to the reMarkable,
  • -N -T do not run a remote command and don’t even allocate a pseudo terminal (we don’t need that),
  • -o "ExitOnForwardFailure yes" adds an option to the ssh connection to fail and exit if it was not able to establish the port forward.
  • -R 22221:localhost:22 does the magic connecting port 22221 on the remote machine with port 22 on the reMarkable.
  • hole@YOURSERVER.TLD the final argument specifies what host and username to connect to, here we connect to YOURSERVER.TLD as user hole.

Making this secure

An important security principle is that of the minimal privilege, the remarkable should only be allowed to open a specific port on our server and nothing else. To do this we will put restrictions on the what is allowed with the reMarkable provides.

Creating a SSH key on the reMarkable

Before we can set restrictions on a key, we need to create it. Because there is no openSSH on the reMarkable but dropbear, we need to use its key generator to make a key in /home/root/.ssh/id_rsa. The second line prints the public part of the key.

dropbearkey -f ./.ssh/id_rsa -t rsa -s 2048
dropbearkey -f ./.ssh/id_rsa -y

Making a dedicated hole user for port forwarding on the server

The remarkable only needs to open a port, and it should not be able to authenticate as one of the actual users on the server. So we create a hole user on the server that will only be used to manage forwarded ports. This user should be created as a system user without a login shell, but with a home directory:

useradd --system --create-home --shell /usr/bin/nologin hole

Now in the home directory of this user /home/hole we will need to make an autorised keys file /home/perforation/.ssh/authorized_keys with the following contents:

restrict,port-forwarding,permitopen="localhost:22221" ssh-rsa BCDAD5PscK...WgTp root@reMarkable

Where BCDAD5PscK...WgTp is replaced with the public key of remarkable. This file ensures that any connection authenticating with the given key will only be able to use portforwarding to listen on port 22221.

Making it convenient

With all this set up we can connect to our remarkable with a jump host. To make this more simple we can add this to our ssh config (~/.ssh/config).

Host remarkableTun
    Hostname localhost
    ProxyJump jumphost
    Port 22221
    User root

Host jumphost
    Hostname YOURSERVER.TLD
    User user

This configuration allows us to connect to the reMarkable with:

ssh remarkableTun

Extra tips and tricks

Getting the correct IP

Sending files to the remarkable can be slow over this SSH tunnel certainly if the server is not on the same network. We can use the following command to find out programmatically what IP address is assigned to the reMarkable. Then we can connect to it directly over the local network. Of course, this does not work if the remarkable is on a different network or a network that drops ssh packets on the local network.

ssh remarkableTun -- /sbin/ip addr | awk -F ' ' '$1 == "inet" && $7 == "wlan0" { print $2 }'

Alternative ports

If f your ssh server does not run on port 22 add -p PORTNUMBER to the end of the ExecStart

The ExecStart command to use when on OpenSSH

I just put this here for archival purposes if site I learned this from goes down.

ExecStart=/usr/bin/ssh -vvv -g -N -T -o "ServerAliveInterval 10" -o "ExitOnForwardFailure yes" -R 22221:localhost:22  hole@YOURSERVER.TLD