System Compromised!

Hi,

A user systemd was created with root privileged on FreeBSD, once we started digging more into this we found the .bash_history file under his home directory which he created in /lib/.systemd, his history explained that he downloaded some log tamper script in order to hide his appearance but fortunately he wasn't able to install it (some binary error occurred). What more interesting is, generally when you create a user via root the following log is created in userlog :
Code:
userlog:2016-02-25 22:36:55 [[B]root[/B]:useradd] systemd(6) home /myuser made
But when we checked about the creation log of that mysterious 'systemd' user, this is what we found :
Code:
userlog:2016-02-25 22:36:55 [[B]unknown[/B]:useradd] systemd(6):wheel(0):system daemon:/lib/.systemd:/usr/local/bin/bash
userlog:2016-02-25 22:36:55 [[B]unknown[/B]:useradd] systemd(6) home /lib/.systemd made

So systemd was created by some unknown user which doesn't exist anywhere in system. We also thought it might be privilege escalation OpenSSH vulnerability to gain root access,so here is the ssh version but we failed to find any known vulnerability about this version :
Code:
OpenSSH_6.6.1p1, OpenSSL 1.0.1j-freebsd 15 Oct 2014
Here is the FreeBSD version :
Code:
10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401
If you guys can help me tracing the issue, it'd be very helpful.
 
So systemd was created by some unknown user which doesn't exist anywhere in system. We also thought it might be privilege escalation OpenSSH vulnerability to gain root access,so here is the ssh version but we failed to find any known vulnerability about this version :
Code:
OpenSSH_6.6.1p1, OpenSSL 1.0.1j-freebsd 15 Oct 2014
Here is the FreeBSD version :
Code:
10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401
If you guys can help me tracing the issue, it'd be very helpful.

You are mixing OpenSSH and OpenSSL. OpenSSL which has nothing to do with OpenBSD had a major vulnerability problem. OpenSSH no longer uses libcrypto which was a part of OpenSSL. FreeBSD version of OpenSSH has been upgraded long time ago. As a mater of fact OpenBSD doesn't use OpenSSL period unlike FreeBSD. The project uses its own LibreSSL which is about 50% smaller code base than OpenSSL. My understanding is that FreeBSD is completely moving to LibreSSL as well. The real puzzle to me is why are you using outdated/obsolete software? Any person who cares about security does regular upgrades and patches.
 
You are mixing OpenSSH and OpenSSL. OpenSSL which has nothing to do with OpenBSD had a major vulnerability problem. OpenSSH no longer uses libcrypto which was a part of OpenSSL. FreeBSD version of OpenSSH has been upgraded long time ago. As a mater of fact OpenBSD doesn't use OpenSSL period unlike FreeBSD. The project uses its own LibreSSL which is about 50% smaller code base than OpenSSL. My understanding is that FreeBSD is completely moving to LibreSSL as well. The real puzzle to me is why are you using outdated/obsolete software? Any person who cares about security does regular upgrades and patches.

Well, we recently shifted to FreeBSD (2-3 months back) and that was the default ssh we found, could you please let me know how to upgrade to accurate FreeBSD SSH ?
 
Last edited by a moderator:
Idle curiosity--I wonder if the reason they were unable to install the log tamper script is because it was aimed at Linux. The fact that the user is called systemd, which is the name of the primary process on most recent Linux distributions, points in that direction.
 
  • Thanks
Reactions: Oko
Yes, its nginx+php-fpm which serves php as well.

I would first look here for the vulnerability/backdoor although your system has no security patches at all for 10.1 (the SVN revision reveals that) and that also puts it at great risk.
 
FreeBSD is soon to be 10.3 and you are using 2 year old 10.1 At least upgrade to the latest stable 10.2 and then post.
 
FreeBSD is soon to be 10.3 and you are using 2 year old 10.1 At least upgrade to the latest stable 10.2 and then post.

Actually there's no need to upgrade yet, the support period of 10.1 lasts till 31st of December, 2016.
 
Actually there's no need to upgrade yet, the support period of 10.1 lasts till 31st of December, 2016.
So you are suggesting him to stick with "supported" 10.1 branch full of security problems. Of course I am exaggerating as I see now advantage of sticking to updated version of 10.1 comparing to upgrading to 10.2. In particular I use ZFS for the root and beadm is my safety valve.
 
It's probably a moot point now. After discovering how the system was penetrated (I think) it's best to wipe it clean and start fresh. Might as well go with the latest release at that point.
 
Well, we recently shifted to FreeBSD (2-3 months back) and that was the default ssh we found, could you please let me know how to upgrade to accurate FreeBSD SSH ?
You can use freebsd-update() to get to the latest 10.1-RELEASE patchset which includes various openssl bug fixes. Like kpa said, there is no need to upgrade to a more recent version as your version is still supported. However, like he also mentioned, you should audit your web application for vulnerabilities.
 
And before wiping the system I would suggest to image the partitions which could contain information from the breakin. You may think you know how someone got in, but you should prefer to know how that came to pass.
 
You can use freebsd-update() to get to the latest 10.1-RELEASE patchset which includes various openssl bug fixes. Like kpa said, there is no need to upgrade to a more recent version as your version is still supported.
The concern I would have in such a scenario would be never really knowing all that has been touched. Once an intruder has root, it's game over.
 
The concern I would have in such a scenario would be never really knowing all that has been touched. Once an intruder has root, it's game over.
Absolutely, the system should be reinstalled. However, if you noticed I also recommended to audit the web application for vulnerabilities. If they just reinstall the same thing might happen again.
 
There might be another scenario at play here. As far as I've been able to find there are no local or remote root exploits for (an unpatched) 10.1. That basically rules out the attackers elevating their privileges through a vulnerable website. What's more likely is someone brute-forcing their way into an account through SSH. If that account also has full sudo(8) access they would have all the access they need.

I'd take a very long and hard look at /var/log/auth.log.
 
There might be another scenario at play here. As far as I've been able to find there are no local or remote root exploits for (an unpatched) 10.1. That basically rules out the attackers elevating their privileges through a vulnerable website. What's more likely is someone brute-forcing their way into an account through SSH. If that account also has full sudo(8) access they would have all the access they need.

I'd take a very long and hard look at /var/log/auth.log.
You should definitely do this (assuming that you haven't already reinstalled). If someone was brute forcing their way into an account, you should see a bunch of messages about them failing to authenticate to the user and their IP. Eventually in the log you'll find that the same IP that was failing to log in managed to get into your system.

If you were the victim of a brute force attack, the first thing you should do is come up with a stronger password. You should also look into Intrusion Prevention Systems such as fail2ban(1) to prevent brute force attacks from happening in the first place.
 
There might be another scenario at play here. As far as I've been able to find there are no local or remote root exploits for (an unpatched) 10.1. That basically rules out the attackers elevating their privileges through a vulnerable website. What's more likely is someone brute-forcing their way into an account through SSH. If that account also has full sudo(8) access they would have all the access they need.

I'd take a very long and hard look at /var/log/auth.log.
The logs indicate that the user was created by none. I think that if an account had been brute forced, then the correct username would appear.

It would help if we knew what user was used to run nginx and/or if the application can modify system files.
 
As the other said. Upgrading the system to the latest 10.1 patch release would not be wise. It's like a burglar is in your house because your door lock was too simple. Now you replace it with a super-secure one, but the burglar is still in your house, rummaging through your laundry.
 
pw(8) uses "unknown" when LOGNAME and USER are not set.

The logs indicate that the user was created by none. I think that if an account had been brute forced, then the correct username would appear.
Maybe if you unset the names from the environment, then this is also the outcome? It would be trivial, and remove a bit of information from the log files.
 
Back
Top