[Tutorial] Intrusion detection using Tripwire

This is a multi-part post because the tutorial is a bit too long for 1 post :)

Synopsis


This is a tutorial which I've been planning to write for approximately 2 years now. I discovered Tripwire back in my Linux days and I was immediately a fan. I like that it utilizes several features to keep its data safe, ranging from encryption right down to minor "security through obscurity". When I had just migrated from Linux to FreeBSD, roughly 3 years ago, Tripwire "sort off" worked. Things have changed, and after revising a LAN server earlier this week I got immediately reminded about how much I like this program. So... time to finally write that tutorial.

Intrusion Detection System aka IDS

An IDS is a program which can collect all kinds of information from the files (and/or directories) which you want it to monitor. This data is then stored into a database and used to check against later in order to detect any possible changes. There are several IDS solutions available, FreeBSD even includes a simple one in its base system: mtree(8). My favorite though is security/tripwire.

Lets start from scratch!

Although Tripwire provides options to automatically generate some of its files during installation I personally prefer not to. In this tutorial we're going to start out from scratch and slowly work our way towards a fully working setup. The underlying reasoning should be obvious: a little bit of paranoia can't hurt when using security software, so instead of relying on automated scripts we're going to actually make things happen ourselves. This also helps us to understand what is happening, and why.

Now, the first thing to know about Tripwire is that it requires 2 aspects to be set up: a configuration file which tells it where to find all the files it needs (and also to customize the way it operates) and the so called policy file. The policy file tells Tripwire what it should monitor and how.

Here's where things become interesting: all the data which Tripwire uses gets encrypted. So good look to you if you suddenly discover that my policy file is /usr/local/etc/tripwire/tw.pol. Even if you manage to obtain this file then it won't do you much good because the file is, as said, encrypted.

Step one: Generate your keys

We will need a key in order to encrypt something, and Tripwire works no different. Tripwire utilizes two keys: a site (or global) key and a local key. Basically the site key is mostly used to handle the configuration files, while the local key is used to sign the database.

So go to the Tripwire configuration directory which is /usr/local/etc/tripwire by default, then issue this command: # twadmin -m G -L local.key -S site.key. This will generate the two key files. By default Tripwire tends to use the hostname for the local key filename, but this is fully customizable. In this tutorial I'll be using local.key, also for easier access.

Tripwire will also be using two passwords. For ease of use you could consider to use the same password for both keyfiles, but I'd still advice you to use different passwords. One password is used to control the Tripwire configuration (site key) while the other is used to protect the database integrity (local key).

Step two: Set up a configuration file

Tripwire needs to know where to find the files it needs. From the key files, the policy file right up to the database file which we'll be eventually creating. All this information is stored in the configuration file. As mentioned above: all files which Tripwire needs will end up getting encrypted, so we're going to start by creating a textfile which will eventually get processed. I prefer to call this file twconfig.txt so that there can be no confusion with other files.

A good place to determine what needs to be configured is the twconfig(5) manualpage. Not only does this explain to you what format we can use (keyword = value) but it also presents us with the minimal required settings:

Code:
   Required Variables
       The following variables must be set in order for Tripwire to operate.
       The values listed below are assigned during installation.

POLFILE         Default = /usr/local/etc/tripwire/tw.pol
DBFILE          Default = /var/lib/tripwire/$(HOSTNAME).twd
REPORTFILE      Default = /var/lib/tripwire/report/$(HOSTNAME)-$(DATE).twr
SITEKEYFILE     Default = /usr/local/etc/tripwire/site.key
LOCALKEYFILE    Default = /usr/local/etc/tripwire/$(HOSTNAME)-local.key
So all we have to do is to copy and paste this into our configuration file and change some of the values. For example; by default Tripwire stores its database in /var/lib/tripwire, as can be seen above, but that's not the right location for FreeBSD. Instead we'll be using: /var/db/tripwire.

Also notice the $(HOSTNAME) substitution in the file above, used for the local key? If you're following my tutorial then this needs to be changed into local.key. So eventually we end up with this:

Code:
## twconfig.txt, Tripwire configuration file

POLFILE = /usr/local/etc/tripwire/tw.pol
DBFILE = /var/db/tripwire/$(HOSTNAME).twd
REPORTFILE = /var/db/tripwire/report/$(HOSTNAME)-$(DATE).twr
SITEKEYFILE = /usr/local/etc/tripwire/site.key
LOCALKEYFILE = /usr/local/etc/tripwire/local.key
However, I strongly suggest that you read over the twconfig(5) manualpage to see if there are other options which you'd like to use.

For example, even though I prefer vi for most of the configuration editing I do I'm actually pointing the EDITOR option to /usr/local/bin/vim, for the simple reason that it has a larger undo buffer as well as an option to split the screen (which makes it easy to work in more places within the same file).

Another option which you might appreciate, especially if we're a little paranoia, is LATEPROMPTING; this ensures that the password prompts during operation will happen as late and as close together as possible, thus reducing the time when the passwords are kept in active memory.

When we're all done its time to create the actual configuration file by encrypting the textfile. Because Tripwire has no configuration yet we have to specify every file we need (including the keyfile) manually. So here goes: # twadmin -m F -S site.key twconfig.txt. You'll be asked for your site passphrase and after that Tripwire will create the configuration file, by default this will be called /usr/local/etc/tripwire/tw.cfg. You can change this if you'd like by using the -c parameter, but in this tutorial we'll be using the defaults.

Step three: Setting up a policy file

Now that we have successfully configured Tripwire it's time to put it to good use. And to do so we'll need to tell it what it should monitor and how. Tripwire can monitor nearly every aspect of a file system entry (file or directory); from access time (+a), modification time (+m), file size (+s) and file type (+t) right down to file or group owner (+u and +g respectfully).

Of course these won't be enough. After all: if you keep track of file size then this gives you no guarantees that nothing changed. Lets say I have a textfile which contains: "Peter likes playing Minecraft", very sensitive information indeed ;) These are 30 characters.

So now lets look at this: "Peter likes playing Warcraft!". I don't like Warcraft, I don't even know if this is actually a game of some sort. But if we were only keeping track of file statistics then we wouldn't even be noticing this change at all. What's that? Access and creation dates of the file you say? That's nothing which touch(1) can't handle. I'll use stat(1) up front to check for the current timestamps, and after I changed the file to my liking I'll use touch to set them back again.

And that's why Tripwire can also generate hash checksums. Using not one but four different methods: CRC32 (+C), Haval (+H), MD5 (+M) and SHA (+S), all of which can be used either individually, grouped or all at once. If you like to learn more about cryptographic hash functions then check out this wikipedia page.

Now, my policy file example consists of several rules where each rule can check one or more files or directories. However, Tripwire is a very flexible program and it allows us to define these rules in more than one way. Instead of one rule which checks multiple files I can also set up one rule per file (or directory). But for clarity sake I'll be focusing on one specific format only, while also briefly mentioning the other format near the end of this section.

So:

Code:
(<rule definition>)
{
    /file/system/entry/to/monitor -> <monitor directives>;
}

Rule definition

Here we configure the attributes of our rule. Things like the rulename, severity and optionally an e-mail address which will be used if a violation has been discovered, this is defined using emailto.

Severity can be a number ranging from 0 (default) to 1,000,000. Its primary use is to control which rules to check. By default all the rules in the policy will be checked against the database, but you could also set it up so that only a specific selection of rules is checked, for example only those which have a severity of 80 and higher.

Monitor directives

Here you configure the properties which I mentioned earlier, they determine which aspects are being monitored and which hash functions are to be used. A directive can be turned on by using a + and turned off, or excluded, by using a -.

Here is a full list of all the directives which you can use, it's actually from the header of my own Tripwire policy file:

Code:
## Directives
# a - Access timestamp                  m - Modification timestamp
# b - Allocated blocks                  n - Number of links
# c - INode timestamp                   p - Permission & filemode bits
# d - ID of device                      r - ID of device pointed to (/dev)
# g - Group ID                          s - File size
# i - INode number                      t - File type
# l - Size increase ('growing')         u - File owner UID
#
# C - CRC32 hash, H - Haval hash, M - MD5 hash, S - SHA hash.
For a complete overview I suggest that you check out the twpolicy(5) manualpage.

Now, before we set up our first rule there are 2 more things which I'd like to address: variables and directives.

Policy variables

As you can see there are 14 monitoring directives which we can use. And trust me when I say that you might be able to remember them if you've been working with Tripwire for a while, but after a few weeks you won't anymore. And then it can become quite bothersome to add a new file or directory to monitor, because what directives are you going to use again?

So to make things a bit easier on us we can also use variables. There are 6 variables defined by default:

Code:
## Predefined
# ReadOnly      Value: +pinugtsdbmCM-rlacSH
# Dynamic       Value: +pinugtd-srlbamcCMSH
# Growing       Value: +pinugtdl-srbamcCMSH
# Device        Value: +pugsdr-intlbamcCMSH
# IgnoreAll     Value: -pinugtsdrlbamcCMSH
# IgnoreNone    Value: +pinugtsdrbamcCMSH-l

And of course we're also free to create our own. For example: if you look closely then you'll notice that IgnoreNone also includes the +a (access time) directive. This basically means that as soon as a file gets accessed then Tripwire will notice it. Can you imagine what will happen if you use this for a directory such as /bin?

So to prevent all that nastiness I created some variables of my own:

Code:
CheckAll = $(IgnoreNone)-a;
CheckLog = +iptugl;

A variable can be used in almost the same way as in a shellscript: $(variable). So if I were to check a directory of binaries and I wanted to check everything but the access time then I'd use this: "/bin -> $(CheckAll);" (without the ""'s obviously). I hope you can agree with me that it's much easier to use and understand than the substitute: "/bin -> +pinugtsdrbamcCMSH-al".

Policy directives

Now, I'm only going to address these briefly because in a regular situation you won't be needing these. But it is good to realize that they exist, because they can make your life a whole lot easier, especially if you're using Tripwire on a server farm or if you need to bugfix your policy file.

@@ifhost, @@else, @@endif. These can be used to include rules in your policy file which are only to be used on specific hosts. This allows you to maintain one policy file and then use that across all your servers. For example:

Code:
@@ifhost pks.intranet.lan
  /usr/local/etc/pksd.conf -> $(CheckAll) (rulename=GPG_conf, severity=80);
@@endif
This would ensure that Tripwire will only check for /usr/local/etc/pksd.conf while it's being used on my pks server. As you can see this rule also looks much different than the template I shared at the start of this section. This is the other format which I mentioned earlier.

@@print and @@error. These can be used to bugfix your policy file, they will output a message to either stdout or stderr which allows you to determine which parts Tripwire is processing.

And finally there's @@section and @@end. You can ignore @@section because its use is pretty redundant. Some package maintainers on Linux use this in their template but it's meaningless because all it basically does is to allow Tripwire to ignore certain policy sections when it's being used on multiple operating systems, Windows in particular. Needless to say: I'm focusing on FreeBSD usage here.

@@end can be useful, this can mark the physical end of the policy file. However its use isn't required.

Our first useful policy file

Code:
## twpolicy.txt

(rulename = "Tripwire config", severity = 100)
{
        /usr/local/sbin/tripwire                -> $(CheckAll);
        /usr/local/sbin/twadmin                 -> $(CheckAll);
        /usr/local/sbin/twprint                 -> $(CheckAll);
        /usr/local/sbin/twtest                  -> $(CheckAll);
        /usr/local/sbin/siggen                  -> $(CheckAll);
        /var/db/tripwire                        -> +iptug (recurse=false);
        /usr/local/etc/tripwire                 -> $(CheckAll);
        !/usr/local/etc/tripwire/tw.pol.bak;
        !/usr/local/etc/tripwire/tw.cfg.bak;
        !/usr/local/etc/tripwire/twconf.txt;
        !/usr/local/etc/tripwire/twconfig.txt;
        !/usr/local/etc/tripwire/twpol.txt;
        !/usr/local/etc/tripwire/twpolicy.txt;
}
Here is the first rule of my own Tripwire policy file. As you can see this rule checks my Tripwire installation itself. You'll probably recognize $(CheckAll); this is a variable which I set at the beginning of my template file. (see further above and also in the template which I'm sharing). And you can also see some manually defined directives (+iptug).

But there are also some things which I haven't mentioned. recurse is an attribute which allows you to control how deep Tripwire should scan a directory. When set to false, like here, it will only check the directory entry itself but not its contents. In the example above I use this to check that the database directory doesn't get tampered with, but without checking the actual database files themselves.

The exclamation mark (!) is a stop point, this literally tells Tripwire to ignore certain files or directories. Here we also see the only small caveat of a Tripwire policy: it's not easy to separate file entries from directory entries. Alas: first I told Tripwire that it should check all the files in its configuration directory (/usr/local/etc/tripwire). However; the text files which we're initially editing are not used by Tripwire. So there's no reason to include these in our checks. The same applies to the files with the .bak extension; the very moment you update your policy or configuration file then Tripwire will also make a backup copy. It won't be actively used, so there really is no need to include it.

If you want to simply copy/paste this rule to get you started with an example then I suggest to include my template. Otherwise replace the $(CheckAll) with either $(IgnoreNone)-a or +pinugtsdrbamcCMSH-al.

Now it's time to actually create our policy file!

Just like with the configuration file we need to tell Tripwire to encrypt it. However, because we already have our configuration file in place there's no need to specify any keys. As such we're now using: # twadmin -m P twpolicy.txt. You'll be asked to supply your password again and after that we're all set.
 

Attachments

Last edited:
Step four: remove obsolete files and initialize the database

The first thing we want to do is to remove any text and backup files: # rm *txt *bak, these won't be used by Tripwire. This is also where 'security through obscurity' comes a bit into play. Because an attacker who is not familiar with Tripwire will have no way to easily check which system areas are being monitored. However, keep in mind that the main function of the keys is to prevent people from easily (re)creating their own configuration, not so much to prevent them from accessing it. You can see so yourself when you use: twadmin -m p. All you need is read access to the site key and the encrypted policy file, after which twadmin will easily decrypt it for you, no root privileges required.

So, to initialize the database: # tripwire -m i. Eventually you'll end up with a .twd file in the /var/db/tripwire directory, this is your database.

Step five: perform a consistency check

Once again I'd like to refer you to the manualpage (tripwire(8)) for more specific information about the several command line parameters. For example, I can well imagine that if you're using Tripwire using a cronjob then you probably don't want any excessive output. For that you'd be using --silent.

But for now we're going to see if it found any changes, use: # tripwire -m c | less. You'll see something like this:

Code:
Parsing policy file: /usr/local/etc/tripwire/tw.pol
*** Processing Unix File System ***
Performing integrity check...
And depending on your policy, the size of your database and the speed of your server this could take a while. Eventually you'll be presented with an overview of everything which Tripwire has discovered about your system. It'll start with an overview showing all the different rules and what was found (any added, removed or modified files).

Here's what it looks like on my end right now:

Code:
===============================================================================
Rule Summary:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                       Severity Level    Added    Removed  Modified
  ---------                       --------------    -----    -------  --------
* Tripwire config                 100               0        0        3
  Ports                           70                0        0        0
* Security services               100               0        0        8
  IRC services                    50                0        0        0
  INN                             20                0        0        0
  Java                            20                0        0        0
  System binaries                 80                0        0        0
  System libraries                80                0        0        0

Total objects scanned:  18874
Total violations found:  11
If you scroll down you'll find an overview of all the changes which were detected. Let's look at my Tripwire config overview:

Code:
-------------------------------------------------------------------------------
Rule Name: Tripwire config (/usr/local/etc/tripwire)
Severity Level: 100
-------------------------------------------------------------------------------

Modified:
"/usr/local/etc/tripwire/macron-local.key"
"/usr/local/etc/tripwire/site.key"
"/usr/local/etc/tripwire/tw.cfg"
So now its time to determine if this was actually a break in attempt, after which we should probably start some kind of security procedure, or if this was intended. If intended then we need to update our database with this new information.

Step six: checking & updating the database

There are two ways to update the database. You can simply tell Tripwire to update it unattended, but I strongly suggest against doing this. Think about it: right now (time of writing) I have to deal with 11 changed files. In this case I know exactly what happened, so I might be tempted to simply update the database.

But what would happen if some intruder was active on my system right now (time of writing)? Then Tripwire would happily include all the stuff he messed up so far in my database, making it quite likely that I'll miss the whole thing entirely.

So we're not going to do that. Instead we'll be checking the database again, but this time also tell Tripwire to start an interactive update mode. This allows us to check exactly what got changed and to determine if Tripwire should update the database or not.

We'll be using the same command as before, but with a new parameter: # tripwire -m c -I.

After a while you'll see the same output as before, but this time all the entries have mark boxes in front of them. This is what my previous violation looks like now:

Code:
-------------------------------------------------------------------------------
Rule Name: Tripwire config (/usr/local/etc/tripwire)
Severity Level: 100
-------------------------------------------------------------------------------

Remove the "x" from the adjacent box to prevent updating the database
with the new values for this object.

Modified:
[x] "/usr/local/etc/tripwire/macron-local.key"
[x] "/usr/local/etc/tripwire/site.key"
[x] "/usr/local/etc/tripwire/tw.cfg"
Now, "modified" doesn't tell me anything of course. Because what got modified exactly? It could be a permission bit, file size, etc. To see this information we need to scroll further down:

Code:
Modified object name:  /usr/local/etc/tripwire/macron-local.key

  Property:            Expected                    Observed
  -------------        -----------                 -----------
  Object Type          Regular File                Regular File
  Device Number        181                         181
  File Device Number   0                           0
  Inode Number         573023                      573023
  Mode                 -rw-r-----                  -rw-r-----
  Num Links            1                           1
  UID                  root (0)                    root (0)
  GID                  wheel (0)                   wheel (0)
  Size                 931                         931
  Modify Time          Sat Jul  2 16:00:10 2016    Sat Jul  2 16:00:10 2016
* Change Time          Sat Jul  2 17:18:25 2016    Sun Jul  3 18:51:04 2016
  Blocks               8                           8
  CRC32                Btrxrs                      Btrxrs
  MD5                  CPUwpuvczWH5jMh/uerwjr      CPUwpuvczWH5jMh/uerwjr
  SHA                  JxPCWMkEc12CHLKTz22k1OKqOW8 JxPCWMkEc12CHLKTz22k1OKqOW8
  HAVAL                Czg0xxM2Ljr14gAGlG/QdW      Czg0xxM2Ljr14gAGlG/QdW

Modified object name:  /usr/local/etc/tripwire/tw.cfg

  Property:            Expected                    Observed
  -------------        -----------                 -----------
  Object Type          Regular File                Regular File
  Device Number        181                         181
  File Device Number   0                           0
  Inode Number         573025                      573025
  Mode                 -rw-r--r--                  -rw-r--r--
  Num Links            1                           1
  UID                  peter (1001)                peter (1001)
  GID                  wheel (0)                   wheel (0)
  Size                 4586                        4586
* Modify Time          Sat Jul  2 17:45:03 2016    Sun Jul  3 16:15:33 2016
* Change Time          Sat Jul  2 17:45:03 2016    Sun Jul  3 16:15:33 2016
  Blocks               16                          16
* CRC32                A8rIx8                      A370Ox
* MD5                  Ay1vsii221Lhm6aUXxvsDL      BR7xiouQyt6bJvKaK7aj4I
* SHA                  Ewi7H3BCCExu5AEOJaYNTPfVZPn KFiEESujeheJzmaLBs9KCLAo8mR
* HAVAL                ChXGGJiXtWa9VjJF9468o5      BONIy+G9zxMbFeTNp7wox6
As you can see the keyfile changed, but not internally. This is actually correct because I renamed it, then renamed it back. And Tripwire detected this. The configuration file is another example: I re-created it, so I encrypted the same configuration file again. Although nothing was edited (see the filesize?) all the hash functions clearly noticed the change itself.

And what about the database integrity?

If you're a little paranoid like me then you might notice that there's one thing missing: what would happen if someone were to tamper with the Tripwire database itself? Now, Tripwire does have some integrity checks of its own in place, but you know the saying: safety first.

Well, for this I suggest to use another security program which I also deem invaluable on any server: security/gnupg1. I prefer version 1 over 2 because the latter uses loadable libraries.

The fun thing about GPG is that it can create detached signatures. And since we told Tripwire to only check the database directory itself we can easily place the signature there as well. So go to /var/db/tripwire and use: # gpg -sb *twd.

If you want to verify the file integrity then all you need is the public key of the user who signed it (root in my example), after which you can use this command: gpg *sig to verify the database integrity (also from within the database directory of course).

And there you have it, my Tripwire tutorial. Hope some of you can put this to good use.
 
Thank You for this excellent tutorial.
Five years later and I noticed that twtest is no long part of tripwire.
So the rule fails on this in twpolicy.txt
/usr/local/sbin/twtest -> $(CheckAll);
 
Back
Top