Refreshing ELK and CERT tools locally, and sharing

setjmp

setjmp

Hi,

Recently I have been crash coursing on some tools and found some outdated in ports, others without the full options (which I needed) and so on. I started updating some of these, and adding those that were absent along the way.

When things are more solid, I intend to pass the existing ones onto the current active maintainers, and properly submit the new ones as well. In the meantime, all in process is accessible at:

https://github.com/setjmp2013/freebsd-ports

Among them are:
yaf (2.7.1 with libp0f support, mostly the original one with a libp0f added port below. Still looking at getting other features working though pretty sure the special devices probably are more linux in nature , not sure yet though, research needed)
logstash (1.5.0 - the file input tag still isn't working, in progress, noting what the original porter went thru.)
kibana4 - not there yet even, though will be. Got it running a lot, just would be handy as a service
silktools (3.10.2)
libp0f (2.0.8) with updated p0f.a as well (needed for yaf --p0f-print --p0f-fingerprints...)
libfixbuf (1.6.2)

By the end I hope to have all the latest parts around the ELK stack, nearly everything around the CERT tools at NETSA, and other items discussed at FloCon... Solid with full capability. And likely others.

What sparked this? Hehe, my main workstation for the last 5 years is becoming obsolete, and looking back out of windows hereafter (was pretty much FreeBSD 90% since about 1994 prior to that). Feels good to command my workstation again (as well as the router/lan servers that have always been BSD :) )

Thanks to everyone that has kept FreeBSD alive and so strong. And porting could never be easier it seems. Thank You Very Much. Feel free to contact me, watch the repo, or suggest a better way as needed :) Some of these will be submitted sooner.

Sincerely,
Eric
 
That's very cool and a nice little stash of security related goodies. Do upstream it when you have a working patch. At least getting the bug in with a rough draft patch at https://bugs.FreeBSD.org/bugzilla/ will help start a dialog between you and the maintainer. If there needs to be any revisions on your PR you and the maintainer working on it together will likely be faster than working on it alone.
 
Sounds good. Today I have realized a few issues and anyone attempting these early should look with care too (yaf-file-mediator particularly coredumped once).

Kibana4-devel actually builds/installs, and does run ok. Though I need to focus more on the rc.d around it which is (non-existant in the distro) as well as mitigating the node dependency. The distro actuall includes one for the particular snapshot (Darwin/MAC rather than freebsd FreeBSD which will not run.). However one field in the sh startup script replacing the path to that node with `which node` works fine. And while the elastic crew seems to focus around node010, it runs fine here on the latest. I think that means of finding node is a little sloppy though.

And I also need to define a rc.d script for it too. I will probably be submitting both Kibana4 pretty quick since they are nearly standalone with only a node dependency anyways. 4.0.2 is the current release, while 4.1.0-devel is ongoing, but pretty stable at this point while they have been busy already with the documentation for release.

The devel distfiles/makesum could potentially change daily or even more often. So that might be an issue, since the runnable dists are from their site built, and not in the Github (source to build with npm, bower, etc).

More will be revealed...

Eric
 
Upstream re-rolling distfiles and changing the checksum daily would be unacceptable. There would have to be a static source to use. Hopefully there is one. There used to be a capability for this but after some discussion on the massive security implications it was rightfully removed.
 
It's the norm for the devel there which is very active and hopefully to a release pretty soon. The biggest changes seem in them getting the documentation towards release at this time. The other option would be to build it from the git repo with npm and bower. Though for the main 4.0.2 it doesn't change (the one not called kibana4-devel)

I can understand that on the security front. Not all distros are well controlled and could become compromised and allow root privs to bad code hehe. That brings another thought on this, a kibana user to restrict it might be wise. Other than log file permissions it would have no issues with that. It's just a node app similar to the typical MEAN stack.

Though for faster buffering/production nginx makes a great proxy to better serve it web as well as enable authentication + SSL
 
I will get the release version cleaned and best case and submitted tonight since it does not change, while I use the devel more often here and when it releases the best cases will be in place for the update of the release version. :)
 
Appears that there was another port of kibana4 submitted on the first and unlike my first port in a couple of years, it's cleaner IMO. In the meantime though I need these tools now and that is the drive. I am carefully looking at libp0f as it is much in common with p0f, however a different build with library which is needed for fingerprinting in yaf/silk. It does create a p0f command, however doesn't create a few other tools from the p0f2 port. I'm thinking it will depend on that and simply include the library instead, headers, and none of the executables (which won't even use the library) and I suspect of the etc fp files are going to be fresher, they will be from the existing p0f build. Hence a addon which does no harm whatsoever to the main tools.

Actually the p0f2 port has the original 2006 p0f.fp rather then the revised 2012 with a few more OS. I'll mitigate that and for function at the moment will have a etc/libp0f folder for the lib one (and the rest of the fingerprints files that should be identical on both).

Hopefully this will be submitted tonight :)
 
You must log in or register to reply here.
Back
Top