Writing Prelude LML rules

In this post, we will cover the basics of adding your own rules to Prelude-LML, which is Prelude's own log monitoring analysis engine. Highly optimized in C, Prelude-LML, comes with numerous rules built-in for everything from SSH authentication to Netscreen firewall rules.

To start we will navigate to the default LML ruleset directory, which is located in /usr/local/etc/prelude-lml/ruleset - unless you specified otherwise.

Our Example

For example purposes, we'll use the made up syslog entry below to follow along with. It shows the date, a host named some_hostame, with a service called mylogger_service, that printed some message.

Aug 22 05:22:05 some_hostname mylogger_service: We have an imortant message here.

Setting Up

The first thing you want to do is have a look at pcre.rules. The pcre.rules file, provides a way to match on certain criteria that all rules in a certain set will all match on, a lot of them are based on services, such as ssh, which allows LML to limit how many log messages are processed against which rules.

Since our service, mylogger_service, is new and no other specific rulesets apply to it, we'll add a line to pcre.rules to only apply our new rule (which we will add to its own file later, called local.rules). Adding the following to pcre.rules will do just this:

regex=mylogger_service; include = local.rules;

What this does, is only process rules in the local.rules file, if the log entry contains "mylogger_service" in it.


Adding a Rule to its own Ruleset

So now we see that we have prepped our new rule file (local.rules), to receive any log entries that contain the line mylogger_service in them. We next need to add rules to our local.rules file for further processing and alerting on any matches. Here is what we will add to local.rules for our example:

# Detect important messages from the mylogger_service.
# LOG:Aug 22 05:22:05 some_hostname mylogger_service: We have an imortant message here.
regex=important message; \
classification.text=Important Message Detected.; \
id=32001; \
revision=1; \
assessment.impact.type=other; \
assessment.impact.severity=low; \
assessment.impact.description=An important message was detected with the mylogger_service; \
last;

Stepping through this example, we see the following:

- A comment line that describes what this rule is about.
- A LOG line that shows an actual example syslog entry for what we are looking for.
- regex, that is what we are matching on, any potential regex can be used here. Such as character classes (\w,\d) or wildcards (.,*,+).
- classification.text, here is the main alert text for this rule
- id, which differentiates on this particular rule
- revision, bump this up by one as you make production edits
- impact type, which can be things such as admin, user, other, etc.
- impact severity, such as low, medium, high
- impact description, a longer description of what most likely is referenced in classification.text
- last, which basically tells LML to stop further processing if this rule matches

Many more IDMEF fields may be used, such as references or process names.

When mylogger_service is seen in a syslog entry that LML processes, it will process this entry against all the rules in the local.rules file (which is how we set this up in pcre.rules). Furthermore, if the entry matches our regex of "important message", we will get an alert with severity of low, a message text of "Important Message Detected", and the various other settings we have set.

Conclusion

This example showed a simple way of adding rules to your LML engines. You may have noticed when looking in pcre.rules, at the top is a "best practices" section on creating/adding LML rules. For much more extensive information, look here.

High Availability Prelude Central Services

I recently posted to the Prelude wiki, a sample configuration for providing high availability for your central Prelude services.

Basically what the configuration provides is a setup across two servers to host these services: Manager, Correlator, Apache, MySQL, and Prewikka. It is purely a fault tolerant scheme, as opposed to a performance booster. Although, you could spread the load to increase performance - such as with a split of the web interface to one while the other provides database interaction for incoming events, or offloading things such as reporting and backups to the secondary.

MySQL v5 is required to avoid potential auto increment collisions when doing multi-master replication. Other than that, there really should be minimal changes needed for using various versions with regards to the other software pieces. Either host in the pair, is capable of taking over as the primary - only caveat is heartbeat is for machine failure, not service failure. So....you will still need your application-level monitoring (ie. Nagios or other snmp-based solution) in place to be notified of service issues.

Prelude Registration Server

As anyone who has used Prelude, you will know that registering a sensor with a Prelude Manager/Relay, is the first step in having your sensor send alerts into your Prelude framework. Usually a combination of, (a) running 'prelude-adduser registration-server' on the manager/relay, and (b) running 'prelude-adduser register' on the sensor you are adding - followed by accepting the registration on the manager, etc.

In this post, I will show a quick way of setting up a pseudo-daemonized instance of the Prelude registration server, that will auto-accept the sensor registration. This comes in handy when you have a bunch of sensors to register, yet you don't want to constantly be going back to the manager console to acknowledge each individual sensor registration.

On the manager side, first install the screen utility.

Continuing on the manager machine, I usually create an init script, that has the process being the following:

/usr/bin/screen -d -m /usr/local/bin/prelude-adduser registration-server prelude-manager --passwd=somepassword --keepalive --no-confirm

What this command says is, have screen fire up this prelude command while detaching the screen session - thus putting it in the background, much like a daemonized process (ie. not running active in your console). The 'prelude-adduser registration-server' command runs using the prelude-manager analyzer profile. The key additions to the command, are the use of a pre-shared password, and the keepalive and no confirm options. The pre-shared password is used by the sensor registering, and the no confirm eliminates the need to accept the sensor registration on the manager each time. Finally, the keepalive option, does not cause the registration server to exit after a single successful registration on the manager side.

Finally, running the following on the sensors needing to register (in this example, a snort sensor):

prelude-adduser register prelude-snort-profile "idmef:w admin:r" 192.168.1.2 --uid snort --gid snort --passwd=somepassword

The above does the normal sensor registration pieces of specifying the profile in use, prelude permissions to use, and user/group to allow access to the sensor profile. The important addition, is the use of the pre-shared password that was specified in the registration server running on our manager.

Prelude-LML as a UDP Listener

If you have ever used Prelude's Log Monitoring Lackey (LML) before, you probably know that the default behavior is to process these log files that happen to be local to the server. What I will show in this post, is how to set Prelude-LML to listen on a UDP port, and accept messages there from Syslog-NG or anything that can transport the messages to it via UDP.

Having Prelude-LML listen for UDP messages, is as easy as adding the following to your startup options:

--format syslog --udp-server=192.168.1.25:10514

Basically, what the above says, is for the Syslog format, listen on the interface with IP address 192.168.1.25 for UDP messages on port 10514. Listening on UDP, can also be specified in the format section in prelude-lml.conf of your choosing, whether a format for Syslog messages or maybe Apache log formats, etc.

So now you have a Prelude-LML instance listening for UDP port 10514 messages on its appropriate network interface. Let us also assume you already collect your Syslog data from all your hosts in your environment centrally via TCP, to a server running Syslog-NG. All you need to add to your Syslog-NG configuration would be something similar to the following:

destination d_prelude_lml{ udp("192.168.1.25" port(10514)); };
log { source(s_tcp); destination(d_prelude_lml); };

The above assumes, you already collect your logs centrally from all hosts via TCP, and have labeled the source as s_tcp in your Syslog-NG configuration - adjust accordingly. We created a new destination call d_prelude_lml, pointing it to use udp with our IP and port combination we setup on our Prelude-LML server. The log statement we have added, combines the source TCP statement, with our new Prelude-LML destination statement to send any logs coming into our Syslog-NG central log server from hosts over TCP out over UDP to our LML instance. Allowing all the logs to be processed via Prelude-LML, but without, for instance, being stored on our LML server.

Prelude-LML is highly configurable, many more options and combinations can be made to tailor it for your environment. I hope this provided one option of adapting it to an architecture that already collects their Syslog data centrally.

Prelude for Event Management (ie. SIM)

The hybrid IDS, or "meta-IDS", as described by the project's founder, makes an excellent choice as a SIM/Event Management tool in what is a sparse area of the Open Source world. Not only is it "good enough" to justify not throwing hundreds of thousands of dollars at an Arcsight or Network Intelligence solution, it has far exceeded my expectations in many areas.

I'd like to commence this first post on Prelude, by detailing five things that I most like about it. In no particular order, let us begin the Prelude Five.

** The Framework **

When Prelude first started out, it was very much an IDS, with a NIDS of its own, etc. But realizing that the framework itself was the crux of this project, was probably the wisest design decision made.

From deploying and configuring numerous sensors, from Samhain to Snort, the Prelude library makes it seamless to connect agents/sensors into the existing framework. TLS encrypted channels secure the data in transit, from your agents to Prelude Managers at a central site or in a relaying configuration to distribute the load. Registration of the individual agents/sensors/relays, allows you to only accept and connect systems that you specifically specify.

The distributed architecture allows for various systems, along the route to your central site, to spool events and data when various relays, etc. are unavailable. Both allowing you to spread out your load, and have fault-tolerance baked in.

** Versatile API **

Have a sensor or tool that has data you want sent to Prelude and it isn't one of the natively supported sensors? No big deal! Prelude's API allows you to use C, Perl, or Python to format your alert/event in IDMEF and pump it into the Prelude framework. Install the Prelude library, format the data appropriately, and use one of the languages to create your client and away you go.

**Prewikka**

If there is one thing that the Open Source world of projects frequently lacks, it is a nice GUI interface. Prewikka, the web interface to Prelude, is one exception to this statement.

The ability to view alerts, drill-down to event data, view agents - and not to mention the additional features in a commercial add-on (that includes graphs, stats, ticketing, reports, etc.). The Python-based code, runs either as a CGI program, mod_python module or within its own self-contained web daemon. All in all, a very well thought out interface that is both usable and constantly improving.

**Analysis Engines**

Two analyzers that come with Prelude, are the Correlator and LML (log monitoring lackey).

Let's say you want to analyze your syslog data or even your windows event logs (exporting them via ntsyslog in a syslog format), this is what LML is for. Heavily weighted in regular expressions, you generate alerts from your logs on failed SSH logins, firewall events, mail system abnormalities, and all the usual suspects that make up a standard log analyzer. But here's the real catch....most of these open source log analyzers are written in Perl, while LML is "highly optimized" in C! I've personally seen LML with the default ruleset not break a sweat on thousands of large syslog messages per second, when SEC (a very fine pattern matching program itself) get brought to its knees with a much smaller, tuned ruleset.

Then there is the Correlator, still technically not released officially, and with a very minimal ruleset at the moment, but nonetheless extremely usable. What would an event management solution be if it couldn't take in Snort alerts, and correlate them with firewall or syslog events. This is just what the Correlator does, by leveraging Prelude's heavy reliance on the IDMEF RFC for event format, allowing correlation across any of the fields.

Best of all, both with Correlator and LML, you can add/change/delete rules as you see fit. No black box, enable or disable only options for you - this is Open Source my friend!

**Modular Plugins**

So you want to pump your data into a database, who wouldn't....so enable the database plugin, make a few changes, and presto. Maybe you don't want to log to the database and you want all your alerts in a flat text file, sure that can be done. Or you could purchase, for minimal dollars, the SMTP alerting plugin to send alerts to you via email.

All the plugins are enabled/disabled in the Prelude Manager, and allow various things to happen with this data you are gathering in from your various sensors. Remove what you don't need, and add what you do - you have full control of this.

If modular plugins aren't enough, you can stack them too. Stacking the plugins allows you to take the benefits of one plugin, say the filtering capabilities of the IDMEF-Criteria plugin, and hook it to the Thresholding plugin (limiting the number of events processed), then hooking that to the Database plugin. Effectively, creating a stack of plugins hooking into the next to manipulate a chain of events as you see fit, on the exact events or data that you require.

As you can see, Prelude has plenty to offer in the Event Management/SIM space, is actively developed and supported, with both an Open Source community and a commercial outfit providing both support and enhancements.