Sunday, June 10, 2007

Prelude-LML as a UDP Listener


If you have ever used Prelude's Log Monitoring Lackey (LML) before, you probably know that the default behavior is to process these log files that happen to be local to the server. What I will show in this post, is how to set Prelude-LML to listen on a UDP port, and accept messages there from Syslog-NG or anything that can transport the messages to it via UDP.

Having Prelude-LML listen for UDP messages, is as easy as adding the following to your startup options:

--format syslog --udp-server=192.168.1.25:10514

Basically, what the above says, is for the Syslog format, listen on the interface with IP address 192.168.1.25 for UDP messages on port 10514. Listening on UDP, can also be specified in the format section in prelude-lml.conf of your choosing, whether a format for Syslog messages or maybe Apache log formats, etc.

So now you have a Prelude-LML instance listening for UDP port 10514 messages on its appropriate network interface. Let us also assume you already collect your Syslog data from all your hosts in your environment centrally via TCP, to a server running Syslog-NG. All you need to add to your Syslog-NG configuration would be something similar to the following:

destination d_prelude_lml{ udp("192.168.1.25" port(10514)); };
log { source(s_tcp); destination(d_prelude_lml); };

The above assumes, you already collect your logs centrally from all hosts via TCP, and have labeled the source as s_tcp in your Syslog-NG configuration - adjust accordingly. We created a new destination call d_prelude_lml, pointing it to use udp with our IP and port combination we setup on our Prelude-LML server. The log statement we have added, combines the source TCP statement, with our new Prelude-LML destination statement to send any logs coming into our Syslog-NG central log server from hosts over TCP out over UDP to our LML instance. Allowing all the logs to be processed via Prelude-LML, but without, for instance, being stored on our LML server.

Prelude-LML is highly configurable, many more options and combinations can be made to tailor it for your environment. I hope this provided one option of adapting it to an architecture that already collects their Syslog data centrally.

No comments:

Post a Comment

Basic Pig usage to process Argus data

Some quick notes on testing out Pig in local mode to process some basic Argus data. Argus Capture a sampling of network traffic with Argus a...