[gentoo-user] Re: syslog-ng configs for separating warnings/errors and different types of traffic

[James]

Tanstaafl <tanstaafl <at> libertytrek.org> writes:


> I'm very interested in what are best practices, and what others do as
> far as separating out different types of messages in their logs.

First list all of your resources you are going to monitor:
webservers? DNS activity/servers?  Security? Specific ports?
Users?  

Networks product a cornicopia of data to collect, monitor, store
and analyze.

> I've always just sent everything to /var/log/messages, and this is not a
> very heavily loaded box so it hasn't been a big problem, but I'm working
> on a new server and would like to do some separation.

Ok, if your network is expanding and you've listed what you 
need to do, then look for tools that will help make sense, quickly
of all of that logged data:

reportmagic, analog, awstats, just to nake a few.

> I'd still like everything to go to /var/log/messages, but I'd like to
> also send certain types of messages to different logs to simplify
> troubleshooting, etc - ie, I often peruse the logs with:

> egrep '(reject|warning|error|fatal|panic):' /var/log/messages

If you use custom (CLI) or scripts, you'll need to think about
collecting that up and what sort of analysis you want/need to run.


> But I'd like to actually feed all of those messages to a separate log,
> for easier tailing.

systemd is clouding these issue versus syslog(ng). so whether or not you are
or are planning to use systemd is also a factor you need to incorporate
into you decision. If you are currently using cron, plan on moving to
"cronie" as it is actively maintained and cron is not.

There are many, many different and valid  approaches to this
issue, so first go out and read about ideas related to what your
need to do (Googling is your friend). Collecting up data, into
a singular file allows you to see what occurs in a chronological
fashion, and is easiest for a small netowrk.

Once you go creating many different log files, you now need to
develop a strategy to priortize what you need to monitor.

Are you reviewing these logs file, by hand?  Dailey, weeking
or real time monitoring?  What is your first priority?
Security? System Admin(resource utilization)?  keeping an ecomerce
server/farm fast and responsive?   Following you hacker budies around
the net? (inside your net?)......  

DEFINE what you need to do first. Then look for tools to ease
the job. Implement, test, refine...... "rinse and repeat".

What you are asking, is a life_long quest for most of us, it's never
done, always there and fundamental to running large amounts of
hardware and software, hopefully in a pristine manner.

Oh yea, learn abount "managed switches" and keeping track of what's
going on inside of your routers, too.

http://sixrevisions.com/tools/10-free-server-network-monitoring-tools-that-kick-ass/

 http://www.jffnms.org/

> Charles

hth,
James