On Fri, 21 May 2010 11:15:58 +0200 Hannes von Haugwitz <han...@vonhaugwitz.com> wrote: > Dan D Niles wrote:
> > suppose you have a program that outputs: > > > > This is a failure test > > > > This would show up a a SECURITY event. It isn't really a SECURITY > > event, so you exclude it in violations.ignore.d. Now it does not show > > up as a SECURITY event, but it also does not show up as a SYSTEM event. > > That behavior is not what I would expect. This (old) bug was marked wonfix, but I think it should be fixed - I suspect many people find logcheck's current design confusing. And I think there is a case for simplification, which i set out below. - which would close this bug. I just wasted a few hours failing to understand the logic in logcheck's 'security events', (only to find i was doing an entirely unrelated stupid thing with my rules - but getting to that realisation took far too long because logcheck is over-complicated) I thought i would write down how logcheck works, which will illustrate why I think it's a more complicated than it needs to be Currently the approach to filtering rules is the following 4 steps - I would suggest that step 2 and 3 are needlessly complicated and not actually what users want. -------------------------------------------------------------------------------------------- 1. Get log entries from logs/journal 1a. Extract recent log entries (function logoutput) 1b. Remove blank lines, sort and remove duplicates (function: none, done in the top-level code) 1c rules are also cleaned - blank lines and comments are removed (cleanrules function) 2. Find all lines that _do_ match patterns in /etc/logcheck/cracking.d (grepoutput function does all of steps 2,3; actual work is done by 'cleanchecked') - im not going to properly describe this but it is similar to step 3 below, although it depends on whether SUPPORT_CRACKING_IGNORE is set to 1 or not. Report anything as "attack alerts" 3. Find all lines that _do_ match patterns in /etc/logcheck/violations.d If a file /etc/logcheck/violations.d/foo finds any matches, then function greplogoutput filters these through /etc/logcheck/violations.ignore.d/foo /etc/logcheck/violations.ignore.d/logcheck-foo if 'foo' is 'logcheck' then: all files /etc/logcheck/violations.ignore.d/* (ie all ignore files filter matches from logcheck, whereas matches from 'su' are only filtered by logcheck-su anything in /etc/violations.d that does _not_ start with 'logcheck' (because matches have/will be separately reported) end if /etc/logcheck/violations.ignore.d/local /etc/logcheck/violations.ignore.d/local-* (ie any file starting 'local') /etc/logcheck/cracking.d/* (because any matches from were reported as cracking attempts at step 2) Within this step (which is all within greplogoutput function) it is completely arbitrary whether a step is skipped if we already know there are no more entries to match or not. the naming convention is baffling and not correctly documented. At the end, results are reported as "security events for foo", except if foo is "logcheck" then just "security events". 4. Now start again with _all_ the log entries from step 1 4a. Filter _out_ anything matching any rules in /etc/logcheck/ignore.d.* (but taking account of REPORTLEVEL, ie if REPORTLEVEL=server then we use ignore.d.paranoid and ignore.d.workstation only) The naming of files does not matter, but things are applies in the order returned by run-parts (Actually files from different reportlevels are concatenated) 4b. Filter _out_ anything left through cracking.d/* and violations.d/* (because any matches from here were dealt with at steps 2 or 3) (this does not use the cracking.ignore.d or violations.ignore.d at all) 4c, Anything left is reported as "system events" - extremely easy to miss that this is different from a "security event" imo --------------------------------------------------------- I think most people assume that step 4 is all that is happening, and i think steps 2 and 3 probably cause more complication than they are worth. They certainly caused this bug report. For example - logcheck-database ships violations.d/logcheck-su and violations.ignore.d/logcheck-su and ignore.d.server/su - these dont work very well out of the box (doesnt the second line of violations.ignore.d/su make all the other lines pointless?). but working out which one to edit and why is baffling. And number of times i have thought i found a bug because i failed to spot a report was a "security event" rather than a "system event" i dont want to think about I would suggest logcheck could be simplified as follows 1. Extract log entries - no need for changes 2. Filter through ignore.d.* as above - no changes. 3. Look for potential violations attempts 3a. Any output from 2 (ie any rules not filtered out) is filtered through /etc/logcheck/important.d/* (I am suggesting merging everything currently in cracking.d and violations.d into important.d. But we could keep them separate if people want. - the naming could be improved here i am sure!) 3b. Any output from 3a is filtered though important.ignore.d/* (again, this would contain what is currently in violations.ignore.d (eg 'su' might need this - flag anything containing /su/ as 'important' unless it was a 'su' from root? but maybe this is not needed at all, as we can just filer out such rules entirely?) 3c - anything left after 3b is reported under the heading "Important log entries" 4. Any output from step 2 is reported under the heading "Log entries" (This means anything in step 3 appears twice. i would actually prefer that, but we could also prevent duplicates by filtering the result of stage 2 though the important.d directory before step 4) This means it's much easier - log entries are reported unless they match a rule in ignore.d. And within the things that are reported, we highlight anything with "Error" or "Failed" or "su" as important using step 2, which i think is similar enough to the current behaviour. This needs very little change to the current code, causes only a small change to the output, simplifies the implementation and would be slightly faster - and the life of users a lot: Regardless of where something is reported, hiding it just means adding a rule. I think this would make it much easier to make changes such as - rather than have rules applied in alphabetical order, allow logcheck to learn which rules actually match entries and apply them first. - instead of applying all rules in ignore.d.*, do not apply rules at all if they are for a package that is not installed (changes like this should happen at the start of a release cycle, and will need tests and documentation updates. Eg debug() needs some improvements to help debug this. And function names could all be improved to describe what the function does - i dont have any code yes, but i think it is quite simple to do) What do you think?
