Am Wed, 25 Feb 2015 10:13:14 +0000 (UTC) schrieb Duncan <1i5t5.dun...@cox.net>:
> Rich Freeman posted on Tue, 24 Feb 2015 16:44:59 -0500 as excerpted: > > > On Tue, Feb 24, 2015 at 3:15 PM, Marc Joliet <mar...@gmx.de> wrote: > >> > >> == Things I have *not* gotten rid of (yet) == > >> > >> I plan on uninstalling syslog-ng, but haven't done so yet. I simply > >> feel better waiting a bit, even though I don't run it anymore. Man, I > >> feel silly after typing that... > > > > Yeah, took me a while to uninstall it, but I don't find much value in > > running both. > > First, welcome to systemd! =:^) I may respond to other parts too but > this one's reasonably easy, while my choice is different, so I'll explain > what and why... > > Intro: I chose to keep syslog-ng running here, while I (of course) have > journald running as well. I believe the way I have configured both plays > to the advantages of each, while discounting the weak areas of each as > the other one covers that area. > > What: > > 1) Journald is configured to use volatile storage (the /run/log/journal > subdir, with /run being tmpfs) ONLY, effectively limiting it to single- > session. Right, I remember that from your thread a while back. > 2) Syslog-ng is configured pretty much as before, except that I have its > systemd USE flag turned on, which builds it with journald support so the > two cooperate a bit better. [...] > Meanwhile, for logs beyond one session, I still prefer traditional text- > based logs, tho I suppose that's partly because that's what I'm most used > to. But there's several advantages to it that I see: Personally, I don't mind the binary format, since the engineering advantages convinced me (constant-time access, lots of meta-data which you can filter by, git-inspired chain of hashes for integrity checking, etc.). > a) If the system crashes, partially corrupted in-the-crash text logs can > be of at least some use after a reboot. Binary journals, not so much. As Rich mentioned, and as I remember reading myself (on the btrfs ML?), the journal will do its best to read back from corrupted logs and is supposedly pretty resilient. > b) As far as I could find, journald has absolutely no mechanism to filter > incoming (pre-storage) log entries, while syslog-ng has had (pre-storage) > filter mechanisms for ages. Journald seems to have all sorts of outgoing > post-storage filter-on-output options, some of which are pretty nifty, > but there have been several times in the past where logs filled up with > "noise", thousands of repeat entries for something or other that occurs > regularly, say once a minute, that I simply do not care about and don't > want in my logs at all, but which I can't directly stop at the source for > whatever reason. One of them was a kernel bug that triggered for a few > kernel cycles, then eventually went away, another was a ksysguard > triggered message, with that ksysguard graph updating every second, etc. > Unfortunately, while journald makes it easy to filter these out on > output, I found no way of preventing their appearance in the journal at > all. > > But syslog-ng lets me dump them without ever actually logging them, or > route them to a different log file if I prefer, keeping my primary logs > clean. =:^) That is a very good point, I hadn't thought of that. However, I think that I prefer the post-filtering approach, since it is in principle the more flexible one. At work I sometimes mix the logs from different units (e.g., units A and B, and then units B, C, and D). But I realised today that journalctl has no negation operator! There is, however, an RFE for it. But regardless of what you use, I think that the worst offenders are services that write logs themselves (I'm looking at you, samba). > As a counter-point, however, at least journald does compression to some > degree, so tens of thousands of identical or nearly identical messages > don't take up as much room as you might expect as they compress very > well. =:^) I guess after the first instance, dups basically consist of a > timestamp and a pointer to the first message. So they don't take much > space. Which is fortunate when some "noise" message starts appearing > once a second... and you're storing it in tmpfs, thus in memory! > Obviously persistent logs would be similarly space efficient, but I still > don't like the idea of not being able to filter out the noise, BEFORE it > hits permanent storage, so I just make sure it /doesn't/ hit permanent > storage. =:^) Yeah, I think that's a good use case for pre-filtering: an overly noisy service whose logs you purposely want to ignore (for example, because you're still testing it - but then again, for that particular case you could probably use systemd-nspawn and log to an isolated journal). > c) I use btrfs for my primary filesystems, and btrfs and journald's > binary-format journals don't play so well together. FWIW, the latest > systemd-219 has lots of improvements for btrfs users, including some new > features that are btrfs-specific (which I never-the-less don't expect to > use right away as they're mostly VM/container oriented and I don't do > much of that sort of thing, but I may well eventually use them), as well > as some changes that should make journaling on btrfs work somewhat > better. So this point should be to some degree eliminated after that > stuff stabilizes tho part of the fix is journald setting nocow on the > journal files and that has other implications I'll skip further > discussion of here. However, by limiting journald to tmpfs only, I still > get the best benefits of systemd/journald integration (like status > including the last few log messages as mentioned above), while > eliminating the entire class of btrfs/journald issues. Best of both > worlds. =:^) Well, I'm on an SSD, but even on the laptop I haven't noticed any performance issues (yet). Then again, I use autodefrag, so that probably helps. What's funny though is that the systemd news file (http://cgit.freedesktop.org/systemd/systemd/tree/NEWS) occasionally refers to non-btrfs file systems as "legacy file sysetms". At least, as a btrfs user I think it's funny :) . > d) Tho I don't do anything with it in my syslog-ng config here, syslog-ng > can actually make use of some of the extra information journald passes to > it, including trusted per-app filtering and routing. One of the problems > that the old syslog approach had was that anything that could log > messages could claim to be anything it wanted, and syslog really couldn't > tell whether it was telling the truth or not, all it could do is sort and > report the messages based on what was claimed. A big bullet-point > feature of systemd/journald is that because systemd enforces service > separation via control-classes and the like, there's a much stronger > guarantee that what systemd/journald says is service X, is REALLY service > X and not service Y! But that's not limited to journald. Syslog-ng has > for several versions now had the ability to filter, route and report on > out-of-log-channel information such as the extra properties that journald > associates with each message, that make features such as systemctl status > reporting the last few log messages from that service possible, because > systemd/journald actually tracks it, AND passes it on to other loggers as > well now, providing they're built to use the API provided by the systemd > library I mentioned above. And again, for gentooers, syslog-ng is built > with that API, against that library, based on USE=systemd. So once > that's there, if desired, you can do all sorts of fancy filtering/log- > routing/etc based on information such as the application/service name, > that systemd passes along to syslog-ng. > > And again, with syslog-ng, this can be done BEFORE the actual logging if > desired, instead of logging everything and filtering only on output, as > journald does. =:^) That's actually pretty cool :) . You're basically using the journal as a log accumulator and syslog-ng for routing and post-processing its messages. Whiles that's not how I want to do things, I still appreciate how cool that is :) (and the fact that the journal can be used that way). > e) I'm simply more comfortable dealing with text-based logs, since I'm > used to them and can use whatever tools I want on them, as well as being > able to split/sort/route to multiple log files in as simple or complex a > setup as I want. =:^) > > Summary: > > For me, splitting up logging duties so journald does the collection, but > only does session journaling to tmpfs, while syslog-ng still handles the > "persistent" logging, gives me the best of both worlds. I love systemd/ > journald features such as log-messages-in-status and having seen how > useful it is, would seriously find it hard to do without. But at the > same time, I can't bring myself to accept persistent logging without the > ability to filter out the noise before it hits persistent storage, and > journal storage on btrfs is problematic anyway, with both problems nicely > solved by handing off to syslog-ng for my persistent storage needs while > keeping journald for same-session use, so that's exactly what I do. =:^) Right :) . Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup
pgpmFGQrVajm0.pgp
Description: Digitale Signatur von OpenPGP
