Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-24 Thread Lennart Poettering
On Sat, 22.04.17 15:29, Andrei Borzenkov ([email protected]) wrote: > 18.04.2017 07:27, Chris Murphy пишет: > > On Mon, Apr 17, 2017 at 10:05 PM, Andrei Borzenkov > > wrote: > >> 18.04.2017 06:50, Chris Murphy пишет: > > > What exactly "changes" mean? Write() syscall? > >>> > >>> filefra

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-22 Thread Andrei Borzenkov
22.04.2017 15:29, Andrei Borzenkov пишет: > 18.04.2017 07:27, Chris Murphy пишет: >> On Mon, Apr 17, 2017 at 10:05 PM, Andrei Borzenkov >> wrote: >>> 18.04.2017 06:50, Chris Murphy пишет: >> > What exactly "changes" mean? Write() syscall? filefrag reported entries increase, it's usi

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-22 Thread Andrei Borzenkov
18.04.2017 07:27, Chris Murphy пишет: > On Mon, Apr 17, 2017 at 10:05 PM, Andrei Borzenkov > wrote: >> 18.04.2017 06:50, Chris Murphy пишет: > What exactly "changes" mean? Write() syscall? >>> >>> filefrag reported entries increase, it's using FIEMAP. >>> >> >> So far it sounds like btrfs a

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-18 Thread Lennart Poettering
On Mon, 17.04.17 21:50, Chris Murphy ([email protected]) wrote: > >> Why do I see so many changes to the journal file, once ever 2-5 > >> seconds? This adds 4096 byte blocks to the file each time, and when > >> cow, that'd explain why there are so many fragments. > >> > > > > > > What exactl

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-18 Thread Lennart Poettering
On Mon, 17.04.17 13:49, Chris Murphy ([email protected]) wrote: > On Mon, Apr 17, 2017 at 11:27 AM, Andrei Borzenkov > wrote: > > 17.04.2017 19:25, Chris Murphy пишет: > >> This explains one system's fragmented journals; but the other system > >> isn't snapshotting journals and I haven't f

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
On Mon, Apr 17, 2017 at 10:05 PM, Andrei Borzenkov wrote: > strace -p $(pgrep systemd-journal) > > You will not see actual writes as file is memory mapped, but it > definitely does not do any fsync() every so often. https://paste.fedoraproject.org/paste/oVT-tsU2sBOdTJaZxGua-15M1UNdIGYhyRLivL9gyd

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
On Mon, Apr 17, 2017 at 10:05 PM, Andrei Borzenkov wrote: >> I have no idea if it's fsync or what. How can I tell? >> > > strace -p $(pgrep systemd-journal) > > You will not see actual writes as file is memory mapped, but it > definitely does not do any fsync() every so often. Also found this.

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
On Mon, Apr 17, 2017 at 10:05 PM, Andrei Borzenkov wrote: > 18.04.2017 06:50, Chris Murphy пишет: >>> What exactly "changes" mean? Write() syscall? >> >> filefrag reported entries increase, it's using FIEMAP. >> > > So far it sounds like btrfs allocates new extent on every write to > journal file

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Andrei Borzenkov
18.04.2017 06:50, Chris Murphy пишет: > On Mon, Apr 17, 2017 at 9:42 PM, Andrei Borzenkov wrote: >> 17.04.2017 22:49, Chris Murphy пишет: >>> On Mon, Apr 17, 2017 at 11:27 AM, Andrei Borzenkov >>> wrote: 17.04.2017 19:25, Chris Murphy пишет: > This explains one system's fragmented journ

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
On Mon, Apr 17, 2017 at 9:42 PM, Andrei Borzenkov wrote: > 17.04.2017 22:49, Chris Murphy пишет: >> On Mon, Apr 17, 2017 at 11:27 AM, Andrei Borzenkov >> wrote: >>> 17.04.2017 19:25, Chris Murphy пишет: This explains one system's fragmented journals; but the other system isn't snapshot

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Andrei Borzenkov
17.04.2017 22:49, Chris Murphy пишет: > On Mon, Apr 17, 2017 at 11:27 AM, Andrei Borzenkov > wrote: >> 17.04.2017 19:25, Chris Murphy пишет: >>> This explains one system's fragmented journals; but the other system >>> isn't snapshotting journals and I haven't figured out why they're so >>> fragme

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
On Mon, Apr 17, 2017 at 11:27 AM, Andrei Borzenkov wrote: > 17.04.2017 19:25, Chris Murphy пишет: >> This explains one system's fragmented journals; but the other system >> isn't snapshotting journals and I haven't figured out why they're so >> fragmented. No snapshots, and they are all +C at crea

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
Here's an example rotated log (Btrfs, NVMe, no compression, default ssd mount option). As you can see it takes up more space on disk than it contains data, so there's a lot of slack space for some reason, despite /etc/systemd/journald.conf being unmodified and thus Compress=Yes. file: system@2547

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Andrei Borzenkov
17.04.2017 19:25, Chris Murphy пишет: > This explains one system's fragmented journals; but the other system > isn't snapshotting journals and I haven't figured out why they're so > fragmented. No snapshots, and they are all +C at create time > (systemd-journald default on Btrfs). Is it possible to

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Chris Murphy
On Mon, Apr 17, 2017 at 3:57 AM, Lennart Poettering wrote: >> I do manual snapshots before software updates, which means new writes >> to these files are subject to COW, but additional writes to the same >> extents are overwrites and are not COW because of chattr +C. I've used >> this same strate

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Kai Krakow
Am Mon, 17 Apr 2017 16:01:48 +0200 schrieb Kai Krakow : > > We also ask btrfs to defrag the file as soon as we mark it as > > archived... > > This makes sense. And I've learned that journal on btrfs works much > better if you use many small files vs. a few big files. I've currently > set the jo

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Kai Krakow
Am Mon, 17 Apr 2017 11:57:21 +0200 schrieb Lennart Poettering : > On Sun, 16.04.17 14:30, Chris Murphy ([email protected]) wrote: > > > Hi, > > > > This is on a Fedora 26 workstation (systemd-233-3.fc26.x86_64) > > that's maybe a couple weeks old and was clean installed. Drive is > > NVMe.

Re: [systemd-devel] journal fragmentation on Btrfs

2017-04-17 Thread Lennart Poettering
On Sun, 16.04.17 14:30, Chris Murphy ([email protected]) wrote: > Hi, > > This is on a Fedora 26 workstation (systemd-233-3.fc26.x86_64) that's > maybe a couple weeks old and was clean installed. Drive is NVMe. > > > # filefrag * > system.journal: 9283 extents found > user-1000.journal: 3

[systemd-devel] journal fragmentation on Btrfs

2017-04-16 Thread Chris Murphy
Hi, This is on a Fedora 26 workstation (systemd-233-3.fc26.x86_64) that's maybe a couple weeks old and was clean installed. Drive is NVMe. # filefrag * system.journal: 9283 extents found user-1000.journal: 3437 extents found # lsattr C-- ./system.journal C-- ./use