On Fri Sep 6, 2024 at 8:32 AM AEST, Johannes Thyssen Tishman wrote:
> 2024-09-05T21:59:43Z Dylan D'Silva:
> > Hello all,
> >
> > Just to chime in here. I wrote that email on the aerc mailing list.
> > For some context. This specific work around is in regards to using
> > MAILDIR and notmuch with aerc. You can use notmuch and aerc without
> > MAILDIR by removing maildir:// and copy-to from accounts.conf but you
> > can only send from one account without some interesting solutions, If
> > you have multiple accounts this isn't ideal. The aerc team has decided
> > this approach is best practice when using aerc, MAILDIR and notmuch.
> > This work around was to provide a working solution for people who want
> > to follow the developments team best practice while using a BSD
> > system.
>
> Where and when exactly was this workaround deemed the best practice by
> the 'aerc team'?
The best practice is in regards to using MAILDIR and notmuch with
multiple email accounts. Not in regards to requiring a larger openfile.
This setup is designed for people who have multiple email accounts. Say
on Protonmail, gmail or other providers and want to pull them all into
aerc and utilise notmuch on them.
It's been mentioned in the mailing list. It's written about on the wiki
and one of the devs even wrote a whole article about using this setup
with isync, etc.
> > Running without a matching openfile results in "Too many open file
> > errors" and the application not functioning.
> >
> > Aerc uses fsnotify for watching files. Linux backend is inotify which
> > supports recurive watching. OpenBSD uses kqueue which doesn't.
> >
> > This was never meant to be an ideal solution. I also mentioned to
> > Johannes, that a better approach would be to shrink your inbox and have
> > a smaller openfile.
> >
> > I'm using aerc on a OpenBSD system as a desktop and I'm the sole user.
> > So I'm not concerned with people opening multiple instances of aerc. I
> > realise everyone might not be in that boat.
> >
> > On Thu Sep 5, 2024 at 11:35 PM AEST, Johannes Thyssen Tishman wrote:
> > > 2024-09-05T12:55:06Z Stuart Henderson:
> > > > On 2024/09/05 14:23, Johannes Thyssen Tishman wrote:
> > > > > 2024-09-05T12:09:51Z Stuart Henderson:
> > > > > > On 2024/09/05 12:52, Johannes Thyssen Tishman wrote:
> > > > > > > The email below was sent to the aerc-discuss mailing list a
> > > > > > > couple of
> > > > > > > days ago. Is it worth adding a pkg/README for this?
> > > > > > >
> > > > > > > 2024-09-03T23:12:23Z "Dylan D'silva" <dylan@dsilva.email>:
> > > > > > > Hello all,
> > > > > > >
> > > > > > > After some digging I've come to a solution. For those on a BSD
> > > > > > > system
> > > > > > > with large MAILDIR, you'll require something similar. Aerc uses
> > > > > > > github.com/fsnotify/fsnotify to monitor changes to files. BSD
> > > > > > > system
> > > > > > > rely on Kqueue which doesn't support recursive watching of files.
> > > > > > ^^
> > > > > > directories
> > > > > > > Therefore you require one file descriptor for each file.
> > > > > >
> > > > > > This does not seem to be a good approach for monitoring emails in
> > > > > > Maildirs on kqueue systems.
> > > > > >
> > > > > > Most of the common changes to a Maildir result in either a new file,
> > > > > > rename, or deletion and not actually changes to the contents of the
> > > > > > files. AFAIK new files won't show up without rescanning the whole
> > > > > > directory - for renamed files I don't think you can find the new
> > > > > > name so again you need to rescan - it seems like really it's just
> > > > > > going to need an efficient way to pick up changes in the dir.
> > > > > >
> > > > > > (I suppose this is not going to work well with NFS either which is
> > > > > > not an uncommon place to store Maildirs..)
> > > > > >
> > > > > > > Assuming you are running aerc from the current account. You can
> > > > > > > change the limit to match your maildir size.
> > > > > > >
> > > > > > > For me, I store my maildir in ~/Mail
> > > > > > > find . -type f | wc -l #counts number of files
> > > > > > > 71514
> > > > > > >
> > > > > > > I run aerc from a staff account. Therefore I updated my
> > > > > > > /etc/login.conf
> > > > > > > under staff to increase the openfile-cur size to 71600 and
> > > > > > > openfile-max
> > > > > > > to 72000
> > > > > > > :openfiles-cur=71600:\
> > > > > > > :openfiles-max=72000:\
> > > > > >
> > > > > > If you're going to do this, I would recommend a "normal"
> > > > > > openfiles-cur,
> > > > > > just raise openfiles-max. Then start such software from a script or
> > > > > > shell alias which uses ulimit -n to raise the limit just for that
> > > > > > software.
> >
> > What do you consider normal? In your comment below, you made comment
> > about not many people running computers from the 80's.
>
> I think Stuart meant that you could leave openfiles-cur with it's
> default value (in case you hadn't changed it before for something else)
> and only increase openfiles-max to your desired value. Then you can
> start aerc from a script like the following:
I understood what Stuart meant in regards to the cur-max comment. You
missed the point I was making. Stuart thinks its pointless to use
cap_mkdb as modern systems are fast enough to handle larger login.conf
files. Yet Stuart thinks an openfile-cur of 128 is sufficient. So I
asked what is his definition of normal. I then pointed to the inotify
change of 2020 in Linux kernel which set a range of 8192 to 104576. Does
he have a standard or something to reference or is this just opinion?
Also this isn't the 90's and OpenBSD devs don't need their minions to
answer for them, and I don't need interpretations. If he doesn't feel
the need to respond that fine.
> #!/bin/sh
>
> ulimit -n 72000
> aerc
How does that solve the mentioned problem of people running multiple
aerc instances?
> > Here is a commit for inotify in 2020 making the openfile limit dynamic
> > between 8192 and 104576.
> >
> > From the commit:
> > "The default value of inotify.max_user_watches sysctl parameter was set
> > to 8192 since the introduction of the inotify feature in 2005 by
> > commit 0eeca28 ("[PATCH] inotify"). Today this value is just too
> > small for many modern usage. As a result, users have to explicitly set
> > it to a larger value to make it work."
> >
> > https://github.com/torvalds/linux/commit/92890123749bafc317bbfacbe0a62ce08d78efb7
> >
> >
> > It's worth mentioning but the syncthing pkg-readme suggest a openfile of
> > 4096.
> >
> > > > > >
> > > > > > I think it would generally be better to run aerc pointed at an IMAP
> > > > > > server though (obviously the IMAP server can just run on the machine
> > > > > > running aerc if wanted).
> >
> > I agree this is a better solution. This was only meant as a work around
> > to follow what the dev team are using.
> > > > > >
> > > > > > > If you use a large login.conf you might want create a database
> > > > > > > version
> > > > > > > with cap_mkdb /etc/login.conf see login.conf(5).
> > > > > >
> > > > > > We are mostly not running on computer systems from the 80s and the
> > > > > > file
> > > > > > is not really slow to parse - however having the db files around
> > > > > > does
> > > > > > cause trouble when someone forgets to rerun cap_mkdb after changing
> > > > > > the
> > > > > > file later. I would not recommend this.
> >
> > The man page for login(5).conf has that comment.
> >
> > > > >
> > > > > Thanks for the feedback Stuart. Should this advice be included in a
> > > > > pkg/README as it is done for syncthing?
> > > >
> > > > It would be better if the situation was improved upstream, but we
> > > > could do...
> > >
> > > I'll check with upstream.
> > >
> > > > Rather than documenting the workaround I think it's probably better if
> > > > we go for this approach. Setting the OS to allow opening of tens of
> > > > thousands of files is very much not ideal, and imagine what it would
> > > > be like if you have a couple of users on the machine, all of whom want
> > > > to run aerc...
> >
> > How about improving Kqueue to support recursive watching?
> >
> > > I agree, this seems more sane. Thank you Stuart :)
> > >
> > > > Index: Makefile
> > > > ===================================================================
> > > > RCS file: /cvs/ports/mail/aerc/Makefile,v
> > > > diff -u -p -r1.16 Makefile
> > > > --- Makefile 31 Jul 2024 18:36:02 -0000 1.16
> > > > +++ Makefile 5 Sep 2024 12:52:52 -0000
> > > > @@ -7,6 +7,7 @@ MODGO_MODNAME = git.sr.ht/~rjarry/aerc
> > > > MODGO_VERSION = v0.0.0-20240729220005-e037c095a049
> > > >
> > > > V = 0.18.2
> > > > +REVISION = 0
> > > > DISTNAME = aerc-$V
> > > >
> > > > CATEGORIES = mail
> > > > Index: pkg/PLIST
> > > > ===================================================================
> > > > RCS file: /cvs/ports/mail/aerc/pkg/PLIST,v
> > > > diff -u -p -r1.9 PLIST
> > > > --- pkg/PLIST 31 Jul 2024 18:36:02 -0000 1.9
> > > > +++ pkg/PLIST 5 Sep 2024 12:52:52 -0000
> > > > @@ -43,4 +43,5 @@ share/aerc/templates/forward_as_body
> > > > share/aerc/templates/new_message
> > > > share/aerc/templates/quoted_reply
> > > > share/applications/aerc.desktop
> > > > +share/doc/pkg-readmes/${PKGSTEM}
> > > > @tag update-desktop-database
> > > > Index: pkg/README
> > > > ===================================================================
> > > > RCS file: pkg/README
> > > > diff -N pkg/README
> > > > --- /dev/null 1 Jan 1970 00:00:00 -0000
> > > > +++ pkg/README 5 Sep 2024 12:52:52 -0000
> > > > @@ -0,0 +1,12 @@
> > > > ++-------------------------------------------------------------------------------
> > > > +| Running ${PKGSTEM} on OpenBSD
> > > > ++-------------------------------------------------------------------------------
> > > > +
> > > > +aerc's Maildir support on BSD uses kqueue(2), which is really intended
> > > > for
> > > > +watching individual files rather than large directories or trees. It
> > > > requires
> > > > +a file handle to be opened for each file in the monitored directories,
> > > > which
> > > > +will soon exceed default limits in the system.
> > > > +
> > > > +It is therefore recommended to run an IMAP daemon (for example Dovecot,
> > > > +courier-imap, imap-uw) and connect over IMAP, even if this is just to
> > > > the
> > > > +local machine.
