Re: glibc THREAD_GSCOPE and excessive gsync_wake ()

2021-05-08 Thread Samuel Thibault
Samuel Thibault, le sam. 08 mai 2021 21:54:34 +0200, a ecrit: > Don't bother looking at the existing implementation, its roots dates > before we really had gsync working. > > Sergey Bugaev, le sam. 08 mai 2021 22:35:23 +0300, a ecrit: > > + do \ > > +{ \ > > + int count; \ > > + cou

Re: glibc THREAD_GSCOPE and excessive gsync_wake ()

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 08 mai 2021 23:03:34 +0300, a ecrit: > On Sat, May 8, 2021 at 10:54 PM Samuel Thibault > wrote: > > Don't bother looking at the existing implementation, its roots dates > > before we really had gsync working. > > Interesting! > > As far as I can see, this implementation w

Re: glibc THREAD_GSCOPE and excessive gsync_wake ()

2021-05-08 Thread Sergey Bugaev
On Sat, May 8, 2021 at 10:54 PM Samuel Thibault wrote: > Don't bother looking at the existing implementation, its roots dates > before we really had gsync working. Interesting! As far as I can see, this implementation was committed (by you) in 2018; and gsync has not seen significant changes sin

Re: glibc THREAD_GSCOPE and excessive gsync_wake ()

2021-05-08 Thread Samuel Thibault
Hello, Don't bother looking at the existing implementation, its roots dates before we really had gsync working. Sergey Bugaev, le sam. 08 mai 2021 22:35:23 +0300, a ecrit: > + do \ > +{ \ > + int count; \ > + count = atomic_exchange_and_add_rel (&GL(dl_thread_gscope_count), -1); >

glibc THREAD_GSCOPE and excessive gsync_wake ()

2021-05-08 Thread Sergey Bugaev
I've noticed that even a simple hello world does a lot of gsync calls: $ rpctrace echo hello world |& grep -c gsync 53 These are in fact all identical gsync_wake () calls on a single address: $ rpctrace echo hello world |& grep gsync | uniq task132(pid18549)->gsync_wake (202820 0 0); This immed

Re: settrans: (os/kern) invalid right

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 08 mai 2021 14:51:22 +0300, a ecrit: > if we failed to set the translator (either > because of this issue, or for any other reason), let's shut the > translator back down instead of leaving it in limbo. We'd rather do that indeed, thanks! Samuel

Re: settrans: (os/kern) invalid right

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 08 mai 2021 14:51:22 +0300, a ecrit: > P.S. There must be something wrong with your mailserver. I only > received a bunch of replies form you just a few minutes ago, and > seemingly so did the archive [0], but your citation indicates that you > have sent the reply promptly af

Re: [PATCH 1/4] libpager: pager_write_page () should not unmap page

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 08 mai 2021 18:31:41 +0300, a ecrit: > Clarify this in the documentation, and fix the storeio implementation. That was indeed quite unclear, most code was assuming it was unmapped, and storeio was following the documentation. Applied, thanks! Samuel > --- > doc/hurd.texi

Re: [PATCH 0/4] _pager_do_write_request fixes

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 08 mai 2021 18:31:40 +0300, a ecrit: > This is the same patch, split into three for readabilty and > reviewability. Thanks! That will be way easier to review indeed. Samuel

[PATCH 3/4] libpager: Track which pages are precious

2021-05-08 Thread Sergey Bugaev
This introduces the PM_PRECIOUS bit, which gets set when a precious page is offered to the kernel using pager_offer_page () with the precious argument set, and cleared when the page is returned back. --- libpager/data-return.c | 3 ++- libpager/offer-page.c | 2 ++ libpager/priv.h| 1 + 3

[PATCH 2/4] libpager: Store omit_data in an array

2021-05-08 Thread Sergey Bugaev
While this is less efficient than using a single bit per page, there is no guarantee that the kernel will not send us more than 32 pages at a time. Indeed, one can easily craft a situation where it will send more by unmapping a large mapping. Also, this function already uses on-stack arrays for tr

[PATCH 0/4] _pager_do_write_request fixes

2021-05-08 Thread Sergey Bugaev
This is the same patch, split into three for readabilty and reviewability. I'm still looking for feedback :) Instead of removing the munmap from _pager_do_write_request (), document that pager_write_page () is *not* supposed to unmap the page (nor keep references to it), and remove the munmap call

[PATCH 1/4] libpager: pager_write_page () should not unmap page

2021-05-08 Thread Sergey Bugaev
Clarify this in the documentation, and fix the storeio implementation. --- doc/hurd.texi | 6 +++--- libpager/pager.h| 6 +++--- storeio/pager.c | 8 +++- tmpfs/pager-stubs.c | 6 +++--- 4 files changed, 12 insertions(+), 14 deletions(-) diff --git a/doc/hurd.texi b/doc/hurd.tex

[PATCH 4/4] libpager: Do not throw away precious pages

2021-05-08 Thread Sergey Bugaev
The kernel invokes memory_object_data_return () with dirty = 0 in two cases: if notification on eviction was requested, or when returning precious pages. Properly distinguish between the two cases, and only throw the clean page away in the former case. --- libpager/data-return.c | 54 +

Re: [RFC PATCH tarfs 0/6] mmap support for tarfs

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 08 mai 2021 16:25:07 +0300, a ecrit: > On Sat, May 8, 2021 at 2:40 PM Samuel Thibault > wrote: > > Ah, seeing zipstores.c I thought zipfiles were supported, so I was > > wondering how since, as you say, they combine both. > > By the way, upstream libstore itself supports g

Re: [RFC PATCH tarfs 0/6] mmap support for tarfs

2021-05-08 Thread Sergey Bugaev
On Sat, May 8, 2021 at 2:40 PM Samuel Thibault wrote: > Ah, seeing zipstores.c I thought zipfiles were supported, so I was > wondering how since, as you say, they combine both. By the way, upstream libstore itself supports gzip & bzip2; I have only briefly looked into why tarfs needs to ship its

Re: And another patch...

2021-05-08 Thread Sergey Bugaev
On Sat, May 8, 2021 at 2:36 PM Samuel Thibault wrote: > > Hello, > > Just to be sure: this is not only because of your previous patch > libpager: Do not flush in-core pages on offer > ? Yes; this issue is completely orthogonal to that fix (but there's an interesting interaction I haven't thought

Re: settrans: (os/kern) invalid right

2021-05-08 Thread Sergey Bugaev
On Sat, May 8, 2021 at 2:39 PM Samuel Thibault wrote: > I pushed the revert, thanks! Thank you! I'm attaching another patch that tries to make settrans handle this case a tiny bit better: if we failed to set the translator (either because of this issue, or for any other reason), let's shut the t

Re: [RFC PATCH tarfs 0/6] mmap support for tarfs

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le sam. 01 mai 2021 20:23:58 +0300, a ecrit: > tarfs can be run with --bzip2 and --gzip options, but not with --zip. Ah, seeing zipstores.c I thought zipfiles were supported, so I was wondering how since, as you say, they combine both. > > > Hmm, but doesn't the page reading/writin

Re: [PATCH 1/2] startup: Mark all essential tasks as our children

2021-05-08 Thread Samuel Thibault
Damien Zammit, le sam. 08 mai 2021 10:19:57 +1000, a ecrit: > This marks all essential tasks as children of startup (or proc?) Please additionally try to remove err = proc_child (procserver, fstask); err = proc_child (procserver, authtask); from launch_core_servers. Otherwise they will be dupli

Re: settrans: (os/kern) invalid right

2021-05-08 Thread Samuel Thibault
Hello, Sergey Bugaev, le ven. 07 mai 2021 15:41:17 +0300, a ecrit: > I don't think I understand the reasoning behind [1]. Perhaps I'm > missing something? I don't see either why not deallocating it. Event if it's a dead name, we want to deallocate it, so as to free the port name. Samuel

Re: settrans: (os/kern) invalid right

2021-05-08 Thread Samuel Thibault
Samuel Thibault, le ven. 07 mai 2021 15:29:49 +0200, a ecrit: > Sergey Bugaev, le ven. 07 mai 2021 15:41:17 +0300, a ecrit: > > I don't think I understand the reasoning behind [1]. Perhaps I'm > > missing something? > > I don't see either why not deallocating it. Event if it's a dead name, > we wa

Re: [PATCH 5/6] libpager: Fix overallocating pagemap

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le jeu. 06 mai 2021 15:56:30 +0300, a ecrit: > The code tried to round up the allocation size to a multiple of page size. > But we actually allocate newsize * sizeof (*p->pagemap) bytes, not newsize > bytes, which meant allocations were sizeof (*p->pagemap) times larger than > they n

Re: And another patch...

2021-05-08 Thread Samuel Thibault
Hello, Just to be sure: this is not only because of your previous patch libpager: Do not flush in-core pages on offer ? Samuel Sergey Bugaev, le jeu. 06 mai 2021 20:58:29 +0300, a ecrit: > There's yet another bug: libpager just throws away clean precious > pages (those previously offered with pa

Re: [PATCH 6/6] libpager: Use libc heap for pagemap

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le ven. 07 mai 2021 12:26:39 +0300, a ecrit: > On Thu, May 6, 2021 at 3:56 PM Sergey Bugaev wrote: > > - newaddr = mmap (0, newsize * sizeof (*p->pagemap), > > - PROT_READ|PROT_WRITE, MAP_ANON, 0, 0); > > + void *newaddr = reallocarray (p->pagemap, off,

Re: [PATCH 6/6] libpager: Use libc heap for pagemap

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le jeu. 06 mai 2021 15:56:31 +0300, a ecrit: > libc already implements the functionality for allocating and managing > memory blocks like the pagemap. Using libc functions gives us some > additional niceties like overflow checking in reallocarray (). it also > means that we will not

Re: [PATCH 4/6] libpager: Store pagemapsize as vm_size_t

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le jeu. 06 mai 2021 15:56:29 +0300, a ecrit: > On a 64-bit system, there can be a lot more pages > than a 32-bit int can fit. Indeed, applied, thanks! > --- > libpager/pagemap.c | 2 +- > libpager/priv.h| 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git

Re: [PATCH 2/6] libpager: Do not flush in-core pages on offer

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le jeu. 06 mai 2021 15:56:27 +0300, a ecrit: > pager_offer_page () is documented to may ignore the offered page if > the kernel already has a copy in core (indeed, that's what Mach does). > > However the current behavior is the inverse of that: it asks the kernel > to flush (i.e. dr

Re: [PATCH 1/6] libpager: Fix mixing up success and error

2021-05-08 Thread Samuel Thibault
Sergey Bugaev, le jeu. 06 mai 2021 15:56:26 +0300, a ecrit: > _pager_pagemap_resize () returns an error or 0 on success, > not a boolean. Quite trivially bogus indeed :) Applied, thanks! > --- > libpager/offer-page.c | 27 ++- > 1 file changed, 14 insertions(+), 13 deleti

Re: Privileged ports to access the arbiter

2021-05-08 Thread Samuel Thibault
Joan Lledó, le jeu. 06 mai 2021 20:17:07 +0200, a ecrit: > El 5/5/21 a les 20:22, Samuel Thibault ha escrit: > > Mmm, no, we only need it for the device_open case. We don't need it for > > falling back on using file_name_lookup (_SERVERS_BUS_PCI) > > And do we need write permissions to perform the