On Sun, 25 Jun 2017 16:41+0300, Konstantin Belousov wrote:
> On Sat, Jun 24, 2017 at 07:19:25PM -0700, Manfred Antar wrote:
> >
> > > On Jun 24, 2017, at 7:04 PM, Konstantin Belousov <kostik...@gmail.com>
> > > wrote:
> > >
> > > On Sat, Jun 24, 2017 at 06:48:03PM -0700, Manfred Antar wrote:
> > >>
> > >>> On Jun 24, 2017, at 6:23 PM, Konstantin Belousov <kostik...@gmail.com>
> > >>> wrote:
> > >>>
> > >>> On Sat, Jun 24, 2017 at 06:08:50PM -0700, Manfred Antar wrote:
> > >>>> New world and kernel r320323
> > >>>> I get a new error or message when using ruby:
> > >>>>
> > >>>>
> > >>>> /usr/local/sbin/portupgrade -av
> > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily
> > >>>> unavailable, scheduling broken
> > >>>>
> > >>>> everything works just this message when using ruby. I recompiled ruby
> > >>>> , still same message
> > >>>>
> > >>>> /usr/local/bin/ruby -v
> > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily
> > >>>> unavailable, scheduling broken
> > >>>> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12]
> > >>>>
> > >>>> Not sure what???s changed, I noticed some commits to vm stuff, maybe
> > >>>> thats it.
> > >>>
> > >>> ktrace your failing ruby invocation, then post output of kdump -H
> > >>> somewhere.
> > >>>
> > >>
> > >> Ok not sure if this is right , but this is what i did:
> > >>
> > >> (tmp)4637}ktrace /usr/local/bin/ruby -v
> > >> <main>: warning: pthread_create failed for timer: Resource temporarily
> > >> unavailable, scheduling broken
> > >> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12]
> > >>
> > >> (tmp)4638}kdump -H -f ./ktrace.out > kdump.txt
> > >>
> > >> you can get kdump.txt at:
> > >>
> > >> http://www.pozo.com/kernel/kdump <http://www.pozo.com/kernel/kdump>.txt
> > >>
> > >> It???s not failing, I don???t think , I can do portupgrade and it works
> > >> fine.
> > >> I just get this new message
> > >
> > > I see what is going on, but it is somewhat strange that it happens.
> > >
> > > Do you run ruby in a jail with old (say, stable/10) libthr ?
> > > Or do you have environment variable LIBPTHREAD_SPLITSTACK_MAIN set in
> > > your environment ?
> > >
> > > Anyway, the rework of the stack grow indeed have incompatibility with the
> > > old (pre-11) libthr, which tries to split main thread stack into smaller
> > > stacks for the new threads. New stack grow code was specifically designed
> > > to prevent this. Some hack would be needed there, to allow reuse of
> > > the main stack gap.
> > >
> >
> > Not running any jails
> > all libraries are current as of today
> > locate libthr. |xargs ls -l
> > -r--r--r-- 1 root wheel 120240 Jun 24 12:50 /lib/libthr.so.3
> > -r--r--r-- 1 root wheel 256072 Jun 24 12:50 /usr/lib/libthr.a
> > lrwxr-xr-x 1 root wheel 21 Jun 24 12:50 /usr/lib/libthr.so ->
> > ../../lib/libthr.so.3
> >
> > ldd /usr/local/bin/ruby
> > -rwxr-xr-x 1 root wheel 2677552 Jun 24 18:22
> > /usr/local/lib/libruby23.so.2
> > -rwxr-xr-x 1 root wheel 43832 Jun 24 19:10
> > /usr/local/lib/libunwind.so.8.0.1
> >
> > /usr/local/bin/ruby:
> > libruby23.so.23 => /usr/local/lib/libruby23.so.23 (0x800a00000)
> > libelf.so.2 => /lib/libelf.so.2 (0x800e9d000)
> > libunwind.so.8 => /usr/local/lib/libunwind.so.8 (0x8010b5000)
> > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8012ce000)
> > libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0x8014d1000)
> > libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016db000)
> > libm.so.5 => /lib/libm.so.5 (0x8018f9000)
> > libthr.so.3 => /lib/libthr.so.3 (0x801b26000)
> > libc.so.7 => /lib/libc.so.7 (0x801d4e000)
> > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802335000)
> > liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80254b000)
> > libkvm.so.7 => /lib/libkvm.so.7 (0x802774000)
> > libutil.so.9 => /lib/libutil.so.9 (0x802982000)
> >
> > no LIBPTHREAD_SPLITSTACK_MAIN set anywhere
> >
> > This is a fresh buildworld - installworld from noon today California time.
> >
> > here are env in make.conf:
> > DEFAULT_VERSIONS= mysql=5.7 apache=2.4 python2=2.7 python3=3.4 ruby=2.3
> > perl5=5.24 php=5.6 tcltk=8.6 samba=4.4 ssl=base ncurses=base
> > MALLOC_PRODUCTION=yes
> > WITH_BDB_VERSION=5
> >
> > env in src.conf:
> > WITHOUT_DYNAMICROOT=yes
> > WITHOUT_UNBOUND=yes
> > WITHOUT_CASPER=yes
> > WITHOUT_CAPSICUM=yes
> > WITHOUT_DMAGENT=yes
> > WITHOUT_PROFILE=yes
> > WITHOUT_TESTS=yes
> > WITHOUT_KERNEL_SYMBOLS=yes
> > WITHOUT_DEBUG_FILES=1
> > WITHOUT_LIB32=yes
> > INSTALL_NODEBUG=yes
> >
> > # Don't die on warnings
> > NO_WERROR=
> > WERROR=
> > KERNCONF=pozo
> > WITH_CCACHE_BUILD=yes
>
> Weird, I do not see where such behaviour could come from. I mean,
> the behaviour of the libthr, assuming that the failing call is from
> libthr and not from the ruby.
>
> Anyway, please try the following. Note that this might be not the final
> fix, but I am interesting in seeing whether it fixed the issue for you.
> I want to see the kdump output from ktrace _without_ -di, same as in
> your first trace.
>
> diff --git a/lib/libc/sys/mprotect.2 b/lib/libc/sys/mprotect.2
> index 3be81c6582a..ca4d6307c44 100644
> --- a/lib/libc/sys/mprotect.2
> +++ b/lib/libc/sys/mprotect.2
> @@ -28,7 +28,7 @@
> .\" @(#)mprotect.2 8.1 (Berkeley) 6/9/93
> .\" $FreeBSD$
> .\"
> -.Dd August 3, 2016
> +.Dd June 24, 2017
> .Dt MPROTECT 2
> .Os
> .Sh NAME
> @@ -80,10 +80,24 @@ and
> arguments is not valid.
> .It Bq Er EACCES
> The calling process was not allowed to change
> -the protection to the value specified by
> -the
> +the protection to the value specified by the
> .Fa prot
> argument.
> +.It Bq Er ENOMEM
> +The virtual address range specified by the
> +.Fa addr
> +and
> +.Fa len
> +arguments specify one or more pages which are not mapped.
> +.It Bq Er ENOMEM
> +The
> +.Fa prot
> +argument specifies
> +.Dv PROT_WRITE
> +on a
> +.Dv MAP_PRIVATE
> +mapping, and it would require more swap space than the system is able to
> +supply for the private pages.
> .El
> .Sh SEE ALSO
> .Xr madvise 2 ,
> diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c
> index 39cf303eaa0..b311b54d0f0 100644
> --- a/sys/vm/vm_map.c
> +++ b/sys/vm/vm_map.c
> @@ -1490,11 +1490,11 @@ vm_map_fixed(vm_map_t map, vm_object_t object,
> vm_ooffset_t offset,
> * If object is non-NULL, ref count must be bumped by caller
> * prior to making call to account for the new entry.
> */
> -int
> -vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset,
> - vm_offset_t *addr, /* IN/OUT */
> - vm_size_t length, vm_offset_t max_addr, int find_space,
> - vm_prot_t prot, vm_prot_t max, int cow)
> +static int
> +vm_map_find1(vm_map_t map, vm_object_t object, vm_ooffset_t offset,
> + vm_offset_t *addr, /* IN/OUT */
> + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot,
> + vm_prot_t max, int cow)
> {
> vm_offset_t alignment, initial_addr, start;
> int result;
> @@ -1556,6 +1556,26 @@ again:
> return (result);
> }
>
> +int
> +vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset,
> + vm_offset_t *addr, /* IN/OUT */
> + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot,
> + vm_prot_t max, int cow)
> +{
> + int rv;
> + bool hinted;
> +
> + hinted = *addr != 0;
> + for (;;) {
> + rv = vm_map_find1(map, object, offset, addr, length, max_addr,
> + find_space, prot, max, cow);
> + if (rv != KERN_NO_SPACE || !hinted)
> + return (rv);
> + *addr = 0;
> + hinted = false;
> + }
> +}
> +
> /*
> * vm_map_simplify_entry:
> *
> @@ -1970,7 +1990,7 @@ vm_map_pmap_enter(vm_map_t map, vm_offset_t addr,
> vm_prot_t prot,
> */
> int
> vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end,
> - vm_prot_t new_prot, boolean_t set_max)
> + vm_prot_t new_prot, boolean_t set_max)
> {
> vm_map_entry_t current, entry;
> vm_object_t obj;
> @@ -1995,7 +2015,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start,
> vm_offset_t end,
> if (vm_map_lookup_entry(map, start, &entry)) {
> vm_map_clip_start(map, entry, start);
> } else {
> - entry = entry->next;
> + vm_map_unlock(map);
> + return (KERN_INVALID_ADDRESS);
> }
>
> /*
> @@ -2003,6 +2024,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start,
> vm_offset_t end,
> */
> for (current = entry; current != &map->header && current->start < end;
> current = current->next) {
> + if ((current->eflags & MAP_ENTRY_GUARD) != 0)
> + continue;
> if (current->eflags & MAP_ENTRY_IS_SUB_MAP) {
> vm_map_unlock(map);
> return (KERN_INVALID_ARGUMENT);
> @@ -2011,6 +2034,11 @@ vm_map_protect(vm_map_t map, vm_offset_t start,
> vm_offset_t end,
> vm_map_unlock(map);
> return (KERN_PROTECTION_FAILURE);
> }
> + if (current->end < end && (current->next == &map->header ||
> + current->next->start > current->end)) {
> + vm_map_unlock(map);
> + return (KERN_INVALID_ADDRESS);
> + }
> }
>
> /*
> @@ -2712,9 +2740,9 @@ vm_map_wire(vm_map_t map, vm_offset_t start,
> vm_offset_t end,
> * If VM_MAP_WIRE_HOLESOK was specified, skip this check.
> */
> next_entry:
> - if (((flags & VM_MAP_WIRE_HOLESOK) == 0) &&
> - (entry->end < end && (entry->next == &map->header ||
> - entry->next->start > entry->end))) {
> + if ((flags & VM_MAP_WIRE_HOLESOK) == 0 &&
> + entry->end < end && (entry->next == &map->header ||
> + entry->next->start > entry->end)) {
> end = entry->end;
> rv = KERN_INVALID_ADDRESS;
> goto done;
> diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c
> index 4d8f6ad9ed7..09912250d52 100644
> --- a/sys/vm/vm_mmap.c
> +++ b/sys/vm/vm_mmap.c
> @@ -607,6 +607,7 @@ kern_mprotect(struct thread *td, uintptr_t addr0, size_t
> size, int prot)
> case KERN_PROTECTION_FAILURE:
> return (EACCES);
> case KERN_RESOURCE_SHORTAGE:
> + case KERN_INVALID_ADDRESS:
> return (ENOMEM);
> }
> return (EINVAL);
I'm seeing similar fallout on r320327, built from scratch while
running r320293:
Script started on Sun Jun 25 17:03:41 2017
# uname -aKU
FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r320327: Sun Jun 25 16:35:02 CEST
2017 r...@freebsd-head-uefi-zfs.bsd.net:/usr/obj/usr/src/sys/VBOX amd64
1200035 1200035
# /sbin/zfs mount -va
Fatal error 'Cannot allocate red zone for initial thread' at line 392 in file
/usr/src/lib/libthr/thread/thr_init.c (errno = 12)
Abort trap (core dumped)
# ls -l /lib/libthr*
-r--r--r-- 1 root wheel 153336 Jun 25 16:39 /lib/libthr.so.3
# ls -l /usr/lib/libthr*
-r--r--r-- 1 root wheel 256064 May 28 21:41 /usr/lib/libthr.a
lrwxr-xr-x 1 root wheel 21 Jun 25 16:39 /usr/lib/libthr.so ->
../../lib/libthr.so.3
-r--r--r-- 1 root wheel 270362 May 28 21:41 /usr/lib/libthr_p.a
-r--r--r-- 1 root wheel 45376 Apr 22 10:39 /usr/lib/libthread_db.a
lrwxr-xr-x 1 root wheel 17 Jun 25 16:39 /usr/lib/libthread_db.so ->
libthread_db.so.3
-r--r--r-- 1 root wheel 38776 Jun 25 16:39 /usr/lib/libthread_db.so.3
-r--r--r-- 1 root wheel 49496 Apr 22 10:39 /usr/lib/libthread_db_p.a
# ldd /sbin/zfs
/sbin/zfs:
libjail.so.1 => /lib/libjail.so.1 (0x800247000)
libnvpair.so.2 => /lib/libnvpair.so.2 (0x80024f000)
libuutil.so.2 => /lib/libuutil.so.2 (0x800269000)
libzfs_core.so.2 => /lib/libzfs_core.so.2 (0x800276000)
libzfs.so.3 => /lib/libzfs.so.3 (0x800281000)
libc.so.7 => /lib/libc.so.7 (0x8002cd000)
libmd.so.6 => /lib/libmd.so.6 (0x8006d1000)
libumem.so.2 => /lib/libumem.so.2 (0x8006eb000)
libutil.so.9 => /lib/libutil.so.9 (0x8006f0000)
libm.so.5 => /lib/libm.so.5 (0x800706000)
libavl.so.2 => /lib/libavl.so.2 (0x800736000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x80073a000)
libgeom.so.5 => /lib/libgeom.so.5 (0x800770000)
libz.so.6 => /lib/libz.so.6 (0x800779000)
libthr.so.3 => /lib/libthr.so.3 (0x800793000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x8007bd000)
# ls -l /sbin/zfs
-r-xr-xr-x 1 root wheel 124296 Jun 25 16:39 /sbin/zfs
# exit
Script done on Sun Jun 25 17:04:32 2017
Although not relevant for this case, neither /usr/lib/libthr.a, nor
/usr/lib/libthr_p.a, nor /usr/lib/libthread_db.a, nor
/usr/lib/libthread_db_p.a has been updated by installworld, nor
removed by delete-old-libs, in a very long time.
Anything else I can contribute? I'll try your patch and see if I can
get any further.
--
+-------------------------------+------------------------------------+
| Vennlig hilsen, | Best regards, |
| Trond Endrestøl, | Trond Endrestøl, |
| IT-ansvarlig, | System administrator, |
| Fagskolen Innlandet, | Gjøvik Technical College, Norway, |
| tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, |
| sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. |
+-------------------------------+------------------------------------+
_______________________________________________
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
