I think kernel should be fixed to return ENFILE in that case.
> pipe()/pipe2() may fail with ENOMEM error during creation in
> pipe_create() -> pipespace(), but it is not mentioned in the
> pipe(2) man page.
>
> The below diff fixes this.
>
> Index: pipe.2
> ===================================================================
> RCS file: /cvs/src/lib/libc/sys/pipe.2,v
> retrieving revision 1.18
> diff -u -p -r1.18 pipe.2
> --- pipe.2 10 Dec 2014 19:30:22 -0000 1.18
> +++ pipe.2 18 Dec 2017 09:32:32 -0000
> @@ -106,6 +106,8 @@ The system file table is full.
> The
> .Fa fildes
> buffer is in an invalid area of the process's address space.
> +.It Bq Er ENOMEM
> +Not enough memory to establish a pipe.
> .El
> .Pp
> In addition,
>