[adding the findutils ML]
On 2019-10-25 09:56, Ray Satiro wrote:
> Recently I tracked down a delay in some scripts to this line:
>
> find / -name filename* 2>/dev/null | head -n 1
>
> It appeared that the searching continued after the first result was
> found. I expected head would terminate and SIGPIPE would be sent which
> would cause find to terminate, but that did not happen. Since I was in
> cygwin I thought maybe it was a Windows issue but I tried in Ubuntu 16
> with an aribtrary file and noticed the same thing. I also tried
> disabling any pipe trap and monitoring with strace. It looks like head
> exits but find continues searching. This was only observable the first
> run since some caching seems to be done which makes subsequent runs
> complete too fast to tell, but if I reset the VM I can reproduce.
>
> owner@ubuntu1604-x64-vm:~$ ( trap '' pipe; find / -name initrd*
> 2>/dev/null | strace -e 'trace=!all' head -n 1)
> /initrd.img
> +++ exited with 0 +++
> (few seconds wait)
'head' just terminates after it has read the given number of lines.
> Since we only need the first line I can just use find options -print
> -quit and skip piping to head. But say we needed the first n results,
> how would I do that with head and get find to terminate rather than
> continue searching?
>
> find (GNU findutils) 4.7.0-git
> head (GNU coreutils) 8.25
... and 'find' properly terminates after receiving EPIPE:
$ strace -ve write find /dev | head -n 1 >/dev/null
write(1, "/dev\n/dev/.blkid.tab\n/dev/.blkid"..., 4096) = 4096
write(1, "chi_HDS721010CLA330_JP2911N03AR0"..., 4096) = -1 EPIPE (Broken pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=21014, si_uid=1000} ---
+++ killed by SIGPIPE +++
As you can see from above strace output, find does some buffering of the
output, i.e., it does not write one line per entry.
Thus, I assume that searching through the files until the first 4k output
buffer is full takes a while on your system.
To confirm, please try with turning this buffering off with 'stdbuf':
$ trace -vfe write stdbuf -o 0 find /dev | head -n 1 >/dev/null
write(1, "/dev\n", 5) = 5
write(1, "/dev/.blkid.tab\n", 16) = -1 EPIPE (Broken pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=21098, si_uid=1000} ---
+++ killed by SIGPIPE +++
With more depth, another kind of buffering comes into play:
'find' uses gnulib's FTS module to read the next directory hierarchy.
This means, that although 'head' might terminate at e.g. 10 items,
'find' already has read the entries for a couple of entries more.
Finally, the situation becomes more complicated if it is not 'find' which
is writing into the broken pipe, but a -exec'ed tool: in that case,
that tool terminates due to EPIPE but find will continue processing
with the next entry:
$ mkdir x && cd x
$ seq 10 | xargs touch
$ find . -exec echo '{}' \; | head -n 3
.
./10
./9
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
find: 'echo' terminated by signal 13
FWIW: To work around the above, one would use the false value of -exec in the
case the command to be run failed:
$ find . -exec echo '{}' \; -o -quit | head -n 3
.
./10
./9
find: 'echo' terminated by signal 13
Does this describe the effects you see?
Have a nice day,
Berny
