Re: interrupted system call when using named pipes on FreeBSD
On 18 January 2013 13:55, Chet Ramey wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 1/18/13 1:30 AM, Mike Frysinger wrote:
>> this is somewhat a continuation of this thread:
>> http://lists.gnu.org/archive/html/bug-bash/2008-10/msg00091.html
>>
>> i've gotten more or less the same report in Gentoo:
>> http://bugs.gentoo.org/447810
>>
>> the simple test case is:
>> $ cat test.sh
>> #!/bin/bash
>> while :; do
>> (:)& (:)& (:)& (:)& (:)& (:)& (:)& (:)& (:)& (:)&
>> while read x ; do : ; done < <(echo foo)
>> done
>>
>> execute `./test.sh` and we see failures pretty much all the time.
>>
>> a simple patch to workaround/fix the issue by Yuta SATOH:
>> --- bash-4.2/redir.c
>> +++ bash-4.2/redir.c
>> @@ -632,7 +632,9 @@
>> }
>>else
>> {
>> - fd = open (filename, flags, mode);
>> + do {
>> + fd = open (filename, flags, mode);
>> + } while ((fd < 0) && (errno == EINTR));
>> #if defined (AFS)
>>if ((fd < 0) && (errno == EACCES))
>> {
>>
>> but we're not sure if this is the route to take ? seems like if bash is
>> handling SIGCHLD, there's no avoiding this sort of check.
>
> Why is open returning -1/EINTR when the SIGCHLD handler is installed with
> SA_RESTART? The intent is that opens get restarted even when bash handles
> SIGCHLD.
SA_RESTART is buggy on some platform/filesystem combinations, for
example Solaris/ZFS with high latency storage or Solaris/NFS are prone
to SA_RESTART not working unless you've installed the latest kernel
patches (this is at least the case with Solaris 10 and 11). However
Oracle now wants money for such patches which makes such fixes not
generally available (certainly not for Universities).
Lionel
Re: interrupted system call when using named pipes on FreeBSD
On 30 January 2013 17:48, Chet Ramey wrote:
>> a simple patch to workaround/fix the issue by Yuta SATOH:
>> --- bash-4.2/redir.c
>> +++ bash-4.2/redir.c
>> @@ -632,7 +632,9 @@
>> }
>>else
>> {
>> - fd = open (filename, flags, mode);
>> + do {
>> + fd = open (filename, flags, mode);
>> + } while ((fd < 0) && (errno == EINTR));
>> #if defined (AFS)
>>if ((fd < 0) && (errno == EACCES))
>> {
>>
>> but we're not sure if this is the route to take ? seems like if bash is
>> handling SIGCHLD, there's no avoiding this sort of check.
>
> It seems like implementations are buggy enough that bash needs to do
> something to work around them, but this isn't exactly it.
>
> The issue with the proposed patch is that it would make it impossible to
> interrupt an open using, say, ^C. There needs to be a check for signals
> in there somewhere. I'll fix it.
Question is whether such buggy platforms like Solaris should be
supported at all by bash. Oracle refuses to make such patches
available in/for the public, meaning that the average system will
suffer from this permanently.
Lionel
Executing shell code on the signal stack, or why is this is a bad idea
Forwarding an interesting posting from Roland Mainz who did an
investigation why signal trap processing in ksh93, bash and dash is
currently not reliable.
Lionel
-- Forwarded message --
From: Roland Mainz
Date: 3 June 2013 15:22
Subject: Re: [ast-developers] Realtime signal issues, revised, still
loosing signals
To: ast-develop...@research.att.com
On Sat, Jun 1, 2013 at 4:17 AM, Cedric Blancher
wrote:
> I've tried to use the realtime signals in ast-ksh 20130524 to see if
> they are reliable how but ran into instabilities again.
> My example I used for testing is this one:
> cut--
[snip]
> cut--
>
> The example should print nothing if realtime signals are working as
> expected. ast-ksh.20130524 however gives me this output:
> got {#rtar[10][*]}=1,expected 10
> got {#rtar[11][*]}=4,expected 10
> got {#rtar[12][*]}=4,expected 10
> got {#rtar[13][*]}=4,expected 10
> got {#rtar[14][*]}=5,expected 10
> got {#rtar[15][*]}=5,expected 10
> (
> [10]=(
> (
> msg=10
> typeset -l -i pid=55918
> )
> )
> [11]=(
> (
> typeset -l -i pid=55916
> )
> (
> typeset -l -i pid=55918
> )
> (
> msg=11
> typeset -l -i pid=55922
> )
> (
> msg=11
> typeset -l -i pid=55925
> )
> )
> [12]=(
> (
> msg=12
> typeset -l -i pid=55918
> )
> (
> msg=12
> typeset -l -i pid=55920
> )
> (
> typeset -l -i pid=55922
> )
> (
> msg=12
> typeset -l -i pid=55925
> )
> )
> [13]=(
> (
> msg=13
> typeset -l -i pid=55917
> )
> (
> msg=13
> typeset -l -i pid=55918
> )
> (
> msg=13
> typeset -l -i pid=55924
> )
> (
> msg=13
> typeset -l -i pid=55925
> )
> )
> [14]=(
> (
> typeset -l -i pid=55916
> )
> (
> msg=14
> typeset -l -i pid=55917
> )
> (
> msg=14
> typeset -l -i pid=55918
> )
> (
> msg=14
> typeset -l -i pid=55924
> )
> (
> msg=14
> typeset -l -i pid=55925
> )
> )
> [15]=(
> (
> msg=15
> typeset -l -i pid=55917
> )
> (
> msg=15
> typeset -l -i pid=55920
> )
> (
> msg=15
> typeset -l -i pid=55923
> )
> (
> msg=15
> typeset -l -i pid=55924
> )
> (
> msg=15
> typeset -l -i pid=55925
> )
> )
> )
>
> More than half of the signals are lost (which is a violation of the
> POSIX realtime spec which mandates that realtime signals must be
> reliable) and some of the array entries aren't even filled out (like
> rtar[14][0].msg is missing).
Grumpf... I spend half the weekend digging and debugging the signal
handling code. Basically the issue is that signal+signal trap handling
in ast-ksh.2013-05-24 is IMO utterly *broken* - while testing I found
that SIGCHLD and SIGRTMIN signals are delivered _reliably_ to the
shell but the implementation then looses between 8%-50% (measured on
SuSE 12.3/Linux/AMD64 and Solaris 11) during signal trap processing.
Example:
I applied the following test patch to ast-ksh.2013-05-24:
-- snip --
diff -r -u src/cmd/ksh93/sh/fault.c src/cmd/ksh93/sh/fault.c
--- src/cmd/ksh93/sh/fault.c 2013-05-22 19:33:13.0 +0200
+++ src/cmd/ksh93/sh/fault.c 2013-06-03 12:41:13.575689611 +0200
@@ -72,6 +72,9 @@
return(action);
}
+volatile int numsigrt_received=0;
+volatile int numsigrt_processed=0;
+
/*
* Most signals caught or ignored by the shell come h
Re: Executing shell code on the signal stack, or why is this is a bad idea
On 6 June 2013 11:29, Lionel Cons wrote: > Forwarding an interesting posting from Roland Mainz who did an > investigation why signal trap processing in ksh93, bash and dash is > currently not reliable. > > Lionel > PS: malloc() from AT&T libast, used by ksh93, is async signal safe. Unless bash's sh_malloc() has a similar functionality you'll need a different solution, such as using signalfd() Lionel
Re: Bash install on Windows 7 using SUA
On 2 July 2013 22:44, Eric Blake wrote: > On 07/02/2013 12:20 PM, Chris Irvine wrote: >> I tried to download the BASH for my Windows 7 computer with SUA (services >> for Unix installed). The latest download script, config.guess, was used and >> the configure command failed with an error. I have attached a picture of >> the error message in a Word document. > > Sheesh - you made people have to read a non-free file format to see a > screenshot, instead of just copying and pasting the text of the error > message itself. Stunts like that cause undue network congestion, and > alienate you from free software developers who try to stay as far away > from proprietary software as possible. > > Your screenshot shows that your build error boils down to: > > loadmsgcat.c: In function `get_sysdep_segment_value': > loadmsgcat.c:727: error: `uintmax_t' undeclared (first use in this function) > > which means your system has incomplete headers, and no one else has ever > tried working around that bug. > > SUA is a rather wimpy porting platform. Have you considered trying > Cygwin instead (see cygwin.com), so that you don't have to put up with > all the non-conforming incomplete POSIX garbage that Microsoft is > pushing with their SUA platform? The problem isn't 'whimpy platform', the problem is that the POSIX APIs implemented by Window's 7 SUA are too old to be usable. SUA needs an update to be competitive. Lionel
Re: bash kill(1) doesn't report errors when $(ulimit -i) is exceeded
On 15 July 2013 15:35, Chet Ramey wrote:
> On 7/14/13 2:38 PM, Cedric Blancher wrote:
>> The tests below is from the ongoing work of David Korn and Roland
>> Mainz to make signals in ksh93 reliable and predictable (so far no
>> shell tested really does it). I derived one of their tests and found
>> that bash doesn't handle realtime signals properly either:
>> bash -c '{ trap ":" RTMIN ; kill -STOP ${BASHPID} ; true ; } & ((
>> pid=$! )) ; sleep 5 ; maxi=$(ulimit -i) ; for ((i=0 ; i < (maxi*2) ;
>> i++)) ; do kill -RTMIN $pid ; done ; kill -CONT $pid ; wait ; true'
>> This prints 1, but IMO should print $(ulimit -i) as integer value and
>> then lots of errors from kill(1) because no further signals can be
>> queued to $pid because ulimit -i is exceeded for that child process.
>
> Two things: First, the script as supplied doesn't print anything.
Any idea why there is this difference?
> Second,
> the kill system call never returns < 0, so the kill builtin won't print
> an error message.
>
> I took your script, added a little debugging code to bash, and ran it.
> It sent 63000+ signals to the stopped process without kill(2) ever
> returning an error.
Either your ulimit -i is greater than 63000 or we have a Linux bug. If
ulimit -i is reached then kill(1) should fail.
Lionel
Re: Bash-4.3-beta2 available for FTP
On 15 October 2013 22:44, Joshuah Hurst wrote: > On Tue, Oct 15, 2013 at 10:36 PM, Chet Ramey wrote: >> On 10/15/13 4:27 PM, Joshuah Hurst wrote: >> > What happened to the patch for cd -@ to handle NFSv4+Windows alternate > streams? ksh93 already has this feature since quite some time and > they're now even extending support even further to the builtin POSIX > commands... >> >> I have not added it to bash yet. > > How long does it take? > >> Do you mean cd -@ as this one in ksh? - - - - - - - - -@ Change into the hidden attribute directory of directory which may also be a file. CDPATH is ignored. Hidden attribute directories are file system and operating system specific. - - - - - - - - That would be great. Where's the patch? >>> >>> Just to be sure, this does use the extended attributes with O_XATTR, right? >> >> Yes, Cedric's patch uses O_XATTR. > > *Great*! :) > > Chester/Cedric: Can you send the patch around so we can ask the Debian > GNU/kOpenSolaris, Solaris and Illumos folks to integrate the patch in > their next bash patch update, please? Chet: We're interested in the patch, too. We're actively use and rely on NFSv4 extended attributes for two of our main projects and I'd like to have a 2nd source of this feature. Primarily we use ksh93, perl and HPC applications for this but I'd like to have a fallback option for ksh93 for now. Lionel
Re: Bash-4.3-beta2 available for FTP
On 16 October 2013 14:19, Lionel Cons wrote: > On 15 October 2013 22:44, Joshuah Hurst wrote: >> On Tue, Oct 15, 2013 at 10:36 PM, Chet Ramey wrote: >>> On 10/15/13 4:27 PM, Joshuah Hurst wrote: >>> >>>>>> What happened to the patch for cd -@ to handle NFSv4+Windows alternate >>>>>> streams? ksh93 already has this feature since quite some time and >>>>>> they're now even extending support even further to the builtin POSIX >>>>>> commands... >>> >>> I have not added it to bash yet. >> >> How long does it take? >> >>> >>>>> >>>>> Do you mean cd -@ as this one in ksh? >>>>> - - - - - - - - >>>>> -@ Change into the hidden attribute directory of directory >>>>> which >>>>> may also be a file. CDPATH is ignored. Hidden attribute >>>>> directories are file system and operating system >>>>> specific. >>>>> - - - - - - - - >>>>> >>>>> That would be great. Where's the patch? >>>> >>>> Just to be sure, this does use the extended attributes with O_XATTR, right? >>> >>> Yes, Cedric's patch uses O_XATTR. >> >> *Great*! :) >> >> Chester/Cedric: Can you send the patch around so we can ask the Debian >> GNU/kOpenSolaris, Solaris and Illumos folks to integrate the patch in >> their next bash patch update, please? > > Chet: We're interested in the patch, too. We're actively use and rely > on NFSv4 extended attributes for two of our main projects and I'd like > to have a 2nd source of this feature. Primarily we use ksh93, perl and > HPC applications for this but I'd like to have a fallback option for > ksh93 for now. Any news on this one? Lionel
Build failure with cd -@ / O_XATTR
On systems with O_XATTR the bash git devel branch fails due a misnamed
buffer used for cd -@ support:
gcc -c -DHAVE_CONFIG_H -DSHELL -I. -I.. -I.. -I../include -I../lib
-I. -DSOLARIS -DDEBUG -g -O2 cd.c || ( rm -f cd.c ; exit 1 )
./cd.def: In function `cdxattr':
./cd.def:219: error: `buff' undeclared (first use in this function)
./cd.def:219: error: (Each undeclared identifier is reported only once
./cd.def:219: error: for each function it appears in.)
gmake[1]: *** [cd.o] Error 1
The fix is straightforward:
diff --git a/builtins/cd.def b/builtins/cd.def
index 498cf88..c448c5a 100644
--- a/builtins/cd.def
+++ b/builtins/cd.def
@@ -194,7 +194,7 @@ cdxattr (dir, ndirp)
{
#if defined (O_XATTR)
int apfd, fd, r, e;
- char buf[11+40+40]; /* construct new `fake' path for pwd */
+ char buff[11+40+40]; /* construct new `fake' path for pwd */
apfd = openat (AT_FDCWD, dir, O_RDONLY|O_NONBLOCK);
if (apfd < 0)
And thanks for cd -@ - keep up the good work :)
Lionel
Re: Bash-4.3-beta2 available for FTP
On 5 November 2013 22:56, Chet Ramey wrote:
>> > I'm interested in the patch if cd -@ file works like in ksh. Or
>> > whatever, just send the patch that I can test it.
>> >
>> > Thank you.
>>
>> Again. Any patch or git pull tarball which we could try?
>
> The cd -@ option is available in the `devel' branch of the git tree on
> savannah:
>
> http://git.savannah.gnu.org/cgit/bash.git/?h=devel
>
> It first appears in the bash-20131025 snapshot.
Chet, thank you
Build instructions:
git clone --branch devel git://git.savannah.gnu.org/bash.git
Apply this patch and then build as usual:
diff --git a/builtins/cd.def b/builtins/cd.def
index 498cf88..c448c5a 100644
--- a/builtins/cd.def
+++ b/builtins/cd.def
@@ -194,7 +194,7 @@ cdxattr (dir, ndirp)
{
#if defined (O_XATTR)
int apfd, fd, r, e;
- char buf[11+40+40]; /* construct new `fake' path for pwd */
+ char buff[11+40+40]; /* construct new `fake' path for pwd */
apfd = openat (AT_FDCWD, dir, O_RDONLY|O_NONBLOCK);
if (apfd < 0)
Lionel
Testcase for cd -@ / O_XATTR
Chet, below is a simple testcase for cd -@ / O_XATTR support:
./bash -c 'set -o errexit ; touch foo ; runat foo "echo hello1
>myxattr" ; cd -@ foo ; exec {n}<>myxattr ; cd -P .. ; cat <&$n ;
true'
This should print 'hello1'.
Any suggestion where this test can be placed?
Lionel
Re: [bug] Problem with cd -@ if filesystem does not support extended attributes (O_XATTR)
On 6 November 2013 13:16, Joshuah Hurst wrote:
> Chet, on Solaris the /devices filesystem does not support extended
> attributes (O_XATTR), yet cd -@ returns 0 (success):
>
> ./bash -c 'cd -@ /devices 2>/dev/null; echo $?'
> 0
> ~/bin/ksh -c 'cd -@ /devices 2>/dev/null; echo $?'
> 1
>
> truss shows that bash gets errno==EINVAL but then uses
> chdir("/devices") instead of returning the error:
> stat64("/devices", 0x08047520) = 0
> open64("/devices", O_RDONLY|O_NONBLOCK) = 3
> openat64(3, ".", O_RDONLY|O_XATTR) Err#22 EINVAL
> close(3)= 0
> chdir("/devices") = 0
> brk(0x0819C000) = 0
>
> Josh
Thanks
The following patch should fix the bugs (build+cd -@ /devices):
===
diff --git a/builtins/cd.def b/builtins/cd.def
index 498cf88..7dd9684 100644
--- a/builtins/cd.def
+++ b/builtins/cd.def
@@ -194,7 +194,7 @@ cdxattr (dir, ndirp)
{
#if defined (O_XATTR)
int apfd, fd, r, e;
- char buf[11+40+40]; /* construct new `fake' path for pwd */
+ char buff[11+40+40]; /* construct new `fake' path for pwd */
apfd = openat (AT_FDCWD, dir, O_RDONLY|O_NONBLOCK);
if (apfd < 0)
@@ -630,7 +630,7 @@ change_to_directory (newdir, nolinks, xattr)
/* We're not in physical mode (nolinks == 0), but we failed to change to
the canonicalized directory name (TDIR). Try what the user passed
verbatim. If we succeed, reinitialize the_current_working_directory. */
- if (chdir (newdir) == 0)
+ if (!xattr && (chdir (newdir) == 0))
{
t = resetpwd ("cd");
if (t == 0)
===
After applying the patch I get:
./bash -c 'cd -@ /devices ; echo $?'
./bash: line 0: cd: /devices: Invalid argument
1 <--- correct
./bash -c 'cd -@ / ; ls -l'
total 2
-r--r--r-- 1 root root 156 Sep 18 05:05 SUNWattr_ro
-rw-r--r-- 1 root root 472 Sep 18 05:05 SUNWattr_rw
Lionel
Re: flag for quiet CDPATH
On 21 January 2014 01:16, Elliott Forney wrote: > I find it a little unpleasant that cd echoes the new working directory > when CDPATH is used to locate the new directory (I already have the > working directory in my prompt). I understand that this is behavior > is mandated by POSIX but I wonder if we could have an option that > disables this. > > Maybe if the `cd' builtin had an option, say `-q', that would cause it > to be quit if CDPATH is used. Then, I could simply > > alias cd='cd -q' > > and put a stop to this. I have attached a proposed patch, any thoughts? Yes, adding yet another option which can be implemented using POSIX behaviour is IMHO code bloat. Just use alias cd='2>"/dev/null" cd ' (yes, POSIX mandates that re-directions can be prefixed and not only postfixed). Lionel
Re: Bash-4.3 available for FTP
Thank you for the cd -@ / O_XATTR support!!
Lionel
On 27 February 2014 14:12, Chet Ramey wrote:
> Introduction
>
>
> The first public release of bash-4.3 is now available with the URLs
>
> ftp://ftp.cwru.edu/pub/bash/bash-4.3.tar.gz
> ftp://ftp.gnu.org/pub/gnu/bash/bash-4.3.tar.gz
>
> and from the usual GNU mirror sites.
>
> This tar file includes the formatted documentation (postscript, dvi, html,
> and nroffed versions of the manual pages).
>
> Diffs from bash-4.2 are not available.
>
> Please use `bashbug' to report bugs with this version. It is built
> and installed at the same time as bash.
>
> Installation
>
>
> Please read the README file first.
>
> Installation instructions are provided in the INSTALL file.
>
> New Features
>
>
> This is the third revision to the fourth major release of bash.
>
> This release fixes many outstanding bugs in bash-4.2 and introduces several
> new features. The most significant bug fix is the reworking of signal
> handling to avoid running signal and trap handlers in a signal handler
> context. This led to issues with glibc, which uses internal locks
> extensively and handles longjmps from user code very poorly.
>
> The most notable new features are the `globasciiranges' shell option, which
> forces the pattern matching code to treat [a-z] as if in the C locale;
> nameref variables and the changes to allow assigning, referencing, and
> unsetting them; improvements to the `direxpand' option introduced in bash-4.2
> patch 29; and allowing negative subscripts when assigning and referencing
> indexed array elements.
>
> There is one incompatible change between bash-4.2 and bash-4.3. Bash now
> performs quote removal on the replacement string in pattern substitution
> (${param/pat/rep}), since the shell treats quotes as special. If you
> have to quote single quotes to get them to be treated literally, the shell
> should perform quote removal on them.
>
> Read the file NEWS in the bash-4.3 distribution for a complete description
> of the new features. A copy of the relevant portions is included below.
>
> Changes have been made to the Readline library being released at the same
> time as bash-4.3, readline-6.3, so that Bash can be linked against an
> already-installed Readline library rather than the private version in
> lib/readline. Only readline-6.2 and later versions are able to provide all
> of the symbols that bash-4.3 requires; earlier versions of the Readline
> library will not work correctly.
>
> A complete list of changes between bash-4.2 and bash-4.3 is available in
> the file CHANGES; the complete list is too large to include in this
> message.
>
> Readline
>
>
> Also available is a new release of the standalone Readline library,
> version 6.3, with its own configuration scripts and Makefiles.
> It can be retrieved with the URLs
>
> ftp://ftp.cwru.edu/pub/bash/readline-6.3.tar.gz
> ftp://ftp.gnu.org/pub/gnu/readline/readline-6.3.tar.gz
>
> and from the usual GNU mirror sites.
>
> Diffs from readline-6.2 are not available.
>
> The formatted Readline documentation is included in the readline
> distribution tar file.
>
> A separate announcement listing the changes in Readline is being
> distributed.
>
> As always, thanks for your help.
>
> Chet
>
> +== NEWS ==+
> This is a terse description of the new features added to bash-4.3 since
> the release of bash-4.2. As always, the manual page (doc/bash.1) is
> the place to look for complete descriptions.
>
> 1. New Features in Bash
>
> a. The `helptopic' completion action now maps to all the help topics, not
> just
> the shell builtins.
>
> b. The `help' builtin no longer does prefix substring matching first, so
> `help read' does not match `readonly', but will do it if exact string
> matching fails.
>
> c. The shell can be compiled to not display a message about processes that
> terminate due to SIGTERM.
>
> d. Non-interactive shells now react to the setting of checkwinsize and set
> LINES and COLUMNS after a foreground job exits.
>
> e. There is a new shell option, `globasciiranges', which, when set to on,
> forces globbing range comparisons to use character ordering as if they
> were run in the C locale.
>
> f. There is a new shell option, `direxpand', which makes filename completion
> expand variables in directory names in the way bash-4.1 did.
>
> g. In Posix mode, the `command' builtin does not change whether or not a
> builtin it shadows is treated as an assignment builtin.
>
> h. The `return' and `exit' builtins accept negative exit status arguments.
>
> i. The word completion code checks whether or not a filename containing a
> shell variable expands to a directory name and appends `/' to the word
> as appropriate. The same code expands shell variables in command names
> when performing command completion.
>
> j. In Posix mode, it is now an error to attempt to define a shell func
