Re: extension of file-test primitives?
On Tue, Aug 22, 2017 at 09:25:44PM -0700, L A Walsh wrote: > Chet Ramey wrote: > > On 8/21/17 9:27 AM, Greg Wooledge wrote: > > > You could write your own helper functions for this: > > > > > > -fx() { test -f "$1" && test -x "$1"; } > > > > This is indeed a quick and easy way to implement desired functionality. > > Shell functions can do a lot. > >Alas, they don't work inside [[...]] with other > file-test operators: > > [[ -fx /bin/ls && -s /bin/ls ]] && echo ok They're not intended to work that way. If you want to test f+x+s then you just make another function: -fxs() { test -f "$1" && test -x "$1" && test -s "$1"; } if -fxs /bin/ls; then ... You could also take the original function and combine it with a test command: if -fx /bin/ls && test -s /bin/ls; then ... Or, since you seem to be infatuated with the [[ command: if -fx /bin/ls && [[ -s /bin/ls ]]; then ... All of your syntactic proposals have problems. First, curly braces. These already have their own semantics, and what you are proposing does not work with those semantics. You proposed: [[ -{f,x} $file ]] With standard curly brace semantics (ignoring [['s stupid magic parser-bending rules for the moment) this expands to: [[ -f -x $file ]] If we use single brackets and set -x we can even witness the shell expanding it thus. wooledg:~$ [ -{f,x} file ] + '[' -f -x file ']' bash: [: -x: binary operator expected Second, you proposed test -fx. But this collides with existing multi-character test options. Specifically, test has the options -e, -f, and -ef. So your proposal of letter-concatenation would take the -e and -f options and squash them together into -ef, which would collide with the "two files are hard links to the same inode" option. Likewise, the unary options -n and -t would combine into -nt which collides with the binary "mtime is newer than" option. Sure, you might argue that there is no reason to combine -e and -f together, since one implies the other; or that no sane person would ever combine -n and -t together; but that's not the point. The point is to avoid reader confusion. It's already bad enough that the -a option is overloaded and has two completely different meanings. Let's not compound that mistake.
Re: extension of file-test primitives?
Greg Wooledge wrote: They're not intended to work that way. If you want to test f+x+s then you just make another function: -fxs() { test -f "$1" && test -x "$1" && test -s "$1"; } How many different single-ops? over 20? That's 20 factorial combos. You wanna include that in a script? um... All of your syntactic proposals have problems. First, curly braces. These already have their own semantics, and what you are proposing does not work with those semantics. You proposed: [[ -{f,x} $file ]] With standard curly brace semantics (ignoring [['s stupid magic parser-bending rules for the moment) this expands to: Nope.. can't ignore '[['s special rules. That's why this can work. Second, you proposed test -fx. I did not -- that was PePa's idea which had nothing to do with my proposal. My initial design avoided that issue specifically for that reason. I spoke out against using a combined op like -ge (as you would suggest for a function name) because it conflicts with existing two character operators. The rest of your argument against merged primitives like funcs -ef, -fe... etc. applies to your using function names with those names as well as PePa's idea. The only form I proposed is -{X,Y,Z} and after Chet pointed out the compat probs with external 'test' binaries, I limited to only being used with '[[' -- which has its own compat rules -- that being the reason I agreed w/Chet to limit it to use with '[[' -- right Chet? ;-)
Re: extension of file-test primitives?
On 08/23/2017 07:36 PM, Greg Wooledge wrote: Second, you proposed test -fx. But this collides with existing multi-character test options. Specifically, test has the options -e, -f, and -ef. So your proposal of letter-concatenation would take the -e and -f options and squash them together into -ef, which would collide with the "two files are hard links to the same inode" option. Likewise, the unary options -n and -t would combine into -nt which collides with the binary "mtime is newer than" option. Sure, you might argue that there is no reason to combine -e and -f together, since one implies the other; or that no sane person would ever combine -n and -t together; but that's not the point. The point is to avoid reader confusion. It's already bad enough that the -a option is overloaded and has two completely different meanings. Let's not compound that mistake. I was the one proposing combining the UNARY operators in a way like -fx, and your -a example shows that there is a precedent. Unary and binary operators are different, so there is no confusion for the parser or for the user. It would just work for the person who would like to use it. The combining would only be allowed (and only make sense) for the unary operators, and it would lead to a much cleaner and more readable source. It is also in line with common GNU combining-short-options syntax. Users might even expect this to work already. This change would not hurt backwards compatibility and enhance bash in a clean way, making things more readable where they would/could be used. Peter
Re: extension of file-test primitives?
On Wed, Aug 23, 2017 at 07:55:55PM +0700, Peter & Kelly Passchier wrote: > I was the one proposing combining the UNARY operators in a way like -fx, and > your -a example shows that there is a precedent. Unary and binary operators > are different, so there is no confusion for the parser or for the user. No, trust me, there *IS* confusion for the users. Try hanging out in #bash for a while and you'll see (unless you gouge your eyes out first).
Re: extension of file-test primitives?
On 8/23/17 8:55 AM, Peter & Kelly Passchier wrote: > Users > might even expect this to work already. Nobody expects this to work already; it never has and there's no reason to think it would. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
On Wed, Aug 23, 2017 at 3:55 PM, L A Walsh wrote: > > > Greg Wooledge wrote: > >> >> >> They're not intended to work that way. If you want to test f+x+s then >> you just make another function: >> >> -fxs() { test -f "$1" && test -x "$1" && test -s "$1"; } >> >> > How many different single-ops? over 20? That's 20 factorial > combos. You wanna include that in a script? um... > You can use a loop, here is hack(ish) function that perhaps work (ie not tested too much) testfile () { local OPTIND=1 f=${!#} while getopts abcdefghLkprsSuwxOGN opt; do case $opt in [abcdefghLkprsSuwxOGN]) test -$opt $f || return 1;; *)return 1;; esac; done } if testfile -fx file;then.
Re: extension of file-test primitives?
On Wed, Aug 23, 2017 at 04:22:09PM +0300, Pierre Gaston wrote: > testfile () { > local OPTIND=1 f=${!#} > while getopts abcdefghLkprsSuwxOGN opt; > do > case $opt in >[abcdefghLkprsSuwxOGN]) test -$opt $f || return 1;; "$f" >*)return 1;; > esac; >done > } > > if testfile -fx file;then. Add the quotes, make opt local too, and I think we have a winner.
Re: extension of file-test primitives?
On 8/23/17 9:34 AM, Greg Wooledge wrote: > On Wed, Aug 23, 2017 at 04:22:09PM +0300, Pierre Gaston wrote: >> testfile () { >> local OPTIND=1 f=${!#} >> while getopts abcdefghLkprsSuwxOGN opt; >> do >> case $opt in >>[abcdefghLkprsSuwxOGN]) test -$opt $f || return 1;; > > "$f" > >>*)return 1;; >> esac; >>done >> } >> >> if testfile -fx file;then. > > Add the quotes, make opt local too, and I think we have a winner. This has the advantage of supporting both syntax options: a single option with multiple operators or a series of options, each with one or more operators, combined with a single operand. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
On August 23, 2017 3:37:51 PM GMT+02:00, Chet Ramey wrote: >On 8/23/17 9:34 AM, Greg Wooledge wrote: >> On Wed, Aug 23, 2017 at 04:22:09PM +0300, Pierre Gaston wrote: >>> testfile () { >>> local OPTIND=1 f=${!#} >>> while getopts abcdefghLkprsSuwxOGN opt; >>> do >>> case $opt in >>>[abcdefghLkprsSuwxOGN]) test -$opt $f || return 1;; >> >> "$f" >> >>>*)return 1;; >>> esac; >>>done >>> } >>> >>> if testfile -fx file;then. >> >> Add the quotes, make opt local too, and I think we have a winner. >This has the advantage of supporting both syntax options: a single >option with multiple operators or a series of options, each with one >or more operators, combined with a single operand. Not really as it changes the meaning of test -f file -a -x file Which I always understood as the correct way of doing this in the first place... The only optimisation would be to possible cache the stat to save on system io. -- Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.
Re: extension of file-test primitives?
On 8/23/17 9:51 AM, Dethrophes wrote: if testfile -fx file;then. >>> >>> Add the quotes, make opt local too, and I think we have a winner. >> This has the advantage of supporting both syntax options: a single >> option with multiple operators or a series of options, each with one >> or more operators, combined with a single operand. > > Not really as it changes the meaning of > test -f file -a -x file It is not a drop-in replacement for test; it is syntactic sugar for compound test operations. If you're concerned about this, add a test before the loop to ensure that there are only two arguments to the function and restrict it to that syntax option only. > Which I always understood as the correct way of doing this in the first > place... It's not as good as multiple test commands: test -f file && test -x file. There's no ambiguity and you get short-circuiting. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
> >> Which I always understood as the correct way of doing this in the >first place... > >It's not as good as multiple test commands: test -f file && test -x >file. >There's no ambiguity and you get short-circuiting. Only if you are using the test built-in, otherwise the latter means 2 spawns/forks however the shell in question calls the test exec. -- Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.
Re: extension of file-test primitives?
On 8/23/17 10:24 AM, Dethrophes wrote: > > >> >>> Which I always understood as the correct way of doing this in the >> first place... >> >> It's not as good as multiple test commands: test -f file && test -x >> file. >> There's no ambiguity and you get short-circuiting. > > Only if you are using the test built-in, otherwise the latter means 2 > spawns/forks however the shell in question calls the test exec. Since bash has a test builtin, this isn't exactly on point. But you have to accept this kind of micro-inefficiency with a shell that sacrifices speed for size. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
On Wed, Aug 23, 2017 at 04:24:55PM +0200, Dethrophes wrote: > > > >> Which I always understood as the correct way of doing this in the > >first place... > > > >It's not as good as multiple test commands: test -f file && test -x > >file. > >There's no ambiguity and you get short-circuiting. > > Only if you are using the test built-in, otherwise the latter means 2 > spawns/forks however the shell in question calls the test exec. The comparison was against "test -f file -a -x file" which is deprecated. The use of "-a" as a logical AND is not mandated by POSIX except in "obsolescent XSI" mode. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html http://pubs.opengroup.org/onlinepubs/9699919799/help/codes.html#OB%20XSI So if you're using test ... -a ... then you're almost certainly relying on test being bash's builtin version. And since this is bug-bash, we generally assume you are using bash. There is also test(1) in GNU coreutils, which currently still supports the binary -a, but the coreutils 8.26 man page says "NOTE: Binary -a and -o are inherently ambiguous. Use 'test EXPR1 && test EXPR2' or 'test EXPR1 || test EXPR2' instead." But, writing a script that relies on test being the one provided by GNU coreutils (or any other version which implements the obsolescent XSI deprecated feature set) is also extremely silly.
Re: extension of file-test primitives?
Well technically I don't *have* to accept the performance penalty. As I can just use the posix comform syntax, which is quicker. And just becasue I was curious as to how much quicker it is. Now admittedly it's not the biggest difference in perf, but I don't see any real point in using a slower one. test_file_1(){ test -f "${1:?Missing Test File}" -a -x "${1}" } test_file_2(){ test -f "${1:?Missing Test File}" && test -x "${1}" } test_case(){ local cnt=${1:?Missing repeat count} local test_func=${2:?Missing Test Function} local test_file=${3:?Missing Test File} echo "${test_func}" "${test_file}" while [ $(((cnt -= 1 ) )) != 0 ]; do "${test_func}" "${test_file}" done } : ${TMPDIR:=/tmp} setup_test(){ touch "${TMPDIR}/file" touch "${TMPDIR}/exec_file" chmod a+x "${TMPDIR}/exec_file" time test_case 1 test_file_1 "${TMPDIR}/file" time test_case 1 test_file_2 "${TMPDIR}/file" time test_case 1 test_file_1 "${TMPDIR}/exec_file" time test_case 1 test_file_2 "${TMPDIR}/exec_file" } ~/ks_qnx/test_bash.sh setup_test test_file_1 /tmp/file real 0m0.132s user 0m0.128s sys 0m0.004s test_file_2 /tmp/file real 0m0.148s user 0m0.116s sys 0m0.028s test_file_1 /tmp/exec_file real 0m0.138s user 0m0.128s sys 0m0.008s test_file_2 /tmp/exec_file real 0m0.153s user 0m0.128s sys 0m0.024s Am 23.08.2017 um 16:27 schrieb Chet Ramey: On 8/23/17 10:24 AM, Dethrophes wrote: Which I always understood as the correct way of doing this in the first place... It's not as good as multiple test commands: test -f file && test -x file. There's no ambiguity and you get short-circuiting. Only if you are using the test built-in, otherwise the latter means 2 spawns/forks however the shell in question calls the test exec. Since bash has a test builtin, this isn't exactly on point. But you have to accept this kind of micro-inefficiency with a shell that sacrifices speed for size.
Re: extension of file-test primitives?
Am 23.08.2017 um 16:46 schrieb Greg Wooledge: On Wed, Aug 23, 2017 at 04:24:55PM +0200, Dethrophes wrote: Which I always understood as the correct way of doing this in the first place... It's not as good as multiple test commands: test -f file && test -x file. There's no ambiguity and you get short-circuiting. Only if you are using the test built-in, otherwise the latter means 2 spawns/forks however the shell in question calls the test exec. The comparison was against "test -f file -a -x file" which is deprecated. The use of "-a" as a logical AND is not mandated by POSIX except in "obsolescent XSI" mode. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html http://pubs.opengroup.org/onlinepubs/9699919799/help/codes.html#OB%20XSI So if you're using test ... -a ... then you're almost certainly relying on test being bash's builtin version. And since this is bug-bash, we generally assume you are using bash. There is also test(1) in GNU coreutils, which currently still supports the binary -a, but the coreutils 8.26 man page says "NOTE: Binary -a and -o are inherently ambiguous. Use 'test EXPR1 && test EXPR2' or 'test EXPR1 || test EXPR2' instead." But, writing a script that relies on test being the one provided by GNU coreutils (or any other version which implements the obsolescent XSI deprecated feature set) is also extremely silly. Ok I wasn't aware of that it is depricated. Having said that it is pretty widely supported. I make use of this in pdksh, bash, dash, ksh and in a couple of test implementations not just the gnu one.
Re: extension of file-test primitives?
On 8/23/17 10:49 AM, dethrophes wrote: > Well technically I don't *have* to accept the performance penalty. > > As I can just use the posix comform syntax, which is quicker. Wait, which posix-conforming syntax? Because your original example, which had five arguments to `test', is explicitly unspecified: >4 arguments: The results are unspecified. unless you're on an XSI-conformant system. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
Yhea I just learned that now, it's been at least a decade since I looked at the posix spec on test. Should probably update the bash help to reflect that as help bash (in my version at least) only says EXPR1 -a EXPR2 True if both expr1 AND expr2 are true. EXPR1 -o EXPR2 True if either expr1 OR expr2 is true. The main oses I work on are XSI conform. Am 23.08.2017 um 17:00 schrieb Chet Ramey: On 8/23/17 10:49 AM, dethrophes wrote: Well technically I don't *have* to accept the performance penalty. As I can just use the posix comform syntax, which is quicker. Wait, which posix-conforming syntax? Because your original example, which had five arguments to `test', is explicitly unspecified: 4 arguments: The results are unspecified. unless you're on an XSI-conformant system.
Re: extension of file-test primitives?
On 8/23/17 11:13 AM, dethrophes wrote: > Yhea I just learned that now, it's been at least a decade since I looked at > the posix spec on test. > > Should probably update the bash help to reflect that > > as help bash (in my version at least) only says > > EXPR1 -a EXPR2 True if both expr1 AND expr2 are true. > EXPR1 -o EXPR2 True if either expr1 OR expr2 is true. Why update the help documentation? Bash supports it. It's just deprecated in the posix standard. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
Ok I inferred incorrectly that bash was also depricating it. And just because I was curious. I added the other syntaxes I'm aware of to see how they compared. looks like [[ -e file && -x file ]], is the quickest. test_file_1(){ test -f "${1}" -a -x "${1}" } test_file_2(){ test -f "${1}" && test -x "${1}" } test_file_3(){ [ -f "${1}" -a -x "${1}" ] } test_file_4(){ [[ -f "${1}" && -x "${1}" ]] } test_file_5(){ [ -f "${1}" ] && [ -x "${1}" ] } test_file_6(){ [[ -f "${1}" ]] && [[ -x "${1}" ]] } test_case(){ local cnt=${1:?Missing repeat count} local test_func=${2} local test_file=${3} echo "${test_func}" "${test_file}" time { while [ $(((cnt -= 1 ) )) != 0 ]; do "${test_func}" "${test_file}" done } } : ${TMPDIR:=/tmp} setup_test(){ touch "${TMPDIR}/file" touch "${TMPDIR}/exec_file" chmod a+x "${TMPDIR}/exec_file" for cFile in "${TMPDIR}/file" "${TMPDIR}/exec_file" ; do for cTest in test_file_{1,2,3,4,5,6} ; do time=$(test_case 1 "${cTest}" "${cFile}" 2>&1) echo ${time} done done } ./test_bash.sh setup_test test_file_1 /tmp/file real 0m0.144s user 0m0.132s sys 0m0.008s test_file_2 /tmp/file real 0m0.146s user 0m0.136s sys 0m0.008s test_file_3 /tmp/file real 0m0.142s user 0m0.136s sys 0m0.004s test_file_4 /tmp/file real 0m0.138s user 0m0.120s sys 0m0.016s test_file_5 /tmp/file real 0m0.172s user 0m0.160s sys 0m0.008s test_file_6 /tmp/file real 0m0.123s user 0m0.100s sys 0m0.020s test_file_1 /tmp/exec_file real 0m0.138s user 0m0.116s sys 0m0.020s test_file_2 /tmp/exec_file real 0m0.151s user 0m0.140s sys 0m0.008s test_file_3 /tmp/exec_file real 0m0.142s user 0m0.140s sys 0m0.000s test_file_4 /tmp/exec_file real 0m0.118s user 0m0.112s sys 0m0.004s test_file_5 /tmp/exec_file real 0m0.162s user 0m0.148s sys 0m0.012s test_file_6 /tmp/exec_file real 0m0.142s user 0m0.132s sys 0m0.004s Am 23.08.2017 um 17:17 schrieb Chet Ramey: On 8/23/17 11:13 AM, dethrophes wrote: Yhea I just learned that now, it's been at least a decade since I looked at the posix spec on test. Should probably update the bash help to reflect that as help bash (in my version at least) only says EXPR1 -a EXPR2 True if both expr1 AND expr2 are true. EXPR1 -o EXPR2 True if either expr1 OR expr2 is true. Why update the help documentation? Bash supports it. It's just deprecated in the posix standard.
Re: extension of file-test primitives?
On 8/23/17 11:54 AM, dethrophes wrote: > Ok I inferred incorrectly that bash was also depricating it. Well, it's supported because it's historical syntax, but that doesn't mean I won't recommend a superior alternative. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: extension of file-test primitives?
Interestingclever, even though not well vetted here, either. Pierre Gaston wrote: You can use a loop, here is hack(ish) function that perhaps work (ie not tested too much): testfile () { local OPTIND=1 f=${!#} while getopts abcdefghLkprsSuwxOGN opt; do case $opt in [abcdefghLkprsSuwxOGN]) test -$opt $f || return 1;; *)return 1;; esac; done } if testfile -fx file;then. However, I am not wild about the way it creates use of two character tests (-ge) that I, Chet and Greg don't like due to confusion with existing infix operators. It's more likely to create maintenance problems just based on that attribute alone. I also don't like having to split it apart from other '[[' operators. FWIW, I always use '[[' ... ']]', if for no other reason than not having to use quotes, as well the consistency of using '&&' and '||' that test and '[' can't use. Trying to remember that something like: > a=1 b=2; test 1 == $a && 2 == $b && echo hi -bash: 2: command not found doesn't work wastes time, as well as > a=1 b=2; test 1 == $a -a 2 == $b && echo hi not working with '[[', where in changing to '[[': > a=1 b=2; [[ 1 == $a -a 2 == $b ]] && echo hi -bash: syntax error in conditional expression -bash: syntax error near `-a' Even parens need special handling, outside of '[['. I'd also have to remember to change the conjunction ops. I.e. changing "-a" to '&&': > a=1 b=2; [[ 1 == $a && 2 == $b ]] && echo hi The [[ operator can be seamlessly changed to numeric ops: > a=1 b=2; (( 1 == $a && 2 == $b )) && echo hi A lot of special casing when updating such usage, as well as it not being readily combinable in [[...]] with other tests. The idea was to add a way to allow a shorthand that wouldn't require secondary knowledge (what does testfile do), or a need for including it.
Re: extension of file-test primitives?
On 08/23/2017 10:17 AM, Chet Ramey wrote: > On 8/23/17 11:13 AM, dethrophes wrote: >> Yhea I just learned that now, it's been at least a decade since I looked at >> the posix spec on test. >> >> Should probably update the bash help to reflect that >> >> as help bash (in my version at least) only says >> >> EXPR1 -a EXPR2 True if both expr1 AND expr2 are true. >> EXPR1 -o EXPR2 True if either expr1 OR expr2 is true. > > Why update the help documentation? Bash supports it. It's just deprecated > in the posix standard. Not just deprecated, but inherently ambiguous. There are situations where you CANNOT tell whether the user meant -a to mean the binary operator or a string being tested against, because POSIX (intentionally) does not specify enough precedence rules in how to parse > 4 arguments. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
Re: extension of file-test primitives?
On Wed, Aug 23, 2017 at 3:57 PM, Eric Blake wrote: > > Not just deprecated, but inherently ambiguous. There are situations > where you CANNOT tell whether the user meant -a to mean the binary > operator or a string being tested against, because POSIX (intentionally) > does not specify enough precedence rules in how to parse > 4 arguments. > > -- > Eric Blake, Principal Software Engineer > Red Hat, Inc. +1-919-301-3266 > Virtualization: qemu.org | libvirt.org > > What would you expect [ ! -a foo ] to do? It's always true, because it treats -a as the logical AND there. And that's with 3 arguments, which is defined but unexpected behavior