[feature request] add associative array support to mapfile
Here is a Bash model of how, associative array support could be added to
mapfile, replacing the need for interpreted while-loop read and logic by
a much more efficient improved mapfile built-in:
#!/usr/bin/env bash
kv_cr_stream () {
printf 'key1=value1\nkey2=value=2\nkey 3=value3\n'
}
kv_null_stream () {
printf 'key1=value1\0key2=value=2\0key\n3=value3\0'
}
declare -A assoc_cr=()
# Load associative array from line records
# Proposed new feature:
# IFS='=' mapfile -A assoc_cr
while IFS='=' read -r k v || [[ -n $k && -n $v ]]; do
assoc_cr[$k]=$v
done < <(kv_cr_stream)
declare -p assoc_cr
declare -A assoc_null=()
# Load associative array from null delimited records
# Proposed new feature:
# IFS='=' mapfile -d '' -A assoc_null
while IFS='=' read -r -d '' k v || [[ -n $k && -n $v ]]; do
assoc_null[$k]=$v
done < <(kv_null_stream)
declare -p assoc_null
Expected output:
declare -A assoc_cr=([key2]="value=2" [key1]="value1" ["key 3"]="value3" )
declare -A assoc_null=([key2]="value=2" [key1]="value1" [$'key\n3']="value3" )
--
Léa Gris
Re: [feature request] add associative array support to mapfile
On 12/17/20 9:29 AM, Léa Gris wrote: Here is a Bash model of how, associative array support could be added to mapfile, replacing the need for interpreted while-loop read and logic by a much more efficient improved mapfile built-in: Thanks for the proposal. I'll consider it for a future version. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
mapfile associative array shim Was: Re: [feature request] add associative array support to mapfile
Here is an include shim to enable associative array support to mapfile
or fall-back to regular mapfile if it has that support already:
#!/usr/bin/env bash
# mapfile_assoc_shim.bash
! { mapfile -A a /dev/null && {
mapfile () {
local k v d=$'\n'
local -n A=${*: -1:1}
[[ ${*: -2:1} = -A ]] && {
[[ ${*:1:1} = -d ]] && d=${*:2:1}
while read -r -d "$d" k v || [[ -n $k && -n $v ]]
do A[$k]=$v
done
true
} || command mapfile "$@"
}
}
Example usage:
. mapfile_assoc_shim.bash
IFS='=' mapfile -d '' -A assoc_null < <(kv_null_stream)
--
Léa Gris
Re: mapfile associative array shim Was: Re: [feature request] add associative array support to mapfile
Shorty shim:
! { mapfile -A _/dev/null&&{ mapfile(){ local k v
d=$'\n';local -n A=${*: -1:1};[ ${*: -2:1} = -A ]&&{ [ ${*:1:1} = -d
]&&d=${*:2:1};while read -rd "$d" k v||[[ -n $k && -n $v ]];do
A[$k]=$v;done;:;}||command mapfile "$@";};}
--
Léa Gris
Re: [feature request] add associative array support to mapfile
On 12/17/20 10:13 AM, Chet Ramey wrote: On 12/17/20 9:29 AM, Léa Gris wrote: Here is a Bash model of how, associative array support could be added to mapfile, replacing the need for interpreted while-loop read and logic by a much more efficient improved mapfile built-in: Thanks for the proposal. I'll consider it for a future version. One problem I see with it is that it assumes the introduction of word splitting into mapfile, with the burden on the user to ensure that there are exactly two resultant fields. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: [feature request] add associative array support to mapfile
you can maybe implent dynamical runtime code decisons like runtime code for splitting definement On Thu, Dec 17, 2020, 17:36 Chet Ramey wrote: > On 12/17/20 10:13 AM, Chet Ramey wrote: > > On 12/17/20 9:29 AM, Léa Gris wrote: > >> Here is a Bash model of how, associative array support could be added > to > >> mapfile, replacing the need for interpreted while-loop read and logic > by > >> a much more efficient improved mapfile built-in: > > > > Thanks for the proposal. I'll consider it for a future version. > > One problem I see with it is that it assumes the introduction of word > splitting into mapfile, with the burden on the user to ensure that there > are exactly two resultant fields. > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ > >
Re: [feature request] add associative array support to mapfile
Le 17/12/2020 à 17:35, Chet Ramey écrivait : One problem I see with it is that it assumes the introduction of word splitting into mapfile, with the burden on the user to ensure that there are exactly two resultant fields. It is just meant to be a replacement to looping on: IFS= read -r -d k v So yes it has only two fields per record/line regardless, the first IFS character will delimit the key from the value, and the value is allowed to contain any of IFS except newline, unless the record delimiter is assigned a different character. The incentive behind this feature request is, to allow mapping key-values pairs from a character stream/file, safely into a Bash's associative array with a single built-in command. These days, people are going out of their way to parse key value pairs out of foreign config files or result streams from jq processed JSON. Maybe (it is clearly open to discussion), the associative array mapping could be allowed to: - skip blank/comment only lines (1) - allow key without value - optionally trim trailing comment (1) Anyway I think some of the above extra parsing would be out of the scope of mapfile, and could be achieved by filtering the stream with sed or other text processor. Optionally but this would go beyond current Bash shell design IMHO, would be to allow expansion/filter modules to plug into the mapfile command for processing specific file syntax, be it JSON, CSV, configfile, inifile..., something like: mapfile -m module_name -A assoc_array
Re: [feature request] add associative array support to mapfile
Le 17/12/2020 à 17:58, Léa Gris écrivait : Maybe (it is clearly open to discussion), the associative array mapping could be allowed to: - skip blank/comment only lines (1) - allow key without value - optionally trim trailing comment (1) Forgot about Bash built-in Regex engine: - allow using a Bash Regex with 2 capture groups for key and value. -- Léa Gris
Re: [feature request] add associative array support to mapfile
On 12/17/20 11:58 AM, Léa Gris wrote: Le 17/12/2020 à 17:35, Chet Ramey écrivait : One problem I see with it is that it assumes the introduction of word splitting into mapfile, with the burden on the user to ensure that there are exactly two resultant fields. It is just meant to be a replacement to looping on: IFS= read -r -d k v So yes it has only two fields per record/line regardless, the first IFS character will delimit the key from the value, and the value is allowed to contain any of IFS except newline, unless the record delimiter is assigned a different character. The incentive behind this feature request is, to allow mapping key-values pairs from a character stream/file, safely into a Bash's associative array with a single built-in command. Then these requirements suggest that the best way to proceed is to add a loadable builtin to handle this case. You could start with the `csv' and `cut' loadable builtins in bash-5.1, and could probably put together a `kv' builtin in a couple of hours. These days, people are going out of their way to parse key value pairs out of foreign config files or result streams from jq processed JSON. Maybe (it is clearly open to discussion), the associative array mapping could be allowed to: - skip blank/comment only lines (1) - allow key without value - optionally trim trailing comment (1) All of this would be better added to a new builtin, since it's all orthogonal to what mapfile provides. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
ulimit -R missing; --help is out of sync
According to the source, -R should be setting RLIMIT_RTTIME, but it does not work: bash-5.0$ ulimit -R bash: ulimit: -R: invalid option ulimit: usage: ulimit [-SHabcdefiklmnpqrstuvxPT] [limit] ..and looking at the above help, I notice letters I never saw. Lets try them? bash-5.0$ ulimit -b bash: ulimit: -b: invalid option ulimit: usage: ulimit [-SHabcdefiklmnpqrstuvxPT] [limit] bash-5.0$ ulimit -k bash: ulimit: -k: invalid option ulimit: usage: ulimit [-SHabcdefiklmnpqrstuvxPT] [limit] bash-5.0$ ulimit -P bash: ulimit: -P: invalid option ulimit: usage: ulimit [-SHabcdefiklmnpqrstuvxPT] [limit] bash-5.0$ ulimit -T bash: ulimit: -T: invalid option ulimit: usage: ulimit [-SHabcdefiklmnpqrstuvxPT] [limit] They probably should not be in help if they don't work.
