First, thanks for taking the time to read and provide your thoughts. This is the real value of the discussion/
Second: I'm NOT trying to argue that there isn't valid use for combining bash/curl/jq, Nor do I suggest adding JSON as first class object to bash (Python/node/Perl/Groovy are way ahead ...). I hope to get feedback from other readers for the news group that may find this approach useful. I'll take the very common use case of using AWS CLI, which does produces JSON response for most calls. Processing the response, while possible with JQ, it is challenging to many junior (and intermediate) developers. In many cases, they fall into the traps that I mentioned above - performance (excessive forking or fork/exec), or code that is hard to read (I've seen really bad code - combining pipes of JQ/awk/sed). I'm trying to address those cases. Almost always they fail to properly handle objects with while space, new-lines, etc. To be practical, I'll try to follow the loadable extension path, and see how much I can get thru that path. Possibly will make sense to continue discussion with a concrete implementation. I believe the necessary commands are: json_read -a data-array -m meta-array -r root-obj Parse the stdin input and into data-array with the items (as described above), and the meta-array with helper information (length, prop list under each node) - to help iterating. json_write -v variable -a data-array -m meta-array -r root The reverse - generate the JSON for associated array following '.' naming convention json_add -a data-array [-m meta-array] [-r root] key1=value1 key2=value2 key3=value3 Helper to add items into associative array representing JSON object, Autodetect type, with the ability to force stringfication using format string. On Sun, Aug 28, 2022 at 3:22 PM John Passaro <john.a.pass...@gmail.com> wrote: > interfacing with an external tool absolutely seems like the correct answer > to me. a fact worth mentioning to back that up is that `jq` exists. billed > as a sed/awk for json, it fills all the functions you'd expect such an > external tool to have and many many more. interfacing from curl to jq to > bash is something i do on a near daily basis. > > https://stedolan.github.io/jq/ > > On Sun, Aug 28, 2022, 09:25 Yair Lenga <yair.le...@gmail.com> wrote: > >> Hi, >> >> Over the last few years, JSON data becomes a integral part of processing. >> In many cases, I find myself having to automate tasks that require >> inspection of JSON response, and in few cases, construction of JSON. So >> far, I've taken one of two approaches: >> * For simple parsing, using 'jq' to extract elements of the JSON >> * For more complex tasks, switching to python or Javascript. >> >> Wanted to get feedback about the following "extensions" to bash that will >> make it easier to work with simple JSON object. To emphasize, the goal is >> NOT to "compete" with Python/Javascript (and other full scale language) - >> just to make it easier to build bash scripts that cover the very common >> use >> case of submitting REST requests with curl (checking results, etc), and to >> perform simple processing of JSON files. >> >> Proposal: >> * Minimal - Lightweight "json parser" that will convert JSON files to bash >> associative array (see below) >> * Convert bash associative array to JSON >> >> To the extent possible, prefer to borrow from jsonpath syntax. >> >> Parsing JSON into an associative array. >> >> Consider the following, showing all possible JSON values (boolean, number, >> string, object and array). >> { >> "b": false, >> "n": 10.2, >> "s: "foobar", >> x: null, >> "o" : { "n": 10.2, "s: "xyz" }, >> "a": [ >> { "n": 10.2, "s: "abc", x: false }, >> { "n": 10.2, "s": "def" x: true}, >> ], >> } >> >> This should be converted into the following array: >> >> ------------------------------------- >> >> # Top level >> [_length] = 6 # Number of keys in object/array >> [_keys] = b n s x o a # Direct keys >> [b] = false >> [n] = 10.2 >> [s] = foobar >> [x] = null >> >> # This is object 'o' >> [o._length] = 2 >> [o._keys] = n s >> [o.n] = 10.2 >> [o.s] = xyz >> >> # Array 'a' >> [a._count] = 2 # Number of elements in array >> >> # Element a[0] (object) >> [a.0._length] = 3 >> [a.0._keys] = n s x >> [a.0.n] = 10.2 >> [a.0.s] = abc >> [a.0_x] = false >> >> ------------------------------------- >> >> I hope that example above is sufficient. There are few other items that >> are >> worth exploring - e.g., how to store the type (specifically, separate the >> quoted strings vs value so that "5.2" is different than 5.2, and "null" is >> different from null. >> >> I will leave the second part to a different post, once I have some >> feedback. I have some prototype that i've written in python - POC - that >> make it possible to write things like >> >> declare -a foo >> curl http://www.api.com/weather/US/10013 | readjson foo >> >> printf "temperature(F) : %.1f Wind(MPH)=%d" ${foo[temp_f]}, ${foo[wind]} >> >> Yair >> >
