[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-04-16 Thread Dom Grigonis
Looks good, I am currently using very low complexity data structures. Could you post some examples how would the data look like for simple stuff like 2d array with 2 coordinate arrays without any extra attributes? Regards, dgpb > On 15 Apr 2024, at 15:38, phili...@loco-labs.io wrote: > > Hell

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-04-15 Thread philippe
Hello, I created a first version of a neutral format for multi-dimensional data (https://nbviewer.org/github/loco-philippe/ntv-numpy/blob/main/example/example_ntv_numpy.ipynb ) and I made available a first version of a package (https://github.com/loco-philippe/ntv-numpy/blob/main/README.md) wi

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-29 Thread Dom Grigonis
Could be of interest to scipp. They already put work to link to https://www.nexusformat.org. Maybe they would be interested in JData as well. — Also, what could be interesting is "format independent standard”. So that there is a general standard of data structures, which is adapted by differen

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-29 Thread philippe
Thank you dom for this encouraging comment ! I agree with these remarks. I will indeed integrate the extensions made by scipp to Xarray. Note: I am also looking for feedback regarding the analysis of tabular structures (e.g. to identify the hidden multidimensional structure): https://github.com

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-28 Thread Dom Grigonis
I sure like the idea of this. I agree that this should be external to numpy. At least until it becomes a standard in a sense that json itself is. And that n-dimensional array should ideally be extended to indexed structures with named dimensions and attributes. To accommodate a superset of: xar

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-28 Thread philippe
Bravo for this very comprehensive work which covers all technical/scientific data structures well ! I think we share the same goal of improving interoperability through the use of neutral formats. However, I see some differences: - I focus efforts more particularly on increasing the semantic le

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-28 Thread fangqq--- via NumPy-Discussion
aside from the previously mentioned ticket https://github.com/numpy/numpy/issues/12481, I also made a similar proposal, posted in 2021 https://github.com/numpy/numpy/issues/20461 https://mail.python.org/archives/list/numpy-discussion@python.org/message/EVQW2PO64464JEN3RQXSCDP32RQDIQFW/ lightwe

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-28 Thread Ralf Gommers
On Tue, Feb 27, 2024 at 3:45 PM wrote: > Thanks Ralf, > > This answers my question about the absence of I/O Numpy format. > > There are three other points related to this format proposal: > > - integration of a semantic level above the number / character formats as > for datetime (e.g. units, poi

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-27 Thread philippe
Thanks Ralf, This answers my question about the absence of I/O Numpy format. There are three other points related to this format proposal: - integration of a semantic level above the number / character formats as for datetime (e.g. units, point / polygon, URI, email, IP, encoding...), - neutra

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-27 Thread Ralf Gommers
On Sun, Feb 25, 2024 at 12:34 AM wrote: > > > Perhaps, like the Pandas package, it should live outside NumPy for a > > while until some wider consensus could emerge. > > Regarding this initial remark, this is indeed a possible option but it > depends on the answer to the question: > > - does Nu

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-24 Thread philippe
Thank you Matti for this response. I completed issue 12481 because in my opinion the format proposal responds to this issue. However, if you think a specific issue is preferable, I can create it. To fully understand the proposed standard, it involves representing multidimensional data that has

[Numpy-discussion] Re: JSON format for multi-dimensional data

2024-02-23 Thread Matti Picus
On 20/2/24 01:24, phili...@loco-labs.io wrote: Hi community, This memo is a proposal to implement a compact and reversible (lossless round-trip) JSON interface for multi-dimensional data and in particular for Numpy (see issue #12481). The links to the documents are at the end of the memo. Th