[Numpy-discussion] JSON format for multi-dimensional data
Hi community, This memo is a proposal to implement a compact and reversible (lossless round-trip) JSON interface for multi-dimensional data and in particular for Numpy (see issue #12481). The links to the documents are at the end of the memo. The JSON-NTV (Named and Typed value) format is a JSON format which integrates a notion of type. This format has also been implemented for tabular data (see NTV-pandas package available in the pandas ecosystem and the PDEP12 specification). . The use of this format has the following advantages: - Taking into account data types not known to Numpy, - Reversible format (lossless round-trip) - Interoperability with other tools for tabular or multi-dimensional data (e.g. pandas, Xarray) - Ease of sharing Json format - Binary coding possible (e.g. CBOR format) - Format integrating data of different nature The associated Jupyter Notebook presents some key points of this proposal (first draft): Summary: - introduction - benefits - multi-dimensionnal data - Multi-dimensional types - Format JSON - Using the NTV format - Equivalence of tabular format and multidimensional format - Astropy specific points - Units and quantities - Coordinates - Tables - Other structures This subject seems important to me (in particular for interoperability issues) and I would like to have your feedback before working on the implementation. Especially, - do you think this “semantic” format is interesting to use? - do you have any particular expectations or subjects that I need to study beforehand? - do you have any examples or test cases to offer me? And of course, any type of remark and comment is welcome. Thanks in advance ! links: - Jupyter notebook : https://nbviewer.org/github/loco-philippe/Environmental-Sensing/blob/main/python/Tests/numpy_tests.ipynb - JSON-NTV format : https://www.ietf.org/archive/id/draft-thomy-json-ntv-02.html - JSON-NTV overview : https://nbviewer.org/github/loco-philippe/NTV/blob/main/example/example_ntv.ipynb - NTV tabular format : https://www.ietf.org/archive/id/draft-thomy-ntv-tab-00.html#name-tabular-structure - NTV-pandas package : https://github.com/loco-philippe/ntv-pandas/blob/main/README.md - NTV-pandas examples : https://nbviewer.org/github/loco-philippe/ntv-pandas/blob/main/example/example_ntv_pandas.ipynb - Pandas specification - PDEP12 : https://pandas.pydata.org/pdeps/0012-compact-and-reversible-JSON-interface.html ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
[Numpy-discussion] Re: Resources of newcomers and some questions about my first PR
On Mon, Feb 19, 2024 at 3:20 PM Amogh Sood wrote: > Hi all. > > I want to thank you all for your contributions and efforts in maintaining > numpy. > numpy has been an indispensable tool during both my graduate and > undergraduate studies. > > While I am no software wizard, going forward, I do want to make efforts to > improve my skills and and also contribute to > projects that I use regularly and benefit from (in my spare time). > > PR Specific Questions: > > The docs linked here https://numpy.org/doc/stable/dev/ have been helpful > in learning how to contribute. > I have also taken some time to review open issues and attempt my first PR. > Feedback and criticism are more than welcome. > I tried following the outlined practices to the best ability. > https://github.com/numpy/numpy/pull/25844 > After submitting the PR, I am looking to remedy the following: > (i) Github actions `Some checks were not successful 1 cancelled, 3 > skipped, 59 successful, and 1 failing checks` specifically > `BLAS tests (Linux) / Test Linux (nightly OpenBLAS) (pull_request)` failed. There are issues with the nightly openblas wheels at the moment, you can ignore these failures. > (ii) I see an unverified tag on the commits associated with my PR ? I > couldn't find the process to remedy this in the docs. You can ignore this too. If you really want the “verified” bubble you’ll need to turn on commit signing but that’s not necessary for numpy (and IMO is mostly security theater). > > General Questions > a) What other resources are available for newcomers (if any) ? > Specifically with respect to mentorship, code review, feedback channels ? You can join the developer slack chat, see https://numpy.org/contribute/ for details on how to join. > b) Are there lists of beginner friend BUG/ENH that are maintained > somewhere ? (e.g. a good first issue tag?) There is a “sprintable” tag. Numpy isn’t really a beginner-friendly library so we don’t want to deceive people by labeling a bug as easy but for an inexperienced developer would be very hard. > > I look forward to learning from and collaborating with this community. > Many thanks for your time and efforts. > ___ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: nathan12...@gmail.com > ___ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
