[Numpy-discussion] Re: JSON format for multi-dimensional data

[Ralf Gommers]

On Sun, Feb 25, 2024 at 12:34 AM  wrote:

>
> > Perhaps, like the Pandas package, it should live outside NumPy for a
> > while until some wider consensus could emerge.
>
> Regarding this initial remark, this is indeed a possible option but it
> depends on the answer to the question:
>
>   - does Numpy want to have a neutral JSON exchange format to exchange
> data with other frameworks (tabular, multidimensional or other)?
>

I'd say it's unlikely. There are a lot of data storage formats; NumPy has
support for almost none of them, and for the few that we do have support
for (e.g. CSV) the reason for having that inside of NumPy is mostly
historical. There are packages like Zarr, h5py, PyTables, scipy.io that
implement support for reading and writing NumPy arrays in a large number of
I/O formats. Typically there is no reason for such code to live inside
NumPy. I'd expect the same to be true for JSON.

That isn't to say that a new JSON-based storage format wouldn't be of
interest to NumPy users - they may very well need it. We do have docs that
mention popular I/O formats, and if yours gets popular we may want to add
it to those docs:
https://numpy.org/devdocs/user/how-to-io.html#write-or-read-large-arrays
(that could use more detail too).

Cheers,
Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: JSON format for multi-dimensional data

[philippe]

Thanks Ralf, 

This answers my question about the absence of I/O Numpy format.

There are three other points related to this format proposal:

- integration of a semantic level above the number / character formats as for 
datetime (e.g. units, point / polygon, URI, email, IP, encoding...),
- neutral format (platform independent) for multidimensional data including 
multi-variables, axes, indexes and metadata,
- finally the conversion of tabular data into multi-dimensional data (dimension 
greater than 2) via a neutral format.

Do these points interest Numpy or would this rather concern applications built 
on a Numpy base?
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] builing numpy on windows

[Ganesh Rajora via NumPy-Discussion]

Hi Team,
I am Ganesh working for an MNC here in India and I am working on customised 
Python where I build set of python modules with python.

I do not install them directly from web but I build each and everything from 
source code.  this is because security concerns at the organisation. 

In similar line I want to get some help from you on building numpy on windows, 
as I did not find any direct reference on how to do that on any of numpy's 
official web. I am facing lots of issues to build that, If you could help me 
out with it or could give me a right point of contact to discuss the issue 
would be great help.
I have posted the issue here on stackoverflow - 
Issues in buildingnumpy from source on Windows

Thanks,Ganesh___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Improved 2DFFT Approach

[camrymanjr--- via NumPy-Discussion]

Good day! 

My name is Alexander Levin.

My colleague and I did a project on optimisation of two-dimensional Fourier 
transform algorithm six months ago. We took your implementation of numpy fft2d 
as a unit of quality. 

In the course of our research we found out that initially mathematically the 
method uses a far from optimal algorithm. As you may know, your operation goes 
first by rows then by columns applying a one-dimensional transformation. After 
spending some time researching mathematical papers on this topic, we found and 
implemented a new approach, the so-called Cooley-Tukey butterfly, optimised by 
Russian mathematicians in 2016.

The output is a completely new approach, where we immediately apply a 
two-dimensional operation and save a serious proportion of time on it. As a 
result, we wrote a C++ package for Python, using Cython as a wrapper. The 
result was the package and an article on Medium describing the process. On 
tests for matrices ranging in size from 2048x512 to 8192x8192, our algorithm 
outperformed the NumPy transformation by an average of 50% in time. 

After discussing this matter with my colleague with whom we did the above 
development, we came to a common desire to share our results with you. Your 
company has been making a huge contribution to the IT community for almost 20 
years on a pro bono basis. We share your philosophy and so we want to make the 
CS world a better place by providing you with our code to optimise the approach 
in an existing operation in your package.

We would like to offer you our development version, though it'll need some 
minor improvements, we'd love to finish them collaboratively with you and hear 
your thoughts regarding what we have discovered and done so far. We'd be 
honored to help you and become NumPy contributors.

I trust that our vision will resonate with you and that you will agree with us. 
I invite you to read our quick article about the process, which I have linked 
below, and our fully functioning package and share your views on our 
contribution to NumPy. We are willing to edit the code to fit your standards, 
as we believe that the best use of our work will be to contribute to the 
development of technology in the world. 

Thank you for your time. We look forward to your response and opinion. 

Medium Article about the process:
https://medium.com/p/7963c3b2f3c9
GitHub Source code Repository:
https://github.com/2D-FFT-Project/2d-fft
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: builing numpy on windows

[Matti Picus]

On 27/2/24 06:09, Ganesh Rajora via NumPy-Discussion wrote:

Hi Team,

I am Ganesh working for an MNC here in India and I am working on 
customised Python where I build set of python modules with python.


I do not install them directly from web but I build each and 
everything from source code.  this is because security concerns at the 
organisation.


In similar line I want to get some help from you on building numpy on 
windows, as I did not find any direct reference on how to do that on 
any of numpy's official web. I am facing lots of issues to build that, 
If you could help me out with it or could give me a right point of 
contact to discuss the issue would be great help.


I have posted the issue here on stackoverflow -

Issues in buildingnumpy from source on Windows 




Thanks,
Ganesh



I posted on the stack overflow link. My guess is you are not building an 
OpenBLAS MSVC import library, but using MSVC to build NumPy.


Matti

___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

[Numpy-discussion] Re: builing numpy on windows

[Ralf Gommers]

On Wed, Feb 28, 2024 at 6:54 AM Ganesh Rajora via NumPy-Discussion <
numpy-discussion@python.org> wrote:

> Hi Team,
>
> I am Ganesh working for an MNC here in India and I am working on
> customised Python where I build set of python modules with python.
>
> I do not install them directly from web but I build each and everything
> from source code.  this is because security concerns at the organisation.
>
> In similar line I want to get some help from you on building numpy on
> windows, as I did not find any direct reference on how to do that on any of
> numpy's official web. I am facing lots of issues to build that, If you
> could help me out with it or could give me a right point of contact to
> discuss the issue would be great help.
>
> I have posted the issue here on stackoverflow -
>
> Issues in buildingnumpy from source on Windows
> 
>

Hi Ganesh. I answered your previous email two days ago, please have a look
at
https://mail.python.org/archives/list/numpy-discussion@python.org/thread/MVAS7DNLOLF6KKUI3WJJXLYEUMLWBZ7N/
.

Beyond that particular issue, the docs at
https://numpy.org/devdocs/building/index.html are up to date and have
specific info on Windows. You can also look at our Windows CI jobs for logs
to see exactly what tools and versions are used:
https://github.com/numpy/numpy/blob/main/.github/workflows/windows.yml.

Cheers,
Ralf
___
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com