On Thu, Aug 25, 2022 at 3:47 PM Qianqian Fang wrote:
> On 8/25/22 12:25, Robert Kern wrote:
>
> I don't quite know what this means. My installed version of `jq`, for
> example, doesn't seem to know what to do with these files.
>
> ❯ jq --version
> jq-1.6
>
> ❯ jq . eye5chunk_bjd_raw.jdb
> parse e
On 8/25/22 12:25, Robert Kern wrote:
No one is really proposing another format, just a minor tweak to the
existing NPY format.
agreed. I was just following the previous comment on alternative formats
(such as hdf5) and pros/cons of npy.
I don't quite know what this means. My installed versi
On Thu, Aug 25, 2022 at 10:45 AM Qianqian Fang wrote:
> I am curious what you and other developers think about adopting
> JSON/binary JSON as a similarly simple, reverse-engineering-able but
> universally parsable array exchange format instead of designing another
> numpy-specific binary format.
Can you give load times for these?
> 8000128 eye5chunk.npy
> 5004297 eye5chunk_bjd_raw.jdb
> 10338 eye5chunk_bjd_zlib.jdb
>2206 eye5chunk_bjd_lzma.jdb
For my case, I'd be curious about the time to add one 1T-entries file to
another.
Thanks,
Bill
--
Phobrain.com
On 2022-08-24 20
I am curious what you and other developers think about adopting
JSON/binary JSON as a similarly simple, reverse-engineering-able but
universally parsable array exchange format instead of designing another
numpy-specific binary format.
I am interested in this topic (as well as thoughts among nu
Hi Matti, hi all,
@Matti: I don’t know what exactly you are referring to (Pull request or the
Github project, links see below). Maybe some clarification is needed, which I
hereby try to do ;)
A .npy file created by some appending process is a regular .npy file and does
not need to be read in c
Sorry for the late reply. Adding a new "*.npy" format feature to allow
writing to the file in chunks is nice but seems a bit limited. As I
understand the proposal, reading the file back can only be done in the
chunks that were originally written. I think other libraries like zar or
h5py have so
