You could put Zarr into a ZIP. But there's little point in using SOZip
for that use case (SOZIP has been merged into master 6 months ago by the
way, in GDAL 3.7.0), since SOZip is for compressing large files. In a
Zarr archive, you would have a lot of small/medium sized files for each
chunk/til
The underlying network file system is opaque to us and can change on occasion.
But recently our team were asked to cull unused files due to inode counts.
We’re excited to explore SOZip on our vector data where random seek is
important to us, but we’re waiting for that branch to be merged into m
On Fri, Dec 8, 2023, at 21:32, Even Rouault wrote:
> yes, poor wording of mine. I meant that if using PREDICTOR=3, one should
> compare with FILTER=DELTA. But looking more closely, they are not strictly
> equivalent. PREDICTOR=3 applies the delta as b[0]-a[0], b[1]-a[1], b[2]-a[2],
> b[3]-a[3] w
Hi,
Could Zarr be used as Sozipped https://gdal.org/programs/sozip.html?
-Jukka Rahkonen-
Lähettäjä: gdal-dev Puolesta Meyer, Jesse R.
(GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev
Lähetetty: perjantai 8. joulukuuta 2023 21.44
Vastaanottaja: Even Rouault ; gdallists
Aihe: R
Unfortunately Zarr has a design choice that won’t work for us: blocks are
individual files on a file system. Our datasets are massive and this will
explode our inode allocations. While we could archive the folder into a zip
archive, it adds a step for anyone to work with the data. Curiously,
Le 08/12/2023 à 20:23, Laurențiu Nicola via gdal-dev a écrit :
On Fri, Dec 8, 2023, at 21:11, Even Rouault wrote:
- GeoTIFF DEFLATE 280 MB
Is it with PREDICTOR=3 ? (If not, then the ZARR datasets should also
be compressed with FILTER=DELTA to have a fair comparison)
I would expect a fa
On Fri, Dec 8, 2023, at 21:11, Even Rouault wrote:
>
>>
>> - GeoTIFF DEFLATE 280 MB
> Is it with PREDICTOR=3 ? (If not, then the ZARR datasets should also be
> compressed with FILTER=DELTA to have a fair comparison)
I would expect a fair comparison to be between PREDICTOR=3 and FILTER=DELTA
- GeoTIFF DEFLATE 280 MB
Is it with PREDICTOR=3 ? (If not, then the ZARR datasets should also be
compressed with FILTER=DELTA to have a fair comparison)
- Zarr BLOSC zlib NONE 281 MB
- Zarr BLOSC zlib BIT 253 MB
- Zarr BLOSC zlib BYTE 249 MB
--
http://www.spatialys.com
My software is f
Hi,
In my (limited) experience, it can really make a difference for floating-point
rasters. Testing with a small one I have on hand (10980x10980x1, Float32), I
get:
- GeoTIFF DEFLATE 280 MB
- Zarr BLOSC zlib NONE 281 MB
- Zarr BLOSC zlib BIT 253 MB
- Zarr BLOSC zlib BYTE 249 MB
Laurentiu
Thanks for the suggestion Even, we’ll see how effective Zarr is for our
datasets.
Jesse
From: Even Rouault
Date: Friday, December 8, 2023 at 12:20 PM
To: "Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC]"
, gdallists
Subject: [EXTERNAL] Re: [gdal-dev] GTiff bit shuffle compr
Jesse,
This would break interoperability with other TIFF readers... Even adding
a new TIFF tag to advertize that bit shuffling is applied would probably
not be a sufficient guard, as existing readers wouldn't read it, and
would just display garbage, which is worth that not being able to open
Hi,
When using horizonal differencing to reduce the numerical range of band data,
the upper bytes in the produced stream are typically 0 which leverages LZ’s
byte based compression model. But the least significant bytes can still have
many significant bits as 0. Unless the whole byte is replic
12 matches
Mail list logo
