Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-29 Thread Simon Eves
We could, but it does import if you're willing to leave it long enough. I guess we could throw an "are you sure?" if the file is large. However, being able to kill it without bringing down the whole server is still desirable! On Thu, Jul 29, 2021 at 11:09 AM Andrew Bell wrote: > Not exactly what

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-29 Thread Andrew Bell
Not exactly what you asked, but can you simply avoid opening the file at all if it's too large rather than start the process and then abort? On Wed, Jul 28, 2021 at 7:10 PM Simon Eves wrote: > Agreed. Unfortunately, we're looking for a quick solution to a customer > complaint. I'll ponder it som

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-29 Thread Mike Flannigan
...@omnisci.com>> > To:gdal-dev@lists.osgeo.org <mailto:to%3agdal-...@lists.osgeo.org> > Subject: [gdal-dev] Large GeoJSONs and aborting file opening > Message-ID: >       mailto:cajf0ktrsasksospv8tba%2bitb%2btql_ui5y4n05wgldw_3gur...@mail.gmail.com>> &

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-29 Thread Mike Flannigan
thub.com Texas is 2.38 GB , but you get a variety of other sizes *From:* gdal-dev on behalf of Mike Flannigan *Sent:* Thursday, July 29, 2021 9:49 AM *To:* gdal-dev@lists.osgeo.org *Subject:* [EXTERNAL] Re: [gdal-dev] Large G

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-29 Thread Simon Eves
n 2 years ago because I used to have problems > with SHP linear files. > > I use QGIS 3.16.8 on Linux Mint. > > > Mike > > > On 7/28/21 2:36 PM, gdal-dev-requ...@lists.osgeo.org wrote: > > Date: Wed, 28 Jul 2021 12:22:12 -0700 > > From: Simon Eves > > To

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-29 Thread Mike Flannigan
@lists.osgeo.org Subject: [gdal-dev] Large GeoJSONs and aborting file opening Message-ID: Content-Type: text/plain; charset="utf-8" Dear All, I am aware that some improvements were made in the 2.3 timeframe with regards to dealing with large GeoJSON files, although even in 3.2,

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-28 Thread Simon Eves
Agreed. Unfortunately, we're looking for a quick solution to a customer complaint. I'll ponder it some more. On Wed, Jul 28, 2021 at 12:28 PM Even Rouault wrote: > Simon, > > I don't think that killing a thread is a safe practice in general. It > would likely result in memory leaks and maybe in

Re: [gdal-dev] Large GeoJSONs and aborting file opening

2021-07-28 Thread Even Rouault
Simon, I don't think that killing a thread is a safe practice in general. It would likely result in memory leaks and maybe in some other inconsistent state that could crash the whole process.  An interesting enhancement for such cases would be to be able to provide a progress / abort callback.

[gdal-dev] Large GeoJSONs and aborting file opening

2021-07-28 Thread Simon Eves
Dear All, I am aware that some improvements were made in the 2.3 timeframe with regards to dealing with large GeoJSON files, although even in 3.2, it's still very slow and memory hungry. Our system allows for aborting imports, but this only works reliably once it has actually got to the stage of