Hi all,
Upgrading to Fiona 1.8.0 won't help the code below because every
outfile.write() call entails a transaction of size 1. The user needs to
pass a sequence or iterator of many records to outfile.writerecords() if
they want to benefit from larger transactions.
On Thu, Nov 1, 2018 at 3:46 AM E
I'm writing file using class ShapeConverter(osmium.SimpleHandler) and ogr
(not fiona)
But, do not know whether transaction are all written instantaneously and
simultaneously or not...
On Thu, Nov 1, 2018 at 8:34 PM koji higuchi wrote:
> Hi Andreas
> I'm writing to a file into my PC disk.
>
> O
Hi Andreas
I'm writing to a file into my PC disk.
On Thu, Nov 1, 2018 at 8:32 PM Andreas Neumann wrote:
> Hi,
>
> Just a quick stupid question: are you writing to a local file system or to
> a network filesystem. The latter can be substantially slower than the
> former, esp. with Geopackage.
>
koji higuchi wrote
> Hi Even
> I managed to write gpkg without ogr without fiona using class
> ShapeConverter(osmium.SimpleHandler).
> still, speed seems slow.
> Any idea to increase it up.
Hi,
Have you realised the importance of making big transactions? You can read
background information from h
Hi,
Just a quick stupid question: are you writing to a local file system or
to a network filesystem. The latter can be substantially slower than the
former, esp. with Geopackage.
Andreas
Am 01.11.18 um 12:10 schrieb koji higuchi:
Hi Even
I managed to write gpkg without ogr without fiona usi
Hi Even
I managed to write gpkg without ogr without fiona using class
ShapeConverter(osmium.SimpleHandler).
still, speed seems slow.
Any idea to increase it up.
On Thu, Nov 1, 2018 at 7:05 PM koji higuchi wrote:
> Hi Even
>
> I am trying to replace using fiona by pure ogr.
> but having difficult
Hi,
Unfortunately I have never used osmium-extract myself but I was remembering
that I had been reading something about doing spatial selection from OSM
pbf. I did even find the thread
https://lists.openstreetmap.org/pipermail/dev/2018-October/030400.html
-Jukka-
koji higuchi wrote
> Hi Jukka
Hi Jukka
Is osmium-extract available in pyosmium?
i'm using windows; and seems difficult to use osmium in windows
On Thu, Nov 1, 2018 at 7:13 PM jratike80 <
jukka.rahko...@maanmittauslaitos.fi> wrote:
> Hi,
>
> That is probably another problem, OSM pbf format does not support effective
> selecti
Hi,
That is probably another problem, OSM pbf format does not support effective
selection by region so selecting a subarea from europe-latest.osm.pbf may be
almost as slow as processing the whole dataset. I would try if
osmium-extract https://docs.osmcode.org/osmium/latest/osmium-extract.html
coul
Hi Even
I am trying to replace using fiona by pure ogr.
but having difficult to collapse the code with class and objects.
If i could extract geom and tags in a ogr pythonic way, then it be better
On Thu, Nov 1, 2018 at 6:45 PM Even Rouault
wrote:
> You are perhaps using a too old version of Fi
Hi, my source file is too big, around 19gb (europe-latest.osm.pbf)
I also tried ogr2ogr for clipping into region as follows.
ogr2ogr -f, gpkg, -clipdst, str(ulx), str(lry), str(lrx), str(uly), fo, fi
but it was also very slow.
On Thu, Nov 1, 2018 at 6:55 PM jratike80 <
jukka.rahko...@maanmittau
koji higuchi wrote
> Hi, following is the code I used:
>
> Import os, osmium, fiona
>
> fi = 'europe-latest.osm.pbf'
> fo = 'europe-latest.dpkg'
>
> drv = 'DPKG'
Hi,
I am not able to evaluate if your code is effective or not. However, I can
give you some numbers for comparison.
Test data http
You are perhaps using a too old version of Fiona. Recent versions insert
features within OGR transactions. Should be fine with latest Fiona 1.8.0
See https://github.com/Toblerity/Fiona/issues/476
> Hi, following is the code I used:
>
> Import os, osmium, fiona
>
> fi = 'europe-latest.osm.pbf'
>
Hi, following is the code I used:
Import os, osmium, fiona
fi = 'europe-latest.osm.pbf'
fo = 'europe-latest.dpkg'
drv = 'DPKG'
crs = {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj':
'longlat'}
schema = {'geometry': 'LineString',
'properties': {'id': 'float', 'n
koji higuchi wrote
> Hi
> I am extracting .osm.pbf file into .gpkg; but the writing rate is very
> slow.
> When I write into .shp, rate is faster but its limit is 4gb.
> So, what other format is better for larger than 4gb requirement?
> Thanks for your ideas.
> Koji
Hi,
Show the exact command tha
Hi
I am extracting .osm.pbf file into .gpkg; but the writing rate is very slow.
When I write into .shp, rate is faster but its limit is 4gb.
So, what other format is better for larger than 4gb requirement?
Thanks for your ideas.
Koji
___
gdal-dev mailing
16 matches
Mail list logo
