Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-15 Thread James Ramm
I agree...could be slightly inelegant to figure out what version of python is used though...A system call to `where python` on windows would return the interpreter on the path, and you could replace the '.exe' with '.dll' (this is assuming python dll is always alongside the exe, but i think this is

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-15 Thread Even Rouault
Sean, > I suspect users will be find it very surprising if `which python` and the > python interpreter used by VRT are not the same Yes, we should probably make some effort to use the same python version as the one that comes as default given the PATH. I've just opened https://trac.osgeo.org/gd

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-15 Thread Sean Gillies
Hi Even, James: I suspect users will be find it very surprising if `which python` and the python interpreter used by VRT are not the same. Imagine starting python in an environment created by virtualenv or conda that contains extension modules like scikit-image, scikit-learn, &c. You import osgeo.

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-15 Thread James Ramm
excellent, working setting the PYTHONSO variable. Trying to match the version of python on the path would be smart, but it is probably 'good enough' to give PYTHONSO some prominence in the documentation - some kind of warning on how python is discovered perhaps? On 14 September 2016 at 20:20, Jame

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread James Ramm
Ah, that makes sense. I'll have to try the config option in the morning, but it sounds like that could be it On 14 Sep 2016 8:06 p.m., "Even Rouault" wrote: > Le mercredi 14 septembre 2016 20:50:04, James Ramm a écrit : > > Yes, it is loading a different pythonThe path printed by sys.path is

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread Even Rouault
Le mercredi 14 septembre 2016 20:50:04, James Ramm a écrit : > Yes, it is loading a different pythonThe path printed by sys.path is > different to if I open the command window and type: > > python > > >>> import sys > >>> print(sys.path) > > Gdal and the python bindings were compiled with vs

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread Even Rouault
Le mercredi 14 septembre 2016 17:24:53, jramm a écrit : > Trying to run this using a function relying on scipy.ndimage... > > When running gdal_translate on the VRT, I get ImportError: No module named > scipy.ndimage > This comes after successfully import numpy. scipy.ndimage will happily > import

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread jramm
ORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","27700"]] 100180.0,5.0,0.0,1215730.0,0.0,-5.0 F:\tif_data\large_sparse.tif 4 TRUE

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread jramm
ORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","27700"]] 100180.0,5.0,0.0,1215730.0,0.0,-5.0 F:\tif_data\large_sparse.tif 4 TRUE

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread Even Rouault
Le mercredi 14 septembre 2016 09:02:07, Rutger a écrit : > Then I guess i'm mistaken in thinking that Python would become a slowdown > in such a case. It's been a while since i tested it. Anyway, thanks for > your comments and efforts to improve the performance. > > I couldnt find any Conda channe

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-14 Thread Rutger
__ > gdal-dev mailing list > gdal-dev@.osgeo > http://lists.osgeo.org/mailman/listinfo/gdal-dev -- View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-VRT-derived-band-pixel-functions-written-in-Python-tp5285323p528573

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread Even Rouault
Le mardi 13 septembre 2016 21:22:20, James Ramm a écrit : > I think you can call SWIG with the -threads argument on the command line so > it will always release the GIL. Could be an easy option if it works That's mostly what I've done. See my other message : https://lists.osgeo.org/pipermail/gdal

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread James Ramm
I think you can call SWIG with the -threads argument on the command line so it will always release the GIL. Could be an easy option if it works On Tuesday, 13 September 2016, Even Rouault wrote: > Le mardi 13 septembre 2016 11:07:39, Rutger a écrit : > > I overlooked the fact that it still moves

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread Even Rouault
Le mardi 13 septembre 2016 11:07:39, Rutger a écrit : > I overlooked the fact that it still moves through Python, is that the > 'only' hurdle preventing parallel IO? Not sure to understand your question. But if you have several sources, you could potentially do parallelized reading of them from t

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread Rutger
you could possibly > parallelize them there. > > > Even > > -- > Spatialys - Geospatial professional services > http://www.spatialys.com > ___ > gdal-dev mailing list > gdal-dev@.osgeo > http://lists.osgeo.org/mai

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread Even Rouault
Le mardi 13 septembre 2016 09:02:09, Rutger a écrit : > Hi Even, > > This is really amazing. I'm becoming more and more a fan of Numba for > number crunching, so this certainly makes my day. As soon as i can find a > Win x64 dev version on a Conda channel I'll give it a try. > > A use case that c

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread Even Rouault
Le lundi 12 septembre 2016 23:38:41, James Ramm a écrit : > Incredible > Really cool initiative. > > I had a play around and wanted a way to be able to write the pixel function > in a 'stand alone' python module and generate the VRT from it. This would > allow independent testing and easier ma

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-13 Thread Rutger
s message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-VRT-derived-band-pixel-functions-written-in-Python-tp5285323p5285450.html Sent from the GDAL - Dev mailing list archive at Nabble.com. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-12 Thread James Ramm
Incredible Really cool initiative. I had a play around and wanted a way to be able to write the pixel function in a 'stand alone' python module and generate the VRT from it. This would allow independent testing and easier maintenance of the python code. It is fairly easy using lxml to build up

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-12 Thread Even Rouault
> I found http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html > to be a good intro to the risks of eval'ing untrusted Python code. > Mentioned in there is a notable attempt to make a secure subset of Python > called "pysandbox", but its developer has since declared it "broken by > de

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-12 Thread Even Rouault
Le lundi 12 septembre 2016 16:59:34, Poughon Victor a écrit : > Hi Even, > > This is a really cool and really impressive feature! > Calling Python code from C++ without development packages as a dependency > sounds like black magic to me. No black magic. Just setting function pointers using dlope

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-12 Thread Poughon Victor
.osgeo.org] de la part de Even Rouault [even.roua...@spatialys.com] Envoyé : lundi 12 septembre 2016 14:31 À : gdal-dev@lists.osgeo.org Objet : [gdal-dev] VRT derived band pixel functions written in Python Hi, I wanted to mention a new (and I think pretty cool ) feature I've added in trunk: the po

Re: [gdal-dev] VRT derived band pixel functions written in Python

2016-09-12 Thread Sean Gillies
Hi Even, On Mon, Sep 12, 2016 at 2:31 PM, Even Rouault wrote: > Hi, > > I wanted to mention a new (and I think pretty cool ) feature I've added in > trunk: the possibility to define pixel functions in Python in a VRT derived > band. > > ... > > There are obvious security concerns in allowing Pyt

[gdal-dev] VRT derived band pixel functions written in Python

2016-09-12 Thread Even Rouault
Hi, I wanted to mention a new (and I think pretty cool ) feature I've added in trunk: the possibility to define pixel functions in Python in a VRT derived band. Let's start with a simple example to multiply a raster by 1.5 : EPSG:26711 440720,60,0,3751320,0,-60 multiply Python