Hi, I have been told that Docker has overhead with everything, with these numbers as evidence:
ubuntu@t:/etc$ time gdalinfo --version GDAL 2.2.3, released 2017/11/20 real 0m0.030s user 0m0.017s sys 0m0.013s ubuntu@t:/etc$ time docker exec -it 742a044b8ef1 gdalinfo --version GDAL 3.1.0dev-19f34295c8c310b6ecb27d2d7aa7ac0a2a2a3d9f, released 2019/11/05 real 0m0.386s user 0m0.023s sys 0m0.030s -Jukka Rahkonen- alex-6 wrote > Hi guys, > > I've been playing quite a lot with "wrapper_GDALWarpDestName" and a .net > core wrapper (https://github.com/MaxRev-Dev/gdal.netcore/). > Everything is running fine, I'm quite happy to see the same behavior on > windows and linux. Sometimes linux is faster for reading and writing > raster, this is great ! .NET Core is (now) a very good option for cross > platform dev. Anyway... > > When calling wrapper_GDALWarpDestName, I'm experiencing an overhead of > 350 to 500 ms on linux that I don't see at all on Windows. > To narrow down the issue, I tried many warp options (-multi; -wo > NUM_THREADS=4; -wm 2000), output drivers (MEM, VRT, NWT_GRD), input file > format (MEM, VRT, NWT_GRD), input and output projections. I also checked > Gdal.GetCacheUsed(), Gdal.HasThreadSupport() and Gdal.GetCacheMax(), > same values between linux and windows. Nothing helps. > > Even with VRT format output, this overhead is present which proves this > is not related to processing. I bet for some sort of initialization of > the warp algorithm with file reading... because 400 ms is far from > nothing. But I'm wondering why this is occurring for every single call > on linux and not on windows... Any option I could try ? My next step is > to profile the linux process but I'm not good at that so any help there > would be great too. > > > OS (version): Windows 10 (1903) > Docker: > Server: Docker Engine - Community > Engine: > Version: 19.03.2 > API version: 1.40 (minimum version 1.12) > Go version: go1.12.8 > Git commit: 6a30dfc > Built: Thu Aug 29 05:32:21 2019 > OS/Arch: linux/amd64 > Experimental: false > > Linux in docker file: debian 9 > (mcr.microsoft.com/dotnet/core/runtime:2.2-stretch-slim) > Package version (core): [e.g. 3.0.1.25] > Package version (runtime windows): [e.g. 3.0.1.25] > Package version (runtime linux): [e.g. 3.0.1.2] > > This is the kind of code I'm running: > > namespace GdalCoreTest > { > class Program > { > static void Main(string[] args) > { > GdalBase.ConfigureAll(); > Console.WriteLine("GDAL configured"); > Console.WriteLine(string.Join('\n', > "GDAL Version: " + Gdal.VersionInfo("RELEASE_NAME"), > "GDAL INFO: " + Gdal.VersionInfo(""))); > WarpFile(); > WarpFile(); > WarpFile(); > } > > public static void WarpFile() > { > Stopwatch sw = new Stopwatch(); > > int sizeX = 1024, sizeY = 1024; > var driver = Gdal.GetDriverByName("MEM"); > string path = $"/vsimem/test.mem"; > string vrtFilePath = null; > using(Dataset ds = driver.Create(path, sizeX, sizeY, 1, > DataType.GDT_UInt32, new string[0])) > { > try > { > //************** > //create a grid > ds.SetGeoTransform(new double[]{0, 30, 0, 0, 0, -30}); > > var ttt = new SpatialReference(null); > ttt.ImportFromProj4("+proj=longlat +datum=WGS84"); > ttt.ExportToWkt(out string latLongWkt, new string[0]); > ds.SetProjection(latLongWkt); > > //write data in the buffer > int[] buffer = new int[sizeX * sizeY]; > for (int x = 0; x < sizeX; x++) > { > for (int y = 0; y < sizeY; y++) > { > buffer[y * sizeX + x] = x + y; > } > } > > ds.GetRasterBand(1).WriteRaster(0, 0, sizeX, sizeY, buffer, > sizeX, > sizeY, 0, 0); > > > //************** > //Prepare for warping > IntPtr[] ptr = {Dataset.getCPtr(ds).Handle}; > GCHandle gcHandle = GCHandle.Alloc(ptr, GCHandleType.Pinned); > var dss = new > SWIGTYPE_p_p_GDALDatasetShadow(gcHandle.AddrOfPinnedObject(), false, > null); > vrtFilePath = $"/vsimem/tile.vrt"; > > //Warp > sw.Start(); > using (Dataset dst = Gdal.wrapper_GDALWarpDestName(vrtFilePath, > 1, > dss, new GDALWarpAppOptions(new[]{"-of", "VRT" }), null,null)) > { > sw.Stop(); > } > } > finally > { > Gdal.Unlink(path); > Gdal.Unlink(vrtFilePath); > } > } > Console.Out.WriteLine($"Warp duration = {sw.ElapsedTicks} ticks > ({sw.ElapsedMilliseconds} ms)."); > } > } > } > > > This is the output I'm getting on windows: > > GDAL configured > GDAL Version: 3.0.1 > GDAL INFO: GDAL 3.0.1, released 2019/06/28 > Warp duration = 221267 ticks (22 ms). > Warp duration = 133395 ticks (13 ms). > Warp duration = 155402 ticks (15 ms). > > This is the output I'm getting on docker/linux: > > GDAL configured > GDAL Version: 3.0.1 > GDAL INFO: GDAL 3.0.1, released 2019/06/28 > Warp duration = 395911725 ticks (395 ms). > Warp duration = 368923383 ticks (368 ms). > Warp duration = 373055090 ticks (373 ms). > > Thanks, > Alex. > _______________________________________________ > gdal-dev mailing list > gdal-dev@.osgeo > https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
