[Python-Dev] os.add_dll_directory and DLL search order
Hi list, I've been trying to get new Python 3.8 builds for the MapServer project - these are SWIG-generated bindings that require many DLLs to import in Python. Up until now this worked by adding a folder to the top of the Windows PATH. I've read through the discussions at https://bugs.python.org/issue36085 and understand the reasoning behind the change. Is the recommended solution in cases of Python wrappers to create a project-specific environment variable and allow users to set this, which in turn the library adds with add_dll_directory? However, there is one feature of using the Windows PATH that I can't seem to replicate with add_dll_directory. The MapServer DLL builds are linked to sqlite3 3.24.0, whereas Python 3.8.2 is linked to 2.6.0. Building with matching versions is not something I can easily change. When using Windows PATH the folder with the newer version could be added to the front of the list to take priority and import worked correctly. This does not seem to be possible with add_dll_directory - the Python sqlite3.dll in C:\Python38\DLLs always takes priority leading to: ImportError: DLL load failed while importing _mapscript: The specified procedure could not be found. I presume I can't remove the C:\Python38\DLLs path, is there another solution to this issue? Thanks in advance for any help, Seth -- web:http://geographika.co.uk twitter: @geographika ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I6JAXBKV4Y7MVF42RJ4XQIPYQMAVXHOF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: os.add_dll_directory and DLL search order
Thanks Steve for the clarifications. As a workaround placing the older sqlite3.dll in the same folder as the _mapscript.pyd file while leaving all the other DLLs in a different folder referenced by add_dll_directory allows for a successful import. For a less hacky fix - after checking the version numbers for sqlite3 again they are not too different so it may be easiest to update the upstream builds to a matching version. Apologies for the noise - I thought this was related to the new add_dll_directory function, but I can break old versions using PATH and mismatched sqlite3 .dlls too! Seth -- web:http://geographika.co.uk twitter: @geographika On Mon, Jun 22, 2020, at 5:46 PM, Steve Dower wrote: > On 22Jun2020 1039, Seth G wrote: > > However, there is one feature of using the Windows PATH that I can't seem > > to replicate with add_dll_directory. > > The MapServer DLL builds are linked to sqlite3 3.24.0, whereas Python 3.8.2 > > is linked to 2.6.0. Building with matching versions is not something I can > > easily change. > > > > When using Windows PATH the folder with the newer version could be added to > > the front of the list to take priority and import worked correctly. This > > does not seem to be possible with add_dll_directory - the Python > > sqlite3.dll in C:\Python38\DLLs always takes priority leading to: > > > > ImportError: DLL load failed while importing _mapscript: The specified > > procedure could not be found. > > > > I presume I can't remove the C:\Python38\DLLs path, is there another > > solution to this issue? > > DLLs should not be in the search path at all - it's searched by sys.path > when importing .pyd files, which are loaded by absolute path and their > dependencies found adjacent. > > What is likely happening here is that _sqlite3.pyd is being imported > before _mapscript, and so there is already a SQLITE3 module in memory. > Like Python, Windows will not attempt to import a second module with the > same name, but will return the original one. > > So your best option here is probably to rebuild sqlite3.dll with as much > of the version number (or some unique string) in the name as you need. > Or you can statically link it into your extension module, assuming you > aren't relying on shared state with other modules in your package. The > latter is probably easier. > > Cheers, > Steve > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SO37EWIVDOKEIQO6MPSR2EVL5EDPLV4J/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: os.add_dll_directory and DLL search order
Thanks Ned. I did double check the docs for sqlite3 after posting and wondering why the versions were so different. I guess the clue should have been the sqlite-3 ! Reading the history of the module I presume sqlite3 has its own module version number as it was integrated from a separate project. Seth -- web:http://geographika.co.uk twitter: @geographika On Mon, Jun 22, 2020, at 6:41 PM, Ned Deily wrote: > On Jun 22, 2020, at 05:39, Seth G wrote: > > However, there is one feature of using the Windows PATH that I can't seem > > to replicate with add_dll_directory. > > The MapServer DLL builds are linked to sqlite3 3.24.0, whereas Python 3.8.2 > > is linked to 2.6.0. Building with matching versions is not something I can > > easily change. > > For what it's worth, you're looking at the wrong value (it's easy to > do!). '2.6.0' is the version number of the sqlite3 module itself, not > of the sqlite3 library. > > > >>> sqlite3.version > '2.6.0' > >>> sqlite3.sqlite_version > '3.31.1' > > -- > Ned Deily > n...@python.org -- [] > > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GMW6VL6MY5CUNAMPGQHSI6JB3PL2KLK7/ Code of Conduct: http://python.org/psf/codeofconduct/
