[Tutor] PMML vs. Pickel
All, have a Python prepayment model that must be integrated into 3rd party vendor software considering translating to PMML. I have been researching this. Any potential downside? Glenn Sent from my iPhone ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] PMML vs. Pickel
The tutor list is for those learning python and its standard library. I suspect you will get a better response asking on the main python list. Alan g. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python C extension - which method?
On Mon, May 7, 2018 at 9:57 AM, Michael C
wrote:
> Sorry I don't understand your suggestion.
>
> Use "ctypes.CDLL" and "ctypes.WinDLL"
>
> this works,
> mydll = cdll.LoadLibrary('test.dll')
> print(mydll.sum(3,2))
>
> and this doens't
> mydll = cdll('test.dll')
> print(mydll.sum(3,2))
>
> What's the syntax of what you suggested?
Loading the library is simply `mydll = ctypes.CDLL('test')`. Note that
CDLL is uppercase. Python is a case-sensitive language.
The base shared library class is CDLL. The constructor takes the
following parameters: name, mode=DEFAULT_MODE, handle=None,
use_errno=False, and use_last_error=False. For the first 3 parameters,
if the shared library `handle` isn't provided, it calls
ctypes._dlopen(mode, name) to load the library.
The CDLL constructor dynamically defines a function-pointer subclass,
with custom _flags_ and _restype_. The _restype_ is c_int (i.e. a
32-bit C integer). The base _flags_ value is _FUNCFLAG_CDECL because
CDLL uses the cdecl calling convention. For the last two constructor
parameters, use_errno includes _FUNCFLAG_USE_ERRNO, and use_last_error
includes _FUNCFLAG_USE_LASTERROR.
The above function pointer class is used by the __getitem__ method of
a CDLL instance, which calls self._FuncPtr((name_or_ordinal, self)) to
return a function pointer instance, e.g. mydll['sum']. (Using a
subscript also allows ordinal look-up, such as mydll[42].) The
function pointer is not cached, so it has to be instantiated ever time
it's accessed this way. On the other hand, the __getattr__ method of
CDLL calls __getitem__ and caches the function pointer via
setattr(self, name, func). For example, with mydll.sum, the `sum`
function pointer is cached.
WinDLL subclasses CDLL to change the default function pointer _flags_
value to _FUNCFLAG_STDCALL because the Windows API uses the stdcall
calling convention, as do many shared libraries on Windows.
ctypes also defines a LibraryLoader class, of which ctypes.cdll and
ctypes.windll are instances, i.e. cdll = LibraryLoader(CDLL). The
constructor simply sets the shared library class as the attribute
self._dlltype. In turn, the LoadLibrary method returns
self._dlltype(name). This is silly because it's extra typing, an extra
call, and prevents passing constructor arguments (e.g.
use_last_error).
Where this goes from silly to smelly is the __getattr__ method of
LibraryLoader. This method makes the mistake of caching loaded
libraries. For example, ctypes.windll.kernel32 caches a
WinDLL('kernel32') instance on ctypes.windll. Remember that
__getattr__ is a fallback method called by __getattribute__. Thus
subsequent references to ctypes.windll.kernel32 will use the cached
library instance. This is bad because multiple packages will end up
sharing the same library instance.
Recall that the __getattr__ method of CDLL caches function pointers.
That's a good thing because function pointer look-up and instantiation
is relatively expensive. I can understand the good intentions of
carrying this over to caching entire library instances across multiple
packages. But the savings is not worth the ensuing chaos of
super-global cached state, especially for commonly used Windows
libraries such as kernel32.dll. The problem is that every function
pointer defines a prototype that consists of its errcheck, restype,
and argtypes properties, but not every package or script will define
these exactly the same, for various reasons. Thus the last one to
define the prototype wins. All other users of the library in the
process will be broken.
In short, avoid ctypes.[cdll | windll].LoadLibrary because it's silly,
and avoid ctypes.[cdll | windll].[library name] because it's smelly.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python C extension - which method?
Sorry I don't understand your suggestion.
Use "ctypes.CDLL" and "ctypes.WinDLL"
this works,
mydll = cdll.LoadLibrary('test.dll')
print(mydll.sum(3,2))
and this doens't
mydll = cdll('test.dll')
print(mydll.sum(3,2))
What's the syntax of what you suggested?
Thanks
On Mon, May 7, 2018 at 3:15 AM, eryk sun wrote:
> On Sun, May 6, 2018 at 12:49 AM, Brad M wrote:
> > If I may ask, what's the difference between these two?
> >
> > 1)
> > import ctypes
> > hello = ctypes.WinDLL('hello', use_last_error=True)
> >
> > 2)
> > from ctypes import cdll
> > hello = cdll.LoadLibrary('hello.dll')
>
> Use ctypes.CDLL and ctypes.WinDLL instead of cdll.LoadLibrary and
> windll.LoadLibrary. The latter is more typing for no benefit and
> prevents using the constructor arguments: handle, mode (POSIX),
> use_errno, and use_last_error (Windows). You need the latter two
> options if the library requires C errno or Windows GetLastError(), in
> which case you should use ctypes.get_errno() or
> ctypes.get_last_error() to get the error values after a C function
> call.
>
> > Both of them can return "1980" from this:
> >
> > hello.c
> >
> > #include
> >
> > __declspec(dllexport) int say_something()
> > {
> > return 1980;
> > }
>
> CDLL is the cdecl calling convention, and WinDLL is stdcall. There is
> no difference in 64-bit Windows (x64 ABI). In 32-bit Windows (x86
> ABI), cdecl has the caller clean the stack (i.e. pop arguments), and
> stdcall has the callee (the called function) clean the stack. cdecl
> allows functions with a variable number of arguments, such as the CRT
> printf function. In MSVC, cdecl is the default convention if you don't
> declare a function as __stdcall. A library can export functions with
> varying calling conventions, so in general you may need to mix CDLL
> and WinDLL.
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Extract main text from HTML document
That looks like a useful combination. Thanks. On 6 May 2018 at 17:32, Mark Lawrence wrote: > On 05/05/18 18:59, Simon Connah wrote: >> >> Hi, >> >> I'm writing a very simple web scraper. It'll download a page from a >> website and then store the result in a database of some sort. The >> problem is that this will obviously include a whole heap of HTML, >> JavaScript and maybe even some CSS. None of which is useful to me. >> >> I was wondering if there was a way in which I could download a web >> page and then just extract the main body of text without all of the >> HTML. >> >> The title is obviously easy but the main body of text could contain >> all sorts of HTML and I'm interested to know how I might go about >> removing the bits that are not needed but still keep the meaning of >> the document intact. >> >> Does anyone have any suggestions on this front at all? >> >> Thanks for any help. >> >> Simon. > > > A combination of requests http://docs.python-requests.org/en/master/ and > beautiful soup https://www.crummy.com/software/BeautifulSoup/bs4/doc/ should > fit the bill. Both are installable with pip and are regarded as best of > breed. > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] use python to log to a remote supercomputer and transfer files
Hello everyone, I need to conduct massive simulation computations using a software called 'lammps' on a remote supercomputer whose operating system is Linux every day. It's extremely annoying to log to the remote supercomputer, upload files to and download files from the supercomputer using WinSCP and Putty on my windows desktop computer. I want to know whether it is possible to write some scripts and let python do these things for me, i.e., I want to log to the remote supercomputer automatically, upload and download files with a simple hit in python (of course with the files specified). Is this possible? If it is possible, which packages do I need? Thanks and best wishes, Rui ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] use python to log to a remote supercomputer and transfer files
On 7 May 2018, at 15:08, ruiyan wrote: > I want to log to the remote supercomputer automatically, upload and download > files If it supports ftp then there is an ftp module. There are also http and ssh modules, depending on the complexity of your needs and the protocols available. Ftp sounds simplest in this case. Alan g. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] use python to log to a remote supercomputer and transfer files
On 05/07/2018 05:53 AM, ruiyan wrote: > Hello everyone, > > > I need to conduct massive simulation computations using a software called > 'lammps' on a remote supercomputer whose operating system is Linux every day. > It's extremely annoying to log to the remote supercomputer, upload files to > and download files from the supercomputer using WinSCP and Putty on my > windows desktop computer. I want to know whether it is possible to write some > scripts and let python do these things for me, i.e., I want to log to the > remote supercomputer automatically, upload and download files with a simple > hit in python (of course with the files specified). Is this possible? If it > is possible, which packages do I need? There are lots of methods for this. If you're already using Putty, you probably want to continue using ssh protocol, so you can look into a Python package called Paramiko. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help with a virtual environment mess
On 05/06/2018 05:16 PM, boB Stepp wrote: On Sun, May 6, 2018 at 11:05 AM, Jim wrote: In a prior thread you guys helped me fix a problem with pip after I upgraded an installed version of python 3.6 on my Mint 18 system. Pip would not run in my python 3.6 virtual environment. The fix was to use synaptic to install python3-distutils. I thought everything was ok until I tried to run a old script from a different VE using python 3.5 which could not import tkinter. I have 4 versions of python on this system: system Python2 = 2.7.12 (default) system Python3 = 3.5.2 (default) a VE called env = 3.5.2 a Ve called env36 = 3.6.5 This is the error I get trying to import tkinter in env, I also get the same error if I try to import it in system python3. jfb@jims-mint18 ~ $ source /home/jfb/EVs/env/bin/activate (env) jfb@jims-mint18 ~ $ python Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. import tkinter as tk Traceback (most recent call last): File "/usr/lib/python3.5/tkinter/__init__.py", line 36, in import _tkinter ImportError: No module named '_tkinter' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.5/tkinter/__init__.py", line 38, in raise ImportError(str(msg) + ', please install the python3-tk package') ImportError: No module named '_tkinter', please install the python3-tk package If I go to synaptic and install the python3-tk it installs version 3.6.5 of the package and I can still not import tkinter in env with python 3.5.2, but I can in env36 with python 3.6.5. As I have not yet tried to play around with virtual environments, I may be about to do more harm than help, but I will plow ahead anyway! ~(:>)) My primitive understanding of installing Python versions that are different from the system Python version into a virtual environment, is that you have to install all dependencies you need from within that virtual environment you created. If I am correct about this then your error messages suggest you need to install the tkinter stuff from within that virtual environment using that virtual environment's pip. Hopefully I am too far off from the truth here, but in any event, I hope this helps you in your problem! My understanding of VE's, based on some feedback from here, is you install install the python you want on the system then use it to install your VE. Then you install what ever you need to the VE. In my case I had a working VE based on python 3.5 then I received an upgrade to the python version 3.6 I had installed. After that I had problems with the 3.5 VE that had been working. Regards, Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] use python to log to a remote supercomputer and transfer files
On 05/07/2018 04:15 PM, Alan Gauld via Tutor wrote: On 7 May 2018, at 15:08, ruiyan wrote: I want to log to the remote supercomputer automatically, upload and download files If it supports ftp then there is an ftp module. There are also http and ssh modules, depending on the complexity of your needs and the protocols available. Ftp sounds simplest in this case. ... but the question is what kind of simplification / comfort increase you are hoping to get from a custom python script of your own compared to what WinSCP or Putty already offer. How are they making things "extremely annoying" for you? You can certainly learn a lot from writing or even just trying to write such a Python script, but in terms of functionality gain vs time invested I doubt the endeavor is worth it. Wolfgang ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help with a virtual environment mess
On 05/07/2018 10:16 AM, Jim wrote:
> My understanding of VE's, based on some feedback from here, is you
> install install the python you want on the system then use it to install
> your VE. Then you install what ever you need to the VE. In my case I had
> a working VE based on python 3.5 then I received an upgrade to the
> python version 3.6 I had installed. After that I had problems with the
> 3.5 VE that had been working.
yes, this happens.
the default behavior for virtualenv is to make links when creating the
VE, which saves copying things but is vulnerable to breakage if the
Python it's linking to receives major changes. In the case of a
distribution upgrading the Python version, a VE constructed against the
old version will break if the links are version-specific. Looking at
one virtualenv I have, {VEPATH}/lib/python3.6 is full of such symlinks,
e.g.:
lrwxrwxrwx. 1 mats mats26 Aug 17 2017 re.py ->
/usr/lib64/python3.6/re.py
Virtualenvs are cheap to recreate, so one approach is to just throw away
the old one and make a new one.
you can also give virtualenv an option (--always-copy) to cause the
created virtualenv to be more self-contained, at a cost of some space
and tiem.
There are plenty of tools for managing python versions and virtualenv.
The python community changed directions a little bit recently, 'venv' is
now the recommended approach:
https://docs.python.org/3/library/venv.html
pyenv can manage different Python versions if you're interested in that.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Need help with a virtual environment mess
On 05/07/2018 12:02 PM, Mats Wichmann wrote:
On 05/07/2018 10:16 AM, Jim wrote:
My understanding of VE's, based on some feedback from here, is you
install install the python you want on the system then use it to install
your VE. Then you install what ever you need to the VE. In my case I had
a working VE based on python 3.5 then I received an upgrade to the
python version 3.6 I had installed. After that I had problems with the
3.5 VE that had been working.
yes, this happens.
the default behavior for virtualenv is to make links when creating the
VE, which saves copying things but is vulnerable to breakage if the
Python it's linking to receives major changes. In the case of a
distribution upgrading the Python version, a VE constructed against the
old version will break if the links are version-specific. Looking at
one virtualenv I have, {VEPATH}/lib/python3.6 is full of such symlinks,
e.g.:
lrwxrwxrwx. 1 mats mats26 Aug 17 2017 re.py ->
/usr/lib64/python3.6/re.py
Virtualenvs are cheap to recreate, so one approach is to just throw away
the old one and make a new one.
My problem is right now the default python3 for my system is also
affected. If I type python3 I will see python version 3.5.2 and I cannot
import tkinter there either.
you can also give virtualenv an option (--always-copy) to cause the
created virtualenv to be more self-contained, at a cost of some space
and tiem.
There are plenty of tools for managing python versions and virtualenv.
The python community changed directions a little bit recently, 'venv' is
now the recommended approach:
https://docs.python.org/3/library/venv.html
That is what I used to setup my VE's.
pyenv can manage different Python versions if you're interested in that.
I am going to see if I can find some log file that would give me a clue
about what happened during the update.
Regards, Jim
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
