Re: [Python-Dev] Making the GIL faster & lighter on Windows
I've often thought of this. The problem is that the GIL uses the regular python "lock" which has to be non-recursive, since it is used for synchronization operations other than mutual exclusion, e.g. one thread going to sleep, and another waking it up. Now, we could easily create another class of locks, a python "mutex" or a "critical section" even, which is allowed (but not required) to be recursive. On other platforms, this could fall back to being the good old lock. Requiring it to be recursive would mean that we would need implementations for all platforms. Which is possible, I suppose, building on the old python lock... For the GIL, we would then use a python "mutex" or "critical section" whichever you prefer. Note that for the GIL, if you use a CriticalSection object, you should initialize its "spincount" to zero, because the GIL is almost always in contention. That is, if you don't get the GIL right away, you won't for a while. I don't know what kernel primitive the Critical Section uses, but if it uses an Event object or something similar, we are in the same soup, so to say, because the CriticalSection's spinlocking feature buys us nothing. K -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Phillip Sitbon Sent: 26. maí 2009 19:49 To: python-dev@python.org Subject: [Python-Dev] Making the GIL faster & lighter on Windows Hi everyone, I'm new to the list but I've been embedding Python and working very closely with the core sources for many years now. I discovered Python a long time ago when I needed to embed a scripting language and found the PHP sources... unreadable ;) Anyway, I'd like to ask something that may have been asked already, so I apologize if this has been covered. Instead of removing the GIL, has anyone thought of making it more lightweight? The current situation for Windows is that the single-thread case is decently fast (via interlocked operations), but it drops to using an event object in the case of contention. (see thread_nt.h) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384: Defining a Stable ABI
[PEP] > Function-like macros (in particular, field access macros) remain > available to applications, but get replaced by function calls > (unless their definition only refers to features of the ABI, such > as the various _Check macros) [MAL] Including Py_INCREF()/Py_DECREF() ? [Nick] >>> I believe so - MvL deliberately left the fields that the ref counting >>> relies on as part of the ABI. [MAL] >> Hmm, another slow-down. [MvL] > ??? Why is "no change" a slow-down? That was just a miscommunication - I misunderstood the sense in which MAL was using "Including". He was referring to the first part of the paragraph from the PEP (most macros become functions), but I answered assuming he was referring to the part in parentheses (some macros get to stay). So to be perfectly clear: the Py_INCREF/Py_DECREF macros are available as part of the stable ABI because they qualify for the PEP's "definition only refers to features of the ABI" exception. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making the GIL faster & lighter on Windows
Martin v. Löwis wrote: > No: fairness in mutex synchronization means that every waiter for the > mutex will eventually acquire it; it won't happen that one thread > starves waiting for the mutex. This is something that the mutex needs to > provide, not the application. CriticalSections are first come first served on Windows, just like a regular mutex. As Phillip already noted, their main limitation is that they don't work cross-process (of course, that's also where they get their extra speed). Since we don't need the cross-process feature and we don't support Win 9x any more, this is certainly an idea worth looking at. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384: Defining a Stable ABI
Nick Coghlan wrote: > [PEP] >> Function-like macros (in particular, field access macros) remain >> available to applications, but get replaced by function calls >> (unless their definition only refers to features of the ABI, such >> as the various _Check macros) > [MAL] > Including Py_INCREF()/Py_DECREF() ? > [Nick] I believe so - MvL deliberately left the fields that the ref counting relies on as part of the ABI. > [MAL] >>> Hmm, another slow-down. > [MvL] >> ??? Why is "no change" a slow-down? > > That was just a miscommunication - I misunderstood the sense in which > MAL was using "Including". He was referring to the first part of the > paragraph from the PEP (most macros become functions), but I answered > assuming he was referring to the part in parentheses (some macros get to > stay). > > So to be perfectly clear: the Py_INCREF/Py_DECREF macros are available > as part of the stable ABI because they qualify for the PEP's "definition > only refers to features of the ABI" exception. Sorry for the confusion. The exclusion clause in the PEP should probably be replaced by an explicit list of macros which are made available. It not necessarily obvious that a macro only uses features made available through the ABI without actually digging through the headers. In the case of Py_INCREF()/Py_DECREF() the macros do use private macros which the ABI omits. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 27 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2009-06-29: EuroPython 2009, Birmingham, UK32 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Arguments of MatchObject in re module
On Tue, May 26, 2009, MRAB wrote:
>
> >>> p = re.compile("foo")
> >>> help(p.match)
> Help on built-in function match:
>
> match(...)
> match(string[, pos[, endpos]]) --> match object or None.
> Matches zero or more characters at the beginning of the string
>
> >>> p.match(string="foo")
>
> Traceback (most recent call last):
> File "", line 1, in
> p.match(string="foo")
> TypeError: Required argument 'pattern' (pos 1) not found
>
> The name of the first argument should be "string", yet it's "pattern".
> Does anyone know if it's anything other than a mistake? Should it be
> fixed in the next version of the re module, or are we just stuck with it
> (and should just change the docstring to match)?
Please file a report on bugs.python.org so this doesn't get lost.
Attaching a suggested patch for _sre.c would be most welcome.
--
Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/
"In many ways, it's a dull language, borrowing solid old concepts from
many other languages & styles: boring syntax, unsurprising semantics,
few automatic coercions, etc etc. But that's one of the things I like
about it." --Tim Peters on Python, 16 Sep 1993
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making the GIL faster & lighter on Windows
On Wed, May 27, 2009 at 4:24 AM, Nick Coghlan wrote: > > CriticalSections are first come first served on Windows, just like a > regular mutex. "Starting with Windows Server 2003 with Service Pack 1 (SP1), threads waiting on a critical section do not acquire the critical section on a first-come, first-serve basis." http://msdn.microsoft.com/en-us/library/ms682530(VS.85).aspx Windows critical sections use events for kernel-level synchronization. The user-mode code basically consists of an interlocked instruction inside the spin loop. When the likelihood of contention is low, a critical section should be a big win because it won't need to switch into the kernel. I suspect that contention will be frequent for the GIL A good description of pre-Vista Windows critical sections can be found here: http://msdn.microsoft.com/en-us/magazine/cc164040.aspx -- Curt Hagenlocher c...@hagenlocher.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making the GIL faster & lighter on Windows
Heads up to those who were following, I did my best to clearly outline the situation and direction in the tracker. http://bugs.python.org/issue6132 It includes a patch that will break the expected behavior of the thread lock object but make it possible to test GIL performance. > Note that for the GIL, if you use a CriticalSection object, you should > initialize its "spincount" to zero, because the GIL is almost always in > contention. That is, if you don't get the GIL right away, you won't for a > while. If I'm not mistaken, calling InitializeCriticalSection rather than InitializeCriticalSectionAndSpinCount (gotta love those long function names) sets the spin count to zero. I could tell when the spin count wasn't zero as far as performance is concerned - spinning is too much of a gamble in most contention situations. > I don't know what kernel primitive the Critical Section uses, but if it uses > an Event object or something similar, we are in the same soup, so to say, > because the CriticalSection's spinlocking feature buys us nothing. Judging from the increase in speed and CPU utilization I've seen, I don't believe this is the case. My guess is that it's something similar to a futex. - Phillip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
