[Python-Dev] Alternative Python VM
It could be interesting. """ - pyvm is * 2 times * faster than Python 2.4. In the source code there is a collection of benchmarks which includes 65 python scripts collected from the internet. At average if Python 2.4 needs 24 hours to do some job, pyvm can do it in 12 hours. - pyvm is a virtual machine that can run Python 2.3/2.4 bytecode. There is a compiler written in python (the 'pyc' program) which is executed by the vm to compile source code to bytecode. It is very easy to do advanced optimizations in python and the compiler produces bytecode of high quality (speed and size). """ http://students.ceid.upatras.gr/~sxanth/pyvm/ Sokolov Yura (funny_falcon) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Skiping searching throw dictionaries of mro() members.
Excuse my english. I have a complex Idea. It can be not worth, but just look at. It touches the classes and inheritance. Base point: We change classes not too often. But every look at a member leads to search throw a __dict__ of all bases in ierarhy till member found. We could cache this search in class.__dict__ itself and check only bases versions. Lets see: we have some classes: __metaclass__=object class a: def meth_a(self): pass class b: def meth_b(self): pass class c(a,b): pass Every class has mro. >>>c.mro() [, , , ] if we call methon from a base class, we look at c.__dict__ and at dict of every inherited class till found the method itself. so c.meth_a() leads to search throw c.__dict__ and a.__dict__ c.meth_b() leads to search throw c.__dict__ , a.__dict__ and b.__dict__ We could cash looks by this way: Store with a class its version. Every time after creation when we change a class (add,remove or chage class member, including __base__, __bases__ and mro) , we increase the version number. Lets call it VERSION Also store a tuple MRO_VERSION=tuple(base.VERSION for base in self.mro()). It changes, when we touches __base__, __bases__ or self.mro) Store with every class member a tuple : MEMBER_VERSION=(,) When we add member to (or change member of) class __dict__ directly, we store in this tuple (MRO_VERSION,class.mro().index(class)). When we search a class member: a) and have not found it in class.__dict__, we search it throw mro() by a classic way. If we've found it (in a class _BASE_), we store it in class.__dict__ with a tuple MEMBER_VERSION=(MRO_VERSION, class.mro().index(_BASE_)). Also we check all of seen base.VERSION to match their cached in MRO_VERSION values. If some is not match, we should update MRO_VERSION (berfore storing founded member in a class.__dict__) and adjust VERSION. If it was not found, we could store a (MRO_VERSION, class.mro().index(class)) with an internal value NotFound or do not store anything, and report that we have not found anything (so we search in an object itself). b) When we've found a class member in class.__dict__ we check versions if MEMBER_VERSION[0]==class.MRO_VERSION (the same object) then we just compare MRO_VERSION[i]==class.mro()[i].VERSION for i in range(MEMBER_VERSION[1]+1) and not search throw mro()[i].__dict__ what we are doing now. If all versions concur, we return found member (or, if we've found previously stored NotFound, we report that we have not found anything) if version of MRO_VERSION[j]!=class.mro()[j].VERSION and class!=class.mro()[j] (here can be a bug, one must think about it more) then we revert to a) point, but search throw [base.__dict__ for base in class.mro()[j:]], (and, of cause, update MRO_VERSION) if MEMBER_VERSION[0]!=class.MRO_VERSION, we reverts to a) point. That's all. PS. We can subclass module from a __builtin__, so we leave one dict lookup. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Skiping searching throw dictionaries of mro() members.
Phillip J. Eby wrote: > At 09:12 PM 9/14/2005 +0400, Sokolov Yura wrote: > >> We could cash looks by this way: >> Store with a class its version. Every time after creation when we change >> a class >> (add,remove or chage class member, including __base__, __bases__ and >> mro) , >> we increase the version number. Lets call it VERSION > > > FYI, there is already a mechanism for new-style classes that notifies > subclasses of changes to the base class; this mechanism could be used > to push cache flushes downward, rather than doing version checking > upward. > > > I didn't know it. So, if this thing works, it should be used. Another think. I do not really know, how emplemented scopes, but could this caching be applied to a scope realisation? So we "derive" current scope from his outer and cache all lookups, that goes out of this scope. Top scope is "derived" from a module and module it self "derived" from builtins. I think, there not many different deep upper lookups in current programs, but many reapeating, and not often we change outer scope, so cache will be fine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Variant of removing GIL.
Excuse my English.
I think I know how to remove GIL Obviously I am an idiot.
First about Py_INCREF and Py_DECREF.
We should not remove GIL at all. We should change it.
It must be "one writer-many reader" in a following semantic:
Lock has a "read-counter" and a "write-counter". Initially both are 0.
When "reader" tries to acquire lock for "read" it sleeps until
"write-counter" is 0.
When he "reader" acquires lock, he increase "read-counter".
When "reader" releases lock, he decreases "read-counter".
One reader will not block other, since he not increases "write-counter".
Reader will sleep, if there is any waiting writers, since they are
increase "write-counter".
When "writer" tries to acquire lock for "write", he increase
"write-counter" and
sleeps until "read-counter" happens 0. For "writers" lock for "write" -
simple lock.
when "writer" release lock, he decrease "write-counter".
When there is no waiting writers, readers arise.
Excuse me for telling obviouse things. I am really reinvent wheel in my
head,
since I was a bad studient.
I think this kind of lock is native for linux (i saw it in a kernel
source, but do not know
is waiting writer locks new readers or not?).
Now, every thread keep an queue of objects to decref. It can be
implemented as array, cause
it will be freed at once.
Initially, every object acquires GIL for "read".
Py_INCREF works as usually,
Py_DECREF places a ref into a queue.
When queue has became full or "100" instruction left ( :-) , it usefull),
thread releases GIL for "read" and acquires for "write",
when he acquire it, he decrefs all objects stored in a queue and clear
queue.
After all he acquires GIL for "read".
But what could we do with changing objects (dicts,lists and another)?
There should be a secondary "one-writer-many-reader" "public-write" GIL
- PWGIL.
SGIL ought to be more complicated, since it should work in RLOCK
semantic for "write" lock.
Lets call this lock ROWMR(reentreed one writer - many reader)
So semantic for ROWMR can be:
When a thread acquires ROWMR lock, it acquires it at a "read" level.
Lets name it "write-level"=0.
While threads "write-level"=0 it is a "reader".
Thread can increase "write-level".
When he turns "write-level" from 0 to 1, he becomes "writer".
while "write-level">0, thread is writer.
Thread can decrease "write-level".
When "write-level" turns from 1 to 0, thread becomes "reader".
With PWGIL :
We can mark every _mutable_ object with a creator thread number.
If mark match current thread number, object is "private" for the thread.
If mark is 0 (or another imposible thread number) object is "public".
If mark !=0 and !=current thread number, object is "alien".
When we access _mutable_ object, we check is it "private"?
If it is, we can do anything without locking.
If it is not and we access for read, we check is it "public".
If yes ("read" of "public"), then we can read it without locking.
If no, we increase "write-level",
if object is "alien", make it "public",
if we need to change object, change it,
decrease "write-level".
Of couse, when we append object to "public" collection, we chould make
it "public",
"write-level" is already increased so we do not make many separate
locks, but
when we then will access thouse object for read, we will not lock for
make it "public".
I don't know, how nested scopes are implemented, but i think it should
be considered as a mutable object.
So there is a small overhead for a single threaded application
( only for comparing 2 numbers)
and in a big part of multithreaded, since we are locking only writting on
_mutable_ _public_ objects. Most part of "public" objects is not
accessed to write
often: they are numbers, classes and mostly-read collections.
And one can optimize a program by accumulating results in a "private"
collection
and then flush it to "public" one.
Also, there may be a statement for explicit increasing "write-level"
around big update
of "public" object and decreasing after it.
PWGIL also must be released and reacquired with every "100" instructions
left,but only if "write-level=0",
it conforms to current GIL semantic.
I think, it must be not released with flushing decref queue, since it
can happen while we are in C code.
And there must be strong think about blocking IO.
Mostly awful situation (at my point of view):
object O is "private" for a thread A.
thread B accesses O and try to mark it "public", so it locks in attempt
of increasing "write-level"
thread A starts to change O (it is in "write-level 0"), and in a C code
it releases PWGIL
(around blocking IO, for example).
thread B becomes "writer", changes object to "public", becomes "reader"
and starts to read O,
returning thread A continue to change O , remaining in a "write-level=0".
But, I think, well written C code should not attemt to make blocking IO
inside of changing non-local objects
(and it does not attempt at the moment, as I guess. Am I
Re: [Python-Dev] Variant of removing GIL.
Corrections: Now, every thread keeps one n queue of objects to incref and second queue to decref. Both can be implemented as array, cause they will be freed at once. Initially, every thread acquires GIL for "read". Py_INCREF places a ref into a incref queue of a thread, Py_DECREF places a ref into a decref queue of a thread. When queue has became full or "100" instruction left ( :-) , it usefull), thread releases GIL for "read" and acquires for "write", when first process acquire it he: walk throw all incref queues in all threads, incref all thouse refs, and clear queues. then walk throw all decref queues in all threads, decref all thouse refs, and clear queues After all he acquires GIL for "read". Other processes could stupidly repeat it founding clear queues. Since there only one thread works as a "garbage collector", we will not loose any incref and decref. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Mixins.
Excuse my english. Is there any pythonic way to make real mixin into a class? Problem: Django model (and may be SQLObject). I have a common peaces of model declaration. Sometimes they are in table definition together, sometimes table have just one of them. I cannot use inheritance, cause Django doesn't handle it in right way (like mixins). Would it be usefull to define new magic class variable __mixins__ for a new style classes, which will accept list of classes and will drop all members of them into definition of class before they went to metaclass. Possible example (my wish in Django's Model): class time_from_to: time_from=meta.DateTimeField(default=datetime(1900,1,1,0,0,0)) time_to=meta.DateTimeField(default=datetime(3000,1,1,0,0,0)) class ident(meta.Model): __mixin__=[time_from_to] ident = meta.TextField() Or, as alternative: class time_from_to(mixin): time_from=meta.DateTimeField(default=datetime(1900,1,1,0,0,0)) time_to=meta.DateTimeField(default=datetime(3000,1,1,0,0,0)) class ident(time_from_to,meta.Model): ident = meta.TextField() You say: "It is problem of Django". Yes, you are right. It is common to use multiple inheritace in place of mixin. But it is handled not all time. Django does not at the moment (I hope, it would). To handle it, library writers, who implements their metaclasses, ought to write similar code again and again. I think, it would be simpler to make mixins in one true standart way. I can be mistaken. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] GIL, Python 3, and MP vs. UP
Ok. While windows already prefers threads, while linux-2.6 improves thread support and speed, while process startup expensive on time and memory, while we ought to dublicate our data and/or use obscure shared memory, while it is much simpler to make threaded program with care just about locks than multiprocessing with inventing interchange protocol (which abuses both CPU - sender and reciever), MULTIPROCESSING RULES!!! And on Opterons: you will win with multiprocessing ONLY if OS will bind process and processor/memory place pair. That would be difficult. Guido van Rossum wrote: > That's an understatement. I expect that *most* problems (even most > problems that we will be programming 10-20 years from now) get little > benefit out of MP. They are allready here. Servers they are. It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. Hardware is cheap, development time is expensive. To make scallable CPython-backed server one needs too much time. And he goes to Java, .NET and others (and rare to Jython, IronPython (in the future)). CPython will not be wide popular without real multithreading. --I want my Apache, written in Python ! :-) --I want my MySQL, written in Python ! ;-) It is a joke. But every joke has a peace of truth (as russian sayes). (Excuse my English) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a conditional expression in Py3.0
Sorry for looking in every hole. Just a suggestion. A= condition and first or second problem is in case when first in (None,0,[],""). May be invent new operator 'take'. take - returns right operator when left evals to True and stops computing condidtional expression. Then we could write: A = condition take first or second. A = x==y take w or s A = z is not None and q!=12 take [] or allowable(z,q) take [(z,q)] or "Impossible" Ok, it might looks ugly. But may be not. - Excuse my english. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a conditional expression in Py3.0
New keyword is so expensive? And why special case for 'then' is better than special case for 'take'? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 3000 and exec
Agree. >>>i=1 >>>def a(): i=2 def b(): print i return b >>>a()() 2 >>>def a(): i=2 def b(): exec "print i" return b >>>a()() 1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pythonic concurrency - offtopic
Offtopic: Microsoft Windows [Version 5.2.3790] (C) Copyright 1985-2003 Microsoft Corp. G:\Working\1>c:\Python24\python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from os import fork Traceback (most recent call last): File "", line 1, in ? ImportError: cannot import name fork >>> ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Autoloading? (Making Queue.Queue easier to use)
May be allow modules to define __getattr__ ? def __getattr__(thing): try: return __some_standart_way__(thing) except AttributeError: if thing=="Queue": import sys from Queue import Queue setattr(sys.modules[__name__],"Queue",Queue) return Queue raise ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pythonic concurrency - offtopic
Josiah Carlson wrote: >Sokolov Yura <[EMAIL PROTECTED]> wrote: > > >>Offtopic: >> >>Microsoft Windows [Version 5.2.3790] >>(C) Copyright 1985-2003 Microsoft Corp. >> >>G:\Working\1>c:\Python24\python >>Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on >>win32 >>Type "help", "copyright", "credits" or "license" for more information. >> >>> from os import fork >>Traceback (most recent call last): >> File "", line 1, in ? >>ImportError: cannot import name fork >> >>> >> >> > >Python for Windows, if I remember correctly, has never supported forking. >This is because the underlying process execution code does not have >support for the standard copy-on-write semantic which makes unix fork >fast. > >Cygwin Python does support fork, but I believe this is through a literal >copying of the memory space, which is far slower than unix fork. > >Until Microsoft adds kernel support for fork, don't expect standard >Windows Python to support it. > > - Josiah > > > > > That is what i mean... sorry for being noisy... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Unifying decimal numbers.
Excuse my English I think, we could just segregate tokens for decimal and real float and make them interoperable. Motivation: Most of us works with business databases - all "floats" are really decimals, algebraic operations should work without float inconsistency and those operations rare so speed is not important. But some of us use floats for speed in scientific and multimedia programs. with from __future__ import Decimal we could: a) interpret regular float constants as decimal b) interpret float constants with suffix 'f' as float (like1.5f 345.2e-5f etc) c) result of operation with decimal operands should be decimal >>> 1.0/3.0 0.3 d) result of operation with float operands should be float >>> 1.0f/3.0f 0.1f e) result of operation with decimal and float should be float (decimal converts into float and operation perfomed) >>> 1.0f/3.0 0.1f >>> 1.0/3.0f 0.1f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (no subject)
> > >Mixing Decimal and float is nearly ALWAYS a user error. Doing it correctly >requires significant expertise in the peculiarities of floating point >representations. > So that I think user should declare floats explicitly (###.###f) - he will fall into float space only if he wish it. >So Python protects the user by throwing exceptions when >attempts are made to mix Decimal and floats. I hate it. I want to get float when I wish to get float. In that case i would like to write #f. I want to stay with decimals by default. (and I want decimals written in C) But it just an opinion of young inexperienced/unpractised man. Excuse my English. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] str.dedent
>>/ msg = textwrap.dedent('''\
>/>/ IDLE's subprocess can't connect to %s:%d. This may be due \
>/>/ to your personal firewall configuration. It is safe to \
>/>/ allow this internal connection because no data is visible on \
>/>/ external ports.''' % address)
>/>/
>/
>Unfortunately, it won't help, since the 'dedent' method won't treat
>those spaces as indentation.
>
So that it would be usefull to implicit parser dedent on string with 'd' prefix
/ msg = d'''\
// IDLE's subprocess can't connect to %s:%d. This may be due \
// to your personal firewall configuration. It is safe to \
// allow this internal connection because no data is visible on \
// external ports.''' % address/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
