Re: Twisted 12.2.0 released
On Sat, Sep 1, 2012 at 7:17 PM, Ashwini Oruganti wrote: > On behalf of Twisted Matrix Laboratories, I am honored to announce the > release of Twisted 12.2. > > Congrats, Ashwini. adoptStreamConnection is much needed feature, happy to see that added to this release. -- Thanks & Regards, Godson Gera http://godson.in -- http://mail.python.org/mailman/listinfo/python-list
get the matched regular expression position in string.
Here is a string : str1="ha,hihi,a,ok" I want to get the position of "," in the str1,Which can count 3,8,14. how can I get it in python ? -- http://mail.python.org/mailman/listinfo/python-list
Re: get the matched regular expression position in string.
On 03/09/2012 09:18, contro opinion wrote: Here is a string : str1="ha,hihi,a,ok" I want to get the position of "," in the str1,Which can count 3,8,14. how can I get it in python ? Write some code using an appropriate string method. If it doesn't work put the minimum piece of code here that demonstrates the problem, what you expect to happen, what actually happened, and if appropriate the complete traceback. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: get the matched regular expression position in string.
On Mon, Sep 3, 2012 at 1:18 AM, contro opinion wrote:
> Subject: get the matched regular expression position in string.
As is often the case in Python, string methods suffice. Particularly
for something so simple, regexes aren't necessary.
> Here is a string :
>
> str1="ha,hihi,a,ok"
>
> I want to get the position of "," in the str1,Which can count 3,8,14.
Note that Python's indices start at 0. Your example output uses 1-based indices.
> how can I get it in python ?
def find_all(string, substr):
start = 0
while True:
try:
pos = string.index(substr, start)
except ValueError:
break
else:
yield pos
start = pos + 1
str1 = "ha,hihi,a,ok"
print list(find_all(str1, ",")) #===> [2, 7, 13]
In the future, I recommend reading the documentation for Python's string type:
http://docs.python.org/library/stdtypes.html#string-methods
What opinion are you contrary ("contro") to anyway?
Regards,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 9/3/2012 2:15 AM, Peter Otten wrote: At least users of wide builds will see a decrease in memory use: Everyone saves because everyone uses large parts of the stdlib. When 3.3 start up in a Windows console, there are 56 modules in sys.modules. With Idle, there are over 130. All the identifiers, all the global, local, and attribute names are present as ascii-only strings. Now multiply that by some reasonable average, keeping in mind that __builtins__ alone has 148 names. Former narrow build users gain less space but also gain the elimination of buggy behavior. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for an IPC solution
Laszlo Nagy wrote: There are just so many IPC modules out there. I'm looking for a solution for developing a new a multi-tier application. The core application will be running on a single computer, so the IPC should be using shared memory (or mmap) and have very short response times. But there will be a tier that will hold application state for clients, and there will be lots of clients. So that tier needs to go to different computers. E.g. the same IPC should also be accessed over TCP/IP. Most messages will be simple data structures, nothing complicated. The ability to run on PyPy would, and also to run on both Windows and Linux would be a plus. I have seen a stand alone cross platform IPC server before that could serve "channels", and send/receive messages using these channels. But I don't remember its name and now I cannot find it. Can somebody please help? Thanks, Laszlo http://pypi.python.org/pypi/execnet ? It sends/receives messages through channels, probably not the only one though. It requires only python installed on the remote machines. JM -- http://mail.python.org/mailman/listinfo/python-list
Re: get the matched regular expression position in string.
2012/9/3 contro opinion : > Here is a string : > str1="ha,hihi,a,ok" > I want to get the position of "," in the str1,Which can count 3,8,14. > how can I get it in python ? > -- > http://mail.python.org/mailman/listinfo/python-list > Hi, you can use re.finditer to match all "," and the start() method of the respective matches; cf.: http://docs.python.org/library/re.html#match-objects >>> import re >>> [m.start() for m in re.finditer(r",", "ha,hihi,a,ok")] [2, 7, 13] >>> [The obtained indices are zero-based, as has already been mentioned.] hth, vbr -- http://mail.python.org/mailman/listinfo/python-list
The opener parameter of Python 3 open() built-in
Does anyone have an example of utilisation? -- http://mail.python.org/mailman/listinfo/python-list
Time Bound Input in python
Hi I though of taking time bound input from user in python using "input" command. it waits fro infinite time , but I want to limit the time after that user input should expire with none. Please help. Thanks & Regard's Vikas Kumar Choudhary Mobile:91-7838594971/9886793145 http://in.linkedin.com/pub/vikas-kumar-choudhary/b/890/9aa http://www.facebook.com/vikaskchoudhary (Yahoo,MSN-Hotmail,Skype) Messenger = vikas.choudhary Please Add Me in Your Messenger List for Better Communication P Please consider the environment before printing this e-mail Do not print this email unless it is absolutely necessary. Save papers, Save tree. Note: This e-mail is confidential and may also be privileged, this is for the intended recipients only. If you are not the intended recipient, please delete the message and notify me immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Notice: All email and instant messages (including attachments) sent to or from This E-mail id , may be retained, monitored and/or reviewed, or authorized law enforcement personnel, without further notice or consent. -- -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
On 09/03/2012 08:32 AM, Marco wrote: > Does anyone have an example of utilisation? As of Python 3.2.3, there is no "opener" parameter in the open() function. http://docs.python.org/py3k/library/functions.html I don't know of any such parameter in earlier or later versions, but I could be wrong there. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
Am 03.09.2012 14:32, schrieb Marco:
> Does anyone have an example of utilisation?
The opener argument is a new 3.3 feature. For example you can use the
feature to implement exclusive creation of a file to avoid symlink attacks.
import os
def opener(file, flags):
return os.open(file, flags | os.O_EXCL)
open("newfile", "w", opener=opener)
--
http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
On 09/03/2012 03:05 PM, Dave Angel wrote: Does anyone have an example of utilisation? As of Python 3.2.3, there is no "opener" parameter in the open() function. http://docs.python.org/py3k/library/functions.html I don't know of any such parameter in earlier or later versions, but I could be wrong there. It's new in Python 3.3: http://docs.python.org/dev/library/functions.html#open -- http://mail.python.org/mailman/listinfo/python-list
simple client data base
Hello all, I am learning to program in python. I have a need to make a program that can store, retrieve, add, and delete client data such as name, address, social, telephone number and similar information. This would be a small client database for my wife who has a home accounting business. I have been reading about lists, tuples, and dictionary data structures in python and I am confused as to which would be more appropriate for a simple database. I know that python has real database capabilities but I'm not there yet and would like to proceed with as simple a structure as possible. Can anyone give me some idea's or tell me which structure would be best to use? Maybe its a combination of structures? I need some help. Thanks for your help. -- http://mail.python.org/mailman/listinfo/python-list
simple client data base
Hi, You can try gadfly ! This system is based on SQL. Loïc > From: [email protected] > Subject: simple client data base > Date: Mon, 3 Sep 2012 10:12:11 -0400 > To: [email protected] > > Hello all, I am learning to program in python. I have a need to make a > program that can store, retrieve, add, and delete client data such as > name, address, social, telephone number and similar information. This > would be a small client database for my wife who has a home accounting > business. > > I have been reading about lists, tuples, and dictionary data > structures in python and I am confused as to which would be more > appropriate for a simple database. > > I know that python has real database capabilities but I'm not there > yet and would like to proceed with as simple a structure as possible. > > Can anyone give me some idea's or tell me which structure would be > best to use? > > Maybe its a combination of structures? I need some help. > > Thanks for your help. > -- > http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
On Tue, Sep 4, 2012 at 12:12 AM, Mark R Rivet wrote: > I have been reading about lists, tuples, and dictionary data > structures in python and I am confused as to which would be more > appropriate for a simple database. I think you're looking at this backwards. A database is for storing information on disk, but lists/tuples/dicts are for manipulating it in memory. You may also be needlessly reinventing the wheel. Aren't there already several million basic contact databases around? Why roll your own? ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
Hi, You can make use of sqlite database also. And for the structure, you can create a single table and different columns for the fields like name, number etc. You can get more details over here. http://docs.python.org/library/sqlite3.html Thanks, Anoop Thomas Mathew atm ___ Life is short, Live it hard. On 3 September 2012 19:51, loïc Lauréote wrote: > > Hi, > > You can try gadfly ! > This system is based on SQL. > > Loïc > * > *** > > > > > From: [email protected] > > Subject: simple client data base > > Date: Mon, 3 Sep 2012 10:12:11 -0400 > > To: [email protected] > > > > > Hello all, I am learning to program in python. I have a need to make a > > program that can store, retrieve, add, and delete client data such as > > name, address, social, telephone number and similar information. This > > would be a small client database for my wife who has a home accounting > > business. > > > > I have been reading about lists, tuples, and dictionary data > > structures in python and I am confused as to which would be more > > appropriate for a simple database. > > > > I know that python has real database capabilities but I'm not there > > yet and would like to proceed with as simple a structure as possible. > > > > Can anyone give me some idea's or tell me which structure would be > > best to use? > > > > Maybe its a combination of structures? I need some help. > > > > Thanks for your help. > > -- > > http://mail.python.org/mailman/listinfo/python-list > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
On Monday, 3 September 2012 15:12:21 UTC+1, Manatee wrote: > Hello all, I am learning to program in python. I have a need to make a > > program that can store, retrieve, add, and delete client data such as > > name, address, social, telephone number and similar information. This > > would be a small client database for my wife who has a home accounting > > business. > > > > I have been reading about lists, tuples, and dictionary data > > structures in python and I am confused as to which would be more > > appropriate for a simple database. > > > > I know that python has real database capabilities but I'm not there > > yet and would like to proceed with as simple a structure as possible. > > > > Can anyone give me some idea's or tell me which structure would be > > best to use? > > > > Maybe its a combination of structures? I need some help. > > > > Thanks for your help. How about the half-way house, sqlite3 which comes with python? hth -- mph -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
On 03/09/2012 15:12, Mark R Rivet wrote: Hello all, I am learning to program in python. I have a need to make a program that can store, retrieve, add, and delete client data such as name, address, social, telephone number and similar information. This would be a small client database for my wife who has a home accounting business. I have been reading about lists, tuples, and dictionary data structures in python and I am confused as to which would be more appropriate for a simple database. I know that python has real database capabilities but I'm not there yet and would like to proceed with as simple a structure as possible. Can anyone give me some idea's or tell me which structure would be best to use? Maybe its a combination of structures? I need some help. Thanks for your help. If you stick with the simple data structures at some point you're going to have to save them to disk with a module like pickle or shelve. IMHO using sqlite is a better option as it comes with Python. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
Chris Angelico wrote: > You may also be needlessly reinventing the wheel. Aren't there already > several million basic contact databases around? Why roll your own? To learn a thing or two, and to stick it to the defeatists ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Time Bound Input in python
On 9/3/2012 3:01 AM Vikas Kumar Choudhary said... Hi I though of taking time bound input from user in python using "input" command. it waits fro infinite time , but I want to limit the time after that user input should expire with none. Please help. Googling yields http://stackoverflow.com/questions/1335507/keyboard-input-with-timeout-in-python among others. Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
In article <[email protected]>, Steven D'Aprano wrote: > > Indexing is O(0) for any string. > > I think you mean O(1) for constant-time lookups. Why settle for constant-time, when you can have zero-time instead :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 03.09.12 04:42, Steven D'Aprano wrote: If you are *seriously* interested in debugging why string code is slower for you, you can start by running the full suite of Python string benchmarks: see the stringbench benchmark in the Tools directory of source installations, or see here: http://hg.python.org/cpython/file/8ff2f4634ed8/Tools/stringbench http://hg.python.org/cpython/file/default/Tools/stringbench However, stringbench is not good tool to measure the effectiveness of new string representation, because it focuses mainly on ASCII strings and comparing strings with bytes. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
On 3 September 2012 15:12, Mark R Rivet wrote:
> Hello all, I am learning to program in python. I have a need to make a
> program that can store, retrieve, add, and delete client data such as
> name, address, social, telephone number and similar information. This
> would be a small client database for my wife who has a home accounting
> business.
>
I would use the sqlite3 module for this (if I wasn't using gmail contacts).
> I have been reading about lists, tuples, and dictionary data
> structures in python and I am confused as to which would be more
> appropriate for a simple database.
>
As already said by Chris these are the types that Python uses to represent
data in memory, rather than on disk. There are a number of ways that you
can use these to represent the information from your database. For example,
you could use a dict of dicts:
>>> contact_db = {} # empty dict
>>> contact_db['john'] = {'alias':'john', 'name':'John Doe', 'email': '
[email protected]'}
>>> contact_db['dave'] = {'alias':'dave', 'name':'Dave Doe', 'email': '
[email protected]'}
>>> contact_db
{'dave': {'alias': 'dave', 'name': 'Dave Doe', 'email': '[email protected]'},
'john': {'alias': 'john', 'name': 'John Doe', 'email': '[email protected]'}}
>>> contact_db['dave']
{'alias': 'dave', 'name': 'Dave Doe', 'email': '[email protected]'}
>>> contact_db['dave']['email']
'[email protected]'
I know that python has real database capabilities but I'm not there
> yet and would like to proceed with as simple a structure as possible.
>
If you don't want to use real database capabilities you could save the data
above into a csv file using the csv module:
>>> import csv
>>> with open('contacts.csv', 'wb') as f:
... writer = csv.DictWriter(f)
... writer.writelines(contact_db.values())
You can then reload the data with:
>>> with open('contacts.csv', 'rb') as f:
... reader = csv.DictReader(f, ['alias', 'name', 'email'])
... new_contact_db = {}
... for row in reader:
... new_contact_db[row['alias']] = row
...
>>> new_contact_db
{'dave': {'alias': 'dave', 'name': 'Dave Doe', 'email': '[email protected]'},
'john': {'alias': 'john', 'name': 'John Doe', 'email': '[email protected]'}}
>>> contact_db == new_contact_db
True
>
> Can anyone give me some idea's or tell me which structure would be
> best to use?
>
The above method for storing the data on disk is simple but not very safe.
If you use it for your wife's business make sure that you are always
keeping backups of the file. Preferably don't overwrite the file directly
but write the data out to a separate file first and then rename the file
(to avoid loss of data if the program has an error while writing).
The obvious way to improve on the above is to use the sqlite3 module to
store the data in an sqlite3 file instead of a csv file. There is one
advantage to using the above over using an sqlite3 database which is that
the data can be edited manually as a text file or using standard
spreadsheet software if necessary.
Oscar
--
http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 03.09.12 04:54, Steven D'Aprano wrote: This means that Python 3.3 will no longer have surrogate pairs. Am I right? As Terry said, basically, yes. Python 3.3 does not need in surrogate pairs, but does not prevent their creation. You can create a surrogate code (U+D800..U+DFFF) intentionally (as you can create a single accent modifier or other senseless alone charcode), but less likely that you will get them unintentionally. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 03.09.12 09:15, Peter Otten wrote: [email protected] wrote: Le dimanche 2 septembre 2012 14:01:18 UTC+2, Serhiy Storchaka a écrit : Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. With a memory gain = 0 since my text contains non-latin-1 characters! I can't confirm this. At least users of wide builds will see a decrease in memory use: And only users of wide builds will see a 20% decrease in speed for this data (with longer strings Python 3.3 will outstrip Python 3.2). This happens because of the inevitable transformation UCS2 -> wchar_t and wchar_t -> UCS2 on platform with 4-bytes wchar_t. On Windows there should be no slowing down. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
> Hello all, I am learning to program in python. I have a need to make a > program that can store, retrieve, add, and delete client data such as > name, address, social, telephone number and similar information. This > would be a small client database for my wife who has a home accounting > business. Python imho would be in need of a really good accounting application as a "demonstrator" for its capabilities. ;-) > I have been reading about lists, tuples, and dictionary data > structures in python and I am confused as to which would be more > appropriate for a simple database. > > I know that python has real database capabilities but I'm not there > yet and would like to proceed with as simple a structure as possible. The list of Python frameworks for rapid development of desktop (i.e. non-Web) database applications currently contains: using PyQt (& Sqlalchemy): Pypapi: www.pypapi.org Camelot: www.python-camelot.com Qtalchemy: www.qtalchemy.org using PyGTK: Sqlkit: sqlkit.argolinux.org (also uses Sqlalchemy) Kiwi: www.async.com.br/projects/kiwi using wxPython: Dabo: www.dabodev.com Defis: sourceforge.net/projects/defis (Russian only) GNUe: www.gnuenterprise.org Pypapi, Camelot, Sqlkit and Dabo seem to be the most active and best documented/supported ones. Sqlalchemy (www.sqlalchemy.org) seems to be "quite useful" for working with databases. Those of the above mentioned frameworks that don't use it do so for historic reasons, because the corresponding project started before Sqlalchemy became known. If you want to rely on not losing your data, you might want to use PostgreSQL (www.postgresql.org) as a storage backend with any of these. Sincerely, Wolfgang P.S.: If anyone knows of frameworks not listed here, thanks for mailing me. -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
On 03.09.12 16:29, Christian Heimes wrote: Am 03.09.2012 14:32, schrieb Marco: Does anyone have an example of utilisation? The opener argument is a new 3.3 feature. For example you can use the feature to implement exclusive creation of a file to avoid symlink attacks. Or you can use new dir_fd argument for same reason (or for performance). for root, dirs, files, rootfd in os.fwalk(topdir): def opener(file, flags): return os.open(file, flags, dir_fd=rootfd) for name in files: with open(name, "r", opener=opener) as f: ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Sun, Sep 2, 2012 at 6:00 AM, Serhiy Storchaka wrote: > On 02.09.12 12:52, Peter Otten wrote: >> >> Ian Kelly wrote: >> >>> Rewriting the example to use locale.strcoll instead: >> >> >> sorted(li, key=functools.cmp_to_key(locale.strcoll)) >> >> >> There is also locale.strxfrm() which you can use directly: >> >> sorted(li, key=locale.strxfrm) > > > Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. Doh! In Python 3.3, strcoll and strxfrm are the same speed, so I guess that the actual optimization I'm seeing here is that in Python 3.3, cmp_to_key(strcoll) has been optimized to return strxfrm. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
On Mon, 03 Sep 2012 18:03:27 +0200, Wolfgang Keller wrote: >> Hello all, I am learning to program in python. I have a need to make a >> program that can store, retrieve, add, and delete client data such as >> name, address, social, telephone number and similar information. This >> would be a small client database for my wife who has a home accounting >> business. > > > > Python imho would be in need of a really good accounting application as > a "demonstrator" for its capabilities. ;-) > > > >> I have been reading about lists, tuples, and dictionary data structures >> in python and I am confused as to which would be more appropriate for a >> simple database. >> >> I know that python has real database capabilities but I'm not there yet >> and would like to proceed with as simple a structure as possible. > > The list of Python frameworks for rapid development of desktop (i.e. > non-Web) database applications currently contains: > > using PyQt (& Sqlalchemy): > Pypapi: www.pypapi.org Camelot: www.python-camelot.com Qtalchemy: > www.qtalchemy.org > > using PyGTK: > Sqlkit: sqlkit.argolinux.org (also uses Sqlalchemy) > Kiwi: www.async.com.br/projects/kiwi > > using wxPython: > Dabo: www.dabodev.com Defis: sourceforge.net/projects/defis (Russian > only) > GNUe: www.gnuenterprise.org > > Pypapi, Camelot, Sqlkit and Dabo seem to be the most active and best > documented/supported ones. > > Sqlalchemy (www.sqlalchemy.org) seems to be "quite useful" for working > with databases. Those of the above mentioned frameworks that don't use > it do so for historic reasons, because the corresponding project started > before Sqlalchemy became known. > > If you want to rely on not losing your data, you might want to use > PostgreSQL (www.postgresql.org) as a storage backend with any of these. Personally, I wouldn't bother with SQLAlchemy for this. I'd just use Python as the front end, PostgreSQL for the database, and psycopg2 for the interface. -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady wrote: > We could use a Python long object for the version index to prevent overflow. > Combined with P. Rubin's idea to count the number of open iterators, most use > cases still wouldn't exceed a single word comparison; we could reset the > counter when there weren't any. We could use a Python long; I just don't think the extra overhead is justified in a data structure that is already highly optimized for speed. Incrementing and testing a C int is *much* faster than doing the same with a Python long. -- http://mail.python.org/mailman/listinfo/python-list
Re: tornado.web ioloop add_timeout eats CPU
Laszlo Nagy wrote:
[...]
> And here is my problem. If I point 5 browsers to the server, then I get
> 2% CPU load (Intel i5 2.8GHz on amd64 Linux). But why? Most of the time,
> the server should be sleeping. cProfile tells this:
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 10.0000.000 845.146 845.146 :1()
>1135775 832.2830.001 832.2830.001 {method 'poll' of
> 'select.epoll' objects}
>
> I have copied out the two relevant rows only. As you can see, total
> runtime was 845 seconds, and 832 seconds were spent in "epoll".
> Apparently, CPU load goes up linearly as I connect more clients. It
> means that 50 connected clients would do 20% CPU load. Which is
> ridiculous, because they don't do anything but wait for messages to be
> processed. Something terribly wrong, but I cannot figure out what?
What's wrong is the 1,135,775 calls to "method 'poll' of
'select.epoll' objects".
With five browsers waiting for messages over 845 seconds, that works
out to each waiting browser inducing 269 epolls per second.
Almost equally important is what the problem is *not*. The problem is
*not* spending the vast majority of time in epoll; that's *good* news.
The problem is *not* that CPU load goes up linearly as we connect more
clients. This is an efficiency problem, not a scaling problem.
So what's the fix? I'm not a Tornado user; I don't have a patch.
Obviously Laszlo's polling strategy is not performing, and the
solution is to adopt the event-driven approach that epoll and Tornado
do well.
-Bryan
--
http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Monday, September 3, 2012 2:30:24 PM UTC-5, Ian wrote: > On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady wrote: > > > We could use a Python long object for the version index to prevent > > overflow. Combined with P. Rubin's idea to count the number of open > > iterators, most use cases still wouldn't exceed a single word comparison; > > we could reset the counter when there weren't any. > > > > We could use a Python long; I just don't think the extra overhead is > > justified in a data structure that is already highly optimized for > > speed. Incrementing and testing a C int is *much* faster than doing > > the same with a Python long. I think the technique would require two python longs and a bool in the set, and a python long in the iterator. One long counts the number of existing (open) iterators. Another counts the version. The bool keeps track of whether an iterator has been created since the last modification, in which case the next modification requires incrementing the version and resetting the flag. -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On 09/03/2012 04:04 PM, Aaron Brady wrote: > On Monday, September 3, 2012 2:30:24 PM UTC-5, Ian wrote: >> On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady wrote: >> >>> We could use a Python long object for the version index to prevent >>> overflow. Combined with P. Rubin's idea to count the number of open >>> iterators, most use cases still wouldn't exceed a single word comparison; >>> we could reset the counter when there weren't any. >> >> >> We could use a Python long; I just don't think the extra overhead is >> >> justified in a data structure that is already highly optimized for >> >> speed. Incrementing and testing a C int is *much* faster than doing >> >> the same with a Python long. > I think the technique would require two python longs and a bool in the set, > and a python long in the iterator. > > One long counts the number of existing (open) iterators. Another counts the > version. The bool keeps track of whether an iterator has been created since > the last modification, in which case the next modification requires > incrementing the version and resetting the flag. I think you're over-engineering the problem. it's a bug if an iterator is used after some change is made to the set it's iterating over. We don't need to catch every possible instance of the bug, that's what testing is for. The point is to "probably" detect it, and for that, all we need is a counter in the set and a counter in the open iterator. Whenever changing the set, increment its count. And whenever iterating, check the two counters. if they don't agree, throw an exception, and destroy the iterator. i suppose that could be done with a flag, but it could just as easily be done by zeroing the pointer to the set. I'd figure a byte or two would be good enough for the counts, but a C uint would be somewhat faster, and wouldn't wrap as quickly. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
If you're just starting to learn python/computer science, why not try setting up section fields in a file that you can parse, with your own database. Then you can parse through, append, delete, etc and this will show you the 'higher' level of db's. Plus, I don't think anyone has mentioned RDBM: http://en.wikipedia.org/wiki/Relational_database_management_system or these from python: www.python.org/dev/peps/pep-0249/ http://wiki.python.org/moin/DatabaseProgramming/ -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for an IPC solution
On Saturday, September 1, 2012 6:25:36 PM UTC+5:30, Wolfgang Keller wrote: > > There are just so many IPC modules out there. I'm looking for a > > > solution for developing a new a multi-tier application. The core > > > application will be running on a single computer, so the IPC should > > > be using shared memory (or mmap) and have very short response times. > > > > Probably the fastest I/RPC implementation for Python should be > > OmniOrbpy: > > > > http://omniorb.sourceforge.net/ > > > > It's cross-platform, language-independent and standard-(Corba-) > > compliant. > > > > > I have seen a stand alone cross platform IPC server before that could > > > serve "channels", and send/receive messages using these channels. But > > > I don't remember its name and now I cannot find it. Can somebody > > > please help? > > > > If it's just for "messaging", Spread should be interesting: > > > > http://www.spread.org/ > > > > Also cross-platform & language-independent. > > > > Sincerely, > > > > Wolfgang Though I'm not the OP, thanks for the info. Will put Spread on my stack to check out ... -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On 27.08.12 22:17, Ian Kelly wrote: May I suggest an alternate approach? Internally tag each set or dict with a "version", which is just a C int. Every time the hash table is modified, increment the version. When an iterator is created, store the current version on the iterator. When the iterator is advanced, check that the iterator version matches the dict/set version. If they're not equal, raise an error. Oh, I'm surprised that this strategy is not used yet. I was sure that it is used. -- http://mail.python.org/mailman/listinfo/python-list
Re: Async client for PostgreSQL?
On Saturday, September 1, 2012 3:28:52 PM UTC-4, Laszlo Nagy wrote: > > Hi > > > > > > does running on tornado imply that you would not consider twisted > > > http://twistedmatrix.com ? > > > > > > If not, twisted has exactly this capability hiding long running > > > queries on whatever db's behind deferToThread(). > > All right, I was reading its documentation > > > > http://twistedmatrix.com/documents/10.1.0/api/twisted.internet.threads.deferToThread.html > > > > It doesn't tell too much about it: "Run a function in a thread and > > return the result as a Deferred.". > You can find more documentation here: http://twistedmatrix.com/documents/current/core/howto/threading.html Also, Twisted has dedicated APIs for interacting with databases asynchronously: http://twistedmatrix.com/documents/current/core/howto/rdbms.html Additionally, there is a non-blocking (rather than thread-based) implementation of the above API available for PostgreSQL: http://pypi.python.org/pypi/txpostgres > > > Run a function but in what thread? Does it create a new thread for every > > invocation? In that case, I don't want to use this. My example case: 10% > > from 100 requests/second deal with a database. But it does not mean that > > one db-related request will do a single db API call only. They will > > almost always do more: start transaction, parse and open query, fetch > > with cursor, close query, open another query etc. then commit > > transaction. 8 API calls to do a quick fetch + update (usually under > > 100msec, but it might be blocked by another transaction for a while...) > > So we are talking about 80 database API calls per seconds at least. It > > would be insane to initialize a new thread for each invocation. And > > wrapping these API calls into a single closure function is not useful > > either, because that function would not be able to safely access the > > state that is stored in the main thread. Unless you protet it with > > locks. But it is whole point of async I/O server: to avoid using slow > > locks, expensive threads and context switching. > > > > Maybe, deferToThread uses a thread pool? But it doesn't say much about > > it. (Am I reading the wrong documentation?) BTW I could try a version > > that uses a thread pool. > > > > It is sad, by the way. We have async I/O servers for Python that can be > > used for large number of clients, but most external modules/extensions > > do not support their I/O loops. Including the extension modules of the > > most popular databases. So yes, you can use Twisted or torandoweb until > > you do not want to call *some* API functions that are blocking. (By > > *some* I mean: much less blocking than non-blocking, but quite a few.) > > We also have synchronous Python servers, but we cannot get rid of the > > GIL, Python threads are expensive and slow, so they cannot be used for a > > large number of clients. And finally, we have messaging services/IPC > > like zeromq. They are probably the most expensive, but they scale very > > well. But you need more money to operate the underlying hardware. I'm > > starting to think that I did not get a quick answer because my use case > > (100 clients) fall into to the "heavy weight" category, and the solution > > is to invest more in the hardware. :-) > > > > Thanks, > > > > Laszlo -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Monday, September 3, 2012 3:28:28 PM UTC-5, Dave Angel wrote: > On 09/03/2012 04:04 PM, Aaron Brady wrote: > > > On Monday, September 3, 2012 2:30:24 PM UTC-5, Ian wrote: > > >> On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady wrote: > > >> > > >>> We could use a Python long object for the version index to prevent > >>> overflow. Combined with P. Rubin's idea to count the number of open > >>> iterators, most use cases still wouldn't exceed a single word comparison; > >>> we could reset the counter when there weren't any. > > >> > > >> > > >> We could use a Python long; I just don't think the extra overhead is > > >> > > >> justified in a data structure that is already highly optimized for > > >> > > >> speed. Incrementing and testing a C int is *much* faster than doing > > >> > > >> the same with a Python long. > > > I think the technique would require two python longs and a bool in the set, > > and a python long in the iterator. > > > > > > One long counts the number of existing (open) iterators. Another counts > > the version. The bool keeps track of whether an iterator has been created > > since the last modification, in which case the next modification requires > > incrementing the version and resetting the flag. > > > > I think you're over-engineering the problem. it's a bug if an iterator > > is used after some change is made to the set it's iterating over. We > > don't need to catch every possible instance of the bug, that's what > > testing is for. The point is to "probably" detect it, and for that, all > > we need is a counter in the set and a counter in the open iterator. > > Whenever changing the set, increment its count. And whenever iterating, > > check the two counters. if they don't agree, throw an exception, and > > destroy the iterator. i suppose that could be done with a flag, but it > > could just as easily be done by zeroing the pointer to the set. > > > > I'd figure a byte or two would be good enough for the counts, but a C > > uint would be somewhat faster, and wouldn't wrap as quickly. > > > > -- > > > > DaveA Hi D. Angel, The serial index constantly reminds me of upper limits. I have the same problem with PHP arrays, though it's not a problem with the language itself. The linked list doesn't have a counter, it invalidates iterators when a modification is made, therefore it's the "correct" structure in my interpretation. But it does seem more precarious comparatively, IMHO. Both strategies solve the problem I posed originally, they both involve trade-offs, and it's too late to include either in 3.3.0. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Mon, 03 Sep 2012 18:26:02 +0300, Serhiy Storchaka wrote: > On 03.09.12 04:42, Steven D'Aprano wrote: >> If you are *seriously* interested in debugging why string code is >> slower for you, you can start by running the full suite of Python >> string benchmarks: see the stringbench benchmark in the Tools directory >> of source installations, or see here: >> >> http://hg.python.org/cpython/file/8ff2f4634ed8/Tools/stringbench > > http://hg.python.org/cpython/file/default/Tools/stringbench > > However, stringbench is not good tool to measure the effectiveness of > new string representation, because it focuses mainly on ASCII strings > and comparing strings with bytes. But it is a good place to start, so you can develop unicode benchmarks. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
On Mon, 03 Sep 2012 15:29:05 +0200, Christian Heimes wrote:
> Am 03.09.2012 14:32, schrieb Marco:
>> Does anyone have an example of utilisation?
>
> The opener argument is a new 3.3 feature. For example you can use the
> feature to implement exclusive creation of a file to avoid symlink
> attacks.
>
> import os
>
> def opener(file, flags):
> return os.open(file, flags | os.O_EXCL)
>
> open("newfile", "w", opener=opener)
Why does the open builtin need this added complexity? Why not just call
os.open directly? Or for more complex openers, just call the opener
directly?
What is the rationale for complicating open instead of telling people to
just call their opener directly?
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Mon, 03 Sep 2012 13:04:23 -0700, Aaron Brady wrote: > On Monday, September 3, 2012 2:30:24 PM UTC-5, Ian wrote: >> On Sun, Sep 2, 2012 at 11:43 AM, Aaron Brady >> wrote: >> >> > We could use a Python long object for the version index to prevent >> > overflow. Combined with P. Rubin's idea to count the number of open >> > iterators, most use cases still wouldn't exceed a single word >> > comparison; we could reset the counter when there weren't any. >> >> We could use a Python long; I just don't think the extra overhead is >> justified in a data structure that is already highly optimized for >> speed. Incrementing and testing a C int is *much* faster than doing >> the same with a Python long. > > I think the technique would require two python longs and a bool in the > set, and a python long in the iterator. > > One long counts the number of existing (open) iterators. Another counts > the version. The bool keeps track of whether an iterator has been > created since the last modification, in which case the next modification > requires incrementing the version and resetting the flag. I think that is over-engineered and could be the difference between having the patch accepted and having it rejected. After all, many people will argue that the existing solution to the problem is good enough. Dicts are extremely common, and your patch increases both the memory usage of every dict, and the overhead of every write operation (__setitem__, __delitem__, update). Only a very few dicts will actually need this overhead, for the rest it is waste. It is important to keep that waste to a minimum or risk having the patch rejected. An unsigned C int can count up to 4,294,967,295. I propose that you say that is enough iterators for anyone, and use a single, simple, version counter in the dict and the iterator. If somebody exceeds that many iterators to a single dict or set, and the version field overflows by exactly 2**32 versions, the results are no worse than they are now. You won't be introducing any new bugs. Complicating the solution is, in my opinion, unnecessary. Why should every set and dict carry the cost of incrementing TWO Python longs and a flag when just a single C int is sufficient for all realistic use-cases? The opportunity for failure is extremely narrow: - I must have an iterator over a dict or set; - and I must have paused the iteration in some way; - and then I must create exactly 2**32 other iterators to the same dict or set; - and at some point modify the dict or set - and then restart the first iterator at which point some items returned by the iterator *may* be duplicated or skipped (depends on the nature of the modifications). -- Steven -- http://mail.python.org/mailman/listinfo/python-list
is there history command in python?
in bash ,history command can let me know every command which i execute in history, is there a same command in python console?if there is no,how can i know the historical inputs? it is not convenient to use direction key( up, or down arrow key) to see my historical inputs. i want an another convenient method. -- http://mail.python.org/mailman/listinfo/python-list
Comparing strings from the back?
There's been a bunch of threads lately about string implementations, and that got me thinking (which is often a dangerous thing). Let's assume you're testing two strings for equality. You've already done the obvious quick tests (i.e they're the same length), and you're down to the O(n) part of comparing every character. I'm wondering if it might be faster to start at the ends of the strings instead of at the beginning? If the strings are indeed equal, it's the same amount of work starting from either end. But, if it turns out that for real-life situations, the ends of strings have more entropy than the beginnings, the odds are you'll discover that they're unequal quicker by starting at the end. It doesn't seem un-plausible that this is the case. For example, most of the filenames I work with begin with "/home/roy/". Most of the strings I use as memcache keys have one of a small number of prefixes. Most of the strings I use as logger names have common leading substrings. Things like credit card and telephone numbers tend to have much more entropy in the trailing digits. On the other hand, hostnames (and thus email addresses) exhibit the opposite pattern. Anyway, it's just a thought. Has anybody studied this for real-life usage patterns? I'm also not sure how this work with all the possible UCS/UTF encodings. With some of them, you may get the encoding semantics wrong if you don't start from the front. -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On 09/03/2012 09:26 PM, Steven D'Aprano wrote: > On Mon, 03 Sep 2012 13:04:23 -0700, Aaron Brady wrote: > >> >> >> I think the technique would require two python longs and a bool in the >> set, and a python long in the iterator. >> >> One long counts the number of existing (open) iterators. Another counts >> the version. The bool keeps track of whether an iterator has been >> created since the last modification, in which case the next modification >> requires incrementing the version and resetting the flag. > I think that is over-engineered and could be the difference between > having the patch accepted and having it rejected. After all, many people > will argue that the existing solution to the problem is good enough. > > Dicts are extremely common, and your patch increases both the memory > usage of every dict, and the overhead of every write operation > (__setitem__, __delitem__, update). Only a very few dicts will actually > need this overhead, for the rest it is waste. It is important to keep > that waste to a minimum or risk having the patch rejected. > > An unsigned C int can count up to 4,294,967,295. I propose that you say > that is enough iterators for anyone, and use a single, simple, version > counter in the dict and the iterator. If somebody exceeds that many > iterators to a single dict or set, I think you have the count confused. it has to be a count of how many changes have been made to the dict or set, not how many iterators exist. > and the version field overflows by > exactly 2**32 versions, the results are no worse than they are now. You > won't be introducing any new bugs. > > Complicating the solution is, in my opinion, unnecessary. Why should > every set and dict carry the cost of incrementing TWO Python longs and a > flag when just a single C int is sufficient for all realistic use-cases? > > The opportunity for failure is extremely narrow: > > - I must have an iterator over a dict or set; > - and I must have paused the iteration in some way; > - and then I must create exactly 2**32 other iterators to the > same dict or set; > - and at some point modify the dict or set > - and then restart the first iterator > > at which point some items returned by the iterator *may* be duplicated or > skipped (depends on the nature of the modifications). > > I agree with almost your entire point, exact that the place where it would fail to detect a bug is when somebody has modified the dict exactly 2**32 times while an iterator is paused. See my own response, of 4:27pm. (my time). -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Mon, 03 Sep 2012 21:50:57 -0400, Dave Angel wrote: > On 09/03/2012 09:26 PM, Steven D'Aprano wrote: >> An unsigned C int can count up to 4,294,967,295. I propose that you say >> that is enough iterators for anyone, and use a single, simple, version >> counter in the dict and the iterator. If somebody exceeds that many >> iterators to a single dict or set, > > I think you have the count confused. it has to be a count of how many > changes have been made to the dict or set, not how many iterators exist. Oops, yes you are absolutely right. It's a version number, not a count of iterators. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Comparing strings from the back?
On Mon, Sep 3, 2012 at 9:54 PM, Roy Smith wrote: > There's been a bunch of threads lately about string implementations, and > that got me t > > On > hinking (which is often a dangerous thing). > > Let's assume you're testing two strings for equality. You've already > done the obvious quick tests (i.e they're the same length), and you're > down to the O(n) part of comparing every character. > > I'm wondering if it might be faster to start at the ends of the strings > instead of at the beginning? If the strings are indeed equal, it's the > same amount of work starting from either end. But, if it turns out that > for real-life situations, the ends of strings have more entropy than the > beginnings, the odds are you'll discover that they're unequal quicker by > starting at the end. > > It doesn't seem un-plausible that this is the case. For example, most > of the filenames I work with begin with "/home/roy/". Most of the > strings I use as memcache keys have one of a small number of prefixes. > Most of the strings I use as logger names have common leading > substrings. Things like credit card and telephone numbers tend to have > much more entropy in the trailing digits. > > On the other hand, hostnames (and thus email addresses) exhibit the > opposite pattern. > > Anyway, it's just a thought. Has anybody studied this for real-life > usage patterns? > > I'm also not sure how this work with all the possible UCS/UTF encodings. > With some of them, you may get the encoding semantics wrong if you don't > start from the front. > -- > http://mail.python.org/mailman/listinfo/python-list > First include len(string)/2, in order to include starting at the center of the string, and threading/weaving by 2 processes out. import timeit do the the rest, and see which has the fastest time. -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -- http://mail.python.org/mailman/listinfo/python-list
Re: Comparing strings from the back?
On Tue, Sep 4, 2012 at 11:54 AM, Roy Smith wrote: > I'm wondering if it might be faster to start at the ends of the strings > instead of at the beginning? > I'm also not sure how this work with all the possible UCS/UTF encodings. > With some of them, you may get the encoding semantics wrong if you don't > start from the front. No problem there; Python uses only fixed-width encodings. Also, any canonical encoding can be safely compared byte-for-byte; two identical Unicode strings will be bit-wise identical in (say) UTF-8. There's issues of cache locality and such that quite possibly mean it's not going to be faster overall, but it wouldn't be difficult to tweak the Python sources, recompile, and run some tests. I'm sure python-dev or python-list will be most happy to discuss some benchmark figures! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Comparing strings from the back?
On Mon, 03 Sep 2012 21:54:01 -0400, Roy Smith wrote: > Let's assume you're testing two strings for equality. You've already > done the obvious quick tests (i.e they're the same length), and you're > down to the O(n) part of comparing every character. > > I'm wondering if it might be faster to start at the ends of the strings > instead of at the beginning? If the strings are indeed equal, it's the > same amount of work starting from either end. But, if it turns out that > for real-life situations, the ends of strings have more entropy than the > beginnings, the odds are you'll discover that they're unequal quicker by > starting at the end. And if the strings have the same end and different beginnings: running jumping cooking then you'll find out quicker by starting at the beginning. No general-purpose string comparison operator can make assumptions about the content of the strings, like "they are nearly always pathnames on Unix-like systems like /home/user/x and /home/user/y". I suppose you could have two different operators, == for "test from the front" and === for "test from the back", and push that decision onto the user, who will then spend hours angsting about the "most efficient" equality test and days painstakingly profiling his data to save four nanoseconds at runtime. Besides, then somebody will say "Yes, but what about the cases where the prefix and the suffix are both equal, but the middle will be different?" and propose a third string-equality operator and then there will be bloodshed. On average, string equality needs to check half the characters in the string. Any individual comparisons may require fewer, or more, than that, but whichever way you scan the string, some comparisons will take more and some will take fewer. With no general way of telling ahead of time which will be which, on average you can't do better than O(N/2) and no complexity justification for picking "start from the end" from "start from the start". -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: is there history command in python?
Look at the readline module or use ipython. Aldrich On Mon, Sep 3, 2012 at 9:47 PM, contro opinion wrote: > in bash ,history command can let me know every command which i execute in > history, > is there a same command in python console?if there is no,how can i know > the historical inputs? > it is not convenient to use direction key( up, or down arrow key) to see > my historical inputs. > i want an another convenient method. > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- http://mail.python.org/mailman/listinfo/python-list
Re: is there history command in python?
Google can help tremendously, if you use it correctly: Used search term 'python console remember history', and got https://www.google.com/search?client=ubuntu&channel=fs&q=python+console+remember+history&ie=utf-8&oe=utf-8, and the first one listed is below: http://stackoverflow.com/questions/947810/how-to-save-a-python-interactive-session > > -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -- http://mail.python.org/mailman/listinfo/python-list
Re: simple client data base
Mark R Rivet wrote:
> Hello all, I am learning to program in python. I have a need to make a
> program that can store, retrieve, add, and delete client data such as
> name, address, social, telephone number and similar information. This
> would be a small client database for my wife who has a home accounting
> business.
>
> I have been reading about lists, tuples, and dictionary data
> structures in python and I am confused as to which would be more
> appropriate for a simple database.
>
> I know that python has real database capabilities but I'm not there
> yet and would like to proceed with as simple a structure as possible.
>
> Can anyone give me some idea's or tell me which structure would be
> best to use?
>
> Maybe its a combination of structures? I need some help.
The data types that would choose are defined by your requirements and how
well a data type meets your requirements.
I can imagine: In a database you would want quick access to data. You would
want to access fields primarily by name. You would also want to filter data
by various criteria.
Therefore, it appears to me that it would be best if your records were
dictionaries or dictionary-like objects, and your recordsets were lists of
records, like so
#!/usr/bin/env python3
from datetime import date
data = [
{
'lastname': 'Doe',
'firstname': 'John',
'sn':'123-451-671-890',
'birthdata': date(2000, 1, 2)
},
{
'lastname': 'Doe',
'firstname': 'Jane',
'sn':'409-212-582-452',
'birthdata': date(2001, 2, 3)
},
]
- You could quickly access the second record with data[1].
- You could access the 'lastname' field of the second record with
data[1]['lastname']
- You could get a list of records where the person is born before 2001 CE
with filter(lambda record: record['birthdate'] < date(2001, 1, 1), data)
The advantage of dictionaries over dictionary-like objects is that they are
easily extensible and that the memory footprint is probably lower (CMIIW);
the disadvantage is slightly more complicated syntax and that you have to
keep track of the keys. Therefore, you might want to consider instantiating
a Record class instead; in its simplest form:
class Record(object):
def __init__(self, lastname, firstname, sn=None, birthdate=None):
self.lastname = lastname
self.firstname = firstname
self.sn = str(sn)
self.birthdate = birthdate
data = [
Record(lastname='Doe', firstname='John', sn='123-451-671-890',
birthdate=date(2000, 1, 2)),
Record(lastname='Doe', firstname='Jane', sn='409-212-582-452',
birthdate=date(2001, 2, 3))
]
- You could access the 'lastname' property of the second record with
data[1].lastname
- You get a list of records where the person is born before 2001 CE with
list(filter(lambda record: record.birthdate < date(2001, 1, 1), data))
(in Python 2.x without list())
However, if you want your program to manipulate the data *persistently*. as
it will probably be needed for business, you will need to also store it
somewhere else than in the volatile memory in which these data structures
are usually stored. The most simple way would be to store and parse the
string representation of the objects.
Production-quality implementations of those and other concepts already
exist, of course, but using something finished and polished does not provide
as much learning experience.
HTH
--
PointedEars
Twitter: @PointedEars2
Please do not Cc: me. / Bitte keine Kopien per E-Mail.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Comparing strings from the back?
On 9/3/2012 9:54 PM, Roy Smith wrote: There's been a bunch of threads lately about string implementations, and that got me thinking (which is often a dangerous thing). Let's assume you're testing two strings for equality. You've already done the obvious quick tests (i.e they're the same length), and you're down to the O(n) part of comparing every character. I'm wondering if it might be faster to start at the ends of the strings instead of at the beginning? I posted the 3.3 str (in)equality compare a few days ago. It first compares length and then kind, then the actual bytes in whatever chunks. If the latter uses the C/system mem compare, that is faster than anything coded in Python or even C. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
Dennis Lee Bieber writes:
> On 04 Sep 2012 01:13:09 GMT, Steven D'Aprano
> declaimed the following in
> gmane.comp.python.general:
> > What is the rationale for complicating [the builtin] open instead of
> > telling people to just call their opener directly?
>
> To avoid the new syntax would mean coding the example as
>
> f = os.fdopen(os.open("newfile", flags | os.O_EXCL), "w")
>
> which does NOT look any cleaner to me... Especially not if "opener" is
> to be used in more than one location.
Exactly. That's not what was asked, though. Steven asked why not call
the opener.
So, having written the opener:
> On Mon, 03 Sep 2012 15:29:05 +0200, Christian Heimes wrote:
> > import os
> >
> > def opener(file, flags):
> > return os.open(file, flags | os.O_EXCL)
why not call that directly?
f = opener(file, flags)
It certainly is cleaner than either of the alternatives so far, and it
doesn't add a parameter to the builtin.
> Furthermore, using "opener" could allow for a localized change to
> affect all open statements in the module -- change file path, open for
> string I/O rather than file I/O, etc.
I don't know of any real-life code which would be significantly improved
by that. Can you point us to some?
--
\ “I find the whole business of religion profoundly interesting. |
`\ But it does mystify me that otherwise intelligent people take |
_o__)it seriously.” —Douglas Adams |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
Re: Comparing strings from the back?
On 2012-09-04 at 02:17:30 +, Steven D'Aprano wrote: > Besides, then somebody will say "Yes, but what about the cases where > the prefix and the suffix are both equal, but the middle will be > different?" and propose a third string-equality operator and > then there will be bloodshed. Lisp has several equality forms, although they're for different kinds or levels of equality rather than for different algorithms for determining that equality. I imagine that a discussion similar to this one spawned many of those forms. I agree with Steven: decisions such as this belong to the application developer, not the standard library. Don't forget the fourth string-equality operator for case-insensitive comparisons and the fifth string-equality operator for strings that are likely to be palindromes. That said, if I really wanted bloodshed, I would propose = for the third string-equality operator! ;-) Dan -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
Ben Finney writes: > So, having written the opener: > > > On Mon, 03 Sep 2012 15:29:05 +0200, Christian Heimes wrote: > > > import os > > > > > > def opener(file, flags): > > > return os.open(file, flags | os.O_EXCL) > > why not call that directly? > > f = opener(file, flags) Ah, because that returns the file descriptor, not the file. I see. -- \ “If nature has made any one thing less susceptible than all | `\others of exclusive property, it is the action of the thinking | _o__) power called an idea” —Thomas Jefferson, 1813-08-13 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
On Mon, 03 Sep 2012 23:19:51 -0400, Dennis Lee Bieber wrote:
> On 04 Sep 2012 01:13:09 GMT, Steven D'Aprano
> declaimed the following in
> gmane.comp.python.general:
>
>
>
>> Why does the open builtin need this added complexity? Why not just call
>> os.open directly? Or for more complex openers, just call the opener
>> directly?
>>
> Because os.open() returns a low-level file descriptor, not a
> Python file object?
Good point.
But you can wrap the call to os.open, as you mention below. The only
complication is that you have to give the mode twice, converting between
low-level O_* integer modes and high-level string modes:
a = os.open('/tmp/foo', os.O_WRONLY | os.O_CREAT)
b = os.fdopen(a, 'w')
But to some degree, you still have to do that with the opener argument,
at least in your own head.
>> What is the rationale for complicating open instead of telling people
>> to just call their opener directly?
>
> To avoid the new syntax would mean coding the example as
>
> f = os.fdopen(os.open("newfile", flags | os.O_EXCL), "w")
>
> which does NOT look any cleaner to me...
Well, I don't know about that. Once you start messing about with low-
level O_* flags, it's never going to exactly be clean no matter what you
do. But I think a one-liner like the above *is* cleaner than a three-
liner like the original:
def opener(file, flags):
return os.open(file, flags | os.O_EXCL)
open("newfile", "w", opener=opener)
although I accept that this is a matter of personal taste.
Particularly if the opener is defined far away from where you eventually
use it. A lambda is arguably better from that perspective:
open("newfile", "w",
opener=lambda file, flags: os.open(file, flags | os.O_EXCL)
)
but none of these solutions are exactly neat or clean. You still have to
mentally translate between string modes and int modes, and make sure
you're not passing the wrong mode:
py> open('junk', 'w').write('hello world')
11
py> open('junk', 'r', opener=lambda file, flags: os.open(file, flags |
os.O_TRUNC)).read() # oops
''
so it's not exactly a high-level interface.
In my opinion, a cleaner, more Pythonic interface would be either:
* allow built-in open to take numeric modes:
open(file, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
* or even more Pythonic, expose those numeric modes using strings:
open(file, 'wx')
That's not as general as an opener, but it covers the common use-case and
for everything else, write a helper function.
> Especially not if "opener" is to be used in more than one location.
The usual idiom for fixing the "used more than once" is "write a helper",
not "add a callback function to a builtin" :)
> Furthermore, using "opener" could
> allow for a localized change to affect all open statements in the module
> -- change file path, open for string I/O rather than file I/O, etc.
A common idiom for that is to shadow open in the module, like this:
_open = open
def open(file, *args):
file = file.lowercase()
return _open(file, *args)
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: The opener parameter of Python 3 open() built-in
Steven D'Aprano writes:
> On Mon, 03 Sep 2012 23:19:51 -0400, Dennis Lee Bieber wrote:
> > f = os.fdopen(os.open("newfile", flags | os.O_EXCL), "w")
> >
> > which does NOT look any cleaner to me...
>
> Well, I don't know about that. Once you start messing about with low-
> level O_* flags, it's never going to exactly be clean no matter what you
> do. But I think a one-liner like the above *is* cleaner than a three-
> liner like the original:
>
> def opener(file, flags):
> return os.open(file, flags | os.O_EXCL)
>
> open("newfile", "w", opener=opener)
>
> although I accept that this is a matter of personal taste.
If the opener has an unhelpful name like ‘opener’, yes.
But if it's named as any function should be named – to say what it does
that's special – then I think the result would be much clearer::
outfile = open("newfile", "w", opener=open_exclusive)
> Particularly if the opener is defined far away from where you
> eventually use it.
Another good reason to name helper functions descriptively.
> * or even more Pythonic, expose those numeric modes using strings:
>
> open(file, 'wx')
Which is, indeed, another improvement in Python 3.3 – the ‘x’ mode for
‘open’ http://docs.python.org/dev/library/functions.html#open>.
--
\“The greatest tragedy in mankind's entire history may be the |
`\ hijacking of morality by religion.” —Arthur C. Clarke, 1991 |
_o__) |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
