Re: py2c - an open source Python to C/C++ is looking for developers
On 02.09.12 06:15, Ramchandra Apte wrote: It converts to *pure* C/C++ *without* using Python or its API so that it can be the same speed as C/C++ How is it implemented long integers? -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
Le jeudi 30 août 2012 17:01:50 UTC+2, Antoine Pitrou a écrit : > > > I honestly suggest you shut up until you have a clue. > Désolé Antoine, I have not the knowledge to dive in the Python code, but I know what is a character. The coding of the characters is a domain per se, independent from the os, from the computer languages. Before spending time to implement a new algorithm, maybe it is better to ask, if there is something better than the actual schemes. I still remember my thoughts when I read the PEP 393 discussion: "this is not logical", "they do no understand typography", "atomic character ???", ... Real world exemples. >>> import libfrancais >>> li = ['noël', 'noir', 'nœud', 'noduleux', \ ... 'noétique', 'noèse', 'noirâtre'] >>> r = libfrancais.sortfr(li) >>> r ['noduleux', 'noël', 'noèse', 'noétique', 'nœud', 'noir', 'noirâtre'] (cf "Le Petit Robert") or The *letters* satisfying the requirements of the "Imprimerie nationale". jmf -- http://mail.python.org/mailman/listinfo/python-list
Re: Async client for PostgreSQL?
Hmm, I was suggesting that you could replace the whole DB driver with a webservice implemented with twisted, if you rule out threads then with ampoule doing it with a process pool and consume this webservice with the tornado side asynchronously. I see. I'm sorry, I misunderstood. So this would be a wrapper around PostgreSQL. Very interesting idea. (In this case, I'm not sure how to safely manage transactions.) production level example thread pool based DB API: Just to give you some ballpark figures, I'm running a game server with a daily peak of about 1500 parallel permanent connections and 50k games played every day (avg game duration 13min, peak request frequency close to 100req/sec) with a lot of statistics going into a MySQL DB on US$2k worth of hardware. Twisted as basis sitting atop FreeBSD, started the latest version in March, its running since then, no restarts, no reboots, no problems. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: py2c - an open source Python to C/C++ is looking for developers
On 02/09/2012 07:16, Ramchandra Apte wrote: On Sunday, 2 September 2012 11:18:38 UTC+5:30, Michael Torrie wrote: On 09/01/2012 09:15 PM, Ramchandra Apte wrote: It converts to *pure* C/C++ *without* using Python or its API so that it can be the same speed as C/C++ Sounds like a fun project for you. I hope you learn a lot doing it. That's reason enough for it. Do you plan to port all the standard python modules as well, though? Because Python modules, both in the standard library and third-party, are the main reasons that I use python. For example, PyGTK. Python is a great glue language. Since it can seamlessly interact with C and be extended in C, speed has never really been an issue for me. I am writing py2c for my OS written in Python (planned) . Sounds like a fun project for you. More fun with more developers. (I need developers) Your earlier quote "Well, it should be quite easy because most of the _modules are either C accelerator (which there is no need to port) or a C wrapper (which should be trivial to port)" means that I won't be touching the project with a 10 foot long disinfected barge pole. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 02/09/2012 08:36, [email protected] wrote: Le jeudi 30 août 2012 17:01:50 UTC+2, Antoine Pitrou a écrit : I honestly suggest you shut up until you have a clue. Désolé Antoine, I have not the knowledge to dive in the Python code, but I know what is a character. You're a character, and from my observations on this thread you're very humorous. YMMV. The coding of the characters is a domain per se, independent from the os, from the computer languages. Before spending time to implement a new algorithm, maybe it is better to ask, if there is something better than the actual schemes. Please write a new PEP indicating how you would correct your perceived deficiencies with PEP 393 and its implementation. I still remember my thoughts when I read the PEP 393 discussion: "this is not logical", "they do no understand typography", "atomic character ???", ... When PEP 393 was first drafted how much input did you give during the acceptance process, if any? Real world exemples. import libfrancais li = ['noël', 'noir', 'nœud', 'noduleux', \ Why the unneeded continuation character, fancy wasting storage space? ... 'noétique', 'noèse', 'noirâtre'] r = libfrancais.sortfr(li) r ['noduleux', 'noël', 'noèse', 'noétique', 'nœud', 'noir', 'noirâtre'] What has sorting foreign words got to do with the internal representaion of the individual characters? (cf "Le Petit Robert") or The *letters* satisfying the requirements of the "Imprimerie nationale". jmf I've just rechecked my calendar and it's definitly not 1st April today. Poor old me I'm baffled as always. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
running Lua in Python
Hi all, I'm looking for a way to run Lua scripts in Python, and also send and receive data between the two. Something like lunatic-python [1] would be ideal. However, so far I haven't been able to build it on the machines it's supposed to run on (macs with OS X Lion) and it seems to be dormant. My requirements are stock OS X Python (which is 2.7) and Lua 5.2. I'm looking for either a way to make lunatic-python work or another tool that would do the job. Thanks in advance. -- Arnaud [1] http://labix.org/lunatic-python -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Sun, Sep 2, 2012 at 1:36 AM, wrote:
> I still remember my thoughts when I read the PEP 393
> discussion: "this is not logical", "they do no understand
> typography", "atomic character ???", ...
That would indicate one of two possibilities. Either:
1) Everybody in the PEP 393 discussion except for you is clueless
about how to implement a Unicode type; or
2) You are clueless about how to implement a Unicode type.
Taking into account Occam's razor, and also that you seem to be unable
or unwilling to offer a solid rationale for those thoughts, I have to
say that I'm currently leaning toward the second possibility.
> Real world exemples.
>
import libfrancais
li = ['noël', 'noir', 'nœud', 'noduleux', \
> ... 'noétique', 'noèse', 'noirâtre']
r = libfrancais.sortfr(li)
r
> ['noduleux', 'noël', 'noèse', 'noétique', 'nœud', 'noir',
> 'noirâtre']
libfrancais does not appear to be publicly available. It's not listed
in PyPI, and googling for "python libfrancais" turns up nothing
relevant.
Rewriting the example to use locale.strcoll instead:
>>> li = ['noël', 'noir', 'nœud', 'noduleux', 'noétique', 'noèse', 'noirâtre']
>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'French_France')
'French_France.1252'
>>> import functools
>>> sorted(li, key=functools.cmp_to_key(locale.strcoll))
['noduleux', 'noël', 'noèse', 'noétique', 'nœud', 'noir', 'noirâtre']
# Python 3.2
>>> import timeit
>>> timeit.repeat("sorted(li, key=functools.cmp_to_key(locale.strcoll))",
>>> "import functools; import locale; li = ['noël', 'noir', 'nœud', 'noduleux',
>>> 'noétique', 'noèse', 'noirâtre']", number=1)
[0.5544277025009592, 0.5370117249557325, 0.5551836677925053]
# Python 3.3
>>> import timeit
>>> timeit.repeat("sorted(li, key=functools.cmp_to_key(locale.strcoll))",
>>> "import functools; import locale; li = ['noël', 'noir', 'nœud', 'noduleux',
>>> 'noétique', 'noèse', 'noirâtre']", number=1)
[0.1421166788364303, 0.12389078130001963, 0.13184190553613462]
As you can see, Python 3.3 is about 77% faster than Python 3.2 on this
example. If this was intended to show that the Python 3.3 Unicode
representation is a regression over the Python 3.2 implementation,
then it's a complete failure as an example.
--
http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On Sun, Sep 2, 2012 at 3:04 AM, Arnaud Delobelle wrote: > Hi all, > > I'm looking for a way to run Lua scripts in Python, and also send and > receive data between the two. Something like lunatic-python [1] would > be ideal. However, so far I haven't been able to build it on the > machines it's supposed to run on (macs with OS X Lion) and it seems to > be dormant. > > My requirements are stock OS X Python (which is 2.7) and Lua 5.2. I'm > looking for either a way to make lunatic-python work or another tool > that would do the job. The simple approach would be to run the Lua scripts with the subprocess module, using JSON (or something equally accessible) as an interchange format. If you need to embed the Lua interpreter in the Python process, and you can't get an existing third-party module to work, consider rolling your own C module for the purpose. My recollection of working with Lua is that it's a very easy language to embed. -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
Or you can use a module made for this task: http://labix.org/lunatic-python http://pypi.python.org/pypi/lupa On Sun, Sep 2, 2012 at 7:24 PM, Ian Kelly wrote: > > On Sun, Sep 2, 2012 at 3:04 AM, Arnaud Delobelle wrote: > > Hi all, > > > > I'm looking for a way to run Lua scripts in Python, and also send and > > receive data between the two. Something like lunatic-python [1] would > > be ideal. However, so far I haven't been able to build it on the > > machines it's supposed to run on (macs with OS X Lion) and it seems to > > be dormant. > > > > My requirements are stock OS X Python (which is 2.7) and Lua 5.2. I'm > > looking for either a way to make lunatic-python work or another tool > > that would do the job. > > The simple approach would be to run the Lua scripts with the > subprocess module, using JSON (or something equally accessible) as an > interchange format. > > If you need to embed the Lua interpreter in the Python process, and > you can't get an existing third-party module to work, consider rolling > your own C module for the purpose. My recollection of working with > Lua is that it's a very easy language to embed. > -- > http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
The method of insert doesn't work with nltk texts: AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert'
I wrote codes to add 'like' at the end of every 3 word in a nltk text as follows: >>> text = nltk.corpus.brown.words(categories = 'news') >>> def hedge(text): for i in range(3,len(text),4): new_text = text.insert(i, 'like') return new_text[:50] >>> hedge(text) Traceback (most recent call last): File "", line 1, in hedge(text) File "", line 3, in hedge new_text = text.insert(i, 'like') AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert' Isn't text in the brown corpus above a list? why doesn't it has attribute 'insert'? Thanks much for your hints. -- http://mail.python.org/mailman/listinfo/python-list
Changing a Value in List of lists
Hi all, I have a doubt regarding how the list work in following case- >>> ls=[[0]*5]*5 >>> ls[1][1]+=1 >>> ls [[0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0]] Here, according to me only one value in ls should be incremented but it is increasing 1 value in all the lists of ls. Why?? -- Regards, Rishabh Dixit MSc.(Tech) Information Systems, BITS Pilani -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
Arnaud Delobelle, 02.09.2012 11:04: > I'm looking for a way to run Lua scripts in Python, and also send and > receive data between the two. Something like lunatic-python [1] would > be ideal. However, so far I haven't been able to build it on the > machines it's supposed to run on (macs with OS X Lion) and it seems to > be dormant. > > My requirements are stock OS X Python (which is 2.7) and Lua 5.2. I'm > looking for either a way to make lunatic-python work or another tool > that would do the job. Try Lupa: http://pypi.python.org/pypi/lupa Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On 2 September 2012 10:39, Alec Taylor wrote: > Or you can use a module made for this task: > > http://labix.org/lunatic-python As I said in my original message, I couldn't get this to work. > http://pypi.python.org/pypi/lupa I'll check this out, thanks. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
Ian Kelly wrote: > Rewriting the example to use locale.strcoll instead: sorted(li, key=functools.cmp_to_key(locale.strcoll)) There is also locale.strxfrm() which you can use directly: sorted(li, key=locale.strxfrm) -- http://mail.python.org/mailman/listinfo/python-list
Re: Changing a Value in List of lists
On Sun, Sep 2, 2012 at 7:44 PM, Rishabh Dixit wrote: > > Hi all, > > I have a doubt regarding how the list work in following case- > ls=[[0]*5]*5 ls[1][1]+=1 ls > [[0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, > 0, 0, 0]] > > > Here, according to me only one value in ls should be incremented but it is > increasing 1 value in all the lists of ls. Why?? When you take the list [[0,0,0,0,0]] and multiply it by 5, you get a list with five references to the same inner list. That's where the confusion is coming from. Then when you change one of them, it changes all five displayed versions, since they're all still the same list. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
[2.5.1] Read each line from txt file, replace, and save?
Hello
This is a newbie question.
I need to read a text file into a variable, loop through each line and
use a regex to substitute some items within the line, and save the
whole variable into a new text file.
This triggers an error when I save the modified variable that contains
all the lines:
==
import re,sys
f = open("C:\\input.txt", "r")
textlines = f.readlines()
f.close()
for line in textlines:
#edit each line
line = "just a test"
#rewrite data to new file
log = open('output.sub','w')
#ERROR: argument 1 must be string or read-only character buffer, not
list
log.write(textlines)
log.close()
==
Should I use another way to read the file, edit each line, and save
the data into a new file?
Thank you.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
I've found the white paper which gives the technical basis for the claims made by jmf so thought I'd better share in order to explain his rationale. http://www.montypython.net/scripts/right-think.php -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: [2.5.1] Read each line from txt file, replace, and save?
On Sun, 02 Sep 2012 12:19:02 +0200, Gilles wrote:
(snip)
Found it:
#rewrite lines to new file
output = open('output.txt','w')
for line in textlines:
#edit each line
line = "just a test"
output.write("%s" % line)
output.close()
--
http://mail.python.org/mailman/listinfo/python-list
Re: [2.5.1] Read each line from txt file, replace, and save?
On 02/09/2012 11:36, Gilles wrote:
On Sun, 02 Sep 2012 12:19:02 +0200, Gilles wrote:
(snip)
Found it:
#rewrite lines to new file
output = open('output.txt','w')
for line in textlines:
#edit each line
line = "just a test"
output.write("%s" % line)
output.close()
IMHO better practice to use the with statement. See
http://docs.python.org/reference/compound_stmts.html#the-with-statement
--
Cheers.
Mark Lawrence.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Changing a Value in List of lists
Hi Chris, Thanks for your reply. I tried this - >>> ls=[0 for i in range(5)] >>> mat=[ls for i in range(5)] >>> mat[1][2]+=1 >>> mat [[0, 0, 1, 0, 0], [0, 0, 1, 0, 0], [0, 0, 1, 0, 0], [0, 0, 1, 0, 0], [0, 0, 1, 0, 0]] I guess here also the same issue is there as ls refers to one list that is being repeated through mat and change in one causes change in others also. Is there any solution to change all elements independently. On Sun, Sep 2, 2012 at 3:14 PM, Rishabh Dixit wrote: > > Hi all, > > I have a doubt regarding how the list work in following case- > > >>> ls=[[0]*5]*5 > >>> ls[1][1]+=1 > >>> ls > [[0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, > 1, 0, 0, 0]] > > > Here, according to me only one value in ls should be incremented but it is > increasing 1 value in all the lists of ls. Why?? > -- > Regards, > Rishabh Dixit > MSc.(Tech) Information Systems, > BITS Pilani > > -- Regards, Rishabh Dixit MSc.(Tech) Information Systems, BITS Pilani -- http://mail.python.org/mailman/listinfo/python-list
Re: Changing a Value in List of lists
Hi, I got it :). We should create a new list every time before adding to a list of list so that instead repeating the reference a new reference generated every time. >>> mat=[[0]*5 for i in range(5)] >>> mat[1][2]+=1 >>> mat [[0, 0, 0, 0, 0], [0, 0, 1, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]] This worked for me!! Thanx :) On Sun, Sep 2, 2012 at 4:30 PM, Rishabh Dixit wrote: > Hi Chris, > > Thanks for your reply. I tried this - > > >>> ls=[0 for i in range(5)] > >>> mat=[ls for i in range(5)] > >>> mat[1][2]+=1 > >>> mat > [[0, 0, 1, 0, 0], [0, 0, 1, 0, 0], [0, 0, 1, 0, 0], [0, 0, 1, 0, 0], [0, > 0, 1, 0, 0]] > > I guess here also the same issue is there as ls refers to one list that is > being repeated through mat and change in one causes change in others also. > > Is there any solution to change all elements independently. > > > > On Sun, Sep 2, 2012 at 3:14 PM, Rishabh Dixit wrote: > >> >> Hi all, >> >> I have a doubt regarding how the list work in following case- >> >> >>> ls=[[0]*5]*5 >> >>> ls[1][1]+=1 >> >>> ls >> [[0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 0], [0, >> 1, 0, 0, 0]] >> >> >> Here, according to me only one value in ls should be incremented but it >> is increasing 1 value in all the lists of ls. Why?? >> -- >> Regards, >> Rishabh Dixit >> MSc.(Tech) Information Systems, >> BITS Pilani >> >> > > > -- > Regards, > Rishabh Dixit > MSc.(Tech) Information Systems, > BITS Pilani > > -- Regards, Rishabh Dixit MSc.(Tech) Information Systems, BITS Pilani -- http://mail.python.org/mailman/listinfo/python-list
Re: The method of insert doesn't work with nltk texts: AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert'
On 09/02/2012 05:39 AM, Token Type wrote: > I wrote codes to add 'like' at the end of every 3 word in a nltk text as > follows: > text = nltk.corpus.brown.words(categories = 'news') def hedge(text): > for i in range(3,len(text),4): > new_text = text.insert(i, 'like') > return new_text[:50] > hedge(text) > Traceback (most recent call last): > File "", line 1, in > hedge(text) > File "", line 3, in hedge > new_text = text.insert(i, 'like') > AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert' > > Isn't text in the brown corpus above a list? why doesn't it has attribute > 'insert'? > I tried to find online documentation for nltk, and although I found the mention of a free online book, I didn't see it. So, some generic comments. The error message is telling you that the object 'text' is not a list, but a "ConcatenatedCorpusView". Perhaps you can look that up in your docs for nltk. But there's quite a bit you can do just with the interpreter. try print type(text) to see the type of text. try dir(text) to see what attributes it has try help(text) to see what docstrings might be built in. Incidentally, if you really think it's a list of words (or that it acts like a list), then 'text' might not be the best name for it. Any reason you didn't just call it words ? -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Logging handler: No output
Hello,
I have a class method that executes a subprocess. There are two loggers in the
class, self.logger for general logging and proclog for process output (stdout
& stderr) logging which should go to stdout and a file:
def start_process(self, command, no_shlex=False, raise_excpt=True,
print_output = True, **kwargs):
cmd = command if no_shlex else shlex.split(command)
# Use an additional logger without formatting for process output.
proclog = logging.getLogger(self.config.tag)
proclog.propagate = False # Process output should not propage to the main
logger
logfile = self._logfilename()
if logfile:
proclog.addHandler(logging.FileHandler(logfile))
if print_output:
proclog.addHandler(logging.StreamHandler(sys.stdout))
self.popen = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, bufsize=0, **kwargs)
while True:
output = self.popen.stdout.readline().decode()
if output == "" and self.popen.poll() != None:
break
proclog.info(output.rstrip("\n"))
ret_code = self.popen.returncode
self.logger.debug("%s returned with %i", command, ret_code)
But neither the FileHandler nor the StreamHandler produce any actual output.
The file is being created but stays empty. If I use a print output in the
while loop it works, so output is catched and the applications stdout in
working. But why the logger proclog catching nothing?
Thanks,
Florian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Changing a Value in List of lists
On 02/09/2012 12:05, Rishabh Dixit wrote: Hi, I got it :). We should create a new list every time before adding to a list of list so that instead repeating the reference a new reference generated every time. Please don't top post on this list as it often destroys the context, thanks. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging handler: No output
Florian Lindner writes: > The file is being created but stays empty. If I use a print output in the > while loop it works, so output is catched and the applications stdout in > working. But why the logger proclog catching nothing? I don't see you setting the log level anyplace in that sample, and you are logging at INFO and DEBUG levels. By default, only WARNING and above only actually produce output. If you want INFO and DEBUG to log, you have to request it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 02.09.12 12:52, Peter Otten wrote: Ian Kelly wrote: Rewriting the example to use locale.strcoll instead: sorted(li, key=functools.cmp_to_key(locale.strcoll)) There is also locale.strxfrm() which you can use directly: sorted(li, key=locale.strxfrm) Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 02/09/2012 13:00, Serhiy Storchaka wrote: On 02.09.12 12:52, Peter Otten wrote: Ian Kelly wrote: Rewriting the example to use locale.strcoll instead: sorted(li, key=functools.cmp_to_key(locale.strcoll)) There is also locale.strxfrm() which you can use directly: sorted(li, key=locale.strxfrm) Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. That's it then I'm giving up with Python. In future I'll be writing everything in machine code to ensure that I get the fastest possible run times. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
In article , Mark Lawrence wrote: > On 02/09/2012 13:00, Serhiy Storchaka wrote: > > On 02.09.12 12:52, Peter Otten wrote: > >> Ian Kelly wrote: > >> > >>> Rewriting the example to use locale.strcoll instead: > >> > >> sorted(li, key=functools.cmp_to_key(locale.strcoll)) > >> > >> There is also locale.strxfrm() which you can use directly: > >> > >> sorted(li, key=locale.strxfrm) > > > > Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. > > > > > > That's it then I'm giving up with Python. In future I'll be writing > everything in machine code to ensure that I get the fastest possible run > times. Feh. You software guys are always too willing to sacrifice performance for convenience. If you really want speed, grab yourself a handful of chips and a soldering iron. -- http://mail.python.org/mailman/listinfo/python-list
Fwd: The method of insert doesn't work with nltk texts: AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert'
First, thanks very much for your kind help. 1)Further more, I test the function of insert. It did work as follows: >>> text = ['The', 'Fulton', 'County', 'Grand'] >>> text.insert(3,'like') >>> text ['The', 'Fulton', 'County', 'like', 'Grand'] >>> 2) I tested the text from nltk. It is list actually. See the following: >>> text = nltk.corpus.brown.words(categories = 'news') >>> text[:10] ['The', 'Fulton', 'County', 'Grand', 'Jury', 'said', 'Friday', 'an', 'investigation', 'of'] How come python tells me that it is not a list by prompting "AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert'"? I am confused. Since we doubt text is not a list, I have to add one more line of code there as follows. Then it seems working. >>> text = nltk.corpus.brown.words(categories = 'news') >>> def hedge(text): text = list(text) for i in range(3,len(text),4): text.insert(i, 'like') return text[:50] >>> hedge(text) ['The', 'Fulton', 'County', 'like', 'Grand', 'Jury', 'said', 'like', 'Friday', 'an', 'investigation', 'like', 'of', "Atlanta's", 'recent', 'like', 'primary', 'election', 'produced', 'like', '``', 'no', 'evidence', 'like', "''", 'that', 'any', 'like', 'irregularities', 'took', 'place', 'like', '.', 'The', 'jury', 'like', 'further', 'said', 'in', 'like', 'term-end', 'presentments', 'that', 'like', 'the', 'City', 'Executive', 'like', 'Committee', ','] Isn't it odd? -- http://mail.python.org/mailman/listinfo/python-list
Re: py2c - an open source Python to C/C++ is looking for developers
On Sunday, 2 September 2012 12:53:35 UTC+5:30, Serhiy Storchaka wrote: > On 02.09.12 06:15, Ramchandra Apte wrote: > > > It converts to *pure* C/C++ *without* using Python or its API so that it > > can be the same speed as C/C++ > > > > How is it implemented long integers? py2c has been just started (long integer multiplacation might use the Python internal multiplication code) (it uses the karatsuba algorithm) -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Sunday, 2 September 2012 17:53:16 UTC+5:30, Mark Lawrence wrote: > On 02/09/2012 13:00, Serhiy Storchaka wrote: > > > On 02.09.12 12:52, Peter Otten wrote: > > >> Ian Kelly wrote: > > >> > > >>> Rewriting the example to use locale.strcoll instead: > > >> > > >> sorted(li, key=functools.cmp_to_key(locale.strcoll)) > > >> > > >> There is also locale.strxfrm() which you can use directly: > > >> > > >> sorted(li, key=locale.strxfrm) > > > > > > Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. > > > > > > > > > > That's it then I'm giving up with Python. In future I'll be writing > > everything in machine code to ensure that I get the fastest possible run > > times. > > > > -- > > Cheers. > > > > Mark Lawrence. please make it *heavily optimized* machine code -- http://mail.python.org/mailman/listinfo/python-list
Re: Fwd: The method of insert doesn't work with nltk texts: AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert'
On 09/02/2012 09:06 AM, John H. Li wrote: > First, thanks very much for your kind help. > > 1)Further more, I test the function of insert. It did work as follows: > text = ['The', 'Fulton', 'County', 'Grand'] text.insert(3,'like') text > ['The', 'Fulton', 'County', 'like', 'Grand'] > 2) I tested the text from nltk. It is list actually. See the following: text = nltk.corpus.brown.words(categories = 'news') text[:10] > ['The', 'Fulton', 'County', 'Grand', 'Jury', 'said', 'Friday', 'an', > 'investigation', 'of'] > > How come python tells me that it is not a list by prompting "AttributeError: > 'ConcatenatedCorpusView' object has no attribute 'insert'"? I am confused. > > Since we doubt text is not a list, I have to add one more line of code > there as follows. Then it seems working. text = nltk.corpus.brown.words(categories = 'news') def hedge(text): > text = list(text) > for i in range(3,len(text),4): > text.insert(i, 'like') > return text[:50] > hedge(text) > ['The', 'Fulton', 'County', 'like', 'Grand', 'Jury', 'said', 'like', > 'Friday', 'an', 'investigation', 'like', 'of', "Atlanta's", 'recent', > 'like', 'primary', 'election', 'produced', 'like', '``', 'no', 'evidence', > 'like', "''", 'that', 'any', 'like', 'irregularities', 'took', 'place', > 'like', '.', 'The', 'jury', 'like', 'further', 'said', 'in', 'like', > 'term-end', 'presentments', 'that', 'like', 'the', 'City', 'Executive', > 'like', 'Committee', ','] > > Isn't it odd? > > Without reading the documentation, or at least the help(), I can't figure it to be odd. If a class wants to support slicing semantics, all it has to do is implement special methods like __getslice__ and __setslice__. If it doesn't document .insert(), then you shouldn't try to call it. Duck-typing. What did you get when you tried type(), dir() and help() ? Did they help. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: The method of insert doesn't work with nltk texts: AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert'
Token Type wrote: > I wrote codes to add 'like' at the end of every 3 word in a nltk text as follows: > > >>> text = nltk.corpus.brown.words(categories = 'news') > >>> def hedge(text): > for i in range(3,len(text),4): > new_text = text.insert(i, 'like') > return new_text[:50] > > >>> hedge(text) > > Traceback (most recent call last): > File "", line 1, in > hedge(text) > File "", line 3, in hedge > new_text = text.insert(i, 'like') > AttributeError: 'ConcatenatedCorpusView' object has no attribute 'insert' > > Isn't text in the brown corpus above a list? why doesn't it has attribute 'insert'? > > Thanks much for your hints. The error message shows that text is not a list. It looks like a list, >>> text = nltk.corpus.brown.words(categories="news") >>> text ['The', 'Fulton', 'County', 'Grand', 'Jury', 'said', ...] but it is actually a nltk.corpus.reader.util.ConcatenatedCorpusView: >>> type(text) The implementer of a class is free to decide what methods he wants to implement. You can get a first impression of the available ones with dir(): >>> dir(text) ['_MAX_REPR_SIZE', '__add__', '__class__', '__cmp__', '__contains__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__hash__', '__init__', '__iter__', '__len__', '__module__', '__mul__', '__new__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_offsets', '_open_piece', '_pieces', 'close', 'count', 'index', 'iterate_from'] As you can see insert() is not among these methods. However, __iter__() is a hint that you can convert the ConcatenatedCorpusView to a list, and that does provide an insert() method. Let's try: >>> text = list(text) >>> type(text) >>> text.insert(0, "yadda") >>> text[:5] ['yadda', 'The', 'Fulton', 'County', 'Grand'] Note that your hedge() function may still not work as you expect: >>> text = ["-"] * 20 >>> text ['-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-'] >>> for i in range(0, len(text), 3): ... text.insert(i, "X") ... >>> text ['X', '-', '-', 'X', '-', '-', 'X', '-', '-', 'X', '-', '-', 'X', '-', '-', 'X', '-', '-', 'X', '-', '-', '-', '-', '-', '-', '-', '-'] That is because the list is growing with every insert() call. One workaround is to start inserting items at the end of the list: >>> text = ["-"] * 20 >>> for i in reversed(range(0, len(text), 3)): ... text.insert(i, "X") ... >>> text ['X', '-', '-', '-', 'X', '-', '-', '-', 'X', '-', '-', '-', 'X', '-', '-', '-', 'X', '-', '-', '-', 'X', '-', '-', '-', 'X', '-', '-'] -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 02/09/2012 14:48, Ramchandra Apte wrote: please make it *heavily optimized* machine code Goes without saying. First thing I'll concentrate on is removing superfluous newlines sent by crappy mail clients or similar. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: [Twisted-Python] Twisted 12.2.0 released
Congratulations to the entire team, particularly Ashwini, and thank you for an awesome release manager job well done, Ashwini :) On Sat, Sep 1, 2012 at 3:47 PM, Ashwini Oruganti wrote: > On behalf of Twisted Matrix Laboratories, I am honored to announce the > release of Twisted 12.2. > > Highlights for this release include: > > * To be able to work on Python3 support, Python 2.5 is no longer > supported. > > * twisted.mail.imap4 now serves BODYSTRUCTURE responses which provide > more information and conform to the IMAP4 RFC more closely. > > * twisted.conch now supports commercial SSH implementations which don't > comply with the IETF standard. > > * twisted.internet.endpoints now provides several new endpoints, > including a > TCP client endpoint that resolves hostnames to IPv6 host addresses. > > * IReactorSocket.adoptStreamConnection, implemented by some reactors, > allows adding an existing established connection to the reactor. > > Starting with the release after 12.2, Twisted will begin requiring > zope.interface 3.6 (as part of Python 3 support). > > This is the last Twisted release supporting Python 2.6 on Windows. > > For more information, see the NEWS file here: > > http://twistedmatrix.com/Releases/Twisted/12.2/NEWS.txt > > > Download it now from: > > http://pypi.python.org/packages/source/T/Twisted/Twisted-12.2.0.tar.bz2 > or > > > http://pypi.python.org/packages/2.6/T/Twisted/Twisted-12.2.0.win32-py2.6.exe > or > > > http://pypi.python.org/packages/2.6/T/Twisted/Twisted-12.2.0.win32-py2.6.msi > or > > > http://pypi.python.org/packages/2.7/T/Twisted/Twisted-12.2.0.win32-py2.7.exe > or > > > http://pypi.python.org/packages/2.7/T/Twisted/Twisted-12.2.0.win32-py2.7.msi > > > Thanks to the supporters of Twisted and to the many contributors for this > release. > > > -- > Ashwini Oruganti > > ___ > Twisted-Python mailing list > [email protected] > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > -- cheers lvh -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging handler: No output
Florian Lindner xgm.de> writes: > But neither the FileHandler nor the StreamHandler produce any actual output. > The file is being created but stays empty. If I use a print output in the > while loop it works, so output is catched and the applications stdout in > working. But why the logger proclog catching nothing? > Paul Rubin's answer looks correct. In addition, you should note that every call to start_process will add a new handler to the logger (I can't tell if the logger could be the same on multiple calls, but it seems likely) and that may produce multiple messages. The rule of thumb is: most code should get loggers and log to them, but adding handlers, setting levels etc. should be only done in one place (typically invoked from a "if __name__ == '__main__'" clause. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
newbie ``print`` question
I can't figure out how to stop the "add a space at the beginning"
behavior of the print function.
>>> print 1,;print 2,
1 2
See the space in between the 1 and the 2 at the output print to the
command console?
The help for print is:
"A space is written before each object is (converted and) written,
unless the output system believes it is positioned at the beginning of
a line."
So it is apparently doing what it is supposed to do.
Is there a way to stop this? Or is there a different function that
will only print what you have in the formatted string?
For example, in MATLAB I only had to do:
>> fprintf('1');fprintf('2')
12
fprintf works the same way if printing to a file. It only puts in
what you explicitly tell it to put in.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 02/09/2012 18:23, gwhite wrote:
I can't figure out how to stop the "add a space at the beginning"
behavior of the print function.
print 1,;print 2,
1 2
See the space in between the 1 and the 2 at the output print to the
command console?
The help for print is:
"A space is written before each object is (converted and) written,
unless the output system believes it is positioned at the beginning of
a line."
So it is apparently doing what it is supposed to do.
Is there a way to stop this? Or is there a different function that
will only print what you have in the formatted string?
For example, in MATLAB I only had to do:
fprintf('1');fprintf('2')
12
fprintf works the same way if printing to a file. It only puts in
what you explicitly tell it to put in.
I'd settle for print 12, YMMV.
--
Cheers.
Mark Lawrence.
--
http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Monday, August 27, 2012 2:17:45 PM UTC-5, Ian wrote:
> On Thu, Aug 23, 2012 at 10:49 AM, Aaron Brady wrote:
>
> > The patch for the above is only 40-60 lines. However it introduces two new
> > concepts.
>
>
>
> Is there a link to the patch?
Please see below. It grew somewhat during development.
> > The first is a "linked list".
SNIP.
> > The second is "uncounted references". The uncounted references are
> > references to "set iterators" exclusively, exist only internally to "set"
> > objects, and are invisible to the rest of the program. The reason for the
> > exception is that iterators are unique in the Python Data Model; iterators
> > consist of a single immutable reference, unlike both immutable types such
> > as strings and numbers, as well as container types. Counted references
> > could be used instead, but would be consistently wasted work for the
> > garbage collector, though the benefit to programmers' peace of mind could
> > be significant.
>
> >
>
> > Please share your opinion! Do you agree that the internal list resolves
> > the inconsistency? Do you agree with the strategy? Do you agree that
> > uncounted references are justified to introduce, or are counted references
> > preferable?
>
>
>
> This feature is a hard sell as it is; I think that adding uncounted
>
> references into the mix is only going to make that worse. May I
>
> suggest an alternate approach? Internally tag each set or dict with a
>
> "version", which is just a C int. Every time the hash table is
>
> modified, increment the version. When an iterator is created, store
>
> the current version on the iterator. When the iterator is advanced,
>
> check that the iterator version matches the dict/set version. If
>
> they're not equal, raise an error.
>
>
>
> This should add less overhead than the linked list without any
>
> concerns about reference counting. It does introduce a small bug in
>
> that an error condition could be "missed", if the version is
>
> incremented a multiple of 2**32 or 2**64 times between iterations --
>
> but how often is that really likely to occur? Bearing in mind that
>
> this error is meant for debugging and not production error handling,
>
> you could even make the version a single byte and I'd still be fine
>
> with that.
>
>
>
> Cheers,
>
> Ian
Hi Ian,
We could use a Python long object for the version index to prevent overflow.
Combined with P. Rubin's idea to count the number of open iterators, most use
cases still wouldn't exceed a single word comparison; we could reset the
counter when there weren't any. Using the linked list collection, modification
operations are expensive in rare cases. Using the version index, iteration is
expensive in rare cases.
I was more interested in the linked list for conceptual reasons, so I developed
it further. Changelog, diff file, test suite, and links are below. The devs
should be aware that a competing patch might be developed. I would be pleased
to hear what everybody thinks of it!
Linked list with uncounted references implementation for Win32.
Added:
- 'set_clear_ex' and 'set_clear_internal_ex' methods, differ in invalidation
and conditional invalidation behavior and return type.. The 'set.clear()'
method and 'tp_clear' type field both called the same method.
- 'set_invalidate_iter_linked' method. Iterate over the iterators of a set,
mark them invalid, and clear the list.
- 'setiter_unlink_internal' method. Remove the iterator from the set's linked
list of iterators.
- 'IterationError', global.
- New fields:
-- PySetObject: setiterobject *iter_linked. Pointer to the first element of
the linked list of the iterators of the set.
-- setiterobject: setiterobject *linked_pred, *linked_succ. Predecessor and
successor nodes in the linked list of iterators of the same set.
-- setiterobject: char ob_valid. Validation status of the iterator.
- Result is compared with original in 'set_intersection_update' and '_multi' to
determine whether to invalidate the list of iterators. Asymptotic running time
is unchanged.
- Pending: add 'tp_clear' field to 'PySetIter_Type'?
- Test script included, 'large numbers' test pending.
6 files changed: { setobject.h, setobject.c, exceptions.c, pyerrors.h,
python3.def, python33stub.def }. Test script 'set_iterator_test.py' new.
Linked list interface and pseudocode 'patch_pseudocode.txt'.
Zip file:
http://home.comcast.net/~castironpi-misc/clpy-0062-set_iterator_patch.zip
Diff file of 3.3.0b2:
http://home.comcast.net/~castironpi-misc/clpy-0062-set_iterator_diff.txt
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sun, Sep 2, 2012 at 1:23 PM, gwhite wrote: > I can't figure out how to stop the "add a space at the beginning" > behavior of the print function. > print 1,;print 2, > 1 2 > > See the space in between the 1 and the 2 at the output print to the > command console? > > The help for print is: > > "A space is written before each object is (converted and) written, > unless the output system believes it is positioned at the beginning of > a line." > > So it is apparently doing what it is supposed to do. > > Is there a way to stop this? Or is there a different function that > will only print what you have in the formatted string? You can do it with string formatting. My example shows 'old style' string formatting syntax. There is a newer style. You can learn about both from the online python docs. >>> print "%d%d" % (1,2) 12 >>> -- Joel Goldstick -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
Am Sun, 02 Sep 2012 10:23:53 -0700 schrieb gwhite:
> I can't figure out how to stop the "add a space at the beginning"
> behavior of the print function.
>
print 1,;print 2,
> 1 2
>
> See the space in between the 1 and the 2 at the output print to the
> command console?
>
> The help for print is:
>
> "A space is written before each object is (converted and) written,
> unless the output system believes it is positioned at the beginning of a
> line."
>
> So it is apparently doing what it is supposed to do.
>
> Is there a way to stop this? Or is there a different function that will
> only print what you have in the formatted string?
>
> For example, in MATLAB I only had to do:
>
>>> fprintf('1');fprintf('2')
> 12
>
> fprintf works the same way if printing to a file. It only puts in
> what you explicitly tell it to put in.
import sys
sys.stdout.write("1"); sys.stdout.write("2")
HTH
Martin
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sun, Sep 2, 2012 at 10:23 AM, gwhite wrote: > I can't figure out how to stop the "add a space at the beginning" > behavior of the print function. > print 1,;print 2, > 1 2 > > See the space in between the 1 and the 2 at the output print to the > command console? > > The help for print is: > > "A space is written before each object is (converted and) written, > unless the output system believes it is positioned at the beginning of > a line." > > So it is apparently doing what it is supposed to do. > > Is there a way to stop this? If you were to use Python 3.x, yes. Otherwise, no. > Or is there a different function that > will only print what you have in the formatted string? Use the .write() method of the sys.stdout file object. http://docs.python.org/library/sys.html#sys.stdout Alternatively, you can stick with `print` and rework your code so that it outputs an entire line at a time (thus, there'd only be 1 argument passed to `print`, so its "spaces between arguments" feature wouldn't come into play). Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: [2.5.1] Read each line from txt file, replace, and save?
On 9/2/2012 6:36 AM, Gilles wrote:
On Sun, 02 Sep 2012 12:19:02 +0200, Gilles wrote:
(snip)
Found it:
#rewrite lines to new file
output = open('output.txt','w')
for line in textlines:
#edit each line
line = "just a test"
output.write("%s" % line)
output.close()
If you process each line separately, there is no reason to read them all
at once. Use the file as an iterator directly. Since line is already a
string, there is no reason to copy it into a new string. Combining these
two changes with Mark's suggestion to use with and we have the following
simple code:
with open('input.txt', 'r') as inp, open('output.txt', 'w') as out:
for line in inp:
out.write(process(line))
where for your example, process(line) == 'just a test\n'
(you need explicit line ending for .write())
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Changing a Value in List of lists
On 9/2/2012 7:17 AM, Mark Lawrence wrote: On 02/09/2012 12:05, Rishabh Dixit wrote: Hi, I got it :). We should create a new list every time before adding to a list of list so that instead repeating the reference a new reference generated every time. Please don't top post on this list as it often destroys the context, thanks. In this particular case, OP could and should have trimmed off everything other than the top post, as it stands alone. Then it would not have been a top post ;-). -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 9/2/2012 1:23 PM, gwhite wrote: I can't figure out how to stop the "add a space at the beginning" behavior of the print function. print 1,;print 2, 1 2 You have discovered why print is a function in 3.x. >>> print(1, 2, sep='') 12 >>> print(1, end=''); print(2, end='') 12 In 2.6 or 2.7, you can add from __future__ import print_function but I recommend using 3.2 or 3.3 unless you have a reason to use older Python. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On 2 September 2012 10:49, Arnaud Delobelle wrote: > On 2 September 2012 10:39, Alec Taylor wrote: >> http://pypi.python.org/pypi/lupa > > I'll check this out, thanks. Mmh it seems to be lua 5.1 and more importantly it seems to require a custom build of Python, which I don't want. So it seems the simplest solution is to run Lua in a child process... -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
Arnaud Delobelle, 02.09.2012 20:34: > On 2 September 2012 10:49, Arnaud Delobelle wrote: >> On 2 September 2012 10:39, Alec Taylor wrote: >>> http://pypi.python.org/pypi/lupa >> >> I'll check this out, thanks. > > Mmh it seems to be lua 5.1 and more importantly it seems to require a > custom build of Python, which I don't want. I have no idea why you say that, for neither of the two. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 10:49 am, mblume wrote:
> Am Sun, 02 Sep 2012 10:23:53 -0700 schrieb gwhite:
>
>
>
>
>
>
>
>
>
> > I can't figure out how to stop the "add a space at the beginning"
> > behavior of the print function.
>
> print 1,;print 2,
> > 1 2
>
> > See the space in between the 1 and the 2 at the output print to the
> > command console?
>
> > The help for print is:
>
> > "A space is written before each object is (converted and) written,
> > unless the output system believes it is positioned at the beginning of a
> > line."
>
> > So it is apparently doing what it is supposed to do.
>
> > Is there a way to stop this? Or is there a different function that will
> > only print what you have in the formatted string?
>
> > For example, in MATLAB I only had to do:
>
> >>> fprintf('1');fprintf('2')
> > 12
>
> > fprintf works the same way if printing to a file. It only puts in
> > what you explicitly tell it to put in.
>
> import sys
>
> sys.stdout.write("1"); sys.stdout.write("2")
>
> HTH
> Martin
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 10:55 am, Chris Rebert wrote: > On Sun, Sep 2, 2012 at 10:23 AM, gwhite wrote: > > I can't figure out how to stop the "add a space at the beginning" > > behavior of the print function. > > print 1,;print 2, > > 1 2 > > > See the space in between the 1 and the 2 at the output print to the > > command console? > > > The help for print is: > > > "A space is written before each object is (converted and) written, > > unless the output system believes it is positioned at the beginning of > > a line." > > > So it is apparently doing what it is supposed to do. > > > Is there a way to stop this? > > If you were to use Python 3.x, yes. Otherwise, no. > > > Or is there a different function that > > will only print what you have in the formatted string? > > Use the .write() method of the sys.stdout file > object.http://docs.python.org/library/sys.html#sys.stdout > > Alternatively, you can stick with `print` and rework your code so that > it outputs an entire line at a time (thus, there'd only be 1 argument > passed to `print`, so its "spaces between arguments" feature wouldn't > come into play). > > Cheers, > Chris Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
Le dimanche 2 septembre 2012 11:07:35 UTC+2, Ian a écrit :
> On Sun, Sep 2, 2012 at 1:36 AM, wrote:
>
> > I still remember my thoughts when I read the PEP 393
>
> > discussion: "this is not logical", "they do no understand
>
> > typography", "atomic character ???", ...
>
>
>
> That would indicate one of two possibilities. Either:
>
>
>
> 1) Everybody in the PEP 393 discussion except for you is clueless
>
> about how to implement a Unicode type; or
>
>
>
> 2) You are clueless about how to implement a Unicode type.
>
>
>
> Taking into account Occam's razor, and also that you seem to be unable
>
> or unwilling to offer a solid rationale for those thoughts, I have to
>
> say that I'm currently leaning toward the second possibility.
>
>
>
>
>
> > Real world exemples.
>
> >
>
> import libfrancais
>
> li = ['noël', 'noir', 'nœud', 'noduleux', \
>
> > ... 'noétique', 'noèse', 'noirâtre']
>
> r = libfrancais.sortfr(li)
>
> r
>
> > ['noduleux', 'noël', 'noèse', 'noétique', 'nœud', 'noir',
>
> > 'noirâtre']
>
>
>
> libfrancais does not appear to be publicly available. It's not listed
>
> in PyPI, and googling for "python libfrancais" turns up nothing
>
> relevant.
>
>
>
> Rewriting the example to use locale.strcoll instead:
>
>
>
> >>> li = ['noël', 'noir', 'nœud', 'noduleux', 'noétique', 'noèse', 'noirâtre']
>
> >>> import locale
>
> >>> locale.setlocale(locale.LC_ALL, 'French_France')
>
> 'French_France.1252'
>
> >>> import functools
>
> >>> sorted(li, key=functools.cmp_to_key(locale.strcoll))
>
> ['noduleux', 'noël', 'noèse', 'noétique', 'nœud', 'noir', 'noirâtre']
>
>
>
> # Python 3.2
>
> >>> import timeit
>
> >>> timeit.repeat("sorted(li, key=functools.cmp_to_key(locale.strcoll))",
> >>> "import functools; import locale; li = ['noël', 'noir', 'nœud',
> >>> 'noduleux', 'noétique', 'noèse', 'noirâtre']", number=1)
>
> [0.5544277025009592, 0.5370117249557325, 0.5551836677925053]
>
>
>
> # Python 3.3
>
> >>> import timeit
>
> >>> timeit.repeat("sorted(li, key=functools.cmp_to_key(locale.strcoll))",
> >>> "import functools; import locale; li = ['noël', 'noir', 'nœud',
> >>> 'noduleux', 'noétique', 'noèse', 'noirâtre']", number=1)
>
> [0.1421166788364303, 0.12389078130001963, 0.13184190553613462]
>
>
> As you can see, Python 3.3 is about 77% faster than Python 3.2 on this
>
> example. If this was intended to show that the Python 3.3 Unicode
>
> representation is a regression over the Python 3.2 implementation,
>
> then it's a complete failure as an example.
- Unfortunately, I got opposite and even much worst results on my win box,
considering
- libfrancais is one of my module and it does a little bit more than
the std sorting tools.
My rationale: very simple.
1) I never heard about something better than sticking with one
of the Unicode coding scheme. (genreral theory)
2) I am not at all convinced by the "new" Py 3.3 algorithm. I'm not the
only one guy, who noticed problems. Arguing, "it is fast enough", is not
a correct answer.
jmf
--
http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
On 2 September 2012 19:42, Stefan Behnel wrote: > Arnaud Delobelle, 02.09.2012 20:34: >> On 2 September 2012 10:49, Arnaud Delobelle wrote: >>> On 2 September 2012 10:39, Alec Taylor wrote: http://pypi.python.org/pypi/lupa >>> >>> I'll check this out, thanks. >> >> Mmh it seems to be lua 5.1 and more importantly it seems to require a >> custom build of Python, which I don't want. > > I have no idea why you say that, for neither of the two. * For the custom build of Python on OS X, this blog post: http://t-p-j.blogspot.co.uk/2010/11/lupa-on-os-x-with-macports-python-26.html which is actually linked to in lupa's own INSTALL.rst file * For Lua 5.1, the official LuaJIT page: http://luajit.org/luajit.html where I quote: """ LuaJIT implements the full set of language features defined by Lua 5.1. """ Of course I'd be glad to be proved wrong on both counts. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 11:33 am, Terry Reedy wrote: > On 9/2/2012 1:23 PM, gwhite wrote: > > > I can't figure out how to stop the "add a space at the beginning" > > behavior of the print function. > > print 1,;print 2, > > 1 2 > > You have discovered why print is a function in 3.x. > >>> print(1, 2, sep='') > 12 > >>> print(1, end=''); print(2, end='') > 12 > > In 2.6 or 2.7, you can add > from __future__ import print_function > but I recommend using 3.2 or 3.3 unless you have a reason to use older > Python. > > -- > Terry Jan Reedy Thanks. I would use 3.2 or 3.3, but I am actually using pythonxy, which hasn't quite gotten there yet, as the bundle of packages has to all play nice together. Or so I am guessing. I have been considering the __future__ technique. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to interact with Windows cmd?
Not sure but have you tried using functions from the os library? For example you could use execl() to execute your commands and pipe the output somewhere-stdin in your case. -- http://mail.python.org/mailman/listinfo/python-list
Re: running Lua in Python
Arnaud Delobelle, 02.09.2012 21:17: > On 2 September 2012 19:42, Stefan Behnel wrote: >> Arnaud Delobelle, 02.09.2012 20:34: >>> On 2 September 2012 10:49, Arnaud Delobelle wrote: On 2 September 2012 10:39, Alec Taylor wrote: > http://pypi.python.org/pypi/lupa I'll check this out, thanks. >>> >>> Mmh it seems to be lua 5.1 and more importantly it seems to require a >>> custom build of Python, which I don't want. >> >> I have no idea why you say that, for neither of the two. > > * For the custom build of Python on OS X, this blog post: > > > http://t-p-j.blogspot.co.uk/2010/11/lupa-on-os-x-with-macports-python-26.html > > which is actually linked to in lupa's own INSTALL.rst file Ah, ok, but that's a) a rather old blog post and it b) refers to LuaJIT, not plain Lua. Just because LuaJIT requires special build flags doesn't mean that Lupa or Lua do. > * For Lua 5.1, the official LuaJIT page: > > http://luajit.org/luajit.html > > where I quote: > > """ > LuaJIT implements the full set of language features defined by Lua 5.1. > """ > > Of course I'd be glad to be proved wrong on both counts. I was told that it works with Lua 5.1 and didn't try it with Lua 5.2 myself. But I think you should, if that's what you want to use. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 10:55 am, Chris Rebert wrote: > On Sun, Sep 2, 2012 at 10:23 AM, gwhite wrote: > > I can't figure out how to stop the "add a space at the beginning" > > behavior of the print function. > > print 1,;print 2, > > 1 2 > > > See the space in between the 1 and the 2 at the output print to the > > command console? > > > The help for print is: > > > "A space is written before each object is (converted and) written, > > unless the output system believes it is positioned at the beginning of > > a line." > > > So it is apparently doing what it is supposed to do. > > > Is there a way to stop this? > > If you were to use Python 3.x, yes. Otherwise, no. > > > Or is there a different function that > > will only print what you have in the formatted string? > > Use the .write() method of the sys.stdout file > object.http://docs.python.org/library/sys.html#sys.stdout > > Alternatively, you can stick with `print` and rework your code so that > it outputs an entire line at a time (thus, there'd only be 1 argument > passed to `print`, so its "spaces between arguments" feature wouldn't > come into play). On the "rework" thing, yes, I suppose I could construct the line as a single string prior to print.There would be things like `for` loops and conditionals to do so. That isn't so unusual. I was actually doing some simple tests of other things -- you know, newbie stuff to make sure I understood what the various ops were doing. I wasn't even thinking about `print` Then I got sidetracked with this item, which I thought was very odd, since it is "deciding for you" to stuff a space in. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 12:26 pm, gwhite wrote: > On Sep 2, 10:55 am, Chris Rebert wrote: > > > > > > > > > > > On Sun, Sep 2, 2012 at 10:23 AM, gwhite wrote: > > > I can't figure out how to stop the "add a space at the beginning" > > > behavior of the print function. > > > print 1,;print 2, > > > 1 2 > > > > See the space in between the 1 and the 2 at the output print to the > > > command console? > > > > The help for print is: > > > > "A space is written before each object is (converted and) written, > > > unless the output system believes it is positioned at the beginning of > > > a line." > > > > So it is apparently doing what it is supposed to do. > > > > Is there a way to stop this? > > > If you were to use Python 3.x, yes. Otherwise, no. > > > > Or is there a different function that > > > will only print what you have in the formatted string? > > > Use the .write() method of the sys.stdout file > > object.http://docs.python.org/library/sys.html#sys.stdout > > > Alternatively, you can stick with `print` and rework your code so that > > it outputs an entire line at a time (thus, there'd only be 1 argument > > passed to `print`, so its "spaces between arguments" feature wouldn't > > come into play). > > On the "rework" thing, yes, I suppose I could construct the line as a > single string prior to print. There would be things like `for` > loops and conditionals to do so. That isn't so unusual. > > I was actually doing some simple tests of other things -- you know, > newbie stuff to make sure I understood what the various ops were > doing. I wasn't even thinking about `print` Then I got sidetracked > with this item, which I thought was very odd, since it is "deciding > for you" to stuff a space in. btw, I also thought the default "add a CR LF" to the end was odd too. But at least that one had a simple way out. -- http://mail.python.org/mailman/listinfo/python-list
Re: PipeController v0.1 - experimental tool to simulate simple UNIX-style pipes in Python
On Saturday, September 1, 2012 9:02:33 PM UTC+5:30, Ramchandra Apte wrote: > On Friday, 31 August 2012 03:27:54 UTC+5:30, vasudevram wrote: > > > I wrote PipeController recently to experiment with doing UNIX-style pipes > > in Python. > > > Doesn't the pipes module already do this? Yes. As Ian Kelly points out here: http://www.mail-archive.com/[email protected]/msg335943.html the pipes module of Python does deal with actual UNIX pipes. But I had mentioned this - Python's pipes module - in my first post, some months ago, about doing UNIX-pipes in Python ( on my blog, here: http://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-style-pipes-in.html ), and that first post was in turn linked to, in the current post under discussion ( i.e. this one: http://jugad2.blogspot.in/2012/08/pipecontroller-v01-released-simulating.html ). To Ian Kelly: > No, that deals with actual Unix pipes. This appears to be about pipelined > processing within a single program and not IPC; the description "Unix-like" > is a bit misleading, IMO. I guess it can be interpreted as a bit misleading, but it was not intentionally so. The way I meant it was that PipeController tries to achieve _roughly_ similar functionality, of composing a program out of components, as UNIX pipes do. (That too, only a small subset, as I mention in my blog post). In the case of UNIX the components are commands, in the case of my Python approach the components are functions. Also, I used the term "UNIX-style pipes", not "UNIX pipes", to stress that it was the overall concept of a pipeline that was being simulated, not the exact design / implementation details, and you can also notice that my first post has links to many different third-party ways of simulating UNIX-pipes in Python, some of which are not like actual UNIX pipes at all, in their implementation. But I agree that your description of PipeController as "pipelined processing within a single program" is more accurate. I could have used a better choice of words but there was no intention to mislead. - Vasudev Ram www.dancingbison.com jugad2.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: stuck in files!!
The best you can do is pipe a string to a .txt document then pipe it again to a wordpad document. You should be able to do that easily with file and open() -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 09/02/2012 03:34 PM, gwhite wrote: > > > btw, I also thought the default "add a CR LF" to the end was odd too. > But at least that one had a simple way out. But it (print on Python 2.x) doesn't, unless you're stuck on Windows. And even then, you can prevent it by using a 'b' in the mode. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 09/02/2012 12:58 PM, [email protected] wrote: > My rationale: very simple. > > 1) I never heard about something better than sticking with one > of the Unicode coding scheme. (genreral theory) > 2) I am not at all convinced by the "new" Py 3.3 algorithm. I'm not the > only one guy, who noticed problems. Arguing, "it is fast enough", is not > a correct answer. If this is true, why were you holding ho Google Go as an example of doing it right? Certainly Google Go doesn't line up with your rational. Go has both Strings and Runes. But strings are UTF-8-encoded bytes strings and Runes are 32-bit integers. They are not interchangeable without a costly encoding and decoding process. Even worse, indexing a Go string to get a "Rune" involves some very costly decoding that has to be done starting at the beginning of the string each time. In the worst case, Python's strings are as slow as Go because Python does the exact same thing as Go, but chooses between three encodings instead of just one. Best case scenario, Python's strings could be much faster than Go's because indexing through 2 of the 3 encodings is O(1) because they are constant-width encodings. If as you say, the latin-1 subset of UTF-8 is used, then UTF-8 indexing is O(1) too, otherwise it's probably O(n). -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 10:45 am, Joel Goldstick wrote: > On Sun, Sep 2, 2012 at 1:23 PM, gwhite wrote: > > I can't figure out how to stop the "add a space at the beginning" > > behavior of the print function. > > print 1,;print 2, > > 1 2 > > > See the space in between the 1 and the 2 at the output print to the > > command console? > > > The help for print is: > > > "A space is written before each object is (converted and) written, > > unless the output system believes it is positioned at the beginning of > > a line." > > > So it is apparently doing what it is supposed to do. > > > Is there a way to stop this? Or is there a different function that > > will only print what you have in the formatted string? > > You can do it with string formatting. My example shows 'old style' > string formatting syntax. There is a newer style. You can learn > about both from the online python docs. > > > > >>> print "%d%d" % (1,2) > 12 For "real" stuff, I've been trying to use that. I saw it in Beazley's latest Essential text. You're right, I can construct a single string and then write it. In fact, it looks like it is a must unless using the 3.2/3.3 version of `print`. -- http://mail.python.org/mailman/listinfo/python-list
Re: stuck in files!!
On 09/02/2012 03:36 PM, me wrote: > The best you can do is pipe a string to a .txt document then pipe it again to > a wordpad document. You should be able to do that easily with file and open() > If your reply is going to break the thread, and if the thread has been stale almost a month, then it seems the least you could do would be to quote some portion of the message you're replying to. As it stands, your message refers to piping a string to two documents, which is patently impractical in Bash. Better switch to Frod. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: How to print a number as if in the python interpreter?
Use repr() print(repr(sum([.1, .1, .1, .1, .1, .1, .1, .1, .1, .1]))) -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 12:43 pm, Dave Angel wrote: > On 09/02/2012 03:34 PM, gwhite wrote: > > > > > > btw, I also thought the default "add a CR LF" to the end was odd too. > > But at least that one had a simple way out. > > But it (print on Python 2.x) doesn't, unless you're stuck on Windows. > And even then, you can prevent it by using a 'b' in the mode. Yes, I'm using windows. What is "'b' in the mode?" The help for print says: A ``'\n'`` character is written at the end, unless the ``print`` statement ends with a comma. This is the only action if the statement contains just the keyword ``print``. So I followed with a comma to stop the default CR LF insertion. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
Well you can convert the ints to str then concatenate them. print "1" + "2" -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 09/02/2012 03:45 PM, Michael Torrie wrote: > : > In the worst case, Python's strings are as slow as Go because Python > does the exact same thing as Go, but chooses between three encodings > instead of just one. Best case scenario, Python's strings could be > much faster than Go's because indexing through 2 of the 3 encodings is > O(1) because they are constant-width encodings. If as you say, the > latin-1 subset of UTF-8 is used, then UTF-8 indexing is O(1) too, > otherwise it's probably O(n). I'm afraid you have it backwards. the Utf-8 version of the latin-1-compatible characters would be variable length. But my understanding of the pep is that the internal one-byte format is simply the lowest order byte of each code point, after assuring that all code points in the particular string are less than 256. That's going to coincidentally resemble latin-1's encoding, but since it's an internal form, the resemblance is irrelevant. Anyway, those one-byte values are going to be O(1), naturally. No encoding involved, and no searching nor expanding. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: stuck in files!!
Didn't see the post date on my reader haha. Sorry about that. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 11:33 am, Terry Reedy wrote:
> On 9/2/2012 1:23 PM, gwhite wrote:
>
> > I can't figure out how to stop the "add a space at the beginning"
> > behavior of the print function.
>
> print 1,;print 2,
> > 1 2
>
> You have discovered why print is a function in 3.x.
> >>> print(1, 2, sep='')
> 12
> >>> print(1, end=''); print(2, end='')
> 12
>
> In 2.6 or 2.7, you can add
> from __future__ import print_function
> but I recommend using 3.2 or 3.3 unless you have a reason to use older
> Python.
It looks like one can avoid pre-construction of the total line with
3.x.
>>> from __future__ import print_function
>>> print('a=%.1f,' % 1.0, end='');print('b=%.1f' % 2.0, end='')
a=1.0,b=2.0
Thanks. It isn't that hard to pre-construct, but regardless, there is
better control in 3.x.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 09/02/2012 03:50 PM, gwhite wrote: > On Sep 2, 12:43 pm, Dave Angel wrote: >> On 09/02/2012 03:34 PM, gwhite wrote: >> >>> >>> btw, I also thought the default "add a CR LF" to the end was odd too. >>> But at least that one had a simple way out. >> But it (print on Python 2.x) doesn't, unless you're stuck on Windows. >> And even then, you can prevent it by using a 'b' in the mode. > Yes, I'm using windows. What is "'b' in the mode?" The help for > print says: > > A ``'\n'`` character is written at the end, unless the ``print`` > statement ends with a comma. This is the only action if the statement > contains just the keyword ``print``. > > So I followed with a comma to stop the default CR LF insertion. > You're correct; the best way to suppress the newline at the end of print is to use the trailing comma. But since print is for lines, it usually is a good default. If you don't want to get any extra characters, just use write(). It takes a string, and outputs exactly what it's given. I assumed you were complaining about the conversion of newline to carriage-return-newline, which is done by default on Windows, and can be suppressed by opening the file with "b" as the mode parameter. -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 02/09/2012 20:58, me wrote: Well you can convert the ints to str then concatenate them. print "1" + "2" Please post other parts of the thread so people can get the context or don't bother posting at all, thanks. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sun, Sep 2, 2012 at 1:24 PM, Mark Lawrence wrote: > On 02/09/2012 20:58, me wrote: >> >> Well you can convert the ints to str then concatenate them. >> >> print "1" + "2" >> > > Please post other parts of the thread so people can get the context or don't > bother posting at all, thanks. Please also adjust your mail client so that your "From" is more informative and less confusing than simply "me". Regards, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 9/2/2012 3:45 PM, Michael Torrie wrote: In the worst case, Python's strings are as slow as Go because Python does the exact same thing as Go, but chooses between three encodings instead of just one. Best case scenario, Python's strings could be much faster than Go's because indexing through 2 of the 3 encodings is O(1) In CPython 3.3, indexing of str text string objects is always O(1) and it is always indexes and counts code points rather than code units. It was the latter for narrow builds in 3.2 and before. As a result, single character (code point) strings had a length of 2 rather than 1 for extended plane characters. 3.3 corrects this. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: stuck in files!!
On 09/02/2012 04:07 PM, me wrote: > Didn't see the post date on my reader haha. Sorry about that. > Well, i see you're also posting responses to recent messages. Great. But please use reply with a proper mail program; your messages are all brand new threads. And could you fix your email address? You're apparently hijacking someone else's domain. If you want to stay anonymous, use the invalid domain, as it's reserved for such purposes. Try me@invalid -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 30.08.12 09:55, Steven D'Aprano wrote: And Python's solution uses those: UCS-2, UCS-4, and UTF-8. I see that this misconception widely spread. In fact Python 3.3 uses four kinds of ready strings. * ASCII. All codes <= U+007F. * UCS1. All codes <= U+00FF, at least one code > U+007F. * UCS2. All codes <= U+, at least one code > U+00FF. * UCS4. All codes <= U+0010, at least one code > U+. Indexing is O(0) for any string. Also the string can optionally cache UTF-8 and wchar_t* representation. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 9/2/2012 3:26 PM, gwhite wrote:
On the "rework" thing, yes, I suppose I could construct the line as a
single string prior to print.There would be things like `for`
loops and conditionals to do so. That isn't so unusual.
The usual idiom is to construct a list of pieces and then join with ''.
>>> print(''.join(['1', '2']))
12
Or map str to a list of objects.
>>> print(''.join(map(str, [1, 2])))
12
You can do either of these in 2.x.
If you use .write, include '\n' at the end of the list (when needed).
Print was designed as a quick and easy way to put lines of text on the
screen. Then people asked for a way to use with with other streams,
hence the >> hack. Then people wanted ways to control the separator and
terminator. As that point, Guido realized that it needed to be a
function, not a statement, with all the options requested.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: PipeController v0.1 - experimental tool to simulate simple UNIX-style pipes in Python
On Monday, September 3, 2012 1:05:03 AM UTC+5:30, vasudevram wrote: > > To Ian Kelly: > > No, that deals with actual Unix pipes. This appears to be about pipelined > > processing within a single program and not IPC; the description "Unix-like" > > is a bit misleading, IMO. > I guess it can be interpreted as a bit misleading, but it was not > intentionally so. The way I meant it was that PipeController tries to achieve > _roughly_ similar functionality, of composing a program out of components, as > UNIX pipes do. To Ian again: Also, I used the word "synchronous" somewhat deliberately (see in my blog post: "PipeController is a tool to experiment with a simple, sequential, synchronous simulation of UNIX-style pipes in Python."). I used the word synchronous to indicate that my PipeController implementation runs the functions one after another (for each item in the input), in contrast to actual Unix pipes, where the different commands in a pipeline can, and IIRC, do, run in parallel / asynchronously, with their IPC being coordinated / managed by the kernel and shell. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 1:16 pm, Dave Angel wrote:
> On 09/02/2012 03:50 PM, gwhite wrote:
>
>
>
>
>
>
>
>
>
> > On Sep 2, 12:43 pm, Dave Angel wrote:
> >> On 09/02/2012 03:34 PM, gwhite wrote:
>
> >>>
> >>> btw, I also thought the default "add a CR LF" to the end was odd too.
> >>> But at least that one had a simple way out.
> >> But it (print on Python 2.x) doesn't, unless you're stuck on Windows.
> >> And even then, you can prevent it by using a 'b' in the mode.
> > Yes, I'm using windows. What is "'b' in the mode?" The help for
> > print says:
>
> > A ``'\n'`` character is written at the end, unless the ``print``
> > statement ends with a comma. This is the only action if the statement
> > contains just the keyword ``print``.
>
> > So I followed with a comma to stop the default CR LF insertion.
>
> You're correct; the best way to suppress the newline at the end of
> print is to use the trailing comma. But since print is for lines, it
> usually is a good default. If you don't want to get any extra
> characters, just use write(). It takes a string, and outputs exactly
> what it's given.
>
> I assumed you were complaining about the conversion of newline to
> carriage-return-newline, which is done by default on Windows, and can be
> suppressed by opening the file with "b" as the mode parameter.
Sorry, I was a little vague on the newline stuff.
In any case, I've learned I should probably avoid the comma, if
looking at 3.x:
>>> from __future__ import print_function
>>> print('a=%.1f,' % 1.0),;print('b=%.1f' % 2.0)
a=1.0,
(None,)
b=2.0
Given the input of several posters, I feel like I should probably be
using the `write` function anyway. (I don't have a problem pre-
constructing strings either.)
But I am a newbie, and "all" the show-and-tell examples/tutorials use
`print`. But this little thread has helped me a lot.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 1:37 pm, Dennis Lee Bieber wrote:
> On Sun, 2 Sep 2012 10:23:53 -0700 (PDT), gwhite
> declaimed the following in gmane.comp.python.general:
>
>
>
> > "A space is written before each object is (converted and) written,
> > unless the output system believes it is positioned at the beginning of
> > a line."
>
> > So it is apparently doing what it is supposed to do.
>
> > Is there a way to stop this? Or is there a different function that
> > will only print what you have in the formatted string?
>
> E:\UserData\Wulfraed\My Documents>python
> ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
> Python 2.7.2 (default, Jun 24 2011, 12:21:10) [MSC v.1500 32 bit
> (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> import sys
> >>> print "1",;sys.stdout.softspace=0;print "2"
> 12
>
> >>> sys.stdout.write("1");sys.stdout.write("2");print
> 12
>
> In Python 2.x, the equivalent of fprint/fprintf is NOT print but
> sys.stdout.write/file_object.write
>
> "print" is just a convenience function that is designed to translate
> its arguments into a textual representation, and then drop it to the
> screen. Part of the convenience is to separate output items by a space
> and to emit a newline at the end.
>
> .write(), OTOH, does no formatting -- not even new lines. You have
> to provide all conversion to text, and line endings.
Thanks. I am starting to get it.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 1:49 pm, Terry Reedy wrote:
> On 9/2/2012 3:26 PM, gwhite wrote:
>
> > On the "rework" thing, yes, I suppose I could construct the line as a
> > single string prior to print. There would be things like `for`
> > loops and conditionals to do so. That isn't so unusual.
>
> The usual idiom is to construct a list of pieces and then join with ''.
>
> >>> print(''.join(['1', '2']))
> 12
>
> Or map str to a list of objects.
>
> >>> print(''.join(map(str, [1, 2])))
> 12
>
> You can do either of these in 2.x.
> If you use .write, include '\n' at the end of the list (when needed).
>
> Print was designed as a quick and easy way to put lines of text on the
> screen. Then people asked for a way to use with with other streams,
> hence the >> hack. Then people wanted ways to control the separator and
> terminator. As that point, Guido realized that it needed to be a
> function, not a statement, with all the options requested.
Thanks again, Terry. There is a lot to the language, I am finding
out. I am a HW engineer, not really a programmer. Python seems a lot
more sophisticated than MATLAB.
I'm kinda thinking `write` is likely to be a little more "stable" than
`print` (if that is the right characterization) when my eventual
switch from 2.7 to 3.x happens. You think?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 02.09.12 23:38, Serhiy Storchaka wrote: Indexing is O(0) for any string. Typo. O(1) -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Mon, Sep 3, 2012 at 7:18 AM, gwhite wrote: > Thanks again, Terry. There is a lot to the language, I am finding > out. I am a HW engineer, not really a programmer. Python seems a lot > more sophisticated than MATLAB. > > I'm kinda thinking `write` is likely to be a little more "stable" than > `print` (if that is the right characterization) when my eventual > switch from 2.7 to 3.x happens. You think? If you're planning to switch, make use of __future__. It's specifically to make that job easier. Once you have a future declaration at the top, print() will be stable across 2.7 and 3.x. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 02/09/2012 21:58, gwhite wrote:
On Sep 2, 1:16 pm, Dave Angel wrote:
On 09/02/2012 03:50 PM, gwhite wrote:
> On Sep 2, 12:43 pm, Dave Angel wrote:
>> On 09/02/2012 03:34 PM, gwhite wrote:
>>>
>>> btw, I also thought the default "add a CR LF" to the end was odd too.
>>> But at least that one had a simple way out.
>> But it (print on Python 2.x) doesn't, unless you're stuck on Windows.
>> And even then, you can prevent it by using a 'b' in the mode.
> Yes, I'm using windows. What is "'b' in the mode?" The help for
> print says:
> A ``'\n'`` character is written at the end, unless the ``print``
> statement ends with a comma. This is the only action if the statement
> contains just the keyword ``print``.
> So I followed with a comma to stop the default CR LF insertion.
You're correct; the best way to suppress the newline at the end of
print is to use the trailing comma. But since print is for lines, it
usually is a good default. If you don't want to get any extra
characters, just use write(). It takes a string, and outputs exactly
what it's given.
I assumed you were complaining about the conversion of newline to
carriage-return-newline, which is done by default on Windows, and can be
suppressed by opening the file with "b" as the mode parameter.
Sorry, I was a little vague on the newline stuff.
In any case, I've learned I should probably avoid the comma, if
looking at 3.x:
from __future__ import print_function
print('a=%.1f,' % 1.0),;print('b=%.1f' % 2.0)
a=1.0,
(None,)
b=2.0
Explanation:
With 'print' as a function, the first 'print' prints the result of
"'a=%.1f,' % 1.0" and then returns None. The trailing comma makes that
into a tuple (None,), which is printed by the interactive interpreter
as such.
In other words:
>>> None,
(None,)
>>>
The second prints the result of "'b=%.1f' % 2.0" and then returns None.
The interactive interpreter, recognising that it's only None, doesn't
bother to print it.
In other words:
>>> None
>>>
Given the input of several posters, I feel like I should probably be
using the `write` function anyway. (I don't have a problem pre-
constructing strings either.)
But I am a newbie, and "all" the show-and-tell examples/tutorials use
`print`. But this little thread has helped me a lot.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 2:47 pm, Chris Angelico wrote: > On Mon, Sep 3, 2012 at 7:18 AM, gwhite wrote: > > Thanks again, Terry. There is a lot to the language, I am finding > > out. I am a HW engineer, not really a programmer. Python seems a lot > > more sophisticated than MATLAB. > > > I'm kinda thinking `write` is likely to be a little more "stable" than > > `print` (if that is the right characterization) when my eventual > > switch from 2.7 to 3.x happens. You think? > > If you're planning to switch, make use of __future__. It's > specifically to make that job easier. Once you have a future > declaration at the top, print() will be stable across 2.7 and 3.x. I guess you're saying 3.x will just ignore: from __future__ import print_function I'll risk being silly, and thus ask: but what if when I get to 3.x there is no __future__, as it is now "present?" Do I need to strip out the line? What would happen when I finally started running 3.3, and a new __future__ was made that broke the old syntax? Do I need to strip out the line? I'm probably over thinking it. I don't know what I am doing. lol! -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Mon, Sep 3, 2012 at 9:20 AM, gwhite wrote: > I guess you're saying 3.x will just ignore: > > from __future__ import print_function > > I'll risk being silly, and thus ask: but what if when I get to 3.x > there is no __future__, as it is now "present?" Do I need to strip > out the line? > > What would happen when I finally started running 3.3, and a new > __future__ was made that broke the old syntax? Do I need to strip out > the line? > > I'm probably over thinking it. I don't know what I am doing. lol! The __future__ statement is guaranteed to be supported forever afterwards. And new ones won't do anything unless you explicitly put them into your code. It's quite a good system, imo. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On Sep 2, 3:43 pm, MRAB wrote:
> On 02/09/2012 21:58, gwhite wrote:
>
>
>
>
>
>
>
> > On Sep 2, 1:16 pm, Dave Angel wrote:
> >> On 09/02/2012 03:50 PM, gwhite wrote:
>
> >> > On Sep 2, 12:43 pm, Dave Angel wrote:
> >> >> On 09/02/2012 03:34 PM, gwhite wrote:
>
> >> >>>
> >> >>> btw, I also thought the default "add a CR LF" to the end was odd too.
> >> >>> But at least that one had a simple way out.
> >> >> But it (print on Python 2.x) doesn't, unless you're stuck on Windows.
> >> >> And even then, you can prevent it by using a 'b' in the mode.
> >> > Yes, I'm using windows. What is "'b' in the mode?" The help for
> >> > print says:
>
> >> > A ``'\n'`` character is written at the end, unless the ``print``
> >> > statement ends with a comma. This is the only action if the statement
> >> > contains just the keyword ``print``.
>
> >> > So I followed with a comma to stop the default CR LF insertion.
>
> >> You're correct; the best way to suppress the newline at the end of
> >> print is to use the trailing comma. But since print is for lines, it
> >> usually is a good default. If you don't want to get any extra
> >> characters, just use write(). It takes a string, and outputs exactly
> >> what it's given.
>
> >> I assumed you were complaining about the conversion of newline to
> >> carriage-return-newline, which is done by default on Windows, and can be
> >> suppressed by opening the file with "b" as the mode parameter.
>
> > Sorry, I was a little vague on the newline stuff.
>
> > In any case, I've learned I should probably avoid the comma, if
> > looking at 3.x:
>
> from __future__ import print_function
> print('a=%.1f,' % 1.0),;print('b=%.1f' % 2.0)
> > a=1.0,
> > (None,)
> > b=2.0
>
> Explanation:
>
> With 'print' as a function, the first 'print' prints the result of
> "'a=%.1f,' % 1.0" and then returns None. The trailing comma makes that
> into a tuple (None,), which is printed by the interactive interpreter
> as such.
>
> In other words:
>
> >>> None,
> (None,)
> >>>
>
> The second prints the result of "'b=%.1f' % 2.0" and then returns None.
> The interactive interpreter, recognising that it's only None, doesn't
> bother to print it.
>
> In other words:
>
> >>> None
Thanks, that makes sense.
It does look like 2.7 & 3.3 don't parse the comma the same way. Just
to repeat, since I didn't give the exact same line to both versions of
Python:
2.7
>>> print('a=%.1f,' % 1.0),;print('b=%.1f' % 2.0)
a=1.0, b=2.0
This has been more informative than I thought it was going to be.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie ``print`` question
On 03/09/2012 00:33, gwhite wrote:
On Sep 2, 3:43 pm, MRAB wrote:
On 02/09/2012 21:58, gwhite wrote:
> On Sep 2, 1:16 pm, Dave Angel wrote:
>> On 09/02/2012 03:50 PM, gwhite wrote:
>> > On Sep 2, 12:43 pm, Dave Angel wrote:
>> >> On 09/02/2012 03:34 PM, gwhite wrote:
>> >>>
>> >>> btw, I also thought the default "add a CR LF" to the end was odd too.
>> >>> But at least that one had a simple way out.
>> >> But it (print on Python 2.x) doesn't, unless you're stuck on Windows.
>> >> And even then, you can prevent it by using a 'b' in the mode.
>> > Yes, I'm using windows. What is "'b' in the mode?" The help for
>> > print says:
>> > A ``'\n'`` character is written at the end, unless the ``print``
>> > statement ends with a comma. This is the only action if the statement
>> > contains just the keyword ``print``.
>> > So I followed with a comma to stop the default CR LF insertion.
>> You're correct; the best way to suppress the newline at the end of
>> print is to use the trailing comma. But since print is for lines, it
>> usually is a good default. If you don't want to get any extra
>> characters, just use write(). It takes a string, and outputs exactly
>> what it's given.
>> I assumed you were complaining about the conversion of newline to
>> carriage-return-newline, which is done by default on Windows, and can be
>> suppressed by opening the file with "b" as the mode parameter.
> Sorry, I was a little vague on the newline stuff.
> In any case, I've learned I should probably avoid the comma, if
> looking at 3.x:
from __future__ import print_function
print('a=%.1f,' % 1.0),;print('b=%.1f' % 2.0)
> a=1.0,
> (None,)
> b=2.0
Explanation:
With 'print' as a function, the first 'print' prints the result of
"'a=%.1f,' % 1.0" and then returns None. The trailing comma makes that
into a tuple (None,), which is printed by the interactive interpreter
as such.
In other words:
>>> None,
(None,)
>>>
The second prints the result of "'b=%.1f' % 2.0" and then returns None.
The interactive interpreter, recognising that it's only None, doesn't
bother to print it.
In other words:
>>> None
Thanks, that makes sense.
It does look like 2.7 & 3.3 don't parse the comma the same way. Just
to repeat, since I didn't give the exact same line to both versions of
Python:
2.7
print('a=%.1f,' % 1.0),;print('b=%.1f' % 2.0)
a=1.0, b=2.0
This has been more informative than I thought it was going to be.
Basically:
In Python 2, 'print' is a statement.
In Python 3, 'print' is a function.
so they _would_ be parsed differently, because they _are_ different.
However, in later versions of Python 2, such as 2.7, adding "from
__future__ import print_function" will make 'print' a function like in
Python 3. (It has no effect in Python 3 because 'print' is already a
function, of course, so if you've converted a Python 2 script which
contains it to Python 3 and just happened to leave it in, it won't
complain either.)
--
http://mail.python.org/mailman/listinfo/python-list
Re: [pyxl] xlrd-0.8.0 .xlsx formatting_info=True not implemented
Albert-Jan Roskam wrote: > Hi, > > As a work-around, you could use the CRAN R package XLConnect, using > RPy or RPy2, to do what you want. IIRC it's based on Java, so it's not > extremely fast. > http://cran.r-project.org/web/packages/XLConnect/vignettes/XLConnect.pdf > This is another package I just saw for the first time > > http://cran.r-project.org/web/packages/xlsx/xlsx.pdf > > Regards, > Albert-Jan hi, many thanks for the suggestions but i've already implemented an alternative using perl and its Spreadsheet::XLSX module. cheers, raf -- http://mail.python.org/mailman/listinfo/python-list
PyPyODBC 0.8.3 released with improved performance and SQLAlchemy support
PyPyODBC - A Pure Python ctypes ODBC module Features -Pure Python, compatible with IronPython and PyPy (tested on Win32) -Almost totally same usage as pyodbc You can simply try pypyodbc in your existing pyodbc powered script with the following changes: #import pyodbc <-- The original pyodbc importing line import pypyodbc as pyodbc # Let pypyodbc "pretend" the pyodbc pyodbc.connect(...)# pypyodbc is now doing pyodbc's job with same APIs! History Version 0.8.3 Sep 1 2012 sql_longvarchar handling fix; performance optimization; Version 0.8.2 Aug 27 2012 Differentiate sql_varchar and sql_longvarchar; Initial support for SQLAlchemy; Version 0.8.1 Aug 26 2012 Fixed the long type parameter issue; Added support for IronPython; Version 0.8 Aug 25 2012 Added getinfo method; Version 0.7 Jul 28 2012 Fixed nchar/ntext/nvarchar string truncat problem ; Version 0.6 Jul 4 2012 Added Cursor.commit() and Cursor.rollback(); Added readonly keyword to connect; Version 0.5 Jun 23 2012 Initial release; -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Sun, 02 Sep 2012 11:58:08 -0700, wxjmfauth wrote: > - Unfortunately, I got opposite and even much worst results on my win > box, considering > - libfrancais is one of my module and it does a little bit more than the > std sorting tools. How do we know that the problem isn't in your module? > My rationale: very simple. > > 1) I never heard about something better than sticking with one of the > Unicode coding scheme. (genreral theory) Your ignorance is not a good reason for abandoning a powerful software technique. 2) I am not at all convinced by > the "new" Py 3.3 algorithm. I'm not the only one guy, who noticed > problems. That's nice. Nobody has yet displayed genuine performance problems, only artificial and platform-dependent slowdowns that are insignificant in practice. If you can demonstrate genuine problems, people will be interested in fixing them. Let me be frank: nobody gives a damn if, for some rare circumstances, some_string.replace(another_string) takes 0.3μs instead of 0.1μs. Overall, considering multiple platforms and dozens of different string operations, PEP 393 is a big win: - many operations are faster - a few operations are a LOT faster - but a very few operations are sometimes slower - many strings will use less memory - sometimes a LOT less memory - no more distinction between wide and narrow builds - characters in the supplementary planes are now, for the first time in Python, treated correctly by default That's six wins versus one loss. > Arguing, "it is fast enough", is not a correct answer. It is *exactly* the correct answer. Nobody is going to revert this just because your script now runs in 5.7ms instead of 5.2ms. Who cares? If you are *seriously* interested in debugging why string code is slower for you, you can start by running the full suite of Python string benchmarks: see the stringbench benchmark in the Tools directory of source installations, or see here: http://hg.python.org/cpython/file/8ff2f4634ed8/Tools/stringbench -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Sun, 02 Sep 2012 23:38:49 +0300, Serhiy Storchaka wrote: > On 30.08.12 09:55, Steven D'Aprano wrote: >> And Python's solution uses those: UCS-2, UCS-4, and UTF-8. > > I see that this misconception widely spread. I am not familiar enough with the C implementation to tell what Python 3.3 actually does, and the PEP assumes a fair amount of familiarity with the CPython source. So I welcome corrections. > In fact Python 3.3 uses four kinds of ready strings. > > * ASCII. All codes <= U+007F. > * UCS1. All codes <= U+00FF, at least one code > U+007F. > * UCS2. All codes <= U+, at least one code > U+00FF. > * UCS4. All codes <= U+0010, at least one code > U+. Where UCS1 is equivalent to Latin-1, correct? UCS2 is what Python 3.2 narrow builds uses for all strings, including codes > U+ using surrogate pairs. UCS4 is what Python 3.2 wide builds uses for all strings. This means that Python 3.3 will no longer have surrogate pairs. Am I right? > Indexing is O(0) for any string. I think you mean O(1) for constant-time lookups. > Also the string can optionally cache UTF-8 and wchar_t* representation. Right, that's the bit that wasn't clear -- the UTF-8 data is a cache, not the canonical representation. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 9/2/2012 9:54 PM, Steven D'Aprano wrote: On Sun, 02 Sep 2012 23:38:49 +0300, Serhiy Storchaka wrote: On 30.08.12 09:55, Steven D'Aprano wrote: And Python's solution uses those: UCS-2, UCS-4, and UTF-8. I see that this misconception widely spread. I am not familiar enough with the C implementation to tell what Python 3.3 actually does, and the PEP assumes a fair amount of familiarity with the CPython source. So I welcome corrections. In fact Python 3.3 uses four kinds of ready strings. * ASCII. All codes <= U+007F. * UCS1. All codes <= U+00FF, at least one code > U+007F. * UCS2. All codes <= U+, at least one code > U+00FF. * UCS4. All codes <= U+0010, at least one code > U+. Where UCS1 is equivalent to Latin-1, correct? UCS2 is what Python 3.2 narrow builds uses for all strings, including codes > U+ using surrogate pairs. UCS4 is what Python 3.2 wide builds uses for all strings. This means that Python 3.3 will no longer have surrogate pairs. Basically, yes. I believe CPython will only use surrogate code points if one requests errors=surrogate-escape on decoding or explicitly puts them in a literal (\u or \U). The consequences fall under the 'consenting adults' policy. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Changing a Value in List of lists
On Sunday, 2 September 2012 23:40:32 UTC+5:30, Terry Reedy wrote: > On 9/2/2012 7:17 AM, Mark Lawrence wrote: > > > On 02/09/2012 12:05, Rishabh Dixit wrote: > > >> Hi, > > >> > > >> I got it :). We should create a new list every time before adding to a > > >> list > > >> of list so that instead repeating the reference a new reference generated > > >> every time. > > >> > > > > > > Please don't top post on this list as it often destroys the context, > > > thanks. > > > > In this particular case, OP could and should have trimmed off everything > > other than the top post, as it stands alone. Then it would not have been > > a top post ;-). > > > > -- > > Terry Jan Reedy Sorry for the inconvenience. I would not do it again. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
Le dimanche 2 septembre 2012 14:01:18 UTC+2, Serhiy Storchaka a écrit : > On 02.09.12 12:52, Peter Otten wrote: > > > Ian Kelly wrote: > > > > > >> Rewriting the example to use locale.strcoll instead: > > > > > > sorted(li, key=functools.cmp_to_key(locale.strcoll)) > > > > > > There is also locale.strxfrm() which you can use directly: > > > > > > sorted(li, key=locale.strxfrm) > > > > Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. With a memory gain = 0 since my text contains non-latin-1 characters! jmf -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 03/09/2012 06:39, [email protected] wrote: Le dimanche 2 septembre 2012 14:01:18 UTC+2, Serhiy Storchaka a écrit : On 02.09.12 12:52, Peter Otten wrote: Ian Kelly wrote: Rewriting the example to use locale.strcoll instead: sorted(li, key=functools.cmp_to_key(locale.strcoll)) There is also locale.strxfrm() which you can use directly: sorted(li, key=locale.strxfrm) Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. With a memory gain = 0 since my text contains non-latin-1 characters! jmf This is getting really funny. Do you make a living writing comedy for big film or TV studios? Your response to Steven D'Aprano's "That's six wins versus one loss." should be hilarious. Or do you not respond to fact based posts? -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
[email protected] wrote: > Le dimanche 2 septembre 2012 14:01:18 UTC+2, Serhiy Storchaka a écrit : >> Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2. > > With a memory gain = 0 since my text contains non-latin-1 characters! I can't confirm this. At least users of wide builds will see a decrease in memory use: $ cat strxfrm_getsize.py import locale import sys print("maxunicode:", sys.maxunicode) locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8") words = [ 'noël', 'noir', 'nœud', 'noduleux', 'noétique', 'noèse', 'noirâtre'] print("total size of original strings:", sum(sys.getsizeof(s) for s in words)) print( "total size of transformed strings:", sum(sys.getsizeof(locale.strxfrm(s)) for s in words)) $ python3.2 strxfrm_getsize.py maxunicode: 1114111 total size of original strings: 584 total size of transformed strings: 980 $ python3.3 strxfrm_getsize.py maxunicode: 1114111 total size of original strings: 509 total size of transformed strings: 483 The situation is more complex than you suppose -- you need less dogma and more experiments ;) -- http://mail.python.org/mailman/listinfo/python-list
tornado.web ioloop add_timeout eats CPU
JavaScript clients (browsers) do long poll requests. Each request can
take up to 10 seconds before the server responds. On the server side,
every client has a queue of messages that needs to be sent to the
client. When the long poll request comes in, the server checks if there
are messages to be sent out. If there are no outgoing messages, then it
does not finish the response, but calls ioloop's add_timeout method for
doing further checks. After 10 seconds (if there are no new messages)
the server returns 304/not modified. If there is a message, then it is
sent back to the client as fast as possible, and the client comes back
with another long poll immediately.
These message queues are used for UI updates and also for instant
messaging. UI must be responsive. For this reason, any message in the
outgoing queue should be sent out to the client within 0.1 seconds.
Sometimes (rarely) lots of messages arrive quickly, and in those cases
it would be good to send them out even faster. What I did is that in the
first 0.1 seconds, I call add_timeout with 0.01 seconds. So if the
outgoing queue is full of messages, then they are delivered quickly.
After 0.1 seconds lapsed, add_timeout is called with 0.1 sec parameter.
So the server load is reduced because most clients are inactive, and
they are going to get callbacks in every 0.1 sec.
Here are the two most important methods of my request handler:
@tornado.web.asynchronous
def post(self):
"""Handle POST requests."""
# Disable caching
self.set_header("Cache-Control","no-cache, must-revalidate")
self.set_header("Expires","Mon, 26 Jul 1997 05:00:00 GMT")
self.poll_start = time.time()
action = self.get_argument("action")
if action=="poll":
self.poll()
elif action=="message":
self.process_incoming(self.get_argument("message"))
else:
self.set_status(400)
self.finish()
def poll(self):
"""Handle POLL request for the browser's message loop.
This method monitors the outgoing message queue, and sends
new messages to the browser when they come in (or until
self.poll_interval seconds elapsed)."""
poll_elapsed = time.time() - self.poll_start
if poll_elapsed<0.1:
poll_step = 0.01
else:
poll_step = 0.1
if poll_elapsedAnd here is my problem. If I point 5 browsers to the server, then I get
2% CPU load (Intel i5 2.8GHz on amd64 Linux). But why? Most of the time,
the server should be sleeping. cProfile tells this:
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.000 845.146 845.146 :1()
1135775 832.2830.001 832.2830.001 {method 'poll' of
'select.epoll' objects}
I have copied out the two relevant rows only. As you can see, total
runtime was 845 seconds, and 832 seconds were spent in "epoll".
Apparently, CPU load goes up linearly as I connect more clients. It
means that 50 connected clients would do 20% CPU load. Which is
ridiculous, because they don't do anything but wait for messages to be
processed. Something terribly wrong, but I cannot figure out what?
Actually I could not try this with 50 clients. If I open 15 clients,
then the server starts dropping connections. (Tried from Firefox and
Chrome too.) If I change the poll() method this way:
else:
print "No messages after %.2f seconds"%poll_elapsed
self.set_status(304)
self.finish()
then I see this in the log:
No messages after 10.01 seconds
ERROR:root:Uncaught exception POST /client (127.0.0.1)
HTTPRequest(protocol='http', host='127.0.0.1:', method='POST',
uri='/client', version='HTTP/1.1', remote_ip='127.0.0.1',
body='_xsrf=df157469a62142d7b28c5a4880dd8478&action=poll',
headers={'Referer': 'http://127.0.0.1:/', 'Content-Length': '50',
'Accept-Language': 'en-us;q=0.8,en;q=0.5', 'Accept-Encoding': 'gzip,
deflate', 'Host': '127.0.0.1:', 'Accept': '*/*', 'User-Agent':
'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101
Firefox/15.0', 'Connection': 'keep-alive', 'X-Requested-With':
'XMLHttpRequest', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache',
'Cookie':
'sid="MS1acHd5b3V1WHFOQU1BbTVmSXJEeVhkLys=|1346652787|e045d786fdb89b73220a2c77ef89572d0c16901e";
_xsrf=df157469a62142d7b28c5a4880dd8478;
xsfr=df157469a62142d7b28c5a4880dd8478', 'Content-Type':
'application/x-www-form-urlencoded; charset=UTF-8'})
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/tornado/stack_context.py",
line 183, in wrapped
callback(*args, **kwargs)
File "/home/gandalf/Python/Projects/test/client.py", line 67, in poll
self.finish()
File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 641, in
finish
self.request.finish()
File "/usr/lib/python2.7/dist-packages/tornado/httpserver.py", line
411, in finish
self.co
