Re: How can I get the ascii code of a charter in python?
On Tue, May 01, 2007 at 02:06:21PM -0700, [EMAIL PROTECTED] wrote:
> How can I get the ascii code of a charter in python?
It's a builtin:
>>> ord('*')
42
Dustin
--
http://mail.python.org/mailman/listinfo/python-list
pre-PEP: Standard Microthreading Pattern
I've been hacking away on this PEP for a while, and there has been some related discussion on python-dev that went into the PEP: http://mail.python.org/pipermail/python-dev/2007-February/070921.html http://mail.python.org/pipermail/python-dev/2007-February/071155.html http://mail.python.org/pipermail/python-dev/2007-February/071181.html I'd love to have feedback on this PEP: - from a user's perspective (would you want to write microthreaded code?) - from an implementer's perspective (and note that I have a reference implementation that's just getting packaged up right now). - from a technical perspective (efficiency, compatibility, etc.) - regarding the two unresolved issues in the text And now, without further ado: PEP: XXX Title: Standard Microthreading Pattern Version: $Revision$ Last-Modified: $Date$ Author: Dustin J. Mitchell <[EMAIL PROTECTED]> Status: Draft Type: Informational Content-Type: text/x-rst Created: 17-Feb-2007 Post-History: Abstract The extended support for generators introduced in PEP 342 [2]_ facilitated a natural way of writing generators that cooperatively yield control to other functions, allowing multitasking within a single OS thread. Such multitasking requires a supporting environment (usually in the form of a scheduler), and several such implementations are in active use. Each has slightly different structural requirements for the task-implementing code. This PEP proposes a single, consistent pattern for such code. This consistency will enable microthreaded code to run in a variety of environments, just as threaded code can currently run atop a variety of OS-level threading libraries. Compliant libraries, e.g., a microthreaded urllib, could then be developed, providing batteries-included functionality for microthreaded software. Definitions === Within this PEP, "microthreading" is defined as interleaving the execution of multiple independent codepaths by cooperatively yielding control periodically in each codepath; a "microthread", then, is one such codepath. This technique and variations on it have also been referred to as "weightless threads", "tasklets", and "greenlets". It is a specilization of the more general event-based multitasking model. Microthreads are conceptually distinct from coroutines. A coroutine can schedule a specific coroutine to be executed when it yields control, even passing a value to that coroutine. Microthreads, however, simply yield to facilitate multitasking, with no knowledge or control of the next codepath to be executed. This PEP addresses two audiences: authors of microthreaded code (users) and designers of microthreaded environments (implementations). Motivation == Application developers usually adopt an event-based architecture in order to exploit asynchronous IO facilities. Asynchronous IO is ideal when the overhead of an OS-level thread for each concurrent IO operation is too high. An event-based architecture is also useful when an application must perform lightweight multitasking -- for example, an email client must both wait for user input and monitor mailboxes for new mail. Implementing this sort of multitasking using threads requires careful use of synchronization primitives throughout the application to ensure correctness. Using common event-based techniques requires storing state on the heap and implementing callbacks for all state transitions, which can quickly become complex. A microthreaded architecture brings the best of both worlds: it allows users to store state in local varables, just as they would in a threaded implementation, while avoiding the need for complex synchronization primitives. An email client, for example, can implement the complex state of an IMAP client in a natural fashion, and reflect that state using natural Python data structures with no need for locking. Several well-developed implementations of cooperative multitasking are currently available, some of which implement microthreading. However, each implementation approaches the subject differently, and it is virtually impossible to share code between implementations. This PEP is intended to bring some measure of consistency to microthreaded code, to facilitate development of libraries and implementation-agnostic code repositories to support microthreaded development. Rationale = Since the introduction of generators in PEP 255 [3]_ and their extension to support coroutines in PEP 342 [2]_, generators have been used to implement various forms of microthreading [1]_: - Example 3 in PEP 342 implements a simple trampoline scheduler. - `Kiwi tasklets`_ (formerly GTasklets) use pre-PEP 342 generators along with an explicit post-yield function call to retrieve any incoming value or exception. - `Twisted Python`_ includes an ``inlineCallbacks`` decorator [#inlineCallbacks]_ which allows a generator t
Re: pre-PEP: Standard Microthreading Pattern
On Tue, May 01, 2007 at 08:34:39PM -0400, Terry Reedy wrote: > Sounds like a good idea to me. Travis Oliphant has spent over a year on > http://www.python.org/dev/peps/pep-3118/ > so array-making-using programs can better operate together. I hope you > have the same patience and persistance. I hope so too! This is still only a pre-PEP, so there's a long way to go. > | .. [2] PEP 342, "Coroutines via Enhanced Generators", van Rossum, Eby > | (http://www.python.org/peps/pep-0342) > > This gave 404 Not Found while this works (/dev added): > http://www.python.org/dev/peps/pep-0342/ I'm going to assume these are related to the noises I'm hearing about ongoing funniness with the website. If someone is wiser than me, please speak up! > | .. [3] PEP 355, "Simple Generators", Schemenauer, Peters, Hetland > | (http://www.python.org/peps/pep-0342) > > 255, not 355 or 342 again: > http://www.python.org/dev/peps/pep-0255/ Thanks -- good catch! Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: pre-PEP: Standard Microthreading Pattern
On Tue, May 01, 2007 at 10:28:08PM -0700, Michele Simionato wrote: > >while i < n: > >latest = (latest[1], latest[0] + latest[1]) > >yield # cooperative yield > There is an infinite loop here! Whoops. To much editing, not enough actual running of example code :) > BTW, in spite of having a great tradition, the Fibonacci example sucks > as a motivation for microthreading programming. You could show a simple > state machine instead. You're absolutely right, and among good company in pointing it out. Thank you for the suggestion of an alternative! Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing a nice formatted csv file
On Wed, May 02, 2007 at 07:28:32AM -0700, redcic wrote:
> Well then how can I format a file ?
for row in rows:
print "".join([ "%-6s" % ("%d," % cell) for cell in row ])
The "%-6s" formats each column to be no less than six characters long;
the "%d," formats the number with a comma after it.
This won't be quite what you want, since you've comma-aligned all of the
fields after the first, but it should be readable.
> > > Whereas what I'd like to get is:
> > > 1,2,3,
> > > 10, 20, 30
> >
> > > which is more readable.
> >
> > cvs files are constructed for efficient processing not formatting so
> > that you can read them easier. If you want a formatted file, then
> > construct one.
--
http://mail.python.org/mailman/listinfo/python-list
Re: gpp (conditional compilation)
On Wed, May 02, 2007 at 10:37:40AM -0700, [EMAIL PROTECTED] wrote: > I'm trying to use the gpp utility (Gnu points to > http://en.nothingisreal.com/wiki/GPP) > to do conditional compilation in Python, and I'm running into a > problem: the same '#' character introduces Python comments and is used > by default to introduce #ifdef etc. lines. The hacks you'll need to wrap gpp in to get it to work will make your head spin. I'm not sure what brought you to gpp, but if you're looking for a basic macro-processing language suitable for the kinds of tasks the C preprocessor performs, I would recommend M4. GNU has an implementation of the language, and it's actually quite widely used as the basis for the GNU autotools suite. It is incredibly flexible (enough so that its learning curve is a bit steep, but not too bad), but that flexibility enables it to work with just about any underlying language. Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: _csv.Error: string with NUL bytes
On Thu, May 03, 2007 at 09:57:38AM -0700, fscked wrote:
> > As Larry said, this most likely means there are null bytes in the CSV file.
> >
> > Ciao,
> > Marc 'BlackJack' Rintsch
>
> How would I go about identifying where it is?
A hex editor might be easiest.
You could also use Python:
print open("filewithnuls").read().replace("\0", ">>>NUL<<<")
Dustin
--
http://mail.python.org/mailman/listinfo/python-list
Re: _csv.Error: string with NUL bytes
On Thu, May 03, 2007 at 10:28:34AM -0700, [EMAIL PROTECTED] wrote:
> On May 3, 10:12 am, [EMAIL PROTECTED] wrote:
> > On Thu, May 03, 2007 at 09:57:38AM -0700, fscked wrote:
> > > > As Larry said, this most likely means there are null bytes in the CSV
> > > > file.
> >
> > > > Ciao,
> > > > Marc 'BlackJack' Rintsch
> >
> > > How would I go about identifying where it is?
> >
> > A hex editor might be easiest.
> >
> > You could also use Python:
> >
> > print open("filewithnuls").read().replace("\0", ">>>NUL<<<")
> >
> > Dustin
>
> Hmm, interesting if I run:
>
> print open("test.csv").read().replace("\0", ">>>NUL<<<")
>
> every single character gets a >>>NUL<<< between them...
>
> What the heck does that mean?
>
> Example, here is the first field in the csv
>
> 89114608511,
>
> the above code produces:
> >>>NUL<<<8>>>NUL<<<9>>>NUL<<<1>>>NUL<<<1>>>NUL<<<4>>>NUL<<<6>>>NUL<<<0>>>NUL<<<8>>>NUL<<<5>>>NUL<<<1>>>NUL<<<1>>>NUL<<<,
I'm guessing that your file is in UTF-16, then -- Windows seems to do
that a lot. It kind of makes it *not* a CSV file, but oh well. Try
print open("test.csv").decode('utf-16').read().replace("\0", ">>>NUL<<<")
I'm not terribly unicode-savvy, so I'll leave it to others to suggest a
way to get the CSV reader to handle such encoding without reading in the
whole file, decoding it, and setting up a StringIO file.
Dustin
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python regular expressions just ain't PCRE
On Sat, May 05, 2007 at 08:52:15AM -0700, Wiseman wrote: > > I believe the current Python re module was written to replace the Python > > wrapping of pcre in order to support unicode. > > I don't know how PCRE was back then, but right now it supports UTF-8 > Unicode patterns and strings, and Unicode character properties. Maybe > it could be reintroduced into Python? I would say this is a case for "rough consensus and working code". With something as big and ugly[1] as a regexp library, I think the "working code" part will be the hard part. So, if you have a patch, there's a decent chance such a thing would be adopted. I'm not sure what your skill level is, but I would suggest studying the code, starting in on a patch for one or more of these features, and then corresponding with the module's maintainers to improve your patch to the point where it can be accepted. Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3131: Supporting Non-ASCII Identifiers
On Sun, May 13, 2007 at 05:44:39PM +0200, "Martin v. L??wis" wrote: > - should non-ASCII identifiers be supported? why? The only objection that comes to mind is that adding such support may make some distinct identifiers visually indistinguishable. IIRC the DNS system has had this problem, leading to much phishing abuse. I don't necessarily think that the objection is strong enough to reject the idea -- programmers using non-ASCII symbols would be responsible for the consequences of their character choice. Dustin -- http://mail.python.org/mailman/listinfo/python-list
[no subject]
Hi there, I'm new to Python and to programming. is this the right place for me to post a beginner question on Python use ? Many thanks. -- http://mail.python.org/mailman/listinfo/python-list
web browsing short cut
Hey guys,
I am new to python. I want to make a shortcut that opens my websites
and re-sizes them to display on different areas on the screen. I looked
around but i had no luck. Is that possible with python? if so can someone
point to to the right direction? Here is what I came up with so far..
Thanks in advance,
--
import webbrowser
# The websites i want to open..
url1 = 'http://www.python.org/'
url2 = 'http://www.google.com/'
url3 = 'http://msn.com/'
# the path to ie
ie = webbrowser.get('c:\\program files\\internet explorer\\iexplore.exe')
ie.open(url1)
ie.open(url2)
ie.open(url3)
--
Dustin
--
http://mail.python.org/mailman/listinfo/python-list
Re: web browsing short cut
Hey, I am looking into Tkinter. But i am not sure if it will actually work. This maybe a crazy idea but i was wondering if i can put a web browser in the frame. I have tried to use Tkinter to resize and place the windows to certain areas of the screen but that's not working or the way im approaching this problem is completely wrong. I want to make a program that will have websites displayed in specific areas of the screen. I was planning on using the program on the big screen. So is it possible to put the web browser inside the frame in Tkinter? On Sat, Jul 2, 2011 at 7:10 PM, Chris Rebert wrote: > On Sat, Jul 2, 2011 at 6:21 PM, Dustin Cheung wrote: > > Hey guys, > > I am new to python. I want to make a shortcut that opens my websites > > and re-sizes them to display on different areas on the screen. I looked > > around but i had no luck. Is that possible with python? if so can someone > > point to to the right direction? Here is what I came up with so far.. > > The window positioning+resizing bit will likely require using > platform-specific APIs. Since you appear to be on Windows, the > relevant library would be pywin32 (http://pypi.python.org/pypi/pywin32 > ). You would use it to invoke some COM API that does window > positioning+resizing. I am unable to give more details as I'm on a > Mac. > > Sidenote: Have you tried Firefox's "Bookmark All Tabs" feature? > > Cheers, > Chris > -- Dustin Cheung -- http://mail.python.org/mailman/listinfo/python-list
Re: web browsing short cut
Okay thanks for the help guys, ill keep you guys posted. On Wed, Jul 6, 2011 at 1:19 PM, Ian wrote: > On 03/07/2011 02:21, Dustin Cheung wrote: > >> Hey guys, >> >> I am new to python. I want to make a shortcut that opens my websites and >> re-sizes them to display on different areas on the screen. I looked around >> but i had no luck. Is that possible with python? if so can someone point to >> to the right direction? Here is what I came up with so far.. >> >> >> I suggest you create a dummy page on your disk with an onload event that > uses javascript to open, size and load all the windows you want. > > Then create a short cut to the dummy page. > > Regards > > Ian > > > > -- > http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list> > -- Dustin Cheung -- http://mail.python.org/mailman/listinfo/python-list
python execution path
I'm wondering if there is a way to get python to show each line as it is executed, sort of like sh -x does for shell programs. Seems like this would be a nice debugging aid. dustin -- http://mail.python.org/mailman/listinfo/python-list
Probably simple syntax error
Hi everyone. This is my first time posting to this newsgroup, and although I maintain my netiquette I might've missed something specific to the newsgroup, so hopefully you can avoid flaming me if I have :) I apologize for the length of this post but I figure the more information the better. My problem is that I'm getting a syntax error in some Python code that looks quite simple. The original code was in Object Pascal as I'm a recent Delphi-turned-Python programmer. I took the code (which is only about 130 lines in OP) and 'translated' it the best I could into Python (ended up being one line shy of 80 when I was done). I can't see any problems with the code but it's coming up with a bunch of errors, which I'm guessing are probably my assuming something is the same in Python as it is in Pascal, and being wrong. Anyway, here's the code I'm having trouble with (the same error comes up several times but this is the first part of the code it shows up in): [code] randomizing_counter = 0 # Put the loop counter for the randomizing to zero. until_val = 36 # Set the "until val" to 36. We'll compare them to make sure we're not at the end of our wordlist_both. while randomizing_counter < until_val: big_randomized_int = RandRange(0,100) # Make a random value and store it. small_randomized_int = big_randomized_int / 100 # Divide that random value and store it in a different variable. small_randomized_int = Round(small_randomized_int, 2) # Round that value to 2 decimal places **weights_array(randomizing_counter) = small_randomized_int # Assign the first randomized value to our first word to be weighted. randomizing_counter = randomizing_counter + 1 # Up the counter and repeat. [/code] The starred line is the one getting the error message: "SyntaxError: can't assign to function call" Now, I do understand what this means. I'm trying to assign to a function instead of the value that the function should create. But since when is weights_array or small_randomizing_int a function? Both are being declared in the code on their first usage. This one has got me stumped, maybe you guys can help me out with it? Thanks, ~Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: Probably simple syntax error
Ah. Thank you everyone. Sorry for not replying earlier, real life got in the way :) Gerry Herron, Tim Delaney, Mark Peters: Thank you. Switching from parentheses to square brackets fixed the code, and yes, Tim, you were right. It was a list I was working with. And thanks for those links Tim. John Machin: Thank you for all the pointers/code fixes there. They'll help alot. Ptn: I was unaware of that period added, Thanks, I'll have to watch out for it. :) And Cameron: Ah, yes. It does reduce the confusion. I do know that square brackets are used for *creating* a dictionary (blah = ["A", "B", "C"], so I figured the same would apply to accessing it (which is why for my list, which I created with parenthesis I assumed I accessed with parenthesis). Thank you =] ~Dustin -- http://mail.python.org/mailman/listinfo/python-list
python references
>>> from Numeric import zeros >>> p=zeros(3) >>> p array([0,0,0]) >>> p[0] 0 >>> x=p[0] >>> x=10 >>> p array([0,0,0]) #actual behavior #array([10,0,0]) #desired behavior I want x to be a C++-esque reference to p[0] for convenience in a vector3 class. i dont want accessor methods. i know python can do this, but it's been a long time since I used it and am unsuccessful in my googling and docreading. a little help please? -- http://mail.python.org/mailman/listinfo/python-list
modifying a list while iterating through
consider the following working loop where Packet is a subclass of list, with Packet.insert(index, iterable) inserting each item in iterable into Packet at consecutive indexes starting at index. i=0 while(ihttp://mail.python.org/mailman/listinfo/python-list
Re: modifying a list while iterating through
On Feb 25, 9:15 pm, [EMAIL PROTECTED] wrote:
> On Feb 25, 5:12 pm, [EMAIL PROTECTED] wrote:
>
> > consider the following working loop where Packet is a subclass of
> > list, with Packet.insert(index, iterable) inserting each item in
> > iterable into Packet at consecutive indexes starting at index.
>
> > i=0
> > while(i > if packet[i:i+5]==Packet("01110"):
> > packet.insert(i, "0")
> > i+=10 #skip the 5 bits inserted, and skip the 5 bits just
> > checked bc overlap should not trigger insertion
> > else: i+=1
>
> > is there a way to do this more elegantly? seems like a big kludge.
>
> If Packet consists of '0's and '1's, then it may be
> easier to convert to, or base the class on str (strings):
>
> packet = "101010011100111010001"
> print "BEFORE ", packet
> li = packet.split("01110")
> packet = "011100".join(li)
> print "AFTER ", packet
>
> --
> Hope this helps,
> Steven
Steven, George,
Thanks for your responses. Yea, that would work.
My original question still stands, though, in situations where a
simple string replacement might not be sufficient. Is there a way to
insert into a list whilst iterating through it?
for example, consider a list of binary values.
for index in range(len(iterable)):
item=iterable[index]
if item==1:
iterable.insert(index,0)
obv this wouldn't work because now all future indexes will be off by
the number of previously inserted items.
using a while loop to fix this ugly and counterintuitive.
--
http://mail.python.org/mailman/listinfo/python-list
Re: refreshing the cache time of a key
On Feb 27, 2008, at 4:17, bharath venkatesh wrote: >is it possible to refresh the cache time of a key with out having > to retrieving the cached data and storing it back in memcache .. for > example if a data is cached for > 1 hour and at the 50th minute from the time this data has been > cached i want to store it in the cache for 1 more hour ..is > there a function to refresh the cache time by knowing the key of > data with out having to do get and set There's been talk of a ``touch'' command, but no work has been done on implementing such a thing. -- Dustin Sallings -- http://mail.python.org/mailman/listinfo/python-list
Standard Asynchronous Python
After seeing David Mertz's talk at PyCon 2012, "Coroutines, event loops, and the history of Python generators" [1], I got thinking again about Python's expressive power for asynchronous programming. Generators, particularly with the addition of 'yield from' and 'return' in PEP 380 [2], allow us to write code that is executed "bit by bit" but still reads naturally. There are a number of frameworks that take advantage of this ability, but each is a little different -- enough so that there's negligible code re-use between these frameworks. I think that's a shame. I proposed a design PEP a while back [3] with the intent of defining a standard way of writing asynchronous code, with the goal of allowing code re-use and bringing users of the frameworks closer together. Ideally, we could have libraries to implement network protocols, database wrappers, subprocess execution, and so on, that would work in any of the available asynchronous frameworks. My proposal met with near-silence, and I didn't pursue it. Instead, I did what any self-respecting hacker would do - I wrote up a framework, uthreads [4], that implemented my idea. This was initially a simple trampoline scheduler, but I eventually refactored it to run atop Twisted, since that's what I use. To my knowledge, it's never been used. I'm considering re-drafting the PEP with the following changes: * De-emphasize the thread emulation aspects, and focus on code-portability issues: * callbacks vs. "blocking" calls (e.g., when accepting incoming connections on a socket, how is my code invoked?) * consistent access to primitives, regardless of framework (e.g., where's the function I call to branch execution?) * nested asynchronous methods * Account for PEP 380 (by making the StopIteration workarounds match PEP 380, and explicitly deprecating them after Python 3.3) * Look forward to a world with software transactional memory [5] by matching that API where appropriate As I get to work on the PEP, I'd like to hear any initial reactions to the idea. Dustin [1] https://us.pycon.org/2012/schedule/presentation/104/ [2] http://www.python.org/dev/peps/pep-0380 [3] http://code.google.com/p/uthreads/source/browse/trunk/microthreading-pep.txt [4] http://code.google.com/p/uthreads/ [5] https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.rst -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Asynchronous Python
The responses have certainly highlighted some errors in emphasis in my approach. * My idea is to propose a design PEP. (Steven, Dennis) I'm not at *all* suggesting including uthreads in the standard library. It's a toy implementation I used to develop my ideas. I think of this as a much smaller idea in the same vein as the DBAPI (PEP 249): a common set of expectations that allows portability. * I'd like to set aside the issue of threads vs. event-driven programming. There are legitimate reasons to do both, and the healthy ecosystem of frameworks for the latter indicates at least some people are interested. My idea is to introduce a tiny bit of coherence across those frameworks. * (Bryan) The Fibonacci example is a simple example of, among other things, a CPU-bound, recursive task -- something that many async frameworks don't handle fairly right now. I will add some text to call that out explicitly. * Regarding generators vs. coroutines (Bryan), I use the terms generator and generator function in the PEP carefully, as that's what the syntactic and runtime concepts are called in Python. I will include a paragraph distinguishing the two. I will need to take up the details of the idea with the developers of the async frameworks themselves, and get some agreement before actually proposing the PEP. However, among this group I'm interested to know whether this is an appropriate use of a design PEP. That's why I posted my old and flawed PEP text, rather than re-drafting first. Thanks for the responses so far! Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Asynchronous Python
Thanks for the second round of responses. I think this gives me some focus - concentrate on the API, talk to the framework developers, and start redrafting the PEP sooner rather than later. Thanks! Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: odt2sphinx 0.2.3 released
On Wed, Sep 12, 2012 at 10:06 AM, wrote: > ߒߤߒߡߜߦߡ ß ß§ And that's why you shouldn't let your kids play with your iPad :) Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: Dictionaries again - where do I make a mistake?
Lad wrote: > Sorting seems to be OK,. > the command > print key,val > prints the proper values > but I can not create Newdict to be sorted properly. > > Where do I make a mistake? > Thank you for help. Dictionaries are unordered -- the order in which items come out is unspecified. It's based on the details of their internal storage mechanism (a hash table), and you can't control it at all. If you need your pairs in a certain order, you'll have to use a list of tuples. Dustin -- http://mail.python.org/mailman/listinfo/python-list
httplib problems -- bug, or am I missing something?
I'm building an interface to Amazon's S3, using httplib. It uses a single object for multiple transactions. What's happening is this: HTTP > PUT /unitest-temp-1161039691 HTTP/1.1 HTTP > Date: Mon, 16 Oct 2006 23:01:32 GMT HTTP > Authorization: AWS <>:KiTWRuq/6aay0bI2J5DkE2TAWD0= HTTP > (end headers) HTTP < HTTP/1.1 200 OK HTTP < content-length: 0 HTTP < x-amz-id-2: 40uQn0OCpTiFcX+LqjMuzG6NnufdUk/.. HTTP < server: AmazonS3 HTTP < x-amz-request-id: FF504E8FD1B86F8C HTTP < location: /unitest-temp-1161039691 HTTP < date: Mon, 16 Oct 2006 23:01:33 GMT HTTPConnection.__state before response.read: Idle HTTPConnection.__response: closed? False length: 0 reading response HTTPConnection.__state after response.read: Idle HTTPConnection.__response: closed? False length: 0 ..later in the same connection.. HTTPConnection.__state before putrequest: Idle HTTPConnection.__response: closed? False length: 0 HTTP > DELETE /unitest-temp-1161039691 HTTP/1.1 HTTP > Date: Mon, 16 Oct 2006 23:01:33 GMT HTTP > Authorization: AWS <>:a5OizuLNwwV7eBUhha0B6rEJ+CQ= HTTP > (end headers) HTTPConnection.__state before getresponse: Request-sent HTTPConnection.__response: closed? False length: 0 File "/usr/lib64/python2.4/httplib.py", line 856, in getresponse raise ResponseNotReady() If the first request does not precede it, the second request is fine. To avoid excessive memory use, I'm calling request.read(16384) repeatedly, instead of just calling request.read(). This seems to be key to the problem -- if I omit the 'amt' argument to read(), then the last line of the first request reads HTTPConnection.__response: closed? True length: 0 and the later call to getresponse() doesn't raise ResponseNotReady. Looking at the source for httplib.HTTPResponse.read, self.close() gets called in the latter (working) case, but not in the former (non-working). It would seem sensible to add 'if self.length == 0: self.close()' to the end of that function (and, in fact, this change makes the whole thing work), but this comment makes me hesitant: # we do not use _safe_read() here because this may be a .will_close # connection, and the user is reading more bytes than will be provided # (for example, reading in 1k chunks) What's going on here? Is this a bug I should report, or am I missing something about how one should use httplib? Thanks for any assistance. Dustin -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling functions
Tommy Grav wrote: > I have a small program that goes something like this > > def funcA() : pass > def funcB() : pass > def funcC() : pass > > def determine(f): > t = f() > return t > > What I would like to do is be able to > > n = determine(funcA) > m = determine(funcB) > > But I can't really figure out how to do this (I think it is > possible :) Except for the spaces after the def's at the top (are those legal?), it should work as written. determine(funcA) results in 'f' being bound to 'funcA'; then 't = f()' results in 'funcA' being called, and its resulting being bound to 't'; 'determine' returns that result, and it's bound to 'n'. Is that not what you wanted? Dustin -- http://mail.python.org/mailman/listinfo/python-list
Redux: Allowing 'return obj' in generators
This question was first brought up in October of 2005[1], and was included in
the "Unresolved Issues" section of my microthreading PEP, which I have quietly
withdrawn from consideration due to lack of community interest.
PEP 255 says
Q. Then why not allow an expression on "return" too?
A. Perhaps we will someday. In Icon, "return expr" means both "I'm
done", and "but I have one final useful value to return too, and
this is it". At the start, and in the absence of compelling uses
for "return expr", it's simply cleaner to use "yield" exclusively
for delivering values.
As those of you who looked at my PEP or are familiar with some of the
implementations will realize, microthreaded functions are syntactically
generator functions, but semantically act as regular functions. There
is a well-defined meaning to 'return x' in such a function: take the
value of x, and use it in the expression where this function was called.
For example:
def read_integer(sock):
txt = yield sock.readline().strip()
try:
return int(txt)
except:
raise AppProtocolError("Expected an integer")
The implementation of the syntax would be similar to that of an
expressionless 'return', but supplying the expression_list to the
StopIteration's 'args' -- this is described quite well in Piet Delport's
post[2].
Given this use-case (and note that I chose an example that will exercise
the interactions of try/except blocks with the StopIteration behavior),
is it time to revisit this issue? BDFL said:
I urge you to leave well enough alone. There's room for extensions
after people have built real systems with the raw material provided by
PEP 342 and 343.[3]
and Nick Coghlan said (to applause from GvR):
I'm starting to think we want to let PEP 342 bake for at least one
release cycle before deciding what (if any) additional behaviour
should be added to generators.[4]
I think we have a decent number of implementations in the wild now
(I have learned of Christopher Stawarz's 'multitask'[5] since last
posting my PEP). With 2.5.1 out, might I suggest this is worth
reconsidering for the 2.6 release?
Dustin
[1]
http://www.python.org/dev/summary/2005-10-01_2005-10-15/#allowing-return-obj-in-generators
[2] http://mail.python.org/pipermail/python-dev/2005-October/056957.html
[3] http://mail.python.org/pipermail/python-dev/2005-October/057119.html
[4] http://mail.python.org/pipermail/python-dev/2005-October/057133.html
[5] http://o2s.csail.mit.edu/o2s-wiki/multitask
--
http://mail.python.org/mailman/listinfo/python-list
SysLogHandler message formatting
I have a daemon that uses the built-in SysLogHandler logging handler
class to log messages to the host's syslog. Unfortunately, I am having
trouble getting it to work with Metalog[1]. At first, I thought the
problem was Metalog's fault because everything works as expected with
syslog-ng. Upon further investigation, I have come to the conclusion
that the SysLogHandler may be sending invalid syslog messages. When I
disabled all filtering in Metalog, my messages started appearing in
Metalog's output logs, but they were displayed inappropriately.
Namely, the entire message was appearing where the program name should
appear. Entries in the output file generally look like this:
%b %d %H:%M:%S [%(ident)s] %(message)s
For example, here is an entry from sshd:
Oct 06 12:19:45 [sshd] Connection from 127.0.0.1 port 34142
In contrast, here is the entry generated by my program:
Oct 06 11:41:05 [INFO Started an Idler for sysattend on
mail.gosupertechs.com] 993 using SSL
Here is the snippet of code I am using to set up the logger:
root_logger = logging.getLogger()
root_logger.setLevel(config.get_value("log_level"))
syslog_hdlr = SysLogHandler(address='/dev/log',
facility=SysLogHandler.LOG_DAEMON)
syslog_hdlr.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(name)s: %(levelname)s %
(message)s')
syslog_hdlr.setFormatter(formatter)
root_logger.addHandler(syslog_hdlr)
logger = logging.getLogger("imapcd.daemon")
logger.debug('test')
I believe that the issue is LogRecords formatted using the Formatter
class, but are not modified into a valid syslog message. I set up a
fake syslog daemon that just listened on /dev/log and echoed
everything it received to the console. When my program logs a message,
it comes through like thist:
<31>imapcd.daemon: DEBUG test
which is exactly how the Formatter formatted the message. Other
programs that log to syslog, on the other hand, send messages that
look like this:
<149>Oct 6 11:17:19 sudo: dhatch : TTY=pts/7 ; PWD=/home/dhatch ;
USER=root ; COMMAND=/usr/bin/whoami
If I adjust the Formatter definition to mimic that, like so:
formatter = logging.Formatter('%(asctime)s %(name)s: %(levelname)s
%(message)s', '%b %e %H:%M:%S')
then all works well.
My question, therefore, is where does this problem lie? Is it a bug in
Metalog that it doesn't properly parse the message, or is it a bug in
SysLogHandler that it doesn't properly format it? I guess it could
also be that if one wants to use the SysLogHandler, then that
particular format string must be used to ensure compatibility with all
syslog daemons. I found a couple of examples[2][3] of people
describing the usage of SysLogHandler, but none of them mention a
problem with formatting.
Any thoughts or suggestions would be appreciated. I will patch my
program to use the latter format string for now, but I would still
like to find the root cause for this problem.
Regards,
Dustin C. Hatch
[1] http://metalog.sourceforge.net/
[2]
http://scottbarnham.com/blog/2008/01/01/sysloghandler-not-writing-to-syslog-with-python-logging/
[3]
http://www.mechanicalcat.net/richard/log/Python/Simple_usage_of_Python_s_logging_module
--
http://mail.python.org/mailman/listinfo/python-list
Re: SysLogHandler message formatting
On Oct 7, 6:18 am, Vinay Sajip wrote: > Thanks for the detailed report. I tried posting a response a couple of times, > but Google appears to have swallowed it ... trying again. Sorry if it results > in > multiple responses. I tried to respond yesterday, but I also noticed this probelm. > The SysLogHandler aims to work within RFC 5424, though that does provide for > some flexibility in message formats. The SysLogHandler sends > formatted > message, as you've observed, with the "formatted message" part encoded in > UTF-8 > with a prepended BOM if it's Unicode. I started to read RFC 5424 yesterday before posting this, but it seemed to describe a much more complex message format than SysLogHandler was producing, so I wasn't sure if that was the reference or not. I may go back and read it completely, now that I know, though. > The existing SysLogHandler code seems to work OK with syslog, syslog-ng and > rsyslog > I agree with this, so I would normally assume that Metalog is the problem and simply stop using it. Unfortunately, I have several systems that use Metalog, and I don't want to introduce any inconsistencies by having one use something else. Switching them all would also be a hassle. > - perhaps these are more forgiving than Metalog in the way they parse > messages, or perhaps it's a Metalog configuration issue. I'd try posting this > issue to the Metalog forum / mailing list to see what feedback they can give. > I will do that; I just wanted to see what the Python stance was first. I will keep you apprised of anything determined through that route. > If you do think it's a bug in SysLogHandler then please create an issue on > bugs.python.org and assign it to me, so I can take a look at it After I communicate with the Metalog community, I will definitely file an issue if necessary. I didn't want to jump the gun, though. > I think using an appropriate Formatter will ensure interoperability in any > particular scenario. I don't especially want to hard-code any format into > SysLogHandler, since a Formatter can be used to set the format flexibly. I think the only problem with doing it that way is that I wasn't initially aware that using a Formatter affected how the message is sent to Syslog, but assumed it only modified the visible portion. I will admit that, until yesterday, I knew nothing of the syslog protocol. Perhaps the best solution would be to simply document the fact that adjusting the message format can affect how the message is parsed by the syslog daemon receiving it. Thank you for your feedback. -- http://mail.python.org/mailman/listinfo/python-list
Re: SysLogHandler message formatting
On Oct 7, 6:18 am, Vinay Sajip wrote: > Thanks for the detailed report. I tried posting a response a couple of times, > but Google appears to have swallowed it ... trying again. Sorry if it results > in > multiple responses. Hmm, I too seem to be experiencing this problem... > The SysLogHandler aims to work within RFC 5424, though that does provide for > some flexibility in message formats. The SysLogHandler sends > formatted > message, as you've observed, with the "formatted message" part encoded in > UTF-8 > with a prepended BOM if it's Unicode. I started to read RFC 5424 yesterday before posting this, but it seemed to describe a much more complex message format than SysLogHandler was producing, so I wasn't sure if that was the reference or not. I may go back and read it completely, now that I know, though. > The existing SysLogHandler code seems to work OK with syslog, syslog-ng and > rsyslog > I agree with this, so I would normally assume that Metalog is the problem and simply stop using it. Unfortunately, I have several systems that use Metalog, and I don't want to introduce any inconsistencies by having one use something else. Switching them all would also be a hassle. > - perhaps these are more forgiving than Metalog in the way they parse > messages, or perhaps it's a Metalog configuration issue. I'd try posting this > issue to the Metalog forum / mailing list to see what feedback they can give. > I will do that; I just wanted to see what the Python stance was first. I will keep you apprised of anything determined through that route. > If you do think it's a bug in SysLogHandler then please create an issue on > bugs.python.org and assign it to me, so I can take a look at it After I communicate with the Metalog community, I will definitely file an issue if necessary. I didn't want to jump the gun, though. > I think using an appropriate Formatter will ensure interoperability in any > particular scenario. I don't especially want to hard-code any format into > SysLogHandler, since a Formatter can be used to set the format flexibly. I think the only problem with doing it that way is that I wasn't initially aware that using a Formatter affected how the message is sent to Syslog, but assumed it only modified the visible portion. I will admit that, until yesterday, I knew nothing of the syslog protocol. Perhaps the best solution would be to simply document the fact that adjusting the message format can affect how the message is parsed by the syslog daemon receiving it. Thank you for your feedback. -- http://mail.python.org/mailman/listinfo/python-list
Re: SysLogHandler message formatting
On Oct 7, 6:18 am, Vinay Sajip wrote: > Thanks for the detailed report. I tried posting a response a couple of times, > but Google appears to have swallowed it ... trying again. Sorry if it results > in > multiple responses. Hmm, I too seem to be experiencing this problem... > The SysLogHandler aims to work within RFC 5424, though that does provide for > some flexibility in message formats. The SysLogHandler sends > formatted > message, as you've observed, with the "formatted message" part encoded in > UTF-8 > with a prepended BOM if it's Unicode. I started to read RFC 5424 yesterday before posting this, but it seemed to describe a much more complex message format than SysLogHandler was producing, so I wasn't sure if that was the reference or not. I may go back and read it completely, now that I know, though. > The existing SysLogHandler code seems to work OK with syslog, syslog-ng and > rsyslog > I agree with this, so I would normally assume that Metalog is the problem and simply stop using it. Unfortunately, I have several systems that use Metalog, and I don't want to introduce any inconsistencies by having one use something else. Switching them all would also be a hassle. > - perhaps these are more forgiving than Metalog in the way they parse > messages, or perhaps it's a Metalog configuration issue. I'd try posting this > issue to the Metalog forum / mailing list to see what feedback they can give. > I will do that; I just wanted to see what the Python stance was first. I will keep you apprised of anything determined through that route. > If you do think it's a bug in SysLogHandler then please create an issue on > bugs.python.org and assign it to me, so I can take a look at it After I communicate with the Metalog community, I will definitely file an issue if necessary. I didn't want to jump the gun, though. > I think using an appropriate Formatter will ensure interoperability in any > particular scenario. I don't especially want to hard-code any format into > SysLogHandler, since a Formatter can be used to set the format flexibly. I think the only problem with doing it that way is that I wasn't initially aware that using a Formatter affected how the message is sent to Syslog, but assumed it only modified the visible portion. I will admit that, until yesterday, I knew nothing of the syslog protocol. Perhaps the best solution would be to simply document the fact that adjusting the message format can affect how the message is parsed by the syslog daemon receiving it. Thank you for your feedback. -- http://mail.python.org/mailman/listinfo/python-list
