Re: [Tutor] Strange Appending
On Thu, 2 Dec 2004, mdcooper wrote:
> Hello,
>
> I am trying to append a list to another list, but everytime I do, the new
> parent list has a new child list, but all the other lists have become the
> same
> as the new child list.
>
> Code:
>
>
> self._f.write(str(self.residue.atoms[int(t[0])-1].element) + ' ')
> for m in t:
> self._f.write(str(m)+' ')
> self._f.write('\n')
>
> self.a.append(t) # WHY DOES THIS NOT WORK?
Hi,
I'm not sure that I understand your question because I don't see all
the code and I don't know what you hope will happen. But ...
append appends the object as a single element.
Try self.a.extend(t)
extend attaches the t list to the end of the list.
Does this give you what you expect?
Marilyn Davis
> print self.a
>
> Output:
>
> [[1, 234, 543]]
> [[1, 234, 548], [1, 234, 548]]
> [[1, 234, 59], [1, 234, 59], [1, 234, 59]]
> [[1, 237, 543], [1, 237, 543], [1, 237, 543], [1, 237, 543]]
>
>
> Can anyone help?
>
> thanks,
>
> Matthew (mdcooper at uvic dot ca)
>
>
> ___
> Tutor maillist - [EMAIL PROTECTED]
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
[Tutor] eval and exec
Hello Tutors, I'm having trouble understanding the difference between eval and exec. Can anyone explain it to me please? Marilyn Davis -- ___ Tutor maillist - [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] eval and exec
Thank you. You guys are great.
I was trying to eval("import %s" % something).
exec("import %s" % something) works just fine and now I understand why.
But, why is this so extremely dangerous?
Marilyn
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] eval and exec
On Sat, 4 Dec 2004, Chad Crabtree wrote:
> Marilyn Davis wrote:
>
> >Thank you. You guys are great.
> >
> >I was trying to eval("import %s" % something).
> >
> >exec("import %s" % something) works just fine and now I understand
> why.
> >
> >But, why is this so extremely dangerous?
> >
> >Marilyn
> >
> >
> Mainly it's only extremely dangerous if it's going to be attacked at
> all. What I mean is it will run any code that it imports this way,
> even
> untrusted code(possibly). Mostly I think that it's difficult to
> debug,
> however if it works you should use it. It seems that many people do
> this at one point or another, and considered I guess inelegent by
> some.
> If security is an issue then this is a very big no no according to
> what
> I've heard.
And Alan said:
> But much better to use the __import__() function for doing that if
> possible... Or simply importing all the modules you might need at the
> beginning, its not a big overhead...
>
> Alan G.
There's something about this that I'm not getting.
Is it more dangerous than having the python interpreter around?
Users will have access to our machine via the web and via email. We
want to be safe against attack.
As I understand it, Apache has modpython, so it runs all the python
code that happens, no matter how many users, with only one copy of the
interpreter in memory. It's sort of a big exec-machine, isn't it?
I want to do the same trick for my Mail Transfer Agent, exim. Exim
has a new feature where you can configure it to talk to an AF_UNIX
socket to get any info it needs. An AF_UNIX socket is file-based and
is not open for outside machines to connect to. So I made a little
python program with a socket and threads so that exim can call the
various python programs that I've written for sorting out mail.
I don't want to introduce insecurity. But also I want to really
understand what the problem is -- especially because I teach python.
And I can't see the security problem, unless there's a security
problem already, like if I allowed incoming email to dictate the
parameters that I send through the socket. The email provides data
for argv[1:] but argv[0] is hard-coded.
And I don't see how web traffic can get there at all.
If we had real users with login rights, then they could get to the
interpreter and wouldn't need my little daemon to wreck havoc -- if I
had my persmissions wrong.
So what am I missing?
Thank you for your help.
Marilyn
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] eval and exec
And how can __import__ be safer? If an attacker can usurp the string
in an exec call, couldn't he usurp the string in an __import__ call?
And couldn't he import untrusted code? It could have a call to exec()
in it? And what about the apply() function? Aren't all of these the
same open door?
I love the exec() call. I love the idea of code that makes and execs
code. I'll make myself obsolete. :^)
Marilyn
On Sun, 5 Dec 2004, Marilyn Davis wrote:
> On Sat, 4 Dec 2004, Chad Crabtree wrote:
>
> > Marilyn Davis wrote:
> >
> > >Thank you. You guys are great.
> > >
> > >I was trying to eval("import %s" % something).
> > >
> > >exec("import %s" % something) works just fine and now I understand
> > why.
> > >
> > >But, why is this so extremely dangerous?
> > >
> > >Marilyn
> > >
> > >
> > Mainly it's only extremely dangerous if it's going to be attacked at
> > all. What I mean is it will run any code that it imports this way,
> > even
> > untrusted code(possibly). Mostly I think that it's difficult to
> > debug,
> > however if it works you should use it. It seems that many people do
> > this at one point or another, and considered I guess inelegent by
> > some.
> > If security is an issue then this is a very big no no according to
> > what
> > I've heard.
>
> And Alan said:
>
> > But much better to use the __import__() function for doing that if
> > possible... Or simply importing all the modules you might need at the
> > beginning, its not a big overhead...
> >
> > Alan G.
>
> There's something about this that I'm not getting.
>
> Is it more dangerous than having the python interpreter around?
>
> Users will have access to our machine via the web and via email. We
> want to be safe against attack.
>
> As I understand it, Apache has modpython, so it runs all the python
> code that happens, no matter how many users, with only one copy of the
> interpreter in memory. It's sort of a big exec-machine, isn't it?
>
> I want to do the same trick for my Mail Transfer Agent, exim. Exim
> has a new feature where you can configure it to talk to an AF_UNIX
> socket to get any info it needs. An AF_UNIX socket is file-based and
> is not open for outside machines to connect to. So I made a little
> python program with a socket and threads so that exim can call the
> various python programs that I've written for sorting out mail.
>
> I don't want to introduce insecurity. But also I want to really
> understand what the problem is -- especially because I teach python.
>
> And I can't see the security problem, unless there's a security
> problem already, like if I allowed incoming email to dictate the
> parameters that I send through the socket. The email provides data
> for argv[1:] but argv[0] is hard-coded.
>
> And I don't see how web traffic can get there at all.
>
> If we had real users with login rights, then they could get to the
> interpreter and wouldn't need my little daemon to wreck havoc -- if I
> had my persmissions wrong.
>
> So what am I missing?
>
> Thank you for your help.
>
> Marilyn
>
> ___
> Tutor maillist - [EMAIL PROTECTED]
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] eval and exec
On Sun, 5 Dec 2004, Danny Yoo wrote:
>
> It pays to see a concrete example of an exploit that has occurred because
> of exec/eval misuse. For example, here's an old one from July 2002:
>
> http://www.securityfocus.com/bid/5255/discussion/
>
> Note that this one was in the Standard Library! We'd expect that the
> folks who implement the Standard Library should know what they are doing.
> And if the Python implementors can have trouble using eval() safely, then
> how much more should we be wary!
>
Thank you. Goodness, pickle would execute this if it unpickled it, as
I understand it:
"S''*__import__('os').system('echo 0wn3d')\np0\n."
Hm. Silly pickle.
>
> If you or your students are interested in security stuff, you may find
> David Wheeler's guide on "Secure Programming for Linux and Unix HOWTO" a
> good start:
>
> http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html
>
Very interesting, thank you.
And thank you everyone for your illuminations.
In our case, where the socket isn't available from outside, and the
call strings are hardcoded, in order to breech an eval() call in my
code, they'd already have the machine in their lap and could do
anything else. Our only ports that are open to the world are for
exim, and apache. Otherwise we let ssh and rysnc through if they are
coming from the right IP. So we're pretty tight.
However, I think it's good engineering policy to keep things as
tightly controlled as possible. So I think I'll take the good advice
given here and use apply() and do all my importing up front.
>
> It contains a small section specifically for Python:
>
> http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/python.html
>
BTW, the author of the Python section doesn't seem to like the
flexibility of Python's function calls.
All these lovely powerful things are security vulnerabilities.
Sigh.
Marilyn
>
> I don't think that we should go completely crazy over security issues:
> this issues are often so subtle that even experts get caught. But even
> so, I think we still have a responsibility to make sure the most
> egregrious security problems never come to fruition. So that's why most
> of us here will say eval() and exec() are evil. *grin*
>
>
> I hope this helps!
>
>
--
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] turning a number into a formated string
Here's one way:
left, right = str(number).split('.')
output = "%04d%d" % (int(left), int(right)) + (4 - len(right)) * '0'
Maybe someone has a more elegant way.
Hope it helps,
Marilyn Davis
On Wed, 15 Dec 2004, Ertl, John wrote:
> I need to take a number and turn it into a formatted string.
> The final output needs to look like when the X is the integer
> part padded on the left and Y is the decimal part padded on the right.
> I figured I could split the number at "." and then use zfill or something
> like this (LEVEL1 = "%04d" % LEVEL1) for the part but I am not sure
> how to right pad the decimal part of the number.
> Example.
> 1 and 1.0 needs to look like 0001 ( I figured I would have to check the
> length of the list made from the split to see if a decimal portion existed)
> 1.1 needs to look like 00011000
> 22.33 needs to look like 00223330
> .22 needs to look like 2200
> Any ideas on the right padding the decimal side using "0"
> Thanks,
> John
> ___
> Tutor maillist - [EMAIL PROTECTED]
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] turning a number into a formated string
On Wed, 15 Dec 2004, Tim Peters wrote:
> ... return ("%09.4f" % n).replace('.', '')
Totally cool.
Marilyn
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
[Tutor] A little Tkinter question
Hi Tutors, I'm reviewing GUIs and wondering: When you pack a component, and you specify fill = BOTH, how is this different from expand = YES? Thank you for any insight. Marilyn Davis ___ Tutor maillist - [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] A little Tkinter question
On Wed, 15 Dec 2004, Gregor Lingl wrote:
>
>
> Marilyn Davis schrieb:
> > Hi Tutors,
> >
> > I'm reviewing GUIs and wondering:
> >
> > When you pack a component, and you specify fill = BOTH, how is this
> > different from expand = YES?
> >
> Hi Marilyn,
> This is a bit tricky and hard to explain, so I recommend playing around
> with this little program from John Grayson's Tkinter book:
>
> from Tkinter import *
>
> class App:
> def __init__(self, master):
> master.geometry("300x200")
> fm = Frame(master)
> Button(fm, text='Left').pack(side=LEFT, fill=BOTH, expand=1)
> Button(fm, text='Center').pack(side=LEFT, fill=BOTH, expand=1)
> Button(fm, text='Right').pack(side=LEFT, fill=BOTH, expand=1)
> fm.pack(fill=BOTH, expand=YES)
>
>
> root = Tk()
> root.option_add('*font', ('verdana', 12, 'bold'))
> root.title("Pack - Example 9")
> display = App(root)
> root.mainloop()
>
> Modify the three lines with Button().pack(...)
> for instance by setting expand = 0 (which is the default value)
>
> Button(fm, text='Left').pack(side=LEFT, fill=BOTH, expand=0)
> Button(fm, text='Center').pack(side=LEFT, fill=BOTH, expand=0)
> Button(fm, text='Right').pack(side=LEFT, fill=BOTH, expand=0)
>
> or by tropping the fill option:
>
> Button(fm, text='Left').pack(side=LEFT, expand=1)
> Button(fm, text='Center').pack(side=LEFT, expand=1)
> Button(fm, text='Right').pack(side=LEFT, expand=1)
>
> and so on.
>
> These Examples are from Chapter 5, several others concerning the packer
> can be found at:
>
> https://secure.manning.com/catalog/view.php?book=grayson&item=source
Thanks for the pointer, Gregor.
I guess that the jist of it is that fill fills all available space.
And expand is about what happens to the widget when its master
expands.
But, the bit of the book I was looking at, I'm afraid, I found to be
difficult to follow. There seemed to be outputs without matching
code, and the numbering system confused me. And he pointed out the
difference between fill alone and fill with expand and they looked the
same.
I guess you are right. One needs to experiment to get the desired
effect.
I have no desired effect. I was just looking for intellectual
understanding.
Thank you.
Marilyn
>
> Hope this helps
> Gregor
>
>
> > Thank you for any insight.
> >
> > Marilyn Davis
> >
> >
> > ___
> > Tutor maillist - [EMAIL PROTECTED]
> > http://mail.python.org/mailman/listinfo/tutor
> >
> >
>
>
--
___
Tutor maillist - [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What am I doing wrong...
Hi Ken, Welcome to python! Adding one line should do it for you: On Thu, 23 Dec 2004, Ken Stevens wrote: > I am a elative new comer to python. I wrote the following test > snippet. > > #!/usr/bin/env python > > def main (): > play_test() > > def play_test (): > print "Hi! -- in play test" main() > > When I do "python test.py" absolutely nothing happens. > > I expect it to do the print statement. > > Thanks in advance for your help. > > Ken > > > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] tempfile
Hello Python Tutors, I'm using tempfile. The doc says it is opened 'w+b' so that it can be read and written without closing and reopening. But, for the life of me, I can't find any way to rewind it so I can read what I wrote. tempfile.mkstemp gives me the file descriptor. Is there a way to make a file object from a file descriptor? Or, is there something in os or fcntl that allows me to rewind from a file descriptor? Thank you for any help you can give. Marilyn Davis ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] tempfile (fwd)
Ooops. I forgot to send to the list. -- Forwarded message -- Date: Thu, 23 Dec 2004 14:53:18 -0800 (PST) From: Marilyn Davis <[EMAIL PROTECTED]> To: QoD SEC <[EMAIL PROTECTED]> Subject: Re: [Tutor] tempfile On Thu, 23 Dec 2004, QoD SEC wrote: > you could use the seek method of the file object to go to the > beginning of the file: Thank you. But my problem was that I had a "file descriptor", a low-level thing, and not a "file object", a high-level thing. A file descriptor doesn't have a seek. But I found the os.fdopen(fd) call. It makes a file object from a file descriptor. So I can do: #!/usr/bin/env python import tempfile import os def do_test(): tmp_fp, tmp_name = tempfile.mkstemp() os.write(tmp_fp, "stuff") file_obj = os.fdopen(tmp_fp) file_obj.seek(0) print os.read(tmp_fp,10) os.close(tmp_fp) do_test() --- and "stuff" comes out. But, the mystery now is: def do_test(): tmp_fp, tmp_name = tempfile.mkstemp() os.write(tmp_fp, "stuff") os.fdopen(tmp_fp).seek(0) print os.read(tmp_fp,10) os.close(tmp_fp) --- gets an OSError: [Errno 9] Bad File Descriptor on os.read(tmp_fp,10) I'm thinking that, if I don't keep a reference to the file object, it gets automatically closed. How pythonic. Thank you for your thoughts. Marilyn > here is what the library reference says about it: > > seek( offset[, whence]) > Set the file's current position, like stdio's fseek(). The whence > argument is optional and defaults to 0 (absolute file positioning); > other values are 1 (seek relative to the current position) and 2 (seek > relative to the file's end). There is no return value. Note that if > the file is opened for appending (mode 'a' or 'a+'), any seek() > operations will be undone at the next write. If the file is only > opened for writing in append mode (mode 'a'), this method is > essentially a no-op, but it remains useful for files opened in append > mode with reading enabled (mode 'a+'). If the file is opened in text > mode (mode 't'), only offsets returned by tell() are legal. Use of > other offsets causes undefined behavior. > > Note that not all file objects are seekable. > > from http://docs.python.org/lib/bltin-file-objects.html > > > On Thu, 23 Dec 2004 14:17:12 -0800 (PST), Marilyn Davis > <[EMAIL PROTECTED]> wrote: > > Hello Python Tutors, > > > > I'm using tempfile. The doc says it is opened 'w+b' so that it can be > > read and written without closing and reopening. > > > > But, for the life of me, I can't find any way to rewind it so I can > > read what I wrote. > > > > tempfile.mkstemp gives me the file descriptor. Is there a way to make > > a file object from a file descriptor? > > > > Or, is there something in os or fcntl that allows me to rewind from a > > file descriptor? > > > > Thank you for any help you can give. > > > > Marilyn Davis > > > > ___ > > Tutor maillist - Tutor@python.org > > http://mail.python.org/mailman/listinfo/tutor > > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Re: tempfile (fwd)
And again. -- Forwarded message -- Date: Thu, 23 Dec 2004 14:55:15 -0800 (PST) From: Marilyn Davis <[EMAIL PROTECTED]> To: Lee Harr <[EMAIL PROTECTED]> Subject: Re: [Tutor] Re: tempfile os.lseek! How did I not find that. Thank you. I'm set. Marilyn On Fri, 24 Dec 2004, Lee Harr wrote: > >I'm using tempfile. The doc says it is opened 'w+b' so that it can be > >read and written without closing and reopening. > > > >But, for the life of me, I can't find any way to rewind it so I can > >read what I wrote. > > > > >>>import tempfile > >>>import os > >>>fd, name = tempfile.mkstemp() > >>>os.write(fd, 'foo') > 3 > >>>os.lseek(fd, 0, 0) > 0L > >>>os.read(fd, 10) > 'foo' > > > > >tempfile.mkstemp gives me the file descriptor. Is there a way to make > >a file object from a file descriptor? > > > > Not that I can see: > http://docs.python.org/lib/os-fd-ops.html > > ... except maybe this, which is not quite the same: > > >>>import tempfile > >>>import os > >>>fd, name = tempfile.mkstemp() > >>>os.write(fd, 'foo') > 3 > >>>f = open(name) > >>>f.read() > 'foo' > > _ > Express yourself instantly with MSN Messenger! Download today it's FREE! > http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] tempfile
On Fri, 24 Dec 2004, Kent Johnson wrote: > Marilyn Davis wrote: > > Hello Python Tutors, > > > > I'm using tempfile. The doc says it is opened 'w+b' so that it can be > > read and written without closing and reopening. > > > > But, for the life of me, I can't find any way to rewind it so I can > > read what I wrote. > > > > tempfile.mkstemp gives me the file descriptor. Is there a way to make > > a file object from a file descriptor? > > Hmm, let's take a look at the docs. > tempfile.TemporaryFile() gives you a file object. Can you use that? Maybe you > are using mkstemp() > because you want the file to persist after it is closed? Right. > > Ok, the docs say TemporaryFile() uses mkstemp() to create its file. So how > does it get a file > object? Is the soure available? Sure, in Python24\Lid\tempfile.py. It uses > os.fdopen() to convert a > file descriptor to a file object. Will that work for you? Looking at the source was a good idea. I'll try to remember that next time I'm stuck. Thank you. Marilyn > > Kent > > > > > Or, is there something in os or fcntl that allows me to rewind from a > > file descriptor? > > > > Thank you for any help you can give. > > > > Marilyn Davis > > > > ___ > > Tutor maillist - Tutor@python.org > > http://mail.python.org/mailman/listinfo/tutor > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
RE: [Tutor] Debugging in emacs
Hi Toby, You still here? I'm finally trying (again) to get debugging going. On Mon, 22 Nov 2004, McLaughlin, Toby wrote: > Thanks Marilyn, > > Debugging in emacs is obviously not the hottest topic around here. > > Luckily, I have something working now. For anyone who is interested, > here is the trick: > > 1. Add the following to your program: > import pdb > pdb.set_trace() > 2. Start an interactive Python buffer (C-c !) This doesn't seem to do anything. But, when I run my program, after following step 1., I'm in the debugger. And it is a beautiful sight. > 3. Run your program (I use C-c C-c) > > Emacs picks up the line numbers from set_trace() and marks the > appropriate line in the source buffer. You can now type 'n' to single > step, as well as issue any of the pdb commands mentioned here: > http://tinyurl.com/3na4u . If I set a breakpoint, then 'c'/'continue' doesn't work. Instead of continuing, it only goes one step. If I don't set a breakpoint, then continue does work. But that's no use. > > The only trouble I have now is that the Python buffer and my source code > keep switching places from top to bottom. A minor annoyance that > hopefully I will rectify soon. This doesn't happen to me. I am using regular ole raw emacs, on a plain terminal, no X. Are you? How are you doing with this? I could sure use a debugger. Marilyn > > Toby McLaughlin. > > > -Original Message- > > From: Marilyn Davis [mailto:[EMAIL PROTECTED] > > Sent: Saturday, 20 November 2004 9:28 AM > > To: McLaughlin, Toby > > Cc: tutor@python.org > > Subject: Re: [Tutor] Debugging in emacs > > > > > > On Fri, 19 Nov 2004, McLaughlin, Toby wrote: > > > > > Hi All, > > > > > > Could somebody suggest a way to debug python in emacs? I'd > > like to be > > > able to single-step while seeing the code marked in some way and > > > possiblly set breakpoints. I read about using pdb from the > > command line > > > but would love to have debugging integrated in emacs. > > Perhaps I should > > > make life easy for myself and use one of the fancy GUI > > editors, but I > > > think it's character building for me to learn emacs. > > > > Boy, you could hear a pin drop on this topic. > > > > I use emacs, even though I have spent some time with idle, because I > > like it much better. I'm real handy in emacs though, and I don't like > > mousing. Mice cause lots of repetitive stress injury. > > > > I spent some time and some email conversations trying to make use of > > the debugger in any environment. I don't remember what happened > > exactly but I gave up. > > > > The thing about an interpreted language is that you can say "print > > my_var" in the code instead of in the debugger. And with emacs, I go > > back and forth between my code and the emacs shell to run it. I'm > > pretty efficient that way. > > > > If anyone has any suggestions about debugging, in idle or anywhere, I > > too would love to hear them. > > > > Thank you. > > > > Marilyn Davis > > > > > > > > > > Thanks, > > > Toby McLaughlin. > > > ___ > > > Tutor maillist - Tutor@python.org > > > http://mail.python.org/mailman/listinfo/tutor > > > > > > > -- > > > > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] flattening a list
On Wed, 12 Jan 2005 [EMAIL PROTECTED] wrote: > Quoting Bill Kranec <[EMAIL PROTECTED]>: > > > I have a list of lists, for example [ [1,2] , [3,4] ], and I would like > > to pass all the elements of that list as arguments to a function (for > > example the intersection of all list elements). Is there a command in > > regular Python to do this? I would like to avoid the hassle and speed > > hit of a loop to extract all the list elements. > > I don't think so... > > There is a recipe on activestate for a flatten function. > > Or you could use a list comprehension: > > >>> arr = zip(range(10), range(10, 20)) > >>> arr > [(0, 10), (1, 11), (2, 12), (3, 13), (4, 14), (5, 15), (6, 16), (7, 17), (8, > 18), (9, 19)] > >>> [x for y in arr for x in y] > [0, 10, 1, 11, 2, 12, 3, 13, 4, 14, 5, 15, 6, 16, 7, 17, 8, 18, 9, 19] > Nice. And there's: >>> arr = zip(range(10), range(10, 20)) >>> arr [(0, 10), (1, 11), (2, 12), (3, 13), (4, 14), (5, 15), (6, 16), (7, 17), (8, 18), (9, 19)] >>> reduce(lambda x,y:x+y, arr) (0, 10, 1, 11, 2, 12, 3, 13, 4, 14, 5, 15, 6, 16, 7, 17, 8, 18, 9, 19) >>> > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] sockets, files, threads
Hello Tutors, I've been banging my head against this bug for 4 days. I can't think of how to ask for help except to give a birds eye view of the situation. I have a python daemon that is listening on a UNIX socket for communication with exim, my Mail Transfer Agent. Each mail message has its own exim process. The exim process speaks to my daemon, who starts a thread which reads everything there is on the socket, and sends and 'OK' to send exim on its way. That exim process finishes quickly because that 'OK' means to dump the message, I am taking responsibility for it from here. When stuff was read from the exim socket, it was stored in a tempfile, so that I could release the exim process, then I lseek to the front of the tempfile and have it handy. I see from all my debugging and logging that the file descriptor for this tempfile is 9. The program then opens a pipe to exim to send mail. I see that the popen2.popen3 call returns 9 for the stdin file descriptor for the pipe. The tempfile (also file descriptor 9) is read for piping into exim and it errors with "Bad file descriptor". Worse yet, the first 5 messages of my test go through the entire process without a problem, and then # 6 hits this -- but only if # 1 message is really big. I'm at a loss. Does anyone have an idea? Marilyn Davis ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
On Wed, 12 Jan 2005, Danny Yoo wrote:
Thank you so much for thinking about this Danny.
> > When stuff was read from the exim socket, it was stored in a tempfile,
> > so that I could release the exim process, then I lseek to the front of
> > the tempfile and have it handy. I see from all my debugging and logging
> > that the file descriptor for this tempfile is 9.
>
> Hi Marilyn,
>
> Question: do you really need to use a tempfile? If your daemon is
> persistent in memory, can it just write to a StringIO object?
Mail messages can be really big. I was thinking that I'd be better
off if I put them on the disk. Am I wrong?
Are we suspecting tempfile? Might it have a shared resource?
> StringIO.StringIO elements act like files too, and may be easier to
> maintain than a tempfile.TemporaryFile. If you can show us how you're
> constructing and using the TemporaryFile, we can try to trace any
> problematic usage.
OK. I'll try to cut-and-paste together something tomorrow. I have to
quit now.
> > The program then opens a pipe to exim to send mail. I see that the
> > popen2.popen3 call returns 9 for the stdin file descriptor for the pipe.
> >
> > The tempfile (also file descriptor 9) is read for piping into exim and
> > it errors with "Bad file descriptor".
>
> Oh! This situation sounds like the 'tempfile' is being closed at some
> point, releasing the file descriptor back to the system. If that is what
> is happening, then that's why the pipe has the same descriptor id: the
> call to pipe() just reuses a free file descriptor.
Yes. So in my logfile, I log every creation, opening and closing of
everything and every write and read and seek. And I just don't see
it.
>
> I'd look for places where the tempfile might be close()d. I'd also look
> for places where your reference to tempfile is reassigned, since that
H. I'll look for reassignments. This is likely. You mean the
tempfile would think its time to close if it got reassigned?
> would also signal a resource collection.
>
>
> How do you deal with threads? Is the temporary file a global resource
> that the threads all touch? If so, have you done any synchronization to
No. Only the thread that creates it reads it.
> make sure that at most one thread can touch the temporary file at a time?
> What are the shared resources for the threads?
I can't think of any -- unless there are hidden things in os.popen? or
tempfile.mkstemp or open.
They do share a logfile. But each entry opens it, appends it, and
closes it.
My threads seem to me to be very independent.
>
>
> The situation you mentioned,
>
> > > Worse yet, the first 5 messages of my test go through the entire
> > > process without a problem, and then # 6 hits this -- but only if # 1
> > > message is really big.
>
> is exactly the sort of thing I'd expect if two threads were contending for
> the same resource, so let's see if the bug has to do with this.
"Resource". That's a file, a socket, a pipe, what else?
Well, I'll sleep and do some code review with your questions in mind.
>
> Best of wishes to you!
Thank you!
I was looking at my use of file objects and file descriptors and I
wrote this sample program and was very surprised by the result --
which makes me think there's something here that I don't understand.
Where did my 'ooo' go?
#! /usr/bin/env python
import os
fobj = open('/tmp/xxx','w')
fobj.write('ooo\n')
fp = fobj.fileno()
os.write(fp,'x\n')
os.close(fp)
##
# OUTPUT:
[EMAIL PROTECTED] tmp]# ./file.py
[EMAIL PROTECTED] tmp]# cat xxx
x
[EMAIL PROTECTED] tmp]#
Thank you so much for your questions and thoughts. We'll see what the
morning brings.
Marilyn
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
Whew! What a trip this bug has been! Danny was exactly right here: > is exactly the sort of thing I'd expect if two threads were > contending for the same resource, so let's see if the bug has to do > with this. This bug drove me nuts. (it's a short drive) So I started wrapping my file openings in lock.acquire(), lock.release() and nothing seemed to change. Imagine my discouragement. So I knuckled down and began to prepare some code for posting here, documenting, cleaning, generating a pleasant log file, ... While doing this, I became more and more convinced that my code is good. Then it hit me. Maybe I should upgrade to 2.4. Maybe I don't have thread-safety happening in my current situation. Maybe I wasn't planning for threads when I compiled 2.3. The new installation MySQL-python tells me that the mysqlclient library is mostly, but not guaranteed, thread-safe and I should use mysqlclient_s. That didn't exist before. So I bring up the latest of everything and things are much worse. My updates to mysql are ignored without an error. I can't run *anything* Now, at that point, on Thursday night, I was so frustrated and discouraged. I was thinking that I should start over in C, that python is a great teaching language, as originally intended, but for production work, forget it. I was so wrong! I don't know how Charlie, my boss and my son, found this, because I googled and googled and couldn't find it, even after I knew what I was looking for. But, the new MySQL-python has changed its behavior with respect to InnoDB tables. This is some relatively new MySQL table type that allows foreign key constraints -- and upon which we totally depend. Now I must do a commit after each update or insert. Then things started to clear up. Adding locks started to make a difference. It turns out that the shared resource causing all my trouble was, indeed, the operating system's file descriptor generation process. I wrapped the same lock around file openings, socket creations, popen calls, and MySQL connectings and my troubles disappeared. However, when, for curiosity, I ran my stuff under 2.3 again, it worked! So I must have missed it that the first few locks did indeed help my situation. It's easy to miss because if any file-descriptor-dependent operation is left outside the lock, it wrecks havoc in all file-descriptor-dependent operations. So the errors looked the same at first glance. But everything is groovy-to-the-max now. I have my MTA deferring messages because of MySQL's "Too many connections" and the python has stepped out of the performance picture for being so fast, so so fast. Or so it looks today. And thank you Alan, for this reminder: > As was said in a recent thread, mixing file objects and > file descriptors is a bad idea. The OS can get confused, > let alone the programmer! Yeh. Gee. I think I started that thread too. Speaking about confused! Thank you so much again, Alan and Danny and other tutors. Knowing that I have this list to turn to got me through this. Marilyn ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
Dearest Tutors, Bah! It's not over yet. I don't know why, but again my file descriptors are being trampled upon now and then. This time I can see in my log that I'm not trampling on them myself, like I used to do, unless I'm making calls to the system that I'm not aware of. And, first I get this worrisome output: close failed: [Errno 9] Bad file descriptor close failed: [Errno 9] Bad file descriptor close failed: [Errno 9] Bad file descriptor coming from nowhere. No traceback, nothing. I traced this down once to discover that it came out of python *after* a raise, after a failed call to open a connection to mysql. In that particular raise clause, I called my facility to pipe mail to exim, opening 3 descriptors. I guessed then that when closing down, it closes down the pipe descriptors that I had already closed? So, if it does this, if it closes down descriptors that I've already closed, it's possible that it's closing down descriptors that have also already been reopened because I'm running this thing like lightning? And the other thought is that there is some other call I make that uses a file descriptor that I'm not thinking of. os.listdir? os.anything? Any new thoughts? Marilyn ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
p.p.s. I have only wrapped my lock around file-descriptor creations. Should I wrap it around closings too? Or the whole open -> close transaction? It sounds like error-prone work to do the latter. What am I missing? What should I be reading to get a better clue? I'll do some googling. Thank you. On Sat, 15 Jan 2005, Marilyn Davis wrote: > Dearest Tutors, > > Bah! It's not over yet. I don't know why, but again my file > descriptors are being trampled upon now and then. > > This time I can see in my log that I'm not trampling on them myself, > like I used to do, unless I'm making calls to the system that I'm not > aware of. > > And, first I get this worrisome output: > > close failed: [Errno 9] Bad file descriptor > close failed: [Errno 9] Bad file descriptor > close failed: [Errno 9] Bad file descriptor > > coming from nowhere. No traceback, nothing. > > I traced this down once to discover that it came out of python *after* > a raise, after a failed call to open a connection to mysql. In that > particular raise clause, I called my facility to pipe mail to exim, > opening 3 descriptors. I guessed then that when closing down, it > closes down the pipe descriptors that I had already closed? > > So, if it does this, if it closes down descriptors that I've already > closed, it's possible that it's closing down descriptors that have > also already been reopened because I'm running this thing like > lightning? > > And the other thought is that there is some other call I make that > uses a file descriptor that I'm not thinking of. os.listdir? > os.anything? > > Any new thoughts? > > Marilyn > > > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
On Sat, 15 Jan 2005, Danny Yoo wrote: > > > > I have only wrapped my lock around file-descriptor creations. Should I > > wrap it around closings too? Or the whole open -> close transaction? > > It sounds like error-prone work to do the latter. What am I missing? > > Hi Marilyn, > > Can you send a link to the source code to the Tutor list? I'm getting the > feeling that there's might be a design problem. Just adding locks > whenever something doesn't work is not a sustainable way to write a > multithreaded application. > > We have to see why your file descriptors being are being shared between > threads. Is there a reason why you need to share them as global > resources? No. And I don't. They are often attributes of instantiations of classes; or they come and go quickly. Thank you for offering to look at the code. Start at: http://www.maildance.com/python/doorman/README Then, the daemon that creates the threads is: http://www.maildance.com/python/doorman/py_daemon.py I'm testing using calls to: http://www.maildance.com/python/doorman/route_mail.py http://www.maildance.com/python/doorman/doorman.py Other modules that open and close file descriptors from there are: http://www.maildance.com/python/doorman/db.py http://www.maildance.com/python/doorman/doorman_log.py http://www.maildance.com/python/doorman/exim.py http://www.maildance.com/python/doorman/move.py I'll be grateful for any improvements you suggest. But, I do know that some modules aren't well-documented, or are hardly documented at all yet. And db.py seems like a mess to me. But I'm not ready to straighten it up yet. But the most important one, py_daemon.py, I hope is very readable. Still though, I should confess, I am feeling a bit dismal about the thread situation, especially since searching around for info on critical code. The examples I found, before I gave up, were all trivial. Also, there is this article: http://linuxgazette.net/107/pai.html Which says that the performance is almost the same with threads as with single-threading. Boo. So suddenly, I have no idea why I'm down this road. We have a huge performance savings from setting up a daemon that reads a socket. But, unless something changes our minds soon, I'll rip threading out of our code. Still though, I am a teacher too, and a student. So any thoughts you have about our code will be treated like the pearls they are. Thank you again. Marilyn > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
Thank you so much Danny. I know how hard it is to look at and comment
on other people's code. You know I teach C and Python and I have to
say, though, that reading students' Python is 100 times easier than
reading their C.
And, I hope you're feeling better. I hate to think of you struggling
through my code when you should be resting.
>
> Hi Marilyn,
>
>
> [Long program comments ahead. Please forgive me if some of the comments
> are overbearing; I'm trying to get over a cold, and I'm very grumpy.
> *grin*]
>
>
> Some comments on the code follow. I'll be focusing on:
>
> > http://www.maildance.com/python/doorman/py_daemon.py
>
> One of the import statements:
>
> ###
> from signal import *
> ###
>
>
> may not be safe. According to:
>
> http://docs.python.org/tut/node8.html#SECTION00841
>
> """Note that in general the practice of importing * from a module or
> package is frowned upon, since it often causes poorly readable code.
Note that it says it is frowned upon, and I almost never do it,
because it 'often causes poorly readable code' -- but not in this
case. There is only signal.signal and signal.alarm and the signal
numbers, none of which are easy to mistake for something else. I have
never seen 'unsafe' --unless you cause yourself a bug by hiding
something from yourself, which would be hard to do in this case.
But, ok, to encourage you to make suggestions, I used the regular
import on signals. No change to the bugs.
> However, it is okay to use it to save typing in interactive sessions, and
> certain modules are designed to export only names that follow certain
> patterns."""
>
>
> There's a block of code in Server.run() that looks problematic:
Darn it! I managed to put up a hybrid version. Here's what I thought
I should have, to wrap a lock around the creation of the file
descriptor for the client socket:
self.server_socket.setblocking(0)
while 1:
if log.level & log.calls:
log.it("fd%d:py_daemon.py: Waiting ...", self.descriptor)
try:
file_lock.acquire()
try:
client_socket, client_addr = self.server_socket.accept()
finally:
file_lock.release()
except socket.error, msg:
if str(msg).find('Resource temporarily unavailable') > -1:
time.sleep(.2)
continue
except (EOFError, KeyboardInterrupt):
self.close_up()
Spawn(client_socket).start()
Not that it matters to your suggestions. But without locks, this
reduces to:
while 1:
if log.level & log.calls:
log.it("fd%d:py_daemon.py: Waiting ...", self.descriptor)
try:
client_socket, client_addr = self.server_socket.accept()
except (EOFError, KeyboardInterrupt):
self.close_up()
Spawn(client_socket).start()
But, I take it, you want me to put Spawn(client_socket).start() in that
try: clause.
>
> The problem is that, as part of program flow, it appears to run after the
> try block. But in one particular case of program flow, 'client_socket'
> will not be set to a valid value. It is better to put that statement a
I don't understand. Which particular case of program flow will
'client_socket' be invalid and yet make it through the socket.accept
call?
> few lines up, right where 'client_socket' is initialized. Like this:
>
> ###
> try:
> client_socket, client_addr = self.server_socket.accept()
> Spawn(client_socket).start()
> except socket.error, msg:
> time.sleep(.5)
> except (EOFError, KeyboardInterrupt):
> self.close_up()
> ###
>
> Not only does this make it more clear where 'client_socket' is being used,
> but it ends up making the code shorter. I've dropped the 'continue'
> statement, as it becomes superfluous when the Spawn() moves into the try's
> body.
But but but, that wraps the whole thread in one try/except clause.
That Spawn() call starts the thread. If a client_socket generates an
error, we don't want the main thread to sleep. All the socket
operations in Spawn.start() are also wrapped in their own (hopefully)
intelligent try/excepts. I thought it was good practice to -not- put
extra stuff in a try/except.
>
> In fact, the placement of that statement may account partially for the
> error message that you were running into earlier. The earlier error
> message:
>
> ###
> close failed: [Errno 9] Bad file descriptor
> ###
>
> can easliy occur if there's an EOFError or KeyboardInterrupt under the
> original code. And although KeyboardInterrupt might be unusual, I'm not
> so sure if EOFError is.
I thought I listed exactly the two errors that come from pounding on
the keyboard. I try to be careful not to mask any tracebacks.
And, the original w
Re: [Tutor] sockets, files, threads
>
> > What about when I do an explicit call to a close inside a __del__. Is
> > that a bad idea?
>
> I usually prefer to add a close() method to my objects that releases
> resources, rather than __del__(), because it's more visible.
>
OK Danny! I found it! When I was almost asleep last night, I knew
what to do and this morning I whittled it down to this situation.
Maybe it can be whittled further but I'm holding up our project and I
can't take any more time right now.
Note that the two variables TO_WHOM and MAILER, I think, can be safely
changed to suit your environment, indeed MAILER='mail' should work for
others.
It happens on 2.4 but not on 2.3. Either 2.3 masks my error, or 2.4
has an error?
I wouldn't be surprised if this is responsible for my thread problem
but we're not going to find out.
#!/usr/bin/env python2.4
'''On my python2.4, this produces:
./test_exim.py
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor
but it runs just fine.
but if I specify python2.3, it runs cleanly.
'''
TO_WHOM = 'marilyn'
MAILER = 'exim'
import popen2
import select
import os
class Exim:
def __init__(self):
self.fdin = None
self.err_flush = []
self.stdout, self.stdin, self.stderr = popen2.popen3('%s -t' % MAILER)
self.fdin = self.stdin.fileno()
self.fdout = self.stdout.fileno()
self.fderr = self.stderr.fileno()
self.outlist = [self.fdout, self.fderr]
self.inlist = [self.fdin]
def __del__(self):
self.close_up()
def close_up(self):
if not self.fdin:
return
count = 0
in_status = os.close(self.fdin)
while 1:
sel = select.select(self.outlist, [], [], 1)
if sel[0] == []:
break
else:
for fd in sel[0]:
r = os.read(fd, 16384)
if r:
self.err_flush += [r]
count = 0
count += 1
if count >= 5:
break
else:
continue
break
self.err_flush = ''.join(self.err_flush)
out_status = os.close(self.fdout)
err_status = os.close(self.fderr)
self.fdin = None
if not in_status:
return
raise RuntimeError, self.err_flush
def write(self, stuff):
while 1:
sel = select.select(self.outlist, self.inlist, [], 1)
if sel[1] != []:
os.write(sel[1][0], stuff)
return
if __name__ == '__main__':
msg = '''To: %s
xx''' % TO_WHOM
p = Exim()
p.write(msg)
del p
On Tue, 18 Jan 2005, Danny Yoo wrote:
>
> [About using the Standard Library]
>
> > And since then, please don't shoot me, but I don't immediately trust the
> > modules. I read them and see how many times they loop through the data,
> > and how many copies of the data they put into memory -- and usually
> > decide to write the simple things I need myself, looping zero times and
> > keeping only one block in memory.
>
> Hmm.. . Do you remember which Standard Library modules you were looking at
> earlier? Perhaps there was some funky stuff happening, in which case we
> should try to fix it, so that no one else runs into the same problems.
Yes. poplib and socket. But, it's not fair to assume that something
funky is happening in there. A poplib should provide a set of tools
for making a pop client and I'm not making a pop client, just pumping
the message from a pop server through my python/mysql magic and into
exim. Similarly with socket. It does buffering and prepares things
that I don't need, but probably lots of people do. So when I say I
don't "trust" stdlib modules, I mean I don't trust them to be simple
enough for my situation.
However, all that said, I do dimly remember that poplib perhaps had
some extra processing that maybe is not needed -- but I could be
wrong.
> In a similar vein, in Spawn.run(), it might be a good idea to explicitely
> call write_and_close() on our FileSocket instance. For example, we can
> add a try/finally in the body of the revised start() method:
>
> ###
> def start(self):
> '''Given the command, provides the function to call.'''
> global TESTING
> function = { 'acl_rcpt.py' : calls.acl_rcpt,
> 'route_mail.py' : calls.route_mail,
> 'route_errors.py' : calls.route_errors,
> 'doorman.py' : calls.doorman,
> 'doorman_errors.py' : calls.doorman_errors,
> 'is_known_to.py' : is_known_to
> }
> if log.level & log.calls:
> log.it('fd%d: %s: %s
Re: [Tutor] sockets, files, threads
Thank you Kent. On Wed, 19 Jan 2005, Kent Johnson wrote: > Marilyn Davis wrote: > >>few lines up, right where 'client_socket' is initialized. Like this: > >> > >>### > >>try: > >>client_socket, client_addr = self.server_socket.accept() > >>Spawn(client_socket).start() > >>except socket.error, msg: > >>time.sleep(.5) > >>except (EOFError, KeyboardInterrupt): > >>self.close_up() > >>### > >> > >>Not only does this make it more clear where 'client_socket' is being used, > >>but it ends up making the code shorter. I've dropped the 'continue' > >>statement, as it becomes superfluous when the Spawn() moves into the try's > >>body. > > > > > > But but but, that wraps the whole thread in one try/except clause. > > That Spawn() call starts the thread. If a client_socket generates an > > error, we don't want the main thread to sleep. All the socket > > operations in Spawn.start() are also wrapped in their own (hopefully) > > intelligent try/excepts. I thought it was good practice to -not- put > > extra stuff in a try/except. > > Use try: except: else: > The else clause is only executed if no exception happens, so the code is > correct; the else is > outside the scope of the try, so you don't catch unexpected exceptions. > > try: > client_socket, client_addr = self.server_socket.accept() > except socket.error, msg: > time.sleep(.5) > except (EOFError, KeyboardInterrupt): > self.close_up() > else: > Spawn(client_socket).start() That makes good sense. But I'm not threading anymore and my call to server_socket.accept now blocks so I'm not sleeping. The only exceptions I want to catch now are those keyboard things, which make me stop. The socket.error I was catching was that there was no connections waiting right now. So my situation is much simpler. Marilyn > > Kent > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
On Wed, 19 Jan 2005, Danny Yoo wrote:
> On Wed, 19 Jan 2005, Marilyn Davis wrote:
>
> > class Exim:
> > def __init__(self):
> > self.fdin = None
> > self.err_flush = []
> > self.stdout, self.stdin, self.stderr = popen2.popen3('%s -t' %
> > MAILER)
> > self.fdin = self.stdin.fileno()
> > self.fdout = self.stdout.fileno()
> > self.fderr = self.stderr.fileno()
>
>
> Hi Marilyn,
>
> You probably don't need to explicitly use the file descriptors here. I see
> that you're using them because of the use of select() later on:
Yippee! This stopped the error. Whew! Odd that it didn't happen on
2.3.
Whatever.
One wonders, how many times do you guys have to tell me to stop mixing
file descriptors and file objects? I guess this was old code, before
you told me that <-- weak excuse.
Having the __del__ doesn't seem to hurt anything, which is a relief.
There are times when I want to call close_up() and not __del__ in my
real code.
>
>
> > Are you well yet? A lot of sick people around these days!
>
>
> Feeling better.
>
>
Good. Take care.
And, thank you some more.
Marilyn
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
Danny! I couldn't resist trying threading again, now that the mysterious single-threading behavior is gone. I didn't put any locks anywhere. And it runs like a champ. Like a super champ. Either I had to put back threading or I had to make a family of socket-readers, or lose some functionality in my test scheme. So I gave it a shot. And, I'm very happy. Thank you again. Marilyn p.s. I hope you're well. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Are you allowed to shoot camels? [kinda OT]
I once heard that Larry Wall said, "Perl is worse than Python because people needed it worse". And I've heard it said that "Perl is the Write-Only language" Marilyn ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
RE: [Tutor] Are you allowed to shoot camels? [kinda OT]
I think Danny was saying that if you don't like:
if var == 'a':
print 'a'
elif var == 'b' or var == 'c':
print 'b or c'
elif var == 'd':
pass
else:
print 'default case'
you might like his dispatch scheme. And it has been mighty nice and
handy for me since he taught me, in some special cases.
I'll whisper that I'm a tiny bit disappointed to see the vaguely
demeaning 'are you joking' theme that has emerged in here. It's
unusual for us to be anything but generous and kind with each other.
I guess this is a hot topic. :^)
Marilyn
On Fri, 4 Feb 2005, Smith, Jeff wrote:
> Now who's joking? Are you saying that
>
> switch var:
> case 'a':
> print 'a'
> case 'b' or 'c':
> print 'b or c'
> case 'd':
> pass
> default:
> print 'default case'
>
> Is less clear and maintainable than
>
> def do_this_function():
> print 'a'
>
> def do_that_function():
> print 'b or c'
>
> def do_pass_function():
> pass
>
> def do_default_function():
> print 'default case'
>
> ftable = { 'a' : do_this_function,
>'b' : do_that_function,
>'c' : do_that_function,
>'d' : do_pass_function }
> ftable.get(var, do_default_function)()
>
> Ugh!
>
>
> -Original Message-
> From: Alan Gauld [mailto:[EMAIL PROTECTED]
> Sent: Friday, February 04, 2005 1:14 PM
> To: Smith, Jeff; Jacob S.; [EMAIL PROTECTED];
> tutor@python.org
> Subject: Re: [Tutor] Are you allowed to shoot camels? [kinda OT]
>
>
> > What you are try to do is "execute a block of code based on the
> value of
> > a single statement." if/elif doesn't do that and thereby introduces
> the
> > possibility of errors.
>
> In that case the best solution is a dictionary jump table.
> That is more maintainable than either and much faster too.
> And its also shorter to write.
> [Especially with lambdas :-)]
>
> > Note that the logic intended is that "on-this." So why force the
> > programmer to rewrite it N times and thereby introduce the
> possibility
> > of N-1 typographical errors...
>
> Thats a fair point although the dictionary solution avoids that. OTOH
> such switches tend to proliferate thropugh code and become a big source
> of maintenance headaches in their own right - multiple update syndrome
> across multiple files potentially.
>
> > Note that I've left out break. I'm not convinced that fall-through is
>
> > an important feature in switch and is usually the culpit in the cases
> > of abuse.
>
> The problem is its so hard to tell when fall though is happening
> intentionally or by accident because someone forgot a break sytatement.
>
> But when it comes to abuuse the fall through mechanism is one of the
> worst offenders in C, its just too tempting to be "clever" with it.
>
> > This is also true for the ternary operator. The desired logic is to
> > assign the value of a variable based on the value of some other
> > variable.
>
> But its not its based on the value of an *expression*. If the test could
> be limited to a single valiable value it might be justified but its an
> arbitrary expression. That makes it a conditional statement, which is
> most clearly represented by an if/else... Well I think so :-)
>
> > I also like Perl's unless statement but really prefer
> > VBs DO/WHILE/UNTIL/LOOP constuct. Nothing beats it for clarity of
> > expression.
>
> I won't argue on either point, Python's minimalist
> approach - there's only one way to do it - means a
> paucity of looping constructs - something I point
> out in my tutorial. But I can live with it for the
> other niceties it brings.
>
> Alan G.
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Are you allowed to shoot camels? [kinda OT]
On Sat, 5 Feb 2005, Alan Gauld wrote: > Marilyn, > > > I'll whisper that I'm a tiny bit disappointed to see the vaguely > > demeaning 'are you joking' theme that has emerged in here. It's > > unusual for us to be anything but generous and kind with each other. > > I guess this is a hot topic. :^) > > Languages (and editors) are always emotional topics on the 'net... > > But I was actually thinking that here we are on a mailing list having > a mini-language war and by internet standards its extremely civilised! > Nobody has called anyone a moron or an idiot yet and an 'are you > joking' is, to me at least, just a term of speech, not an insult. > Certainly if anyone has been offended I apologise profusely it > was never meant thus. This is a great list. Please don't apologize. Super-super-sensitive people like me are reluctant to post if they sense ridicule, even if it wasn't intended. So I'm sorry for my sensitivity. Thank you again for all the great help I've received, and the safe place to expose my ignorance. Marilyn > > Alan G. > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] str.split and quotes
Hi Tutors, I need a little help with this, if anyone has the time and inclination: >>> s = 'Hi "Python Tutors" please help' >>> s.split() ['Hi', '"Python', 'Tutors"', 'please', 'help'] >>> I wish it would leave the stuff in quotes in tact: ['Hi', '"Python Tutors"', 'please', 'help'] Any suggestions? Thank you. Marilyn Davis ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
RE: [Tutor] str.split and quotes
On Wed, 6 Apr 2005, Tony Meyer wrote:
> > >>> s = 'Hi "Python Tutors" please help'
> > >>> s.split()
> > ['Hi', '"Python', 'Tutors"', 'please', 'help']
> > >>>
> >
> > I wish it would leave the stuff in quotes in tact:
> >
> > ['Hi', '"Python Tutors"', 'please', 'help']
>
> You can do this with a regular expression:
>
> >>> import re
> >>> re.findall(r'\".*\"|[^ ]+', s)
> ['Hi', '"Python Tutors"', 'please', 'help']
>
> The regular expression says to find patterns that are either a quote (\")
> then any number of any characters (.*)then a quote (/") or (|) more than one
> of any character except a space ([^ ]).
Yes! I'll do that. Thank you, especially, for the explanation.
Thanks folks for all the suggestions.
Python attracts the best people!
Marilyn Davis
>
> Or you can just join them back up again:
>
> >>> combined = []
> >>> b = []
> >>> for a in s.split():
> ... if '"' in a:
> ... if combined:
> ... combined.append(a)
> ... b.append(" ".join(combined))
> ... combined = []
> ... else:
> ... combined.append(a)
> ... else:
> ... b.append(a)
> ...
> >>> b
> ['Hi', '"Python Tutors"', 'please', 'help']
>
> (There are probably tidier ways of doing that).
>
> Or you can do the split yourself:
>
> def split_no_quotes(s):
> index_start = 0
> index_end = 0
> in_quotes = False
> result = []
> while index_end < len(s):
> if s[index_end] == '"':
> in_quotes = not in_quotes
> if s[index_end] == ' ' and not in_quotes:
> result.append(s[index_start:index_end])
> index_start = index_end + 1
> index_end += 1
> if s[-1] != ' ':
> result.append(s[index_start:index_end])
> return result
>
> >>> print split_no_quotes(s)
> ['Hi', '"Python Tutors"', 'please', 'help']
>
> =Tony.Meyer
>
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] str.split and quotes
On Thu, 7 Apr 2005, Danny Yoo wrote:
>
>
> On Wed, 6 Apr 2005, Kent Johnson wrote:
>
> > s = 'Hi "Python Tutors" please help'
> > s.split()
> > >
> > > ['Hi', '"Python', 'Tutors"', 'please', 'help']
> > >
> > >
> > > I wish it would leave the stuff in quotes in tact:
> > >
> > > ['Hi', '"Python Tutors"', 'please', 'help']
> >
> > You can do this easily with the csv module. The only complication is
> > that the string has to be wrapped in a StringIO to turn it into a
> > file-like object.
>
>
> Hello!
>
> A variation of Kent's approach might be to use the 'tokenize' module:
>
> http://www.python.org/doc/lib/module-tokenize.html
>
> which takes advantage of Python's tokenizer itself to break lines into
> chunks of tokens. If you intend your input to be broken up just like
> Python tokens, the 'tokenize' module might be ok:
>
> ##
> >>> import tokenize
> >>> from StringIO import StringIO
> >>> def getListOfTokens(s):
> ... results = []
> ... for tokenTuple in tokenize.generate_tokens(StringIO(s).readline):
> ... results.append(tokenTuple[1])
> ... return results
> ...
> >>> getListOfTokens('Hi "Python Tutors" please help')
> ['Hi', '"Python Tutors"', 'please', 'help', '']
> ##
>
> (The last token, the empty string, is EOF, which can be filtered out if we
> use the token.ISEOF() function.)
>
In my context, I expect exactly 8 tokens so the extra '' wouldn't be
noticed.
>
> I'm not sure if this is appropriate for Marilyn's purposes though, but I
> thought I might just toss it out. *grin*
Thank you Danny. Very interesting. Both approaches are perfect for
me.
Is there a reason to prefer one over the other? Is one faster? I
compiled my regular expression to make it quicker.
What a rich language! So many choices.
Marilyn
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
RE: [Tutor] str.split and quotes
On Fri, 8 Apr 2005, Tony Meyer wrote: > > Is there a reason to prefer one over the other? Is one > > faster? I compiled my regular expression to make it quicker. > > With Python 2.4 I get these results (all imports are factored out, all give > the same result except for CSV which strips the "s) with timeit.py: > > Own split: 26.8668364275 > Tokenize:78.8295112926 > Rejoin: 11.237671827 > Re: 13.9386123097 > Re compiled: 8.19355839918 > CSV: 23.3710904598 Wow Tony. That is so generous of you to do this experiment. It must be a great re engine. I wondered if tokenize was resource-hungry. It goes so far around the bush. But I'm glad to know about it. Thank you again, Danny. > > Of course, speed isn't everything (or you wouldn't be using Python). > Readability is probably the most important factor - I'd say that a re > (particularly a verbose re) would be the most readable, followed by using > the CSV module, followed by writing your own split function. Since re is > also the fastest of the methods suggested so far, it seems like a good > choice. Yes! To everything you say. And there's the added feature that I don't have to go back and change that code now! And so little typing in the first place. > > > What a rich language! So many choices. > > Somewhat ironically, one of the tenets of Python is "there should be one-- > and preferably only one --obvious way to do it." (type "import this" at an In this case, there is: regular expressions. :^) "Obvious" doesn't mean we can, necessarily, all see it immediately, or that even any one of us can see it without study, thought and inspiration, but, it means we can all see it once it's pointed out. Or maybe I'm wrong. Thank you, you guys. Marilyn > interactive prompt). > > =Tony.Meyer > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] str.split and quotes
On Fri, 8 Apr 2005, Kent Johnson wrote:
> Marilyn Davis wrote:
> > Is there a reason to prefer one over the other? Is one faster? I
> > compiled my regular expression to make it quicker.
>
> The only way to know which is faster is to time them both. The timeit module
> makes it pretty easy to
> do this.
>
> Here is a simple example of using timeit for a different problem; you can
> adapt it to your own needs:
Thank you so much again.
This will be handy.
Marilyn
>
> d = dict( ((i,i,i), i) for i in range(1000))
>
> def withItems(d):
> for k,v in d.iteritems():
> pass
>
>
> def withKeys(d):
> for k in d:
> d[k]
>
>
> from timeit import Timer
>
> for fn in [withItems, withKeys]:
> name = fn.__name__
> timer = Timer('%s(d)' % name, 'from __main__ import d, %s' % name)
> print name, timer.timeit(1000)
>
>
> Kent
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] str.split and quotes
On Fri, 8 Apr 2005, C Smith wrote: > Tony wrote: > With Python 2.4 I get these results (all imports are factored > out, all give > the same result except for CSV which strips the "s) with > timeit.py: > > Just a note here in terms of results. Although the results are all the > same and they work for the case where there is single quoted phrase with > more than one word in it, the split_and_rejoin has problems for the case > where there is only a single word in quotes (since the trailing quote > that you are assuming is in another word occurs in the same word and thus > is never found). e.g. 'I said "wow" when I saw the result' will give it > problems. > > The re expression has a problem if there is more than one quoted string > in the string (e.g. 'if "this" and "that" are quoted'). The reason is > that the expression is "greedy" and matches the longest string of > characters between quote marks: in '"this" and "that"' the entire string > will be matched rather than the first "this". The fix is to use the > non-greedy pattern: > > re.compile(r'\".*?\"|[^ ]+') Yes, thank you. I did discover it to be too greedy and added the ? Thank you for giving it another thought. This will be the best snippet of code in my whole application. Marilyn > > Note the ? after the * > > /c > > > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] inexplicable MySQLdb behavior, INSERT data doesn't appear
On Tue, 19 Apr 2005, Kent Johnson wrote:
> Sounds like you need to commit() the initial cursor.execute()
Yes. When we upgraded to 2.4, we had to add a commit(), so my own execute
looks like:
self.cursor.execute(this,args)
did = self.connection.affected_rows()
self.connection.commit()
return did
Hope it helps.
Marilyn Davis
>
> Kent
>
> [EMAIL PROTECTED] wrote:
> > hi all, while recently trying to insert some data into the following
> > table:
> >
> > # stores unique course definitions
> > CREATE TABLE adminCourses (
> > ID TINYINT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
> > Code CHAR(6),
> > Title VARCHAR(55),
> > Units TINYINT UNSIGNED
> > ) TYPE = InnoDB;
> >
> > I got a 1L value when I ran the following:
> >
> > cursor.execute("INSERT INTO adminCourses(ID, Code, Title, Units) VALUES
> > (NULL, 'EAT100', 'Josue', 30);")
> >
> > indicating no errors. Then when I cursor.execute("Select * from
> > adminCourses") I can see the datum, but when I go to MySQL shell and
> > "Select * from adminCourses;" there are 0 records ! If I then execute the
> > aforementioned INSERT statement while still in MySQL shell, and then the
> > SELECT statement, I will see the lone datum but the ID number will be
> > offset by how many times I executed the INSERT statement via MySQLdb
> > (e.g., if I cursor.execute the INSERT statement 5 times, then go to MySQL
> > shell to run the INSERT statement, when I do a "SELECT * from
> > adminCourses" the solo datum will appear prefixed by an ID number of 6).
> > Am I doing something wrong ?
> >
> > ___
> > Tutor maillist - Tutor@python.org
> > http://mail.python.org/mailman/listinfo/tutor
> >
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] function runs w/o error but produces no output (?) - possible spoiler
On Fri, 6 May 2005, D. Hartley wrote:
> Hey guys,
>
> I wrote the following function, and if I do it line-by-line in the
> interpreter, each line works exactly as it should. However, when I
> run the function by its name, nothing happens. It doesnt print the
> print statement, it doesnt give me an error, it just goes to the next
> >>> line. This is probably an easy fix, but I traced through it about
> twelve times and can't figure out why it's not doing what it should.
> The pdb debugger didnt give me anything I could actually read.
>
> Any ideas?
>
> def urlsearch():
> x = 12345
> count = 0
> #pdb.set_trace()
> while count > 10:
If count is set to 0 above, then this will never happen because 0 is
not > 10.
> param = urllib.urlencode({'nothing': x})
> f =
> urllib.urlopen("http://www.pythonchallenge.com/pc/def/linkedlist.php?%s";
> % param)
> text = f.read()
> print text
> lister = string.split(text)
> x = lister[-1]
> count = count + 1
>
> Thanks :)
> ~Denise
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Planning a program with algorithm?
On Mon, 30 May 2005, . , wrote: > I'm learning python by reading 'Python Programming for the Absolute > Beginner' by Michael Dawson. > > And I'm doing chapter4. In the book it says it's recommended to plan a prog. > with pseudocode. > > Can i just ignore it? Yes! Python is a formalized pseudocode. Go for it! Marilyn > > Thanks. > > _ > FREE pop-up blocking with the new MSN Toolbar - get it now! > http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/ > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] is this a bug global i ???
On Tue, 14 Jun 2005, Pujo Aji wrote: > I have code like this: > > class A: > def __init__(self,j): > self.j = j > > def something(self): > print self.j > print i# PROBLEM is here there is no var i in class A > but it works ??? > > if __name__ == '__main__': > i = 10 > a = A(5) > a.something() > > I don't define global i but it will takes var i from outside of class A. > > Can somebody explain this ??? The i is 'global' by placement so it can be read. But you can't assign it in a.something(). If you do: class A: def __init__(self,j): self.j = j def something(self): print self.j i = 11 # makes a new local i in something print i if __name__ == '__main__': i = 10 a = A(5) a.something() print 'i = ', i # prints the same old global i = 10 You'll find you made a new i in something and your i = 10 remains the same. But, if you want to change the global i, in something, then it's time for the global declaration: def something(self): global i print self.j i = 11# changes the global i print i Hope this helps. I had the same confusion long ago and this list helped me. Marilyn Davis > > pujo > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Inherit from int?
Hello Tutors, I'm stumped. This silly bit of code doesn't work. I expect the output to be 8, not 18. What am I missing? #!/usr/bin/env python '''An Under10 class, just to fiddle with inheriting from int.''' class Under10(int): def __init__(self, number): number %= 10 int.__init__(self, number) if __name__ == '__main__': n = Under10(18) print n ''' $ ./new_style.py 18 ''' Any ideas? Thank you. Marilyn Davis ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Inherit from int?
Thank you everyone. I, indeed, found it in your book, Wes, after I knew it was something extra special. The explanations here are great! Thank you everyone. Marilyn On Sun, 13 May 2007, wesley chun wrote: > > I'm stumped. This silly bit of code doesn't work. I expect the > > output to be 8, not 18. What am I missing? > > > > class Under10(int): > > def __init__(self, number): > > number %= 10 > > int.__init__(self, number) > > marilyn, > > i agree with most of the earlier replies... you need to use __new__() > instead of __init__() in order to "tweak" the original value before > the instance of the immutable object is created. once it's "frozen," > you're stuck. note that __new__() is a class method, so you'll need a > variable for the class (instead of self for the instance). > > also recall that __init__() is (the 1st method) called *after* an > instance has been created, which for you, would be too late. in > practice, i don't think __init__() is ever used for deriving from > immutable types. does anyone have a counterexample? > > (since i know you've been reading Core Python, you can take a look at > my example of subclassing an immutable type on p.552.) :-) > > cheers, > -- wesley > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > "Core Python Programming", Prentice Hall, (c)2007,2001 > http://corepython.com > > wesley.j.chun :: wescpy-at-gmail.com > python training and technical consulting > cyberweb.consulting : silicon valley, ca > http://cyberwebconsulting.com > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Quoting trouble
Dear Tutors,
I'm having a problem and going around in circles. I'm on Python 2.4.1.
This odd address line comes in email:
>>> line = '''"ma >> \"[EMAIL PROTECTED]"" <[EMAIL PROTECTED]>'''
It makes sense to me, and to the MTA. I want to use it to create a
variable:
>>> exec('h_to = %s' % line)
Traceback (most recent call last):
File "", line 1, in ?
File "", line 1
h_to = "ma >> "[EMAIL PROTECTED]"" <[EMAIL PROTECTED]>
^
SyntaxError: invalid syntax
So, how do I keep the \" in there?
I've tried a bunch of things but can't find anything that works.
Heeelp. Please.
Thank you.
Marilyn Davis
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Quoting trouble
On Fri, 20 Jan 2006, Kent Johnson wrote:
Thank you so much Kent. This is *very* helpful.
> Marilyn Davis wrote:
> > Dear Tutors,
> >
> > I'm having a problem and going around in circles. I'm on Python 2.4.1.
> >
> > This odd address line comes in email:
> >
> >
> >>>>line = '''"ma >> \"[EMAIL PROTECTED]"" <[EMAIL PROTECTED]>'''
> >
> >
> > It makes sense to me, and to the MTA. I want to use it to create a
> > variable:
> >
> >
> >>>>exec('h_to = %s' % line)
> >
> > Traceback (most recent call last):
> > File "", line 1, in ?
> > File "", line 1
> > h_to = "ma >> "[EMAIL PROTECTED]"" <[EMAIL PROTECTED]>
> > ^
> > SyntaxError: invalid syntax
> >
> > So, how do I keep the \" in there?
>
> Why are you using exec? What's wrong with
>h_to = line
> ??
Explanation below.
>
> If you really need the exec then you need to put some quotes in there
> somewhere, either
>exec('h_to = '''%s''' ' % line)
>>> exec('h_to = '''%s''' % line ')
...
It seems to want more.
> or
>exec('h_to = %s' % repr(line))
This one does!
>>> exec('h_to = %s' % repr(line))
>>> h_to
'"ma >> "[EMAIL PROTECTED]"" <[EMAIL PROTECTED]>'
Why I'm using exec:
I'm using exec so that I have a flexible scheme for collecting
headers. Maybe you have a better suggestion?
I have a dictionary that starts:
significant_headers = {
'Cc: ':None,
'CC: ':None,
'From: ':None,
'Return-path: ':None,
'Sender: ':None,
'Subject: ':None,
'To: ':None}
and
heads_to_find = significant_headers.keys()
So I collect the header texts into the dictionary items:
for each in heads_to_find:
if not significant_headers[each] \
and text.startswith(each):
significant_headers[each] =
text[len(each):].strip().replace('"','\\"').replace('\n',' ')
break
And then later, I make variables that match the variables that exim,
my MTA, uses. For each header in the message exim provides a $h_to,
$h_from, etc. I like my variable names to match exim's as much as
possible:
for each in significant_headers.keys():
this = '''self.h_%s = "%s"''' % \
(each[:-2].lower().replace('-','_'),
repr(significant_headers[each]))
exec(this)
If I decide that I need to collect another header, I can just add it
to my dicitonary.
And now, with your help, I don't get any exceptions ... so far.
But, if there's another way to do this, I'd appreciate any improvement.
Thank you so much.
Marilyn
>
> Kent
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Quoting trouble
On Fri, 20 Jan 2006, Python wrote:
> On Fri, 2006-01-20 at 13:50 -0800, Marilyn Davis wrote:
> > for each in significant_headers.keys():
> > this = '''self.h_%s = "%s"''' % \
> >(each[:-2].lower().replace('-','_'),
> > repr(significant_headers[each]))
> > exec(this)
>
> So you are using exec to process an assignment statement. The setattr
> builtin function will do what you want.
>
> http://docs.python.org/lib/built-in-funcs.html#l2h-64
>
> so your code could wind up looking something like:
>
> setattr( self,
> 'h_' + each[:-2].lower().replace('-','_'),
> significant_headers[each]
> )
>
> Looking at that code, you can pull the key and value from
> significant_headers in the for statement. That is:
>
> for key,val in significant_headers.items():
> setattr( self,
> 'h_' + key[:-2].lower().replace('-','_'),
> val
> )
>
> Now the line that actually determines the attribute name looks pretty
> ugly. I would recommend writing a function to replace that operation
> with an understandable function name (perhaps key2name). That would
> result in:
> setattr(self, key2name(key), val)
Very nice indeed!
>
>
> Hope this helps. The ease with which Python allows you to attach names
> to values is one of the great features of the language.
Yes. Thank you so much. I'll fix it and I'm glad to have it in my
bag of tricks.
Marilyn
>
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] A question
On Sun, 5 Feb 2006, Bian Alex wrote:
> How can I get a string from a random file.
> For Example:
> Delete On : Copy
> Owner : Bn
> Personalized: 5
> PersonalizedName: MyDocuments
>
> I want to get the string after 'Owner' ( In example is 'Bn')
>
> 'import re' ?
Yes, but maybe it's simpler to use the builtin functions, and slicing:
text = open('the_file').readlines() <-- now the_file is in text as a string
start = text.find('Owner :') + 8 <-- start is the index into the text
where Owner starts + 8 to point
to the text you want.
end = text.find('\n', start) <-- end is where the newline is, after
start.
found = text[start:end].strip() <-- found has the text you want, with
any spaces stripped off.
I hope it helps. I didn't test it.
Marilyn Davis
>
> Pls help.
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] A question
On Sun, 5 Feb 2006, Marilyn Davis wrote:
> On Sun, 5 Feb 2006, Bian Alex wrote:
>
> > How can I get a string from a random file.
> > For Example:
> > Delete On : Copy
> > Owner : Bn
> > Personalized: 5
> > PersonalizedName: MyDocuments
> >
> > I want to get the string after 'Owner' ( In example is 'Bn')
> >
> > 'import re' ?
>
> Yes, but maybe it's simpler to use the builtin functions, and slicing:
>
> text = open('the_file').readlines() <-- now the_file is in text as a string
Ooops. This should be:
text = open('the_file').read()
Maybe someone else will show you the readlines() solution. I have to
run.
Good luck.
Marilyn
>
> start = text.find('Owner :') + 8 <-- start is the index into the text
> where Owner starts + 8 to point
> to the text you want.
> end = text.find('\n', start) <-- end is where the newline is, after
> start.
> found = text[start:end].strip() <-- found has the text you want, with
> any spaces stripped off.
>
> I hope it helps. I didn't test it.
>
> Marilyn Davis
>
> >
> > Pls help.
> >
>
>
--
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] IDE - Editors - Python
On Mon, 6 Feb 2006, Danny Yoo wrote: > >Avoid debuggers like a plague. If someone applies for a job > > with us and starts talking about their proficiency in > > debuggers, the interview stops right there and we keep looking. I can see that bragging about "proficiency in debuggers" would be a strange tactic in an interview, demonstrating an off-center focus. But the debugger is sure handy now and then. I try to *think* first. That usually finds the problem. Sometimes one or two print statements or an assert sorts it all out. But, sometimes I'm stuck, and using the debugger is the quickest way to unstick me. So you wouldn't hire me? Your loss. :^) While people are talking like this, and about IDE's, the thing I miss in the Python debugger is the ability to attach commands to a breakpoint. Does anyone know how to do that? I use Linux and keystroked emacs to avoid mousing. And I use a macro in emacs to get around not knowing how to attach commands to breakpoints. Marilyn > > Hi Tim, > > Seriously? I know that the implication is that sufficient test cases and > design will ferret out bugs, but this attitude toward debuggers surprises > me. Steve McConnell, author of Code Complete, makes it a point to > recommend running any new code through a debugger just to force the > programmer to dig though the abstractions to see what the program's > actually doing at a low level. > > In particular, I've found a debugger invaluable in diving through old C > code that I have not written. Admittedly, I don't use debuggers in > Python, but I do see the value in forcing oneself to jump levels of > abstraction. But maybe this approach is obsolete now and I'm just an old > fuddy-duddy. *grin* > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] re and MULTILINE
Hello Tutors, I'm trying to get a grip on MULTILINE and I guess I don't have it. Here's some code: #!/usr/bin/env python import re def sub_it(mo): return 'xxx' def test(re_str, data): return re.sub(re_str, sub_it, data, re.MULTILINE) if __name__ == '__main__': data = '''Betty Boop:245-836-2837:6937 Ware Road, Milton, PA 93756:9/21/46:43500 Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700 James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:45000 ''' re_str = r'''(d+)$''' print test(re_str, data) re_str = r'''(d+)''' print test(re_str, data) ''' ./re_test2.py Betty Boop:245-836-2837:6937 Ware Road, Milton, PA 93756:9/21/46:43500 Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700 James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:xxx Betty Boop:xxx-xxx-xxx:xxx Ware Road, Milton, PA xxx:xxx/xxx/xxx:43500 Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700 James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:45000 ''' If I don't anchor it with '$', it gets all the digit-groups in the first line except the last one. Why not the last one? Whay not the other lines? If I do anchor it, it only gets the last group on the last line. What's up with that? What I really want, is to mess with each of the last digit-groups on each line. But I can't find them. The exercise is from Ellie Quigley's "Perl by Example" Thank you for any help. Marilyn Davis ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] re and MULTILINE
On Tue, 20 Feb 2007, Kent Johnson wrote: > Marilyn Davis wrote: > > Hello Tutors, > > > > I'm trying to get a grip on MULTILINE and I guess I don't have it. > > > > Here's some code: > > > > #!/usr/bin/env python > > import re > > > > def sub_it(mo): > > return 'xxx' > > > > def test(re_str, data): > > return re.sub(re_str, sub_it, data, re.MULTILINE) > > > The fourth argument to re.sub() is a count, not flags. You have to > compile the regex and pass the MULTILINE flag to the compile, or include > the flag in the actual regex with *|(?m)|*. Thank you so much!! Duh. Geeso, I looked at that for hours. I should have asked for help sooner. Gratefully, Marilyn > > Kent > > > > > > if __name__ == '__main__': > > data = '''Betty Boop:245-836-2837:6937 Ware Road, Milton, PA > > 93756:9/21/46:43500 > > Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700 > > James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:45000 > > ''' > > re_str = r'''(d+)$''' > > print test(re_str, data) > > > > re_str = r'''(d+)''' > > print test(re_str, data) > > > > ''' > > ./re_test2.py > > Betty Boop:245-836-2837:6937 Ware Road, Milton, PA 93756:9/21/46:43500 > > Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700 > > James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:xxx > > > > Betty Boop:xxx-xxx-xxx:xxx Ware Road, Milton, PA xxx:xxx/xxx/xxx:43500 > > Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700 > > James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:45000 > > ''' > > > > If I don't anchor it with '$', it gets all the digit-groups in the first > > line except the last one. Why not the last one? Whay not the other lines? > > > > If I do anchor it, it only gets the last group on the last line. What's up > > with that? > > > > What I really want, is to mess with each of the last digit-groups on each > > line. But I can't find them. > > > > The exercise is from Ellie Quigley's "Perl by Example" > > > > Thank you for any help. > > > > Marilyn Davis > > > > > > ___ > > Tutor maillist - Tutor@python.org > > http://mail.python.org/mailman/listinfo/tutor > > > > > > -- ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] New Style Classes, __getitem__ and iteration
Hi Tutors and Tutees,
I've been teaching Python quite a while and a brilliant student asked a
question that made me realize a big hole in my understanding.
I think it is really magical that, when I define __getitem__ on classic
classes, the built-in iterator uses it.
But, when I make the same class as a new style class, I lose this behavior.
I didn't realize that something was lost in defining a new style class.
Maybe it's something else that I'm missing.
Thank you for your insights.
Marilyn
p.s.
Here's some code that demonstrates my confusion:
#!/usr/bin/env python
class Circle:
def __init__(self, data, times):
"""Put the 'data' in a circle that goes around 'times' times."""
self.data = data
self.times = times
def __getitem__(self, i):
"""circle[i] --> Circle.__getitem__(circle, i)."""
l_self = len(self)
if i >= self.times * l_self:
raise IndexError, \
"Error raised: Circle object only goes around %d times"\
% self.times
return self.data[i % l_self]
def __len__(self):
return len(self.data)
class NewCircle(list):
def __init__(self, data, times):
list.__init__(self, data)
self.times = times
def __getitem__(self, i):
l_self = len(self)
if i >= self.times * l_self:
raise IndexError, \
"Error raised: NewCircle object only goes around %d times"\
% self.times
return list.__getitem__(self, i % l_self)
def main():
circle = Circle("around", 3)
print sorted(circle)
new_circle = NewCircle("around", 3)
print sorted(new_circle)
main()
"""
OUTPUT:
$ ./circle_question.py
['a', 'a', 'a', 'd', 'd', 'd', 'n', 'n', 'n', 'o', 'o', 'o', 'r', 'r',
'r', 'u', 'u', 'u']
['a', 'd', 'n', 'o', 'r', 'u']
$
"""
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reading only a few specific lines of a file
(I'm sorry for the duplicate, Alan.)
On Fri, May 23, 2008 3:49 am, Alan Gauld wrote:
> "Jason Conner" <[EMAIL PROTECTED]> wrote
>
>
>> building a program that will take a text file, search for a string in
>> that text file. Once it finds the string its looking for, I want to put
>> the next five lines of text into a variable. Here's what I have so far:
>>
>> def loadItem(self, objectToLoad): x = 0 wordList = [] fobj = file()
>
> This does nothing. file() is just an alias for open()
>
>
>
>> fobj.open("itemconfig.txt", 'rU')
>>
>> for line in fobj.readline():
>
> And you can iterate over the file itself so this could become:
>
>
> for line in file(("itemconfig.txt", 'rU'):
>
>
>> if line == objectToLoad: while x != 5 for word in line.split():
>> wordList.append(word) x += 1
>
> This isn't quite what you said above in that you didn't mention
> storing the individual words...
>
>>
>> thing = item(wordList[0], wordList[1], wordList[2], wordList[3],
>> wordList[4])
>
> But this could just be:
>
>
> thing = line.split[:5] # use slicing to get first 4 items
>
>> itemList.append(thing)
>
> or even make it all one line here...
>
> Now to get the next 4 lines as well set a flag/counter. say linecount.
> Set it to 0 at the top then
>
>
> linecount = 0 for line in file("itemconfig.txt", 'rU'): if line ==
> objectToLoad or linecount > 0: if linecount == 0: linecount = 5 # set
> linecount on first line itemList.append(line.split()[:5])
> linecount -= 1
I always go for readability. So I like "for x in range(5):" to go 5
times. I'm not sure what we should be doing 5 times either.
Similarly, for readability, I choose the if/elif/else form of switch
replacement.
Also, I don't like to see the obfuscated forms of the conditional
operator. Heck, I'm not even crazy about the Python conditional in 2.5.
I like to say "The point of a programming language is to communicate with
other engineers (in a language that also the computer understands)."
I hope you like these thoughts.
Marilyn Davis
>
>
> HTH,
>
>
> --
> Alan Gauld
> Author of the Learn to Program web site
> http://www.freenetpages.co.uk/hp/alan.gauld
>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Readability, in general: was Re: Reading only a few specific lines of a file
On Fri, May 23, 2008 10:06 am, Alan Gauld wrote:
> "Marilyn Davis" <[EMAIL PROTECTED]> wrote
>
>
>> (I'm sorry for the duplicate, Alan.)
>>
>
> I didn't see a duplicate! ;-)
>
>
>>> linecount = 0 for line in file("itemconfig.txt", 'rU'): if line ==
>>> objectToLoad or linecount > 0: if linecount == 0: linecount = 5
>>> itemList.append(line.split()[:5])
>>> linecount -= 1
>>
>> I always go for readability. So I like "for x in range(5):" to go 5
>> times. I'm not sure what we should be doing 5 times either.
>
> You could use a nested for loop but that would involve the use
> of an iterator.next() function which I didn't want to confuse the issue
> with, although John has pointed it out now anyway. Since the outer for
> loop is doing the next() call for us it didn't seem necessary.
>
>> Similarly, for readability, I choose the if/elif/else form of switch
>> replacement.
>
> Are you referring to the other thread?
> There is no switch here.
>
>
>> Also, I don't like to see the obfuscated forms of the conditional
>> operator. Heck, I'm not even crazy about the Python conditional in 2.5.
>>
>
> Similarly there is no conditional statements here? Again are
> you refering to the other recent thread?
Yes, I am referring to other threads. I have time, right now, to read and
learn.
Maybe I should have said the same thing in each thread. I'm sorry.
Marilyn
>
>> I like to say "The point of a programming language is to communicate
>> with other engineers (in a language that also the computer understands)."
>>
>
> I agree. :-)
>
>
> Alan G.
>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] finding special character string
On Sun, June 1, 2008 10:30 am, Alan Gauld wrote:
> "Kent Johnson" <[EMAIL PROTECTED]> wrote
>
>
>> Assuming the strings are non-overlapping, i.e. the closing "." of
>> one string is not the opening "." of another, you can find them all with
>> import re re.findall(r'\..*?\.', text)
>
> Hmm, my regex hopelessness comes to the fore again.
> Why a *?
> I would probably have guessed the pattern to be
>
>
> \..+?\.
>
>
> What am I missing?
I think you're right.
Maybe the specification means there should be word boundaries?
r"\b\..+?\.\b"
This little program includes a function to test regular expressions. I
find it helpful and I give it to students. I think such programs exist on
the net.
#!/usr/bin/env python
"""This exercise is from the book "Perl by Example" by Ellie Quigley.
The exercise in Ellie's book asks us to print the city and state
where Norma lives.
I used this little program to develop the regular expression."""
import re
import sys
def ReTest(re_str, data, flags):
"""Test the re_str against the data with flags.
If it doesn't find a hit, try again with one character trimmed off
the end, and again, and again, until a hit is found. Then give
a report.
"""
for i in range(len(re_str), 0, -1):
try:
m = re.search(re_str[:i], data, flags)
m.groups() # generate an error
except:
continue
else:
print "This much worked:"
print re_str[:i]
print "It broke here:"
print re_str[i:]
break
def main():
data = """Tommy Savage:408-724-0140:1222 Oxbow Court, Sunnyvale, CA
94087:5/19/66:34200
Lesle Kerstin:408-456-1234:4 Harvard Square, Boston, MA 02133:4/22/62:52600
JonDeLoach:408-253-3122:123 Park St., San Jose, CA 94086:7/25/53:85100
Ephram Hardy:293-259-5395:235 Carlton Lane, Joliet, IL 73858:8/12/20:56700
Betty Boop:245-836-2837:6937 Ware Road, Milton, PA 93756:9/21/46:43500
Wilhelm Kopf:846-836-2837:6937 Ware Road, Milton, PA 93756:9/21/46:43500
Norma Corder:397-857-2735:74 Pine Street, Dearborn, MI 23874:3/28/45:245700
James Ikeda:834-938-8376:23445 Aster Ave., Allentown, NJ 83745:12/1/38:45000
Lori Gortz:327-832-5728:3465 Mirlo Street, Peabody, MA 34756:10/2/76:35200
Barbara Kerz:385-573-8326:832 Pnce Drive, Gary, IN 83756:12/15/46:26850
"""
re_str = r"""
^%s# Line starts with the name
\b # followed by a non-word character
(?:# Un-captured group
[^:]+? # of non-colons
:){2} # followed by a colon, twice
x # a mistake!!!
\d+? # some digits
[ ]+ # one or spaces in []
(?P# capturing a group
# named town. This sequence cannot be
# split for comments.
[^:\d] # with no colons or digits
+?)# one or more times
\d # a digit ends the match
""" % 'Norma'
ReTest(re_str, data, re.VERBOSE + re.MULTILINE)
if __name__ == '__main__':
main()
"""
$ ./re_test.py
This much worked:
^Norma# Line starts with the name
\b # followed by a non-word character
(?:# Un-captured group
[^:]+? # of non-colons
:){2} # followed by a colon, twice
It broke here:
x # a mistake!!!
\d+? # some digits
[ ]+ # one or spaces in []
(?P# capturing a group
# named town. This sequence cannot be
# split for comments.
[^:\d] # with no colons or digits
+?)# one or more times
\d # a digit ends the match
"""
Marilyn Davis
>
>
> Alan G.
>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] finding special character string
On Sun, June 1, 2008 4:58 pm, Kent Johnson wrote: > On Sun, Jun 1, 2008 at 2:04 PM, Marilyn Davis <[EMAIL PROTECTED]> > wrote: > >> On Sun, June 1, 2008 10:30 am, Alan Gauld wrote: >> >> >>> "Kent Johnson" <[EMAIL PROTECTED]> wrote >>> >>> >>> >>>> Assuming the strings are non-overlapping, i.e. the closing "." of >>>> one string is not the opening "." of another, you can find them all >>>> with import re re.findall(r'\..*?\.', text) >>> >>> Hmm, my regex hopelessness comes to the fore again. >>> Why a *? >>> I would probably have guessed the pattern to be >>> >>> >>> >>> \..+?\. >>> > > Mine will find two adjacent periods with no intervening text, yours > won't. Whether this makes any difference to the OP, or which is correct, I > don't know. Yeh, we need a better spec. I was wondering if the stuff between the text ought not include white space, or even a word boundary. A character class might be better, if we knew. Anyhow, I think we wore out the student. :^) Marilyn Davis > >> This little program includes a function to test regular expressions. I >> find it helpful and I give it to students. I think such programs >> exist on the net. > > I am always surprised at how many regex testers there are, because a > simple tester comes with Python. It is called redemo.py; look for it in the > Tools/scripts or bin/ directory installed with Python. On my > Mac it is at > /Library/Frameworks/Python.framework/Versions/2.5/bin/redemo.py. > > > Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] finding special character string
On Tue, June 3, 2008 2:13 am, Dinesh B Vadhia wrote:
> Yes, I'm happy because I found a non-regex way to solve the problem (see
> below).
>
> No, I'm not a student or worn out but wish I was back at college and
> partying!
>
> Yes, this is an interesting problem and here is the requirement:
>
>
> - A text document contains special words that start and end with a period
> ("."), the word between the start and end periods contain no punctuation
> or spaces except a hyphen in some special words. - Examples of special
> words include ".thrfore.", ".because.", '.music-sharp.", ".music-flat.",
> ".dbd.", ".vertline.", ".uparw.", ".hoarfrost." etc.
> - In most cases, the special words have a space (" ") before and after.
Here I might think that I should capture leading and trailing white space
and be careful to not put two spaces in a row in the result.
> - In some cases, a special word will be followed by one or two other
> special words eg. ".dbd..vertline." or ".music-flat..dbd..vertline." - In
> some cases, a special word will be followed by an ordinary word (with or
> without punctuation) eg. ".music-flat.mozart" or ".vertline.isn't" - A
> special word followed by an ordinary word (with or without punctuation)
> could be the end of a sentence and hence have a full-stop (".") eg.
> ".music-flat.mozart." or ".vertline.isn't."
> - The number of characters in a special word excluding the two periods is
> > 1
> - Find and remove all special words from the text document (by processing
> one line at a time)
>
> How did I solve it? I found a list of all the special words, created a
And here I might think that finding a list of all special words, unless
you found them using Python on the text file, isn't quite what was
intended.
It is really hard to write specifications.
It seems like you learned a lot!
Marilyn Davis
> set of special words and then checked if each word in the text belonged
> to the set of special words. If we assume that the list of special words
> doesn't exist then the problem is interesting in itself to solve.
>
> Cheers!
>
>
> Dinesh
>
>
>
> -
> ---
>
>
> Date: Sun, 1 Jun 2008 21:56:26 -0400
> From: "Kent Johnson" <[EMAIL PROTECTED]>
> Subject: Re: [Tutor] finding special character string
> To: "Marilyn Davis" <[EMAIL PROTECTED]>
> Cc: tutor@python.org
> Message-ID:
> <[EMAIL PROTECTED]>
> Content-Type: text/plain; charset=ISO-8859-1
>
>
> On Sun, Jun 1, 2008 at 9:41 PM, Marilyn Davis <[EMAIL PROTECTED]>
> wrote:
>
>
>> Yeh, we need a better spec. I was wondering if the stuff between the
>> text ought not include white space, or even a word boundary. A
>> character class might be better, if we knew.
>
> Hmm, yes, my regex will find many ordinary sentences in plain text.
>
>
>> Anyhow, I think we wore out the student. :^)
>>
>
> He went away happy after my first reply.
>
>
> Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Intercepting methods calls
On Tue, June 3, 2008 10:16 pm, Alan Gauld wrote: > "Laureano Arcanio" <[EMAIL PROTECTED]> wrote > > >> Is there any way to intercept calls to methods ? like the >> __setattribute__ >> medthod does ? > > There are several variations on how to do that kind of thing. > Can you give us some more background on what you are > trying to achieve that makes you need to do that? Great question, Alan. But, whatever the answer, I'd like to see the variations listed, if you have time. Showing my ignorance, I can only think of inheritance/overriding. Or maybe I don't understand the question. Marilyn Davis > > Then we can offer advice as to the best approach. > > > Alan G > > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] [Fwd: Re: Intercepting methods calls]
I keep forgetting to reply-to the list. I wish it was the default.
On Wed, June 4, 2008 11:21 am, Andreas Kostyrka wrote:
> Simple (untested in the mailer typed):
>
>
> class A: def __getattr__(self, key): if key.startswith("user"): def func():
> return key[4:] return func raise AttributeError
>
> assert A().userabc() == "abc"
>
> Something like that?
Yes it works! Brilliant.
You listed __init__ and I'm not sure I know what you mean.
I was thinking of not having access to the original class/method so I did
this awful thing:
a_def.py
class A:
def SaySomething(self):
print 'this'
import a_def
def SayElse(dummy=None):
print 'that'
a_def.A.SaySomething = SayElse
a_def.A().SaySomething()
---
and 'that' came out. I guess the A class could have been written to
prevent that from happening.
Thank you for the info and thoughts.
Marilyn Davis
>
>
>
> On Wednesday 04 June 2008 19:12:16 you wrote:
>
>> Sorry, i doesn't explain my question how i should.
>>
>>
>> What I'm trying to accomplish is to add methods dynamically at class
>> instances. but i what to create them whe a A().someMethod() is called (
>> someMethod doesn't exist )
>>
>> Like as with attributes.
>>
>>
>> So far i try with ( at instance and at class level ):
>>
>>
>> def foo(): return lambda: 'foo'
>>
>> class A(object): pass
>>
>> a = A() a.foo = foo()
>>
>> ---
>>
>>
>> def foo(): return lambda self: 'foo'
>>
>> class A(object): pass
>>
>> A.foo = foo()
>> a= A()
>>
>> This works, but i really like to be able of bound those functions using
>> a sintax like:
>>
>> a = A() a.someMethod()# This should bound the somethod to a
>> function.
>>
>>
>> Thaks in advance
>>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [Fwd: Re: Intercepting methods calls]
On Fri, June 6, 2008 3:37 pm, Andreas Kostyrka wrote: > On Friday 06 June 2008 18:19:23 you wrote: > >> On Thu, June 5, 2008 9:39 am, Andreas Kostyrka wrote: >> >>> On Thursday 05 June 2008 00:18:55 Marilyn Davis wrote: >>> >>>> You listed __init__ and I'm not sure I know what you mean. >>>> >>> >>> Well, __init__ can assign attributes to the instance that are >>> callable. >> >> Oh, well, Python is such a wide-open system. It keeps blowing my >> mind. > > Well, there is basically nothing that you cannot force the object system > of Python to do, almost. > > E.g. transplanting methods from class A to class B: > > > B.__dict__["method"] = A.__dict__["method"] Oh dear. > > > Or deriving a class from itself (although a better name would be "from a > class that happened to be named the same). > > class A: pass class A(A): pass print A.__bases__ Oh dear. > > Now, as an observation, you will notice that most experienced Python > developers usually stay away from all this "magic" stuff. Rules of thumb > include "explicit is good, implicit is bad", and so on. Clearly stuff > tagged as "you really should know what you are doing". > >> >> I'm thinking that if you can provide an __init__ you can intercept >> methods the regular way, by providing them the regular way, and >> overriding them. > > Nope, I mentioned it, because this has been way back, when Python was > young and slow, this was a technique to speed up method lookup. > > Actually, it seems to work with py2.5 too, for oldstyle classes: > > > [EMAIL PROTECTED]:~/Videos$ python -m timeit -s 'class A:' -s ' def > m(self): pass' > 1000 loops, best of 3: 0.0675 usec per loop > [EMAIL PROTECTED]:~/Videos$ python -m timeit -s 'class A:' -s ' def > m(self): pass' -s 'a=A()' 'a.m()' > 100 loops, best of 3: 0.747 usec per loop > [EMAIL PROTECTED]:~/Videos$ python -m timeit -s 'class A:' -s ' def > m(self): pass' -s 'a=A()' -s 'a.m=a.m' 'a.m()' > 100 loops, best of 3: 0.575 usec per loop Wow. > [EMAIL PROTECTED]:~/Videos$ python -m timeit -s 'class A(object):' -s ' > def m(self): pass' -s 'a=A()' 'a.m()' 100 loops, best of 3: 0.671 usec > per loop [EMAIL PROTECTED]:~/Videos$ python -m timeit -s 'class A(object):' > -s ' def m(self): pass' -s 'a=A()' -s 'a.m=a.m' 'a.m()' > 100 loops, best of 3: 0.641 usec per loop > > > The reason for this is, that the instance dictionary is the first place > that Python looks when looking for a.m. I see. Well, that is interesting. I'm glad I asked. > >> >> Thank you Andreas. This has been really interesting and instructive. >> > > Yeah, but please consider my comment. I think every developer should > have seen the possibilities, but in most cases forget about that ugly > stuff. You will get a headache, especially if you try to use the full > "power" that Python provides. One can usually be quite productive without > relying on these mechanisms. Yes, and if you consider the poor readers of your code, and/or if speed isn't so critical, it would be best to forget it. But, thank you for showing this to me. Do you have anything else really interesting to share? Marilyn Davis > > Andreas ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to read a program
This is pretty cool: http://aspn.activestate.com/ASPN/Python/Cookbook/ Marilyn Davis On Fri, June 6, 2008 4:34 pm, Anthony Parks wrote: > that sounds like good advice, but i think what i meant is something along > the lines of: > > "what are particularly great programs to *read*. not like great software, > but great source code. somewhat like treating source code as a > literature, what are the classics? > > On Fri, Jun 6, 2008 at 7:16 PM, Alan Gauld <[EMAIL PROTECTED]> > wrote: > > >> >> "Anthony Parks" <[EMAIL PROTECTED]> wrote >> >> >> which is insanely detailed). are there any specific pieces of software >>> written in python you would recommend a novice to read? programs that >>> illustrate python at its finest, beautiful stretches of code? thanks >>> for any help >>> >> >> You could start with the Python standard library. Many of the modules >> there are fairly sparecly documented, partly because they are quite well >> written! >> >> Then look at the tools that ship with Python. >> >> >> Then just search SourceForge for python projects. >> >> >> Alan G >> >> >> ___ >> Tutor maillist - Tutor@python.org >> http://mail.python.org/mailman/listinfo/tutor >> >> > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] doc string format ?
On Sun, June 8, 2008 1:54 am, dave selby wrote:
> Hi All,
>
>
> I am trying to improve my code quality and have started using doc
> strings. What format do you guys use for your doc strings ? I have 'made
> up' the following general format ...
Your made-up format looks good. It's great that you give thought to it.
I've seen the style-guide where it is suggested that the first line of the
doc string for a function is a very short description, starting with a
verb, like you did. That description should fit on one line.
Then there's a blank line.
Then there's the bigger explanation, calling instructions, return values,
exceptions raised, copyright notice, maybe.
A good thing is to type help("your_module") in the interpreter so that you
can see if you like the doc string and if it meshes nicely with what the
help facility provides for free.
But, mostly, I have to say that your instinct is good.
Marilyn Davis
>
> Cheers
>
>
> Dave
>
>
>
>
> def gen_vhost(kmotion_dir): """
> Generate the kmotion vhost file from vhost_template expanding %directory%
> strings to their full paths as defined in kmotion.rc
>
> arguments : kmotion_dir ... the 'root' directory of kmotion
>
> exceptions:
> exit ... if kmotion.rc cannot be read
>
> returns:
> """
> parser = ConfigParser.SafeConfigParser() parsed =
> parser.read('%s/core/kmotion.rc' % kmotion_dir) try:
> images_dbase_dir = parser.get('dirs', 'images_dbase_dir') port =
> parser.get('misc', 'port') LDAP_enabled = parser.get('LDAP', 'enabled') in
> ['true',
> 'True', 'yes', 'Yes']
> LDAP_url = parser.get('LDAP', 'AuthLDAPUrl')
> except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
>
> --
>
>
> Please avoid sending me Word or PowerPoint attachments.
> See http://www.gnu.org/philosophy/no-word-attachments.html
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] static methods and class methods
On Tue, June 10, 2008 9:17 pm, Christopher Spears wrote: > I am reading Wesley Chun's "Core Python Programming" (2nd Edition) and > have reached the part on static and class methods. I typed in the > following to demonstrate the difference between the two methods: > >>>> class TestClassMethod: > ... def foo(cls): > ... print 'calling class method foo()' > ... print 'foo() is part of class:',cls.__name__ > ... foo = classmethod(foo) > ... > > >>>> class TestStaticMethod: > ... def foo(): > ... print 'calling static method foo()' > ... foo = staticmethod(foo) > ... > >>>> tsm = TestStaticMethod() TestStaticMethod.foo() >>>> > calling static method foo() >>>> tcm = TestClassMethod() TestClassMethod.foo() >>>> > calling class method foo() foo() is part of class: TestClassMethod >>>> tcm.foo > 0xb7da0f2c>> > >>>> > > According to the author, the result for typing in 'tcm.foo' is You forgot to put () after the method name. Wesley has them. Class methods and static methods can be called, even when you don't have an instance of the class. I think that that's all there is to it. With the class method, the interpreter provides the class itself as the first argument. With the static, nothing comes in. What else? Marilyn Davis > > > calling class method foo() foo() is part of class: TestClassMethod > > Did I do something wrong or is this an error on the book's part? > Intuitively, the answer I received makes more sense to me. I am still > unsure of the difference of static and class methods. Can someone > enlighten me? > > Thanks! > > > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] powerball
On Wed, June 11, 2008 9:58 am, max baseman wrote: Hi Max, Here's your code with the indents preserved. I'll make my comments with >'s. from random import randrange wins=0 win=[] count =0 while count !=5: > To loop 5 times it is better to use: > for count in range(5): > stuff you want to do > Then all the loop controlling happens in one place and it is hard to > mess up. number=randrange(55)+1 win.append(number) count=count+1 powerball=randrange(42)+1 count=0 win.sort() while count !=146107962: numbers=[] count2=0 while count2 !=5: number=randrange(55)+1 > Yep, indeed, here you made the easy mistake of forgetting to + 1 to > count2. So, it probably spun forever right there. > I also don't see that "count" is every changed, so that would be another > spin. All these would go away by using 'for' 'in' and 'range'. > It looks like a fun program and a good start. > Marilyn Davis if number in win: numbers.append(number) else: print "lose" break numbers.sort() ball=randrange(42)+1 if ball==powerball: print "win" print print print win, powerball > as a fun little project i scripted a powerball program. it seems to have a > bug in it that i cant find. > > from random import randrange wins=0 win=[] count =0 while count !=5: > number=randrange(55)+1 win.append(number) count=count+1 > powerball=randrange(42)+1 count=0 win.sort() while count !=146107962: > numbers=[] count2=0 while count2 !=5: number=randrange(55)+1 if number in > win: > numbers.append(number) else: > print "lose" break numbers.sort() ball=randrange(42)+1 if ball==powerball: > print "win" print print print win, powerball > > > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] [Fwd: Re: powerball]
Hi Max and everyone, Max, your reply-to only went to me. You have to work harder to get it to go to the whole list. Anyway, my advice is to post your new program and see what people say. Marilyn Original Message Subject: Re: [Tutor] powerball From:"max baseman" <[EMAIL PROTECTED]> Date:Wed, June 11, 2008 7:22 pm To: "Marilyn Davis" <[EMAIL PROTECTED]> -- thank you for your help i beleave i have fixed this bug and made the script slightly shorter yet i seem to have run into another bug i cant find - the program now just eats my ram and doesn't seem to run correctly now it displays no output and nearly freezes my computer i'm still looking for the reason but if you have the time i would appreciate you advise - thanks On Jun 11, 2008, at 12:36 PM, Marilyn Davis wrote: > On Wed, June 11, 2008 9:58 am, max baseman wrote: > > Hi Max, > > Here's your code with the indents preserved. I'll make my comments > with >'s. > > from random import randrange > wins=0 > win=[] > count =0 > while count !=5: > >> To loop 5 times it is better to use: > >> for count in range(5): >> stuff you want to do > >> Then all the loop controlling happens in one place and it is hard to >> mess up. > > number=randrange(55)+1 > win.append(number) > count=count+1 > powerball=randrange(42)+1 > count=0 > win.sort() > while count !=146107962: > numbers=[] > count2=0 > while count2 !=5: > number=randrange(55)+1 > >> Yep, indeed, here you made the easy mistake of forgetting to + 1 to >> count2. So, it probably spun forever right there. > >> I also don't see that "count" is every changed, so that would be >> another >> spin. All these would go away by using 'for' 'in' and 'range'. > >> It looks like a fun program and a good start. > >> Marilyn Davis > > if number in win: > numbers.append(number) > else: > print "lose" > break > numbers.sort() > ball=randrange(42)+1 > if ball==powerball: > print "win" > print > print > print win, powerball > >> as a fun little project i scripted a powerball program. it seems >> to have a >> bug in it that i cant find. >> >> from random import randrange wins=0 win=[] count =0 while count !=5: >> number=randrange(55)+1 win.append(number) count=count+1 >> powerball=randrange(42)+1 count=0 win.sort() while count !=146107962: >> numbers=[] count2=0 while count2 !=5: number=randrange(55)+1 if >> number in >> win: >> numbers.append(number) else: >> print "lose" break numbers.sort() ball=randrange(42)+1 if >> ball==powerball: >> print "win" print print print win, powerball >> >> >> >> >> ___ >> Tutor maillist - Tutor@python.org >> http://mail.python.org/mailman/listinfo/tutor > > from random import randrange wins=0 win=[] count=0 for count in range(5): win.append(randrange(55)+1) powerball=randrange(42)+1 count=0 for count in range(146107962): numbers=[] count2=0 for count2 in range(5): number=randrange(55)+1 if number in win: numbers.append(number) else: print "lose" break numbers.sort() win.sort() ball=randrange(42)+1 if ball == powerball: print "win" win=win+1 print print print win, powerball___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] powerball
On Wed, June 11, 2008 9:42 pm, max baseman wrote: > hello and thank you everyone for your help. I apologize for my ignorance > when it came to knowledge of what the powerball is it is not as wide > spread as i had thought it was > > the powerball is a form of lottery their are 5 numbers between 1 and 55, > and then their is the powerball between 1 and 42 players win different > amounts of money depending on how many numbers they had right the program > i wrote uses the number 146107962 as the amount of tickets in play i > found this number on the powerball stat as the probability of getting > jackpot 1 / 146107962 included is the finished and working script if > anyone would like to use or play with it. > Hi Max, The program looks great. You're getting real pythonic. Since you posted it, I'll add some comments to help you spiff it up and make it faster. Again, I'll use the > for comments: from random import randrange # for creating random numbers wins=0 # to see how many people would win win=[] # the winning numbers go here count=0 > You don't need count=0. Python sets the whole thing up for you > with just that "for": for count in range(5): # powerball uses 5 numbers win.append(randrange(55)+1) powerball=randrange(42)+1 count=0 while count != 146107962: # point of information useing for count in range(146107962) will overlaoad with this large a number > Then you might like xrange(big_number). It generates the numbers, > one at a time. numbers=[] # guesses count2=0 for count2 in range(5): number=randrange(55)+1 > With randrange, you can get duplicates. You can get > [23, 2, 14, 23, 2] which I'm guessing can't > happen in a real powerball game. (I don't know powerball either.) > You might like the sample() function in the random library. if number in win: # if the number is in the winning numbers continue numbers.append(number) else: print "lose" # else quite while ahead break > There you went to the next player. It's a good place > for a comment. numbers.sort() > You don't seem to need those numbers sorted. win.sort() > Since win is not changing, you might want to sort it only once, > back when you made that list. ball=randrange(42)+1 #picks the powerball if ball == powerball: print "win" wins=wins+1 > else? it's quiet. count=count+1 print print wins,"winners with", win, powerball > What happens wrong? I didn't run it? Does it just fall quiet for a > while? You might try putting in some extra print statements to see > what is exactly happening. > Are you a student? What grade? > > > seeing as i have left the python mailing list in the past any comments or > ideas would be appreciated if sent to this email - thank you > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] do I need f.close()
On Thu, June 12, 2008 4:32 pm, Alan Gauld wrote:
> "dave selby" <[EMAIL PROTECTED]> wrote
>
>
>> The whole topic came up because I just finished reading 'learning
>> python' 3rd edition OReilly as a refresher where there are multiple
>> instances of suggesting that you do the exact opposite eg ...
>
> LP is a tutorial book so does not always teach industry strength
> programming practice (in common with most tutorials!)
>
>> [line.rstrip() for line in open('myfile')] ... p361
>> for line in open('script1.py') ... p261& p276 where it is described as
>> 'best practice' for reading files line by line
>>
>
> Its the common idiom and for reading files not too bad.
> I certainly use this sometimes but if its critical I will move
> the open outside the loop:
>
> f = open(...) for line in f:
>
> or
>
> [line.rstrip() for line in f]
>
>
> But thats just so that I can detect and correct missing
> files if necessary etc. Its not becauise of closure issues. I'm fairly
> happy about letting the os close read only files, its really for writing
> that you want to be explicit.
Alan, will the file close, even if it was opened for writing, when the
program ends? I know it stays open if you're interactive, but otherwise
too?
Marilyn Davis
>
> Alan G.
>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Dictionary within a dictionary
On Fri, June 13, 2008 7:23 am, [EMAIL PROTECTED] wrote:
> Hi,
> Following Alan's post, what I was trying to do is to understand how I
> can return the sub-item within the same space, if it makes sense ;)
>
> For example, in my 3 one-to-many lists, I want to generate this list:
When see nested builtin data structures, I always think it's time to think
of making a few classes instead.
Python tutors, is this a good suggestion?
Marilyn Davis
>
>
> [{'id': ,
> 'category': [{'id': ,
> 'sub-category': [ {'id': ,
> 'title': ,
> 'is_selected':
> }
> ...]
> 'title': ,
> 'is_selected':
> }
> ...]
> 'title': ,
> 'is_selected': ,
> }
> ...]
>
>
> Reason for this is that I want to create a navigational menu for a web
> template so that when you click on group_id the value is sent and this
> pulls the 'categories' and when you click on the category then the sub-
> categories are listed.
>
> Thanks
> David
>
>
>
>
> __
>
>
> Find out what Tiscali can do for you - http://www.tiscali.co.uk/services
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] [Fwd: Re: Dictionary within a dictionary]
On Fri, June 13, 2008 4:09 pm, Alan Gauld wrote:
> "Marilyn Davis" <[EMAIL PROTECTED]> wrote
>
>
>> When see nested builtin data structures, I always think it's time to
>> think of making a few classes instead.
>
> It could be, especially if you are about to write a bunch of
> functions to access those structures. But equally if the structures
> accurately reflect the problem and acess does not require any complex
> processing then I'm happy to use the structures.
>
> In a case like this I'd still tend to move to a database rather
> than an OOP solution, although I might use classes to hold the returned
> data.
That sounds like the best plan.
>
> I'm still not clear why the OP wants to hold the data in
> memory.
I don't know either. Maybe s/he wants to use shelve or something; maybe
s/he has not yet learned classes and databases.
So thank you for the reality check.
Marilyn Davis
>
> Alan G
>
>>
>> Python tutors, is this a good suggestion?
>>
>>
>> Marilyn Davis
>>
>>
>>>
>>>
>>> [{'id': ,
>>> 'category': [{'id': ,
>>> 'sub-category': [ {'id': ,
>>> 'title': ,
>>> 'is_selected':
>>> }
>>> ...]
>>> 'title': ,
>>> 'is_selected':
>>> }
>>> ...]
>>> 'title': ,
>>> 'is_selected': ,
>>> }
>>> ...]
>>>
>>>
>>>
>>> Reason for this is that I want to create a navigational menu for a
>>> web template so that when you click on group_id the value is sent and
>>> this pulls the 'categories' and when you click on the category then
>>> the sub- categories are listed.
>>>
>>> Thanks
>>> David
>>>
>>>
>>>
>>>
>>>
>>> __
>>>
>>>
>>>
>>> Find out what Tiscali can do for you -
>>> http://www.tiscali.co.uk/services
>>>
>>>
>>>
>>> ___
>>> Tutor maillist - Tutor@python.org
>>> http://mail.python.org/mailman/listinfo/tutor
>>>
>>
>>
>>
>>
>> ___
>> Tutor maillist - Tutor@python.org
>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] [Fwd: Re: From Newbie]
On Sun, June 22, 2008 7:55 am, [EMAIL PROTECTED] wrote:
> hi Danny,
>
>
> As far as i am aware you must declare your variable first, something like
> a=0
That's a good thought. But he did initialize his variable.
The '!' and the '=' in '!=' are all smashed together without spaces.
So:
> while a ! = 0:
Should be:
while a != 0:
That should get you going again.
Marilyn Davis
>
> The same would go for s
>
>
> hope that helps
>
> paul
>
> On Sun Jun 22 10:45 , Danny Laya sent:
>
>
>
>
>
> Hi ! I have learned wiki tutor for non-programmer and I found some hill
> that stopping me. In Non-Programmer's Tutorial for Python/Count to 10,
> wiki ask me to write this code : a = 1 s = 0 print 'Enter Numbers to add to
> the sum.' print 'Enter 0 to quit.' while a != 0: print 'Current Sum:', s
a =
> int(raw_input('Number? ')) s = s + a print 'Total Sum =', s
>
> But when i write while a != 0: and then i press enter,
> python terminal tell me :
>>>> while a ! = 0:
> File "", line 1
> while a ! = 0: ^
> SyntaxError: invalid syntax
>
>
> Can you
find my mistake, guys ? Sorry to bother you, I try to
> find the answer in google, but I can't found the answer. Please help me
> soon guys, whatever your answer. If you don't want to answer my question,
> please give me some site that could answer this newbie question. Thank's.
>
>
>
>
>
>
>
>
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Astonishing timing result
On Tue, June 24, 2008 2:06 pm, [EMAIL PROTECTED] wrote: > - Message from [EMAIL PROTECTED] - > Date: Tue, 24 Jun 2008 13:44:00 -0700 > From: Dick Moores <[EMAIL PROTECTED]> > > >> At 12:44 PM 6/24/2008, Kent Johnson wrote: >> >>> On Tue, Jun 24, 2008 at 1:43 PM, Dick Moores <[EMAIL PROTECTED]> wrote: >>> >>>> Output: >>>> t1 is 0.000104, no function t2 is 5.87e-006, function explicit t3 is >>>> 0.000126, function imported >>>> t1/t2 is 17.8 t1/t3 is 0.827 t3/t2 is 21.5 >>>> >>>> Now, I'd heard that code in a function runs faster than the same >>>> >>> code not in >>>> a function, but even so I was surprised at the t1/t2 ratio of 17.8. >>>> >>>> >>>> The astonishing (to me, anyway) result was the t3/t2 ratio. I had >>>> no idea that importing from mycalc slowed a script down at all, let >>>> alone by a factor of 21! >>> >>> Note that t1 and t3 are pretty close to each other. Perhaps you >>> should be suspicious of t2. What if __name__ != '__main__' ? >> >> With that, >> t1 is 0.000104, no function t2 is 0.000117, function explicit t3 is >> 0.000113, function imported >> t1/t2 is 0.885 t1/t3 is 0.914 t3/t2 is 0.969 >> >> Explain? Does this mean that if __name__ == "__main__" takes the extra time? and that that's brings t2 in line with the others? and that the difference represents the time it takes to set up a code-block? Something like that? Marilyn Davis >> >> >> Dick >> > > > Hey Dick, > > > I'm not too clear on what it is that you want explained. > > > It seems to me that the difference between t2 and t3 is 1) is so small > as to be most likely due to (effectively) random fluctuations of your > environment (the demands that other processes were making on your system > at the time) and 2) so small so as to not be worth worrying about > (<http://c2.com/cgi/wiki?PrematureOptimization>). > > > I'd further wager that if you repeat the timing a few times, you'll > find that on some runs t2 is less than t3. > > Best, > > > Brian vdB > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] [Fwd: Re: Astonishing timing result]
On Tue, June 24, 2008 5:52 pm, Dick Moores wrote: > At 05:35 PM 6/24/2008, Kent Johnson wrote: > >> On Tue, Jun 24, 2008 at 5:20 PM, Dick Moores <[EMAIL PROTECTED]> wrote: >> >> >>> Basically, I'm not worried, just curious. Not about the small >>> differences, but why did the use of the standardif __name__ == >>> '__main__' result >>> it such speed? >> >> Because __name__ is not equal to "__main__", so you were basically >> skipping the whole test. > > Ah. Ah. So the difference we see is the whole sort. That makes sense. Thank you for the understanding. Has anyone ever timed the difference between using a function that was imported with: from my_module import MyFunction and: import my_module and then my_module.MyFunction() Also, if anyone is still wondering, a "code block" is the stuff that is indented. The stuff between a ':' and the unindent. Marilyn Davis > > >> The most common cause of unexpected timing >> results is tests that don't do what you think they do. >> >> The test code is not run in the main module. You can dig into the >> timeit module if you want the details. > > OK, I'll dig. > > > Thanks, > > > Dick > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Astonishing timing result
On Tue, June 24, 2008 10:16 pm, Dick Moores wrote:
> At 07:00 PM 6/24/2008, Marilyn Davis wrote:
>
>
>> Has anyone ever timed the difference between using a function that was
>> imported with:
>>
>> from my_module import MyFunction
>>
>> and:
>>
>>
>> import my_module
>
> Here are 2 comparisons: <http://py77.python.pastebin.com/f53ab3769>,
> and <http://py77.python.pastebin.com/f68346b28>
>
> I don't see a significant difference.
Good. Thank you.
I'm attaching another astonishing timing result, also wrong.
It's probably always true that if a timing result is astonishing, there's
a mistake somewhere, maybe in your thinking.
This one compares using os.popen, os.listdir, and subprocess.Popen.
Marilyn Davis
>
>
> Dick
>
>
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
#!/usr/bin/env python
"""lab13_1.py -- Adding up the file sizes in the current directory,
three ways, and comparing them."""
import os
import subprocess
__pychecker__ = 'no-local'
def AccuracyTest():
print "os.listdir:", AddFilesOsListdir()
print "os.popen: ", AddFilesOsPopen()
print "subprocess:", AddFilesSubprocess()
def AddFilesOsListdir():
total = 0
files = os.listdir('.')
for f in files:
if os.path.isdir('./' + f):
continue
total += os.path.getsize('./' + f)
return total
def AddFilesOsPopen():
return TotalLsSize(os.popen("ls -al"))
def AddFilesSubprocess():
return TotalLsSize(subprocess.Popen(["ls", "-al"],
stdout=subprocess.PIPE).stdout)
def ProfileTest():
for i in range(100):
AddFilesOsListdir()
AddFilesOsPopen()
AddFilesSubprocess()
def TotalLsSize(file_obj):
total = 0
for line in file_obj:
if line[0] == 'd':
continue
parts = line.split()
if len(parts) != 9:
continue
total += int(parts[4])
return total
def main():
AccuracyTest()
import profile
profile.run('ProfileTest()')
if __name__ == '__main__':
main()
"""
$ lab13_1.py
os.listdir: 26298
os.popen: 26298
subprocess: 26298
30376 function calls in 1.872 CPU seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1010.0040.0000.0040.000 :0(WEXITSTATUS)
1010.0120.0000.0120.000 :0(WIFEXITED)
1010.0000.0000.0000.000 :0(WIFSIGNALED)
770.0040.0000.0040.000 :0(append)
3000.0160.0000.0160.000 :0(close)
2000.0120.0000.0120.000 :0(fcntl)
1000.0040.0000.0040.000 :0(fdopen)
1000.0680.0010.0680.001 :0(fork)
2000.0120.0000.0120.000 :0(isinstance)
54000.1000.0000.1000.000 :0(len)
1000.0320.0000.0320.000 :0(listdir)
2000.0000.0000.0000.000 :0(pipe)
1000.1080.0010.1080.001 :0(popen)
10.0000.0000.0000.000 :0(range)
1000.0160.0000.0160.000 :0(read)
780.0040.0000.0040.000 :0(remove)
10.0040.0040.0040.004 :0(setprofile)
54000.1040.0000.1040.000 :0(split)
53000.2360.0000.2360.000 :0(stat)
1780.0040.0000.0040.000 :0(waitpid)
10.0000.0001.8681.868 :1()
1000.1560.0020.8720.009 lab13_1.py:12(AddFilesOsListdir)
1000.0240.0000.4400.004 lab13_1.py:21(AddFilesOsPopen)
1000.0360.0000.5400.005 lab13_1.py:24(AddFilesSubprocess)
... The rest of the output is irrelevant.
It seems fishy that os.listdir() takes longer than both subprocess.Popen()
and os.popen(). Maybe somehow we are comparing apples and oranges?
$ """___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] unicode help
Hi, I'm still on Python 2.6 and I'm trying to work some unicode handling. I've spent some hours on this snippet of code, trying to follow PEP 0263, since the error tells me to see it. I've tried other docs too and I am still clueless. The code works, except for the comment at the end. I would be very grateful for some help. #!/usr/bin/env python '''Unicode handling for 2.6. ''' class Currency(float): def __str__(self): value = self.__class__.symbol + float.__str__(self) return value class Yen(Currency): symbol = unichr(165) class Pound(Currency): symbol = unichr(163) def main(): y = Yen(100) print unicode(y) p = Pound(100) print unicode(p) main() """ ¥100.0 £100.0 """ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unicode help
Thank you Alexandre for your quick reply. I tried your suggestion (again) and I still get: ./uni.py File "./uni.py", line 20 SyntaxError: Non-ASCII character '\xa5' in file ./uni.py on line 21, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details Can you suggest a different encoding? Or a way to figure out what it should be? Or am I making and re-making some stupid mistake? I run on emacs under SUSE. Marilyn p.s. The code is now: #!/usr/bin/env python # coding utf-8 '''Unicode handling for 2.6. ''' class Currency(float): def __str__(self): value = self.__class__.symbol + float.__str__(self) return value class Yen(Currency): symbol = unichr(165) def main(): y = Yen(100) print unicode(y) main() """ ¥100.0 """ On Sat, May 28, 2011 2:21 pm, Alexandre Conrad wrote: > When Python loads your file from your file system, it assumes all > characters in the file are ASCII. But when it hits non-ASCII characters > (currency symbols), Python doesn't know how to interpret > it. So you can give Python a hint by putting at the top of your file the > encoding of your file: > > After the shebang (1st line), add the following comment: > # coding: utf-8 > > > (or whatever encoding your file is saved to, I think it depends on > your file system, usually utf-8 by default on Linux) > > HTH, > > > > 2011/5/28 Marilyn Davis : > >> Hi, >> >> >> I'm still on Python 2.6 and I'm trying to work some unicode handling. >> >> >> I've spent some hours on this snippet of code, trying to follow PEP >> 0263, >> since the error tells me to see it. I've tried other docs too and I am >> still clueless. >> >> The code works, except for the comment at the end. >> >> >> I would be very grateful for some help. >> >> >> >> #!/usr/bin/env python >> '''Unicode handling for 2.6. >> ''' >> class Currency(float): def __str__(self): >> value = self.__class__.symbol + float.__str__(self) >> return value >> >> >> class Yen(Currency): symbol = unichr(165) >> >> >> class Pound(Currency): symbol = unichr(163) >> >> >> def main(): y = Yen(100) >> print unicode(y) >> p = Pound(100) >> print unicode(p) >> >> >> main() >> >> """ >> ¥100.0 >> £100.0 >> """ >> >> >> >> >> ___ >> Tutor maillist - Tutor@python.org >> To unsubscribe or change subscription options: >> http://mail.python.org/mailman/listinfo/tutor >> >> > > > > -- > Alex | twitter.com/alexconrad > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unicode help
Thank you Martin, This: #!/usr/bin/env python # -*- coding: utf8 -*- '''Unicode handling for 2.6. ''' [rest of module deleted] produces an emacs warning: Warning (mule): Invalid coding system `utf8' is specified for the current buffer/file by the :coding tag. It is highly recommended to fix it before writing to a file. But, if I save anyway, and run, I get this: ./uni.py File "./uni.py", line 13 SyntaxError: 'utf8' codec can't decode byte 0xa5 in position 0: unexpected code byte but, on a hunch, I tried # -*- coding: utf-8 -*- and emacs and python were very happy. Thank you thank you thank you. Now I can enjoy my Saturday. Marilyn On Sat, May 28, 2011 3:00 pm, Martin A. Brown wrote: > Hello there, > > > : I'm still on Python 2.6 and I'm trying to work some unicode > : handling. > : > : I've spent some hours on this snippet of code, trying to follow > : PEP 0263, since the error tells me to see it. I've tried other > : docs too and I am still clueless. > > > OK, so this is PEP 0263. http://www.python.org/dev/peps/pep-0263/ > > > Did you miss these lines? > > > To define a source code encoding, a magic comment must > be placed into the source files either as first or second line in the file, > such as: > > Or was it the lack of an explicit example for UTF-8 in the PEP? > > > Try adding a single line to your script, as the second line. That > should make your script look like: > > #! /usr/bin/env python > # -*- coding: utf8 -*- > > > You might wonder why on earth you have to do this. The interpreter > cannot safely assume that your editor (any arbitrary text editor) knows how > to create/save anything other than ASCII without this (slightly hackish) > hint. > > Good luck, > > > -Martin > > > -- > Martin A. Brown > http://linux-ip.net/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unicode help
Aye, thank you. I do like that syntax better. Sometimes it's time to just quit and try again later when I'm not so frustrated. That's when I make silly bugs. But, we got it! Thank you again. I think it's a nifty hack. M On Sat, May 28, 2011 3:53 pm, Alexandre Conrad wrote: > Marilyn, > > > You miss-typed the line, it should have a semicolon right after the > word "coding", such as: > > # coding: utf-8 > > > not > > # coding utf-8 > > > as you showed from your file. > > The syntax suggested syntax # -*- coding: utf8 -*- by Martin is > equivalent, but I always have a hard time remembering it from the top of my > head when I create a new Python file. So I just use: # coding: utf-8 > > > > 2011/5/28 Marilyn Davis : > >> Thank you Alexandre for your quick reply. >> >> >> I tried your suggestion (again) and I still get: >> >> >> ./uni.py >> File "./uni.py", line 20 >> SyntaxError: Non-ASCII character '\xa5' in file ./uni.py on line 21, but >> no encoding declared; see http://www.python.org/peps/pep-0263.html for >> details >> >> Can you suggest a different encoding? Or a way to figure out what it >> should be? >> >> Or am I making and re-making some stupid mistake? >> >> >> I run on emacs under SUSE. >> >> >> Marilyn >> >> >> p.s. The code is now: >> >> #!/usr/bin/env python >> # coding utf-8 >> '''Unicode handling for 2.6. >> ''' >> class Currency(float): def __str__(self): >> value = self.__class__.symbol + float.__str__(self) >> return value >> >> >> class Yen(Currency): symbol = unichr(165) >> >> >> def main(): y = Yen(100) >> print unicode(y) >> >> >> main() >> >> """ >> ¥100.0 >> """ >> >> >> >> >> >> >> >> On Sat, May 28, 2011 2:21 pm, Alexandre Conrad wrote: >> >> >>> When Python loads your file from your file system, it assumes all >>> characters in the file are ASCII. But when it hits non-ASCII >>> characters (currency symbols), Python doesn't know how to interpret >>> it. So you can give Python a hint by putting at the top of your file >>> the encoding of your file: >>> >>> After the shebang (1st line), add the following comment: >>> # coding: utf-8 >>> >>> >>> >>> (or whatever encoding your file is saved to, I think it depends on >>> your file system, usually utf-8 by default on Linux) >>> >>> HTH, >>> >>> >>> >>> >>> 2011/5/28 Marilyn Davis : >>> >>> >>>> Hi, >>>> >>>> >>>> >>>> I'm still on Python 2.6 and I'm trying to work some unicode >>>> handling. >>>> >>>> >>>> I've spent some hours on this snippet of code, trying to follow PEP >>>> 0263, >>>> since the error tells me to see it. I've tried other docs too and I >>>> am still clueless. >>>> >>>> The code works, except for the comment at the end. >>>> >>>> >>>> >>>> I would be very grateful for some help. >>>> >>>> >>>> >>>> >>>> #!/usr/bin/env python >>>> '''Unicode handling for 2.6. >>>> ''' >>>> class Currency(float): def __str__(self): value = >>>> self.__class__.symbol + float.__str__(self) return value >>>> >>>> >>>> >>>> class Yen(Currency): symbol = unichr(165) >>>> >>>> >>>> class Pound(Currency): symbol = unichr(163) >>>> >>>> >>>> def main(): y = Yen(100) print unicode(y) >>>> p = Pound(100) >>>> print unicode(p) >>>> >>>> >>>> >>>> main() >>>> >>>> """ >>>> ¥100.0 >>>> £100.0 >>>> """ >>>> >>>> >>>> >>>> >>>> >>>> ___ >>>> Tutor maillist - Tutor@python.org >>>> To unsubscribe or change subscription options: >>>> http://mail.python.org/mailman/listinfo/tutor >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Alex | twitter.com/alexconrad >>> >>> >> >> >> > > > > -- > Alex | twitter.com/alexconrad > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Finding error from os.system(cmd)
Hi Kann,
So you are saying that you printed your command, then ran it at the
terminal prompt, and it ran ok?
You might want to look at the subprocess library so that you can collect
stderr from the process. The online documentation is great, with lots of
examples.
BTW, on your line 6, you str(some-stuff-that-is-already-a-str), which is a
waste, and a little confusion for your reader.
I hope you find some help in these suggestions.
Marilyn Davis
On Mon, May 30, 2011 5:28 am, Kann Vearasilp wrote:
> Dear all,
>
>
> I tried using python to execute some external java program in my code.
> My problem is the os.system(cmd) was not working properly while
> executing 'java' from terminal worked just fine. I am not sure what is
> wrong here. Is there a way to print out/detect error in my code for this
> case?
>
>>>>>
>
> 1 import os
> 2
> 3 def create_image():
> 4 path = os.path.abspath('tmp/medusa')
> 5 medusa = os.path.abspath('mirnaworkbench/Medusa/Medusa.jar')
> 6 cmd = str('java -cp ' + medusa + '
> medusa.batchoperations.BatchOperations ' + path) 7 os.system(cmd)
>
>
>>>>>
>
> Thanks,
>
>
> Kann
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Importing classes when needed
If we are coding via a vote, I'd be with Alan. If Timo adds non-parser modules, and they get through his glob filter, then surely his code will break with a nice error statement and that would remind him of his convention. Or maybe it would just give a verbose report and go on to the next file. Marilyn On Mon, May 30, 2011 1:57 pm, Alan Gauld wrote: > "Alexandre Conrad" wrote > > >>> Why not use the os functions to read the file names >>> dynamically and build the list that way? Provided the files use a >>> standard naming scheme you don't need to change the init code. >> >> I wouldn't do that. If Timo adds non-parser modules in that >> directory (say some utils.py file for all parsers to use), then you have >> to maintain a list of excluded files > > I did say he would have to use a naming convention and > that in turn implies the use of glob to filter the names returned. But I > agree it has risks but then, so does relying on always editing init.py > > I just prefer to avoid introducing hard to remember > maintenance tasks when I can do it automatically, but of course you need to > remember the naming convention. But hopefully a quick 'ls' will do that > for you... > > Alan G. > > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] checking if a variable is an integer?
You'll like:
>>> isinstance(1, int)
True
>>> isinstance(1.5, int)
False
There are infinite primes. Proofs abound on the Internet.
But, that's not the problem.
Yes, the problem is possible for any finite number. And it's a nice
little practice problem. Remember that you can skip the testing for all
even numbers. They aren't prime. That'll cut your run time in half,
about.
Marilyn Davis
On Tue, May 31, 2011 2:40 pm, Rachel-Mikel ArceJaeger wrote:
> Isn't one of the unsolved millenium prize problems one that includes the
> ability to find all of the prime numbers? I'm not sure if your program is
> possible if the input number is large.
>
> But to check if a number x is an int, just do this:
>
>
> x == int(x)
>
>
> Rachel
>
>
>
> On Tue, May 31, 2011 at 2:38 PM, Hugo Arts wrote:
>
>
>> On Tue, May 31, 2011 at 11:30 PM, Joel Goldstick
>> wrote:
>>
>>>
>>>
>>>
>> http://stackoverflow.com/questions/1265665/python-check-if-a-string-rep
>> resents-an-int-without-using-try-except
>>>
>>> def RepresentsInt(s):
>>>
>>> try:
>>> int(s)
>>>
>>> return True except ValueError:
>>>
>>> return False
>>>
>>>>>> print RepresentsInt("+123")
>>>
>>> True
>>>
>>>>>> print RepresentsInt("10.0")
>>>
>>> False
>>>
>>>
>>
>> For strings, that works, but not for integers:
>>
>>
>>>>> int(10.0)
>> 10
>>
>>
>> And if you want to check if one number is divisible by another, you're
>> not going to call it on strings.
>>
>> A better way is to use the modulo operator, which gives the remainder
>> when dividing:
>>
>>>>> a = 6 a % 3
>> 0
>>
>>>>> a % 4
>> 2
>>
>>
>> So, if the remainder is zero the left operand is divisible by the right
>> one.
>>
>> Hugo
>> ___
>> Tutor maillist - Tutor@python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] __init__.py question
I don't really understand why __init__.py is necessary -- except that it makes the packaging scheme work. The Python Manual by Guido van Rossum and Fred L. Drake says: ... this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on in the module search path. An example would probably set me straight. Can anyone help with that? Marilyn Davis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] __init__.py question
Thank you Válas Péter. I still don't get it, or how __init__.py alters the scenario you describe. I know that sometimes I'm really dense so please be patient with me. If I have a string.py in my local directory, it will be imported when I "import string" because the search path, sys.path, (which I have control of), has my local directory as the first place to search by default, and no __init__.py is involved. If my string is not a .py but package/directory itself, it will be imported and searched into if it has a __init__.py. In either case, how can I get to the system's string.py by putting __init__.py somewhere? If my string.py is in a package, I have to put its parent directory on my sys.path for it to be found, and the parent directory won't need an __init__.py There's something I'm missing because I think you simply can't call something string.py unless you are willing to give up the library string.py. I can see that a person could have a local string directory that they don't want imported because it has some non-code in it, or something. But the __init__.py facility is a big deal to be there for the purpose of allowing this when it is, IMHO, an architectural mistake. There must be something I'm missing or some wrong thinking described here because Guido is so smart, and it's still in 3.x. If anyone has an insight, I would really appreciate some help on this point, which has bothered me a bit for years. Marilyn Davis On Wed, June 1, 2011 12:26 am, Válas Péter wrote: > I think it means that you may have a subdirectory named "string" in your > actual directory where you store e.g. your string-related scripts, and you > have a Python package named "string" _somewhere_else_, but in the module > search path. The subdirectory of the actual directory will have priority > bacause of its place, and will prevent you of importing the string > package. > > 2011/6/1 Marilyn Davis > > >> I don't really understand why __init__.py is necessary -- except that >> it makes the packaging scheme work. >> >> The Python Manual by Guido van Rossum and Fred L. Drake says: >> >> >> ... this is done to prevent directories with a common name, such as >> string, from unintentionally hiding valid modules that occur later on in >> the module search path. >> >> An example would probably set me straight. Can anyone help with that? >> >> >> Marilyn Davis >> >> >> ___ >> Tutor maillist - Tutor@python.org >> To unsubscribe or change subscription options: >> http://mail.python.org/mailman/listinfo/tutor >> ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] __init__.py question
Maybe I'm getting what you say, Alexandre and Ramit. When you import logging, it imports string, but it won't if you have a string of your own already imported. So, when logging depends on string, it'll get mine and crash. But __init__.py helps what in this scenario? The only scenario it helps, that I can understand so far, is if I make a string directory that isn't part of a package that I'm making, but is in its directory structure, accidentally using a library name that I want. I guess I don't feel very satisfied because I would be happy with the crash so that I can fix up my naming/architecture mistake rather than depend on, and struggle with, the __init__.py scheme. I teach Python and this is always a hard part for people. It might help if I understand it better. Marilyn Davis On Wed, June 1, 2011 11:52 am, Alexandre Conrad wrote: > As a side note, remember that imported modules are cached under > sys.modules. If you import something, it will be looked up in sys.modules > and use the one in memory if it exists. If it doesn't exist, it will > iterate over sys.path and look for your module/package. If it doesn't find > it, you will get an ImportError. If it finds it, it will add it to > sys.modules. So any future imports for the same namespace won't be > re-imported from disk, it's already here in memory. > > Say I *do* have a local package called string. After I import it, say > I want to import the "logging" module. Well, it will break: > > > $ python > Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) > [GCC 4.4.5] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>>> import string # my local string package print string > > >>>> import logging > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python2.6/logging/__init__.py", line 62, in > elif string.lower(__file__[-4:]) in ['.pyc', '.pyo']: AttributeError: > 'module' object has no attribute 'lower' > > > When "logging" is being imported, internally it also imports standard > Python "string" module, which is *already* in sys.modules (your own). > > > So imagine if Python also imported any directory without a __init__.py > file, system wide, you might have random surprises. > > > 2011/6/1 Alexandre Conrad : > >> 2011/5/31 Marilyn Davis : >> >>> I don't really understand why __init__.py is necessary -- except that >>> it makes the packaging scheme work. >>> >>> The Python Manual by Guido van Rossum and Fred L. Drake says: >>> >>> >>> ... this is done to prevent directories with a common name, such as >>> string, from unintentionally hiding valid modules that occur later on >>> in the module search path. >> >> I think it is just to avoids unexpected conflicting namespaces that >> could happen by accidentally importing non-Python directories. By >> explicitly putting a __init__.py module in your directory, you are >> telling Python you want your directory to be considered as a Python >> package and to be imported in your namespace. If you have a conflicting >> namespace, it means you explicitly wanted this to happen, then you have >> to deal with it (or fix it if unintended). >> >> -- >> Alex | twitter.com/alexconrad >> >> > > > > -- > Alex | twitter.com/alexconrad ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] __init__.py question
Thank you for your careful explanation, Steven. I guess there's nothing more for me to know. While I think it is a mistake to put directories in your package directory structure that aren't part of the package, that seems to be all it's about. Well, I guess I don't feel quite so stupid. Thank you folks. Marilyn Davis On Wed, June 1, 2011 5:11 pm, Steven D'Aprano wrote: > Marilyn Davis wrote: > [...] > >> There's something I'm missing because I think you simply can't call >> something string.py unless you are willing to give up the library >> string.py. > > No, you're right about that. Python's search path is "first match wins". > Whether it's a package or a module, it will match before the library > string.py. Adding __init__.py doesn't change that. > > (Unless you re-arrange sys.path so that library modules come first.) > > > However, you can have a directory called "string", containing arbitrary > .py files. So you could drop this in your path: > > > > string/ +-- a.py > +-- b.py > > > and add string to the path so that the directory is searched, and still > get the standard string.py. (I haven't actually tried this, but it should > work.) > > Only if you put a file __init__.py into the directory will it shadow the > standard library string.py. > > Personally, I think the emphasis given in the docs about shadowing > standard library files is misplaced. I think that is actually irrelevant. I > think a more useful explanation for the need for __init__.py is simple: > how else could Python distinguish a package from a mere directory > containing Python files? > > A mere directory is not searched unless it is explicitly added to the > path; a mere directory cannot be imported. A package can be. But since a > package *is* a directory at the file system level, how should Python > distinguish them? > > I suppose Python could have the rule "the directory must end in .py to > be considered a package". But then, when you import the main package > instead of a submodule, which file should Python read? There needs to be a > standard naming convention for the file to be treated as the top-level > module. But since you have such a file, there's no longer any need for > adding a .py to the end of the directory name, since the standard file is > sufficient to determine that the directory is a package. > > That file could have been called anything, __init__.py was chosen by > analogy with __init__ methods in classes and because it's unlikely to clash > with any name package authors might choose for their own use. > > > > -- > Steven > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] checking if a variable is an integer?
True that! This is the nicest, most patient, most knowledgeable, technical list I've ever experienced. That's why I keep coming back when I am hopelessly confused. Thank you. Marilyn Davis On Thu, June 2, 2011 7:04 am, Prasad, Ramit wrote: >> If not, he can google the bits he doesn't understand, or ask. We won't >> bite! > > Unless he asks for it ;) > > > Ramit > > > > > Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology > 712 Main Street | Houston, TX 77002 > work phone: 713 - 216 - 5423 > > > This communication is for informational purposes only. It is not > intended as an offer or solicitation for the purchase or sale of any > financial instrument or as an official confirmation of any transaction. > All market prices, data and other information are not > warranted as to completeness or accuracy and are subject to change without > notice. Any comments or statements made herein do not necessarily reflect > those of JPMorgan Chase & Co., its subsidiaries and affiliates. > > This transmission may contain information that is privileged, > confidential, legally privileged, and/or exempt from disclosure under > applicable law. If you are not the intended recipient, you are hereby > notified that any disclosure, copying, distribution, or use of the > information contained herein (including any reliance thereon) is STRICTLY > PROHIBITED. Although this transmission and any > attachments are believed to be free of any virus or other defect that might > affect any computer system into which it is received and opened, it is the > responsibility of the recipient to ensure that it is virus free and no > responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and > affiliates, as applicable, for any loss or damage arising in any way from > its use. If you received this transmission in error, please immediately > contact the sender and destroy the material in its entirety, whether in > electronic or hard copy format. Thank you. > > Please refer to http://www.jpmorgan.com/pages/disclosures for > disclosures relating to European legal entities. > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] __repr__ and __str__
Hello Python Tutors, A student has asked a question that has me stumped. We are using 2.7. Looking at this code: #!/usr/bin/python class MyList(list): def __str__(self): return """Here are your data: %s """ % list.__str__(self) def main(): a = MyList([1,2]) print a if __name__ == "__main__": main() ___ We get the expected output: Here are your data: [1, 2] But if we add the special method: def __repr__(self): return "MyList(%s)" % (list.__str__(self)) we get: Traceback (most recent call last): File "./stack2.py", line 17, in main() File "./stack2.py", line 14, in main print a File "./stack2.py", line 10, in __str__ """ % list.__str__(self) File "./stack2.py", line 5, in __repr__ return "MyList(%s)" % (list.__str__(self)) File "./stack2.py", line 5, in __repr__ return "MyList(%s)" % (list.__str__(self)) and on and on to the: RuntimeError: maximum recursion depth exceeded -- From https://docs.python.org/2/reference/datamodel.html: If a class defines __repr__() but not __str__(), then __repr__() is also used when an “informal” string representation of instances of that class is required. __ So ????? __str__ is defined and works just fine unless we also define __repr__. What am I missing? Thank you for any help. Marilyn Davis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] __repr__ and __str__
Thank you so much Alan and Steve. We are good now. Marilyn On Tue, June 30, 2015 6:10 am, Steven D'Aprano wrote: > On Mon, Jun 29, 2015 at 11:47:45PM -0700, Marilyn Davis wrote: > >> Hello Python Tutors, >> >> >> A student has asked a question that has me stumped. We are using 2.7. >> > > Ooh, nice one! That had me stumped for a while too, but I think I have > the solution. I wrote this class to investigate: > > class MyList(list): def __str__(self): print "called __str__" return > list.__str__(self) def __repr__(self): print "called __repr__" return > list.__repr__(self) > > > > Then I ran this in the interactive interpreter: > > > py> L = MyList([1, 2, 3]) py> repr(L) called __repr__ '[1, 2, 3]' > py> str(L) called __str__ called __repr__ '[1, 2, 3]' > > > > So what appears to be happening is that list.__str__ calls __repr__. My > guess is that it is written something like this: > > # built-in list > class list: def __str__(self): return self.__repr__() > > > > -- > Steve > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unicode help
Hi, Last year, I was helped so that this ran nicely on my 2.6: #! /usr/bin/env python # -*- coding: utf-8 -*- # necessary for python not to complain about "¥" symbol = unichr(165) print unicode(symbol) --- end of code --- But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get: bash-3.2$ ./uni_test.py ./uni_test.py Traceback (most recent call last): File "./uni_test.py", line 6, in print unicode(symbol) UnicodeEncodeError: 'ascii' codec can't encode character u'\xa5' in position 0: ordinal not in range(128) bash-3.2$ Can anyone please help? It says 'ascii' codec? Shouldn't it be seeing 'utf-8'? I can't imagine it matters but I'm on an imac now and before I was on Ubuntu. Thank you, Marilyn Davis On Sat, May 28, 2011 2:17 pm, Marilyn Davis wrote: > Thank you Martin, > > > This: > > > #!/usr/bin/env python > # -*- coding: utf8 -*- > '''Unicode handling for 2.6. > ''' > [rest of module deleted] > > > produces an emacs warning: > > Warning (mule): Invalid coding system `utf8' is specified > for the current buffer/file by the :coding tag. It is highly recommended to > fix it before writing to a file. > > But, if I save anyway, and run, I get this: > > > ./uni.py > File "./uni.py", line 13 > SyntaxError: 'utf8' codec can't decode byte 0xa5 in position 0: unexpected > code byte > > but, on a hunch, I tried > > # -*- coding: utf-8 -*- > > > and emacs and python were very happy. Thank you thank you thank you. > > Now I can enjoy my Saturday. > > > Marilyn > > > > > > On Sat, May 28, 2011 3:00 pm, Martin A. Brown wrote: > > >> Hello there, >> >> >> >> : I'm still on Python 2.6 and I'm trying to work some unicode >> : handling. >> : >> : I've spent some hours on this snippet of code, trying to follow >> : PEP 0263, since the error tells me to see it. I've tried other >> : docs too and I am still clueless. >> >> >> >> OK, so this is PEP 0263. http://www.python.org/dev/peps/pep-0263/ >> >> >> >> Did you miss these lines? >> >> >> >> To define a source code encoding, a magic comment must >> be placed into the source files either as first or second line in the >> file, such as: >> >> Or was it the lack of an explicit example for UTF-8 in the PEP? >> >> >> >> Try adding a single line to your script, as the second line. That >> should make your script look like: >> >> #! /usr/bin/env python >> # -*- coding: utf8 -*- >> >> >> >> You might wonder why on earth you have to do this. The interpreter >> cannot safely assume that your editor (any arbitrary text editor) knows >> how to create/save anything other than ASCII without this (slightly >> hackish) hint. >> >> Good luck, >> >> >> >> -Martin >> >> >> >> -- >> Martin A. Brown >> http://linux-ip.net/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unicode help
Thank you, Dave, for looking at my problem, and for correcting me on my top posting. See below: On Wed, November 14, 2012 12:34 pm, Dave Angel wrote: > On 11/14/2012 03:10 PM, Marilyn Davis wrote: > >> Hi, >> >> >> Last year, I was helped so that this ran nicely on my 2.6: >> >> >> #! /usr/bin/env python >> # -*- coding: utf-8 -*- >> # necessary for python not to complain about "¥" >> >> >> symbol = unichr(165) print unicode(symbol) >> >> --- end of code --- >> >> >> But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get: >> >> >> bash-3.2$ ./uni_test.py ./uni_test.py >> Traceback (most recent call last): >> File "./uni_test.py", line 6, in >> print unicode(symbol) UnicodeEncodeError: 'ascii' codec can't encode >> character u'\xa5' in position 0: ordinal not in range(128) bash-3.2$ >> >> Can anyone please help? It says 'ascii' codec? Shouldn't it be seeing >> 'utf-8'? >> >> >> I can't imagine it matters but I'm on an imac now and before I was on >> Ubuntu. >> >> >> Thank you, >> >> >> Marilyn Davis >> >> >> >> >> > > You top-posted your message. If you need to quote something, please put > your new text after what you're quoting. > > Try the following in your 2.7 interpreter: > > >>>> import sys print sys.stdout.encoding > UTF-8 > > > If you don't see UTF-8, then there's a discrepancy between what you > think the terminal is doing, and what Python thinks. You're right, I get this: bash-3.2$ python Python 2.7.3 (v2.7.3:70274d53c1dd, Apr 9 2012, 20:52:43) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sys import sys >>> print sys.stdout.encoding print sys.stdout.encoding US-ASCII >>> > > Somebody familiar with the Mac might be able to tell you how to fix it > right, but you could try sys.stdout.encoding = "UTF-8" > > and see if it changes your symptoms. and, your good idea didn't work: >>> sys.stdout.encoding="UTF-8" sys.stdout.encoding="UTF-8" Traceback (most recent call last): File "", line 1, in TypeError: readonly attribute >>> Goodness! I didn't expect it to be a Mac thing. So, on a Windows machine, running Python 2.6.6, sys.stdout.encoding is 'cp1252', yet the code runs fine. On Ubuntu with 2.7, it's 'UTF-8' and it runs just fine. I find this most mysterious. Thank you for any help to get it running on my Mac. Marilyn > > -- > > > DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unicode help
On Wed, November 14, 2012 1:07 pm, Marilyn Davis wrote: > Thank you, Dave, for looking at my problem, and for correcting me on my > top posting. > > See below: > > > On Wed, November 14, 2012 12:34 pm, Dave Angel wrote: > > >> On 11/14/2012 03:10 PM, Marilyn Davis wrote: >> >> >>> Hi, >>> >>> >>> >>> Last year, I was helped so that this ran nicely on my 2.6: >>> >>> >>> >>> #! /usr/bin/env python >>> # -*- coding: utf-8 -*- >>> # necessary for python not to complain about "¥" >>> >>> >>> >>> symbol = unichr(165) print unicode(symbol) >>> >>> --- end of code --- >>> >>> >>> >>> But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get: >>> >>> >>> >>> bash-3.2$ ./uni_test.py ./uni_test.py Traceback (most recent call >>> last): >>> File "./uni_test.py", line 6, in >>> print unicode(symbol) UnicodeEncodeError: 'ascii' codec can't encode >>> character u'\xa5' in position 0: ordinal not in range(128) bash-3.2$ >>> >>> Can anyone please help? It says 'ascii' codec? Shouldn't it be >>> seeing 'utf-8'? >>> >>> >>> >>> I can't imagine it matters but I'm on an imac now and before I was on >>> Ubuntu. >>> >>> >>> >>> Thank you, >>> >>> >>> >>> Marilyn Davis >>> >>> >>> >>> >>> >>> >> >> You top-posted your message. If you need to quote something, please >> put your new text after what you're quoting. >> >> Try the following in your 2.7 interpreter: >> >> >> >>>>> import sys print sys.stdout.encoding >> UTF-8 >> >> >> >> If you don't see UTF-8, then there's a discrepancy between what you >> think the terminal is doing, and what Python thinks. > > You're right, I get this: > > > bash-3.2$ python Python 2.7.3 (v2.7.3:70274d53c1dd, Apr 9 2012, 20:52:43) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>>> import sys > import sys >>>> print sys.stdout.encoding > print sys.stdout.encoding US-ASCII > >>>> > > >> >> Somebody familiar with the Mac might be able to tell you how to fix it >> right, but you could try sys.stdout.encoding = "UTF-8" >> >> and see if it changes your symptoms. > > and, your good idea didn't work: > >>>> sys.stdout.encoding="UTF-8" > sys.stdout.encoding="UTF-8" Traceback (most recent call last): > File "", line 1, in > TypeError: readonly attribute > >>>> > > Goodness! I didn't expect it to be a Mac thing. > > > So, on a Windows machine, running Python 2.6.6, sys.stdout.encoding is > 'cp1252', yet the code runs fine. > > > On Ubuntu with 2.7, it's 'UTF-8' and it runs just fine. > > > I find this most mysterious. > > > Thank you for any help to get it running on my Mac. > > > Marilyn > I found this site: http://hints.macworld.com/article.php?story=20100713130450549 and that fixes it. I would never guessed that! Marilyn ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] sqlite3 making a spurious duplicate?
#!/usr/bin/env python3
"""
Hello Tutors,
I can't figure out why the FillWithStars() function puts Canopus in the db
twice.
What am I missing?
Thank you for any help.
Marilyn Davis
p.s. That Reset(db_name) is in there so that you can run it over and over
if you want.
---
"""
import os, sqlite3, subprocess
def Reset(db_name):
os.system("rm " + db_name)
db_process = subprocess.Popen(("sqlite3", db_name),
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
for send in (b'.tables', ):
returned = db_process.communicate(send)
assert returned == (b'', b'')
def FillWithStars():
with sqlite3.connect("stars.db") as connection:
connection.executescript("""
CREATE TABLE brightest(
name,
constellation,
apparent_magnitude,
absolute_magnitude,
distance);
INSERT INTO brightest VALUES("Canopus", "Carina", -0.72, -2.5,
74);
""")
connection.executemany("INSERT INTO brightest VALUES(?, ?, ?, ?, ?)",
[("Arcturus", "Bootes", -0.04, 0.2, 34),])
stored_stars = connection.execute("SELECT * FROM BRIGHTEST")
for star in stored_stars:
print(star)
def main():
Reset("stars.db")
FillWithStars()
if __name__ == '__main__':
main()
"""Output:
bash-3.2$ ./why3.py
('Canopus', 'Carina', -0.72, -2.5, 74)
('Canopus', 'Carina', -0.72, -2.5, 74)
('Arcturus', 'Bootes', -0.04, 0.2, 34)
bash-3.2$
"""
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sqlite3 making a spurious duplicate?
Thank you Alan, Steven and Peter,
So, this call:
connection.execute("SELECT * FROM BRIGHTEST")
returns a , not a regular python
sequence. I did not know that. And, the connection must still be alive
when you iterate it.
That is a very important tidbit of info.
The fix is to listify the Cursor object, or iterate while still in the
context. Interesting. Closing the context and opening a new one fixes it
too:
with sqlite3.connect("stars.db") as connection:
connection.executescript("""
CREATE TABLE brightest(
name,
constellation,
apparent_magnitude,
absolute_magnitude,
distance);
INSERT INTO brightest VALUES("Canopus", "Carina", -0.72, -2.5,
74);
""")
connection.executemany("INSERT INTO brightest VALUES(?, ?, ?, ?, ?)",
[("Arcturus", "Bootes", -0.04, 0.2, 34),])
with sqlite3.connect("stars.db") as connection:
stored_stars = connection.execute("SELECT * FROM BRIGHTEST")
for star in stored_stars:
print(star)
---
But starting a new context, instead, before the executemany does not fix
it! We still get the duplicate Canopus in the iterable, even though it
was inserted into the data a different context.
I suspect that this is not an intended behavior.
Another question: is it a good principle to try to only put SQL in the
context, and then regular python outside that context? It suits my
instinct but maybe it is just my superstition?
Thank you so much.
Marilyn
p.s. Thank you, Steven, for liking the post. You guys prove what I try
to pound into students: pretty code that does not work is much better than
hard-to-read code, even if it works -- because with pretty code, people
are happy to help you, and with working bad code, when it must be
modified, you have a mess.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
