Re: [Tutor] Running a dos program with python

[Martin Walsh]

Armstrong, Richard J. wrote:
> Hello all,
> 
> This is my first post to the Tutor@python.org 
> mailing list. I am in the process of switching from Matlab to Python and
> there is one task that I am having a hard time doing and cannot find the
> answer on the web. I want to write a script in python that will open up
> a windows dos program, send three inputs (file names) into program and
> then run it. I know one way to open up the dos program with
> os.system(r”c:\shake91.txt”) but cannot do the rest.
> 
> Use the subprocess module:
> 
> http://docs.python.org/library/subprocess.html
> 
> untested, but should work:
> subprocess.Popen([r'c:\shake91.txt', 'param1', 'paramN-1', 'paramN'])
> 
> if you want to communicate with the process you can add ,
> stdout=subprocess.PIPE, stdin=subprocess.PIPE) to the function call.
> 
> Check the docs for more info.
> 
> HTH,
> 
> Wayne
>>
>>
> Wayne,
> 
> It kindof works. I wrote
> subprocess.Popen([r'c:\shake91.exe', 'FLAC.txt', 'a.txt', 'b.txt'])
> 
> The dos program pops up and if I hit the enter key three times then it
> runs. How can I add these three “enters” into the script?

Then perhaps something more like this (untested) ...

from subprocess import Popen, PIPE
proc = Popen([r'c:\shake91.exe'], stdin=PIPE, stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate('FLAC.txt\na.txt\nb.txt\n')

If that doesn't work you may need something like 'expect' for windows
(or, pexpect and cygwin).

HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] accessing Postgres db results by column name

[Martin Walsh]

Serdar Tumgoren wrote:
> Hey everyone,
> 
> Ricardo was nice enough to post his solution as a recipe on ActiveState.
> For anyone interested in bookmarking it, here's the link:
> 
> http://code.activestate.com/recipes/577186-accessing-cursors-by-field-name/
> 
> Serdar
> 

I really like Ricardo's solution ... attribute access is a nice touch,
bookmarking it now.

FWIW, it would seem that psycopg2 also has a DictCursor (and
DictConnection).

http://initd.org/psycopg/docs/extras.html

HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PIL problem

[Martin Walsh]

Jim Byrnes wrote:
> Running Unbuntu 9.10. The Synaptic Pkg Mgr reports python-imaging -
> 1.1.6-3ubuntu1 - Python Imaging Library is installed.
> 
> But trying to import PhotoImage gives these results:
> 
> 
 from ImageTk import PhotoImage
> Traceback (most recent call last):
>   File "", line 1, in 
> ImportError: No module named ImageTk
> 
> What have I gotten wrong?
> 

Apparently, ImageTk is part of a separate ubuntu package called
python-imaging-tk.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with choices for new database program

[Martin Walsh]

On 07/03/2010 10:25 AM, Jim Byrnes wrote:
> Jeff Johnson wrote:

[snip]

>> http://dabodev.com/
>>
>> Please check it out. And go to www.leafe.com and subscribe to the
>> dabo-user email list.
> 
> I would like to try out Dabo, but I don't see it in the Ubuntu
> repositories and I would like to avoid using svn if I can.  I didn't
> subscribe to the mailing list but I did read the archives and saw a
> thread about making a deb package.  It seems to have ended in April
> without a clear resolution.
> 
> So is there a package available so I can use the Ubuntu package manager
> to install it?

Unfortunately, after poking around a bit it would seem the only reliable
way of installing dabo for Linux at the moment is checking out trunk
from the project's subversion repository. Someone better informed should
feel free to set the record straight, if I am mistaken.

If your interest in a deb package is mainly the ability to uninstall,
then I'd recommend using virtualenv[1] until a suitable deb package is
released. The steps would be roughly this (untested) ...

$ sudo apt-get install python-reportlab python-wxgtk2.8
$ sudo apt-get install subversion python-virtualenv
$ virtualenv daboenv
$ cd daboenv
$ source bin/activate # this is important
# now we install dabo as recommended, adapted from:
#   http://wiki.dabodev.com/InstallationOnLinux
(daboenv)$
(daboenv)$ mkdir src && cd src
(daboenv)$ svn co http://svn.dabodev.com/dabo/trunk dabo
(daboenv)$ cd dabo
(daboenv)$ python setup.py install # no sudo!
# and run the demo to verify the installation
(daboenv)$ demo/DaboDemo.py

...

Hmm, this might seem like a lot of work -- but by using this method,
dabo is installed under daboenv and not in the system-wide site-packages
-- particularly useful for evaluation, IMO. YMMV.

HTH,
Marty

[1] http://virtualenv.openplans.org/
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Reply All Dilemma of This List

[Martin Walsh]

Wayne Watson wrote:
> I belong to many, many forums, Yahoo Groups, (Usenet) newsgroups, and
> mail lists. Probably 100 or more. I think it's fair to say that none of
> them but this one has an implicit "Reply All". For newsgroups and mail
> lists, I just press my Mozilla Seamonkey mailer Reply button and the
> resulting message is ready to be seen by everyone, a single address.
> Here a Reply goes only to the poster, none to Tutor. Elsewhere, for
> e-mail-like posts, if I really want to make a special effort to single
> out the poster too, "Reply All" works to additionally get it directly to
> them (actually they'd get two messages directly) and the entire list. 
> For YGs and forums, the  "Reply All" is implicit in a response.
> 
> Since this group, in my world, is unique in these matters, I'll just
> offer the following header for a mail list I belong to, the ASTC, for
> someone's consideration. I suppose that someone might be whoever created
> this mail list. It' definitely different than used here, and no one uses
> "Reply All" to my knowledge.
> 
> Maybe they can figure out if it has applicability here.

This is a contentious topic which comes up at least once a year on this
list. A search of the archives will turn up some interesting debate,
most likely. FWIW, I like the behavior of this list as opposed to others.

You may find these additional references illuminating ... you may not.
http://effbot.org/pyfaq/tutor-why-do-my-replies-go-to-the-person-who-sent-the-message-and-not-to-the-list.htm
http://woozle.org/~neale/papers/reply-to-still-harmful.html

HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Changing the Attribute of a Variable

[Martin Walsh]

Marc Tompkins wrote:
> Also - config_var_list is a tuple of lists.  (I'm guessing you intended
> to make it a list of lists - that's what the name indicates, after all -
> but putting it between "( )" makes it a tuple.)  

Sound advice, but a subtle clarification is warranted I think. It's the
comma(s) that make a tuple not the parens (and an absence of square
brackets, I suppose). Consider the following:

In [1]: a = (1)

In [2]: a
Out[2]: 1

In [3]: type(a)
Out[3]: 

In [4]: b = (1,)

In [5]: b
Out[5]: (1,)

In [6]: type(b)
Out[6]: 

In [7]: c = 1, 2, 3

In [8]: c
Out[8]: (1, 2, 3)

In [9]: type(c)
Out[9]: 

...

Wayne, I second Marc's advice that you're making it hard on yourself.
Understandable to a degree, if you are trying to avoid major
modification to inherited code.

But, loading and saving configuration data is a 'solved' problem in that
there are many choices of ready-made tools to help you accomplish the
task. And, I don't see anything in your description that would indicate
a custom solution is necessary -- except perhaps for the learning
experience, almost never a bad idea, IMHO.

Marc has already pointed out ConfigObj, which is excellent, but I
thought I would also suggest ConfigParser -- part of the standard lib,
if a bit less feature-full.

Back to your original question, and I'm probably overstating the obvious
at this point, but the underlying problem is that the operation causing
the exception is expecting a datetime.time object and getting a str. So,
it's less about adding a strftime attribute to the str object, and more
about 'converting' the str into a datetime.time object.

Short of re-writing for ConfigObj (which provides a type conversion and
validation mechanism), or pickle, or similar -- you'll need to work out
how to convert between str and datetime.time. Here are some (untested)
examples:

def time_to_str(t):
return t.strftime('%H:%M:%S')

def str_to_time(s):
h, m, s = [int(u) for u in s.split(':')]
return datetime.time(h, m, s)

HTH,
Marty

PS. You can avoid posting images of tracebacks by enabling 'Quick Edit'
mode in your windows command prompt. More info here:
http://support.microsoft.com/kb/282301

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Simple CGI script and Apache configuration

[Martin Walsh]

wormwood_3 wrote:
> Hello all,

Hi Sam,

> I'll try to give as much detail as I can, but this is a somewhat vague
> problem. I have a very simple script that I would like to implement as a
> CGI script, just so I can hit a URL and get some output. However, after
> following a number of tutorials, I am still seeing some very odd
> results. I am almost sure it's in my Apache configuration, but I figured
> a few people on this list would likely know what the minimal related
> Apache config should be. (The Apache docs are pretty daunting...)
> 
> Local version wise, I am on Ubuntu 8.10, with Apache 2.2.9 and Python
> 2.5.2. I have libapache2-mod-python installed. Apache config is out of
> the box, along with:
> 
> ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

You need more than this to make apache cgi work, I think.

Firstly, mod_cgi should be loaded -- look for a symlink named cgi.load
or some such, in /etc/apache2/mods-enabled. Run 'a2enmod cgi' if you
don't have one. Secondly, ExecCGI is usually enabled using the Options
directive within a Directory definition -- but, less commonly, you might
see something like 'AddHandler cgi-program .py' in an apache or site
config.

Of course, the script needs to be executable by the apache user (which
would be 'www-data' on ubuntu, IIRC), and contain an appropriate shebang
(#!) on the first line -- but it sounds like you have that covered.

Both 'Options ExecCGI' and 'Addhandler cgi-program .py' are allowed in
.htaccess also, given an appropriate AllowOverride directive for the
path in question. Something to look for on the working system, if all
else fails.

You do *not* need mod python to run python cgi scripts.

> In /var/www/cgi-bin, I have hello.py :
> 
> #!/usr/bin/python
> import cgitb
> cgitb.enable()
> 
> print "Content-type: text/html"
> print
> print ""
> print "Hello!"
> print ""
> 
> Reload, hit http://localhost/cgi-bin/hello.py in a browser, I get
> centered text just fine. Now I want to do this same process on my remote
> webserver. On there, I am on Ubuntu 7.10, with Apache 2.2.4 and Python
> 2.5.1. I add:
> 
> ScriptAlias /python/ "/var/www/samuelhuckins.com/python"

You can try appending something like this (untested):


  AllowOverride None
  Options ExecCGI
  # or, Options +ExecCGI to merge
  # with options from parent dir(s)
  Order allow,deny
  Allow from all


> 
> Reload, hit http://samuelhuckins.com/python/hello.py, and I get a 404?
> The perms and ownership on the file is the same as in other directories.
> Do I need to add some sort of handler, with mod_python.publisher? I
> think I am just missing one of the basics of this whole process.

Hmmm, interesting. It's unlikely that any of my advice will help you
with a 404. With an incomplete apache cgi config, the response I'd
expect would be either a 403 (Forbidden), or the script itself in plain
text. Do the logs provide any additional information?

Re-check your spelling. A 404 with vanilla apache config might just
indicate a typo. When you say 'Reload', I assume you mean the apache
daemon (ie. /etc/init.d/apache2 reload or apache2ctl reload)?

Again, you do *not* need mod python to run python cgi scripts.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Simple CGI script and Apache configuration

[Martin Walsh]

wormwood_3 wrote:
> Thanks for all the suggestions! I tried to go through them, and will
> relate what results I encountered. I changed my Apache config to:
> 
>  
>  AllowOverride None
>  Options ExecCGI
>  Order allow,deny
>  Allow from all
>  
> 
> I in fact did not have the cgi module enabled, so I did that. Then I ran
> "sudo /etc/init.d/apache2 reload", and hit
> http://samuelhuckins.com/cgi-bin/hello.py, which contains simply:
> 
> #!/usr/bin/python
> print "Content-type: text/html"
> print
> print ""
> print "Hello!"
> print ""
> 
> I get prompted to download the file, but it does not execute or appear
> in plain text. The logs just show the request being made. What is the
> missing element to get this script to execute?
> 

When you look at the downloaded file, is it your python script?

Looks like you changed the path where you're keeping your cgi script,
did you update the ScriptAlias directive to suit?

You may find this more helpful ...
http://httpd.apache.org/docs/2.0/howto/cgi.html

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Simple CGI script and Apache configuration

[Martin Walsh]

wormwood_3 wrote:
> I wasn't sure if that was needed, so I took it out, sorry about that. I put
> 
> ScriptAlias /cgi-bin/ "/var/www/samuelhuckins.com/cgi-bin/"
>  
> in place, reloaded, and it works! I think the problem throughout was
> that I was mixing up what was necessary between CGI and mod_python.

The apache2 documentation isn't clear about the relationship between
ExecCGI and ScriptAlias (rather, it's not clear to me) and unless I've
missed something, seems to imply that either ScriptAlias or ExecCGI
alone should be sufficient. Unfortunately, I don't have time to
experiment. IIRC, all of the vanilla apache configs I've worked with
recently include both in definitions for cgi-bin.

In any case, glad it worked.

> 
> If you'd like a random programming epigram served up by this new config,
> check out: http://samuelhuckins.com/cgi-bin/qotd.py

Very cool, thanks.

> 
> -Sam
> wormwood_3 wrote:
>> Thanks for all the suggestions! I tried to go through them, and will
>> relate what results I encountered. I changed my Apache config to:
>>
>>  
>>  AllowOverride None
>>  Options ExecCGI
>>  Order allow,deny
>>  Allow from all
>>  
>>
>> I in fact did not have the cgi module enabled, so I did that. Then I ran
>> "sudo /etc/init.d/apache2 reload", and hit
>> http://samuelhuckins.com/cgi-bin/hello.py, which contains simply:
>>
>> #!/usr/bin/python
>> print "Content-type: text/html"
>> print
>> print ""
>> print "Hello!"
>> print ""
>>
>> I get prompted to download the file, but it does not execute or appear
>> in plain text. The logs just show the request being made. What is the
>> missing element to get this script to execute?
>>
> 
> When you look at the downloaded file, is it your python script?
> 
> Looks like you changed the path where you're keeping your cgi script,
> did you update the ScriptAlias directive to suit?
> 
> You may find this more helpful ...
> http://httpd.apache.org/docs/2.0/howto/cgi.html
> 
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] re.format a file

[Martin Walsh]

A.T.Hofkamp wrote:
> prasad rao wrote:
>> helloThank you Lie and Kent.
>> I forgot  about newline character and the fact that string can be sliced.
>> Thanks for your timely help
>> BTW I have gone through  the Python library reference and find no
>> examples
>> in fileinput module.
> 
> The fileinput module only deals with reading and writing data from/to
> files, it does not deal with manipulating that data.
> 
> How to manipulate strings is in the 'strings' or 'text' section of a
> tutorial.
> 
>> z=fileinput.input(file,inplace=1)
>> for line in  z:
>> ???if len(line)<60:pass
>> ???if len(line)>60:
>> ??line=line[:60]+'\n'+line[60:]
>> Is it the right way to do?
> 
> A nice step forward, I'd say.
> 
> Did you consider what to do with ridiculous long lines, eg 200, 500,
> 1000 or 1 characters long?
> If you want to deal with them, you'd need to repeatedly split the line.
> You could use a while loop for it.

Or if the lines resemble paragraphs, then one might use the textwrap
module which breaks on word boundaries by default I think, and provides
several options for tweaking -- perhaps not what the OP is looking for.

# untested
import textwrap
for line in z:
line = textwrap.fill(line, width=60)

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] probelm pyhton shell doesnt open help please

[Martin Walsh]

mustafa akkoc wrote:
> 
> it gives this message socket error 
>

IDLE's subprocess didn't make a connection. Either IDLE can't start a
subprocess or personal firewall software is blocking the connection.


IIRC, this was once a known issue with IDLE when combined with the
windows firewall service, or when running multiple instances of IDLE
(perhaps inadvertently). But, I'm having difficulty tracking down the
pertinent bug report(s) -- maybe these have been fixed? or I'm just
tired, probably the latter.

A couple of suggestions from memory ...
1. Check the process/task list for errant pythonw.exe processes, and end
them.
2. Launch IDLE with the -n flag from a terminal (command prompt).
3. Report back to the list with your results, and include the python and
windows version info if you continue to have trouble.

HTH,
Marty

PS. Keep in mind, some of us won't see images you post to the list so
you should generally include a text version of error messages whenever
possible. Or at least, note when you've included an image so that those
with sufficient interest can make the extra effort to view it, if
necessary.


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] merging dictionary values based on key

[Martin Walsh]

> greg whittier wrote:
>> On Thu, Mar 12, 2009 at 4:24 PM, ski  wrote:
>> mylist = [{'a': 'x123', 'b':'12'}, {'a': 'x234', 'b': 'd33', 'c':
>> 'a23'}, {'a': 'x234', 'c': 'XX123'}  ]
>>> where mylist has nth number of dictionaries and i want to merge the
>>> values
>>> of the keys that are the same?
>>>
>>
>> If I understand what you mean by merging, I  think you want
>>
>> mylist = [{'a': 'x123', 'b':'12'}, {'a': 'x234', 'b': 'd33', 'c':
>> 'a23'}, {'a': 'x234', 'c': 'XX123'}  ]
>> merged_dict = {}
>> for dictionary in mylist:
>> for key, value in dictionary.items():
>> merged_dict.setdefault(key,[]).append(value)

Or similarly with defaultdict:

from collections import defaultdict

merged_dict = defaultdict(list)
for d in mylist:
for k, v in d.items():
merged_dict[k].append(v)

ski wrote:
> how would you do this for a specific key instead of all the keys?

alist = [d['a'] for d in mylist if d.has_key('a')]

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modifying Grayson's Example 5_14

[Martin Walsh]

Wayne Watson wrote:
...

> it.  It works pretty well, but puts up a a few probably top level
> windows that are blank. How do I get around them, and is there anything

...

> 
> root = Tk()

Try adding this,

root.withdraw()

> dialog = GetPassword(root)

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modifying Grayson's Example 5_14

[Martin Walsh]

> Martin Walsh wrote:
>> Wayne Watson wrote:
>> 
>>
>>
>>> it.  It works pretty well, but puts up a a few probably top level
>>> windows that are blank. How do I get around them, and is there anything
>>>
>>
>> 
>>
>>
>>> root = Tk()
>>>
>>
>> Try adding this,
>>
>> root.withdraw()
>>
>>
>>> dialog = GetPassword(root)
>>>
>>
>> HTH,
>> Marty

Wayne Watson wrote:
> That worked. Why the "extra" blank window though?

I'm no Tkinter expert -- far from it. But I would assume that
GetPassword, as a subclass of Dialog, creates a new window, and thus you
have no need for the root window. root.withdraw() hides the root window
from view, and allows the application to continue running. I imagine
there are other ways to accomplish the same thing.

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fun with Label and Entry--Why NoneType?

[Martin Walsh]

Wayne Watson wrote:
> The program below is derived from an example in Grayson for showing how
> one might a dialog for entering passwords. The structure seems just like
> the original, down to the use of self, Label and Entry plus . Yet the
> print "here" statement produces:
> here None 
> I'm missing something. The NoneType causes the print of the self.lat to
> fail with get().
> 
> The original returns something from the corresponding body function, but
> taking it out does nothing in either program. The original program is
> posted above under "Modifying Grayson's Example 5_14".
> 
> # Derived from Grayson 5_14.py
> from   Tkinter import *
> from   tkSimpleDialog import Dialog
> import tkSimpleDialog
> import tkMessageBox
> #import Pmw
> 
> class DialogPrototype(Dialog):
> 
> def body(self, master):
> self.title("Enter Site Data")
> Label(master, text='Latitude:').grid(row=0, sticky=W)
> self.lat=Entry(master, width=12).grid(row=0, column=1)

This is where you diverge from the Grayson example. What you're saying
is that self.lat should be set to the result of Entry(...).grid(...)
which is always None, presumably. What I think you want is self.lat to
be a reference to the Entry widget itself. Try this,

  self.lat = Entry(master, width=12)
  self.lat.grid(row=0, column=1)

>
> Label(master, text='Longitude:').grid(row=0, column=2)
> self.long=Entry(master, width=12).grid(row=0, column=3)

... and,
  self.long = Entry(master, width=12)
  self.long.grid(row=0, column=3)

> print "here", self.long,type(self.long)
> return
>
> def apply(self):
> print "apply"
> print self.lat.get()
> print self.long.get()
> 
> print "setting"
> lat=1.0
> long=0.0

Is that the way you indented the above, really? I suppose it could be
intentional, but probably not.

> 
> root = Tk()
> root.withdraw()
> dialog = DialogPrototype(root)

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Executing a C Program from RH Linux in Python for Win

[Martin Walsh]

Wayne Watson wrote:
> If you can execute a C program compiled on a Linux with SWIG, then
> that's what I'm looking for. There's really no RH dependency according
> to the above posts. If it were compiled on Debian or Ubuntu, it appears
> it would not make any difference. That is, one could execute a RH
> executable from C on Ubuntu.

Yeah, probably -- if it's a static build, or if the dependencies
(required libraries/versions) are installed, assuming the program has
dependencies. But, I suppose we may be drifting a bit OT.

> 
> Is there a simple example of this in action from a Python program and
> some small C Linux executable program?

http://docs.python.org/library/subprocess.html

The subprocess module is commonly recommended for this type of task (as
opposed to os.system, etc). In fact, I believe Alan already suggested it
in this thread. And speaking of ... Alan's tutorial has several very
good examples of using the subprocess module (OS topic under
Manipulating Processes).

http://www.freenetpages.co.uk/hp/alan.gauld/tutos.htm

---

I'll hazard a wild guess that you don't really want SWIG based on your
original question, and subsequent comments. IIUC, SWIG is intended to
ease the creation of a python wrapper (extension module) for existing
C/C++ code. And, I'm not sure you've given enough information about the
C program to determine if SWIG would be useful. Regardless, I suggest
you get a feel for running an external program using python first.

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] PyCon Videos

[Martin Walsh]

Hi All,

Not sure if it's common knowledge, particularly for those who didn't
make it to PyCon this year, but all of the talks were recorded and will
be available online in good time, thanks to Carl Karsten and his merry
band of A/V volunteers. I can't even begin to grasp how much work is
required to accomplish such a monumental task.

I noticed this morning that the tutorials, are making their way to the
blip.tv site, including "Internet Programming with Python" presented by
our very own Wesley Chun. And I presume "An Introduction to
Object-Oriented Programming", announced on this list a couple months ago
by Michael Goldwasser, will be available soon as well. I'm really
looking forward to watching both.

Not a replacement for attending a PyCon in person -- which I highly
recommend for anyone interested in python, no matter your skill level --
but certainly the next best thing.

http://pycon.blip.tv/posts?view=archive&nsfw=dc

It seems, many presenters included links to additional material (code
examples, slides, etc) on the scheduled-talks page [1] -- not sure about
the tutorials [2], but it doesn't look like it.

[1] http://us.pycon.org/2009/conference/schedule/
[2] http://us.pycon.org/2009/tutorials/schedule/

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] list to string and string to list

[Martin Walsh]

johnf wrote:
> On Thursday 16 April 2009 05:04:39 pm Alan Gauld wrote:
>> "johnf"  wrote
>>
 I want to save the list to the field and when I retrieve the string
 convert
 it back to a list.

 But this does NOT work.
 mylist=[1,2,3,4]
 mystr=str(mylist)

 newlist= list(mystr)

 I keep thinking there must be a simple way of get this done.
>>> Is this a good way?
>>> newlist = eval(mystr)
>> eval has all sorts of security implications so I wouldn't recommend
>> it where you are reading data fropm an external source.
>>
>> One thing that might work is this:
> L = [1,2,3,4,5]
> s1 = ','.join(str(n) for n in L)
> s1
>> '1,2,3,4,5'
>>
> newlist = [int(n) for n in s1.split(',')]
> newlist
>> [1, 2, 3, 4, 5]
>>
>> Provided your original data doesn't have commas to start with
>> it should work, I think... And the data needs to be pretty
>> homogenous to allow a single conversion function.
> 
> Kent Johnson suggested 
> 
> newlist = map(int, mystr[1:-1].split(','))
> 

Maybe that's a question in disguise, but I would think both suggestions
are valid.

Another, less valid, suggestion would be to pickle or shelve the list
before storing it in the database -- although this poses similar
security implications as eval. And the resulting string is not
particularly easy to read, if that's important to you.

import cPickle as pickle
mylist = [1,2,3,4]

mystr = pickle.dumps(mylist)
# '(lp1\nI1\naI2\naI3\naI4\na.'

newlist = pickle.loads(mystr)
# [1, 2, 3, 4]

HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] list to string and string to list

[Martin Walsh]

Martin Walsh wrote:
> johnf wrote:
>> On Thursday 16 April 2009 05:04:39 pm Alan Gauld wrote:
>>> "johnf"  wrote
>>>
>>>>> I want to save the list to the field and when I retrieve the string
>>>>> convert
>>>>> it back to a list.
>>>>>
>>>>> But this does NOT work.
>>>>> mylist=[1,2,3,4]
>>>>> mystr=str(mylist)
>>>>>
>>>>> newlist= list(mystr)
>>>>>
>>>>> I keep thinking there must be a simple way of get this done.
>>>> Is this a good way?
>>>> newlist = eval(mystr)
>>> eval has all sorts of security implications so I wouldn't recommend
>>> it where you are reading data fropm an external source.
>>>
>>> One thing that might work is this:
>>>>>> L = [1,2,3,4,5]
>>>>>> s1 = ','.join(str(n) for n in L)
>>>>>> s1
>>> '1,2,3,4,5'
>>>
>>>>>> newlist = [int(n) for n in s1.split(',')]
>>>>>> newlist
>>> [1, 2, 3, 4, 5]
>>>
>>> Provided your original data doesn't have commas to start with
>>> it should work, I think... And the data needs to be pretty
>>> homogenous to allow a single conversion function.
>> Kent Johnson suggested 
>>
>> newlist = map(int, mystr[1:-1].split(','))
>>
> 
> Maybe that's a question in disguise, but I would think both suggestions
> are valid.
> 
> Another, less valid, suggestion would be to pickle or shelve the list
> before storing it in the database -- although this poses similar
> security implications as eval. And the resulting string is not
> particularly easy to read, if that's important to you.
> 
> import cPickle as pickle
> mylist = [1,2,3,4]
> 
> mystr = pickle.dumps(mylist)
> # '(lp1\nI1\naI2\naI3\naI4\na.'
> 
> newlist = pickle.loads(mystr)
> # [1, 2, 3, 4]

Sorry to reply to my own post, but the json module in python 2.6+
(formerly 3rd party, simplejson) might work for your purposes also. But
I must confess, I'm not that familiar.

import json

s = json.dumps([1, 2, 3, 4])
# '[1, 2, 3, 4]'
l = json.loads(s)
# [1, 2, 3, 4]

> 
> HTH,
> Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Sending a disconnect after openssl s_client command?

[Martin Walsh]

Kayvan Sarikhani wrote:
> Tutors,
> 
>   I'm working on a script to verify whether a particular website
> supports SSLv2 via the following:
> 
> --- BEGIN ---
> #!/usr/bin/python
> import os, re
> 
> checkssl_out = open('checkssl.txt','w')
> 
> website = 'somewebsitename'
> sslv2 = 'Protocol  : SSLv2'
> 
> print 'Checking:', website
> 
> checksslv2 = os.popen('openssl s_client -ssl2 -connect
> somewebsitename:443').read().strip()
> 
> if re.search(sslv2, checksslv2) == None:
> print >> checkssl_out, website, 'does NOT support SSLv2'
> else:
> print >> checkssl_out, website, 'supports: SSLv2'
> 
> checkssl_out.close()
> --- END ---
> 
>   It works, but the problem is that OpenSSL does not automatically
> disconnect after end of input. I was curious if there's a way to send a
> CTRL-C at the end of the command, so that it *does* capture the output,
> and breaks after it. Any suggestions or help is appreciated!

You can do something like the following (untested) to simulate a CTRL-C,
but I'd recommend against it, as I doubt it will work as you expect ...

import os, signal
from subprocess import Popen, PIPE

openssl_cmd = 'openssl s_client -ssl2 -connect somewebsitename:443'
openssl = Popen(openssl_cmd, shell=True, stdout=PIPE, stderr=PIPE)

os.kill(openssl.pid, signal.SIGINT)

# dead, I bet, before any output is generated
stdout, stderr = openssl.communicate()


Instead, you may want to try to mimic this command-line behavior ...

echo "GET /" | openssl s_client -ssl2 -connect somewebsitename:443

... in which case, you can try something like this ...

from subprocess import Popen, PIPE

openssl_cmd = 'openssl s_client -ssl2 -connect somewebsitename:443'
openssl = Popen(
  openssl_cmd, shell=True, stdout=PIPE, stderr=PIPE, stdin=PIPE
)
stdout, stderr = openssl.communicate('GET /')

Alternatively, if you're using python 2.6 and above, it looks like you
can do something similar with a few lines of code, and the ssl module
from the standard lib ...

# untested!
import ssl
try:
cert = ssl.get_server_certificate(
('somewebsitename', 443), ssl.PROTOCOL_SSLv2
)
except ssl.SSLError, ex:
# site may not support sslv2
...

HTH,
Marty

> 
> K
> 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] String Encoding problem

[Martin Walsh]

Matt wrote:
> Hey everyone,
> 
> I'm hoping someone here can help me solve an odd problem (bug?). I'm
> having trouble with string encoding, object deletion, and the xml.etree
> library. If this isn't the right list to be posting this question,
> please let me know. I'm new to Python and don't know of any other "help
> me" Python mailing lists. I have tried debugging this ad-infinitem.
> Anyway, at the bottom of this e-mail you will find the code of a python
> file. This is a gross over-simplification of my code, with little
> exception handling so that the errors are obvious.
> 
> Running this interactively, if you finish off with 'del db', it exits
> fine and creates a skeleton xml file called 'db.xml' with text ' />'. However, if you instead CTRL-D, it throws at exception while
> quitting and then leaves an empty 'db.xml' which won't work. Can anyone
> here help me figure out why this is?
> 
> Stuff I've done:
> I've traced this down to the self.commit() call in __del__. The
> stacktrace and a few print statements injected into xml.etree leads me
> to the call 'root'.encode('us-ascii') throwing a LookupError on line 751
> of xml.etree.ElementTree. This makes no sense to me, since it works fine
> normally.

The environment available to __del__ methods during program termination
is wonky, and apparently not very consistent either. I can't say that I
completely understand it myself, perhaps someone else can provide a
better explanation for both of us, but some of the causes are described
in the documentation:

http://docs.python.org/reference/datamodel.html#object.__del__

What is your rationale for using __del__? Are you trying to force a
'commit()' call on Database instances when your program terminates -- in
the case of an unhandled exception, for example?

HTH,
Marty

> 
> Thank you very much. Any and all help or pointers are appreciated.
> 
> ~Matt
> 
>  db.py ###
> from xml.etree import ElementTree as ET
> import os
> 
> class Database(object):
> def __init__(self, path):
> self.__dbpath = path## Path to the database
> self.load()
> def __del__(self):
> ## FIXME: Known bug:
> ##  del db at command line works properly
> ##  Ctrl-D, when there is no db file present, results in a
> LookupError
> ##and empty xml file
> from StringIO import StringIO
> from traceback import print_exc
> trace = StringIO()
> try:
> print 5
> self.commit()
> print 7
> except Exception:
> print_exc(100, trace)
> print trace.getvalue()
> def load(self):
> if os.path.exists(self.__dbpath):
> self.root = ET.parse(self.__dbpath).getroot()
> else:
> self.root = ET.Element("root")
> def commit(self):
> ET.ElementTree(self.root).write(self.__dbpath)
> db = Database('db.xml')

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] String Encoding problem

[Martin Walsh]

Forwarding to the list. Matt, perhaps you can repost in plain text, my
mail client seems to have mangled your source ...

Strax-Haber, Matthew (LARC-D320) wrote:
>> *From: *Martin Walsh 
>>
>> The environment available to __del__ methods during program termination
>> is wonky, and apparently not very consistent either. I can't say that I
>> completely understand it myself, perhaps someone else can provide a
>> better explanation for both of us, but some of the causes are described
>> in the documentation:
>>
>> http://docs.python.org/reference/datamodel.html#object.__del__
>>
>> What is your rationale for using __del__? Are you trying to force a
>> 'commit()' call on Database instances when your program terminates -- in
>> the case of an unhandled exception, for example?
> 
> Perhaps I oversimplified a bit. In my actual code, there is a database
> class and an operator class. The actual structure is this:
> 
> In operator:
> def __init__(self, path, saveDB=True, cleanUp=True):
>'''Constructor'''## Calculate filesystem paths
>self.WORK_DIR= path + '.tmp'DB_PATH= path
> + '.xml'self.SAVE_DB= saveDB## finish(): Delete
> unnecessary files created by run?self.CLEANUP= cleanUp##
> finish(): Delete database at end of run?## Make sure we
> have a working directory (exception on failed write)if not
> os.path.isdir(self.WORK_DIR):os.mkdir(self.WORK_DIR)
> 
>self._db = DB.Database(DB_PATH)
> ## SOME OTHER ENVIRONMENT SETUP STUFF
> def _cleanUpEnvironment(self):  try:## Delete
> temp files created for this runfor path,dirs,files in
> os.walk(self.WORK_DIR, topdown=False):for f in files:
>os.unlink(os.path.join(path,f))for d in dirs:
>os.rmdir(os.path.join(path,d))os.rmdir(self.WORK_DIR)
>except:print >>sys.stderr, 'Could not delete temp
> files; left at:'print >>sys.stderr, self.WORK_DIRdef
> finish(self):'''Clean up and finish the run (write out to
> the database)'''if self.SAVE_DB:self._db.commit()
>if self.CLEANUP:self._cleanUpEnvironment()
> def __del__(self):## FIXME: Known bug:##  del t at
> command line works properly##  Ctrl-D, when there is no db file
> present, results in a LookupErrorself.finish()
> if __name__ == '__main__':
> printHelp()
> ## Provide tab completion to the userimport readline,
> rlcompleterreadline.parse_and_bind('tab: complete')
> t= OperatorClassName(os.path.splitext(__file__)[0])
> 
> In database:
> def commit(self):'''Write the database back to the file'''
>## Noticed exceptions: IOError
>ET.ElementTree(self.root).write(self.__dbpath)
> 
> The operator also has a series of functions that collect data and enter
> it into the database. Here’s the usage pattern I want to have:
> 
> User calls Operator.py. User runs a bunch of functions of the form
> t.(). When the user quits, the database will save or not
> save depending on the status of t.SAVE_DB (which some functions in the
> operator will programmatically change).
> 
>>
>> HTH,
>> Marty
>>
>>>
>>> Thank you very much. Any and all help or pointers are appreciated.
>>>
>>> ~Matt
>>>
>>>  db.py ###
>>> from xml.etree import ElementTree as ET
>>> import os
>>>
>>> class Database(object):
>>> def __init__(self, path):
>>> self.__dbpath = path## Path to the database
>>> self.load()
>>> def __del__(self):
>>> ## FIXME: Known bug:
>>> ##  del db at command line works properly
>>> ##  Ctrl-D, when there is no db file present, results in a
>>> LookupError
>>> ##and empty xml file
>>> from StringIO import StringIO
>>> from traceback import print_exc
>>> trace = StringIO()
>>> try:
>>> print 5
>>> self.commit()
>>> print 7
>>> except Exception:
>>> print_exc(100, trace)
>>> print trace.getvalue()
>>> def load(self):
>>> if os.path.exists(self.__dbpath):
>>>   

Re: [Tutor] PDF to text conversion

[Martin Walsh]

Robert Berman wrote:
> Hello Emad,
> 
> I have seriously looked at the documentation associated with pyPDF. This
> seems to have the page as its smallest element of work, and what i need
> is a line by line process to go from .PDF format to Text. I don't think
> pyPDF will meet my needs but thank you for bringing it to my attention.
> 
> Thanks,
> 
> 
> Robert Berman

Have you looked at pdfminer?

http://www.unixuser.org/~euske/python/pdfminer/index.html

Looks promising.

HTH,
Marty


> 
> Emad Nawfal (عماد نوفل) wrote:
>>
>>
>> On Tue, Apr 21, 2009 at 12:54 PM, bob gailer > > wrote:
>>
>> Robert Berman wrote:
>>
>> Hi,
>>
>> I must convert a history file in PDF format that goes from May
>> of 1988 to current date.  Readings are taken twice weekly and
>> consist of the date taken mm/dd/yy and the results appearing
>> as a 10 character numeric + special characters sequence. This
>> is obviously an easy setup for a very small database
>>  application with the date as the key, the result string as
>> the data.
>>
>> My problem is converting the PDF file into a text file which I
>> can then read and process. I do not see any free python
>> libraries having this capacity. I did see a PDFPILOT program
>> for Windows but this application is being developed on Linux
>> and should also run on Windows; so I do not want to
>> incorporate a Windows only application.
>>
>> I do not think i am breaking any new frontiers with this
>> application. Have any of you worked with such a library, or do
>> you know of one or two I can download and work with?
>> Hopefully, they have reasonable documentation.
>>
>>
>> If this is a one-time conversion just use the save as text feature
>> of adobe reader.
>>
>>
>>
>> My development environment is:
>>
>> Python
>> Linux
>> Ubuntu version 8.10
>>
>>
>> Thanks for any help  you might be able to offer.
>>
>>
>> Robert Berman
>> ___
>> Tutor maillist  -  Tutor@python.org 
>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>>
>> -- Bob Gailer
>> Chapel Hill NC
>> 919-636-4239
>>
>> ___
>> Tutor maillist  -  Tutor@python.org 
>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>>
>> I tried pyPdf once, just for fun, and it was nice:
>> http://pybrary.net/pyPdf/
>> -- 
>> لا أعرف مظلوما تواطأ الناس علي هضمه ولا زهدوا في إنصافه
>> كالحقيقة.محمد الغزالي
>> "No victim has ever been more repressed and alienated than the truth"
>>
>> Emad Soliman Nawfal
>> Indiana University, Bloomington
>> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Sending a disconnect after openssl s_client command?

[Martin Walsh]

Kayvan Sarikhani wrote:
> On Mon, Apr 20, 2009 at 1:17 PM, Martin Walsh  <mailto:mwa...@mwalsh.org>> wrote:
> 
> from subprocess import Popen, PIPE
> 
> openssl_cmd = 'openssl s_client -ssl2 -connect somewebsitename:443'
> openssl = Popen(
>  openssl_cmd, shell=True, stdout=PIPE, stderr=PIPE, stdin=PIPE
> )
> stdout, stderr = openssl.communicate('GET /')
> 
> Alternatively, if you're using python 2.6 and above, it looks like you
> can do something similar with a few lines of code, and the ssl module
> from the standard lib ...
> 
> # untested!
> import ssl
> try:
>cert = ssl.get_server_certificate(
>('somewebsitename', 443), ssl.PROTOCOL_SSLv2
>)
> except ssl.SSLError, ex:
># site may not support sslv2
>...
> 
> HTH,
> Marty
> 
>  
> Thanks Marty; this does indeed help...it just also means I need to
> really learn how subprocess works. ;) I wish I could claim to be using
> 2.6, but unfortunately the most current version at work is Python
> 2.5.2...most boxes here are even below, and I can't convince them to
> upgrade. Ah, well.

Yep, subprocess is the way to go.

In that case, if you're not offended by the extra dependency, then you
might be interested in http://pypi.python.org/pypi/ssl, which appears to
be a backport of the 2.6 ssl module.

I haven't tried it myself, but it has a get_server_certificate helper
also, so I'd expect it to work the same way. Although, you'll probably
want to explore in greater detail the properties of the exception that
is raised by a site not supporting sslv2. When I tried I received an
SSLError(errno=6) for a server configured w/o sslv2.

> 
> Thanks again though!

You're welcome, glad it helped. :)

Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] finding mismatched or unpaired html tags

[Martin Walsh]

A.T.Hofkamp wrote:
> Dinesh B Vadhia wrote:
>> I'm processing tens of thousands of html files and a few of them
>> contain mismatched tags and ElementTree throws the error:
>>
>> "Unexpected error opening J:/F2/663/blahblah.html: mismatched tag:
>> line 124, column 8"
>>
>> I now want to scan each file and simply identify each mismatched or
>> unpaired
> tags (by line number) in each file. I've read the ElementTree docs and
> cannot
> see anything obvious how to do this. I know this is a common problem but
> feeling a bit clueless here - any ideas?
>>
> 
> Don't use elementTree, use BeautifulSoup instead.
> 
> elementTree expects perfect input, typically generated by another computer.
> BeautifulSoup is designed to handle your everyday HTML page, filled with
> errors of all possible kinds.

But it also modifies the source html by default, adding closing tags,
etc. Important to know, I suppose, if you intend to re-write the html
files you parse with BeautifulSoup.

Also, unless you're running python 3.0 or greater, use the 3.0.x series
of BeautifulSoup -- otherwise you may run into the same issue.

http://www.crummy.com/software/BeautifulSoup/3.1-problems.html

HTH,
Marty





___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] finding mismatched or unpaired html tags

[Martin Walsh]

Dinesh B Vadhia wrote:
> A.T. / Marty
>  
> I'd prefer that the html parser didn't replace the missing tags as I
> want to know where and what the problems are.  Also, the source html
> documents were generated by another computer ie. they are not web page
> documents.  My sense is that it is only a few files out of tens of
> thousands.  Cheers ...
>  
> Dinesh

If this is a one time task, write a script to iterate over the html
files, and collect the traceback info from those that throw a
'mismatched tag' error. Based on your example below, it appears to
contain the line number. You'd only get one error per file per run, but
you can run it until there are no errors remaining. I hope that makes
sense.

HTH,
Marty

>  
>  
> 
> Message: 7
> Date: Tue, 28 Apr 2009 08:54:33 -0500
> From: Martin Walsh 
> Subject: Re: [Tutor] finding mismatched or unpaired html tags
> To: "tutor@python.org" 
> Message-ID: <49f70a99.3050...@mwalsh.org>
> Content-Type: text/plain; charset=us-ascii
> 
> A.T.Hofkamp wrote:
>> Dinesh B Vadhia wrote:
>>> I'm processing tens of thousands of html files and a few of them
>>> contain mismatched tags and ElementTree throws the error:
>>>
>>> "Unexpected error opening J:/F2/663/blahblah.html: mismatched tag:
>>> line 124, column 8"
>>>
>>> I now want to scan each file and simply identify each mismatched or
>>> unpaired
>> tags (by line number) in each file. I've read the ElementTree docs and
>> cannot
>> see anything obvious how to do this. I know this is a common problem but
>> feeling a bit clueless here - any ideas?
>>>
>>
>> Don't use elementTree, use BeautifulSoup instead.
>>
>> elementTree expects perfect input, typically generated by another
> computer.
>> BeautifulSoup is designed to handle your everyday HTML page, filled with
>> errors of all possible kinds.
> 
> But it also modifies the source html by default, adding closing tags,
> etc. Important to know, I suppose, if you intend to re-write the html
> files you parse with BeautifulSoup.
> 
> Also, unless you're running python 3.0 or greater, use the 3.0.x series
> of BeautifulSoup -- otherwise you may run into the same issue.
> 
> http://www.crummy.com/software/BeautifulSoup/3.1-problems.html
> 
> HTH,
> Marty
> 
> 
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Add newline's, wrap, a long string

[Martin Walsh]

David wrote:
> I am getting information from .txt files and posting them in fields on a
> web site. I need to break up single strings so they are around 80
> characters then a new line because when I enter the info to the form on
> the website it has fields and it errors out with such a long string.
> 
> here is a sample of the code;
> 
> #!/usr/bin/python
> import subprocess
> import os
> 
> u_e = subprocess.Popen(
> 'grep USE /tmp/comprookie2000/emerge_info.txt |head -n1|cut -d\\"-f2',

Did you copy and paste this faithfully? The 'cut -d\\"-f2' looks a bit
odd. Is the delimiter a " (double-quote)? Perhaps you left out a space
before the -f2?

> shell=True, stdout=subprocess.PIPE,)
> os.waitpid(u_e.pid, 0)

'u_e.wait()' would wait the way you intend, as well, I believe.

> USE = u_e.stdout.read().strip()

Or, if you use the communicate() method of the Popen object, the wait is
implicit. As in,

stdout, stderr = u_e.communicate()

... or perhaps ...

USE = u_e.communicate()[0].strip()

... but, you don't need to use subprocess at all. How about (untested),

# grep USE /tmp/comprookie2000/emerge_info.txt |head -n1|cut -d\" -f2
infof = open('/tmp/comprookie2000/emerge_info.txt')
for line in infof:
if 'USE' in line:
USE = line.split('"')[1]
break
else:
USE = ''
infof.close()

> L = len(USE)
> print L
> print USE
> 
> L returns 1337

cosmic :)

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Add newline's, wrap, a long string

[Martin Walsh]

David wrote:
> vince spicer wrote:
>> first, grabbing output from an external command try:
>>
>> import commands
>>
>> USE = commands.getoutput('grep USE /tmp/comprookie2000/emege_info.txt
>> |head -n1|cut -d\\"-f2')
>>  
>> then you can wrap strings,
>>
>> import textwrap
>>
>> Lines = textwrap.wrap(USE, 80) # return a list
>>
>> so in short:
>>
>> import commands, textwrap
>> data = textwrap.wrap(commands.getoutput('my command'), 80)
>>
>>
>>
>> Vince
> Thanks Vince,
> I could not get command to work, but I did not try very hard;
> ["cut: the delimiter must be a single character Try `cut --help' for
> more", 'information. head: write error: Broken pipe']

Ah, I see. This error is most likely due to the typo (missing space
before -f2).

> 
> But textwrap did the trick, here is what I came up with;
> 
> #!/usr/bin/python
> 
> import subprocess
> import os
> import textwrap
> import string
> 
> def subopen():
> u_e = subprocess.Popen(
> 'grep USE /tmp/comprookie2000/emerge_info.txt |head -n1|cut
> -d\\" -f2',
> shell=True, stdout=subprocess.PIPE,)
> os.waitpid(u_e.pid, 0)
> USE = u_e.stdout.read().strip()
> L = textwrap.wrap(USE, 80) # return a list
> Lines = string.join(L, '\n')

Just one more comment, string.join is deprecated, yet join is a method
of str objects. So ...

  Lines = '\n'.join(L)

... or use textwrap.fill which returns a string with the newlines
already in place ...

  Lines = textwrap.fill(USE, 80)

HTH,
Marty

> fname = 'usetest.txt'
> fobj = open(fname, 'w')
> fobj.write(Lines)
> fobj.close
> 
> subopen()
> 
> Here is the output;
> http://linuxcrazy.pastebin.com/m66105e3
> 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Add newline's, wrap, a long string

[Martin Walsh]

David wrote:
> Martin Walsh wrote:
>>> ... but, you don't need to use subprocess at all. How about (untested),
>>>
>>> # grep USE /tmp/comprookie2000/emerge_info.txt |head -n1|cut -d\" -f2
>>> infof = open('/tmp/comprookie2000/emerge_info.txt')
>>> for line in infof:
>>> if 'USE' in line:
>>> USE = line.split('"')[1]
>>> break
>>> else:
>>> USE = ''
>>> infof.close()
>>>
> 
> Thanks Martin works perfect. So I understand correctly, line.split('"')
> splits the list on " so [1] grabs after USE = " and up till [2] starts.

That's right, in other words it emulates the behavior of 'cut -d\" -f2'.
Python having 0-based indexing, you use 1 instead of 2, etc.

> Again thanks, I like the comments :)
> -david
> 

Welcome, glad it helped.

Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Add newline's, wrap, a long string

[Martin Walsh]

Sander Sweers wrote:
> 2009/4/29 David :
>> Here is the whole program so far, what it does is it logs into a druple web
>> site and posts. I would like to make it better, as you can see I do the same
>> thing over and over.
>>
>> http://linuxcrazy.pastebin.com/m7689c088
> 
> What you can do is define all the variables upfront. This way you can
> get rid of the else. Below is an example how you can do this with only
> looping once over the fle.
> 
> CBUILD = ''
> ACCEPT_KEYWORDS = ''
> 
> for line in fname:
> if 'ACCEPT_KEYWORDS' in line:
> output = line.split('"')[1]
> ACCEPT_KEYWORDS = textwrap.fill(output, 80)
> 
> if 'CBUILD' in line:
> output = line.split('"')[1]
> CBUILD = textwrap.fill(output, 80)


What is particularly important to note here is that Sander is iterating
over the file 'fname' only once. When you loop thru the lines of a file
with this form it is consumed, and you get nothing back for subsequent
attempts, unless you re-open the file, or explicitly move back to the
beginning, ie. fname.seek(0).


HTH,
Marty


> 
> etc etc
> 
> Greets
> Sander
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] returning the entire line when regex matches

[Martin Walsh]

Nick Burgess wrote:
> So far the script works fine, it avoids printing the lines i want and
> I can add new domain names as needed. It looks like this:
> 
> #!/usr/bin/python
> import re
> 
> outFile = open('outFile.dat', 'w')
> log = file("log.dat", 'r').read().split('Source') # Set the line delimiter
> for line in log:
> if not re.search(r'notneeded.com|notneeded1.com',line):
> outFile.write(line)

There is a subtle problem here -- the '.' means match any single
character. I suppose it's unlikely to bite you, but it could -- for
example, a line containing a domain named notneeded12com.net would
match. You should probably escape the dot, and while you're at it
compile the regular expression.

# untested
pattern = re.compile(r'notneeded\.com|notneeded1\.com')
for line in log:
if not pattern.search(line):
outFile.write(line)

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python popen command using cat > textfile .... how to terminate

[Martin Walsh]

MK wrote:
> Hi there,
> 
> i am using this code to send an "cat > ThisIsMyUrl" with popen.
> Of cos cat now waits for the CTRL+D command. 
> How can i send this command ?
> 
> def console_command(cmd):
>   print cmd
>   console = os.popen(cmd,"r")
>   output = console.read()
>   console.close()
>   return output
> 
> command="cat > " + working_dir + "/" + subdir + "www.thisismyurl.com"
> console_command(command)

Ignoring the subprocess module for a moment, you could use os.popen2
instead of os.popen and then close the stdin fd to simulate a CTRL-D(EOF).

stdin, stdout = os.popen2(cmd)
stdin.write('This is line 1.\n')
stdin.close() # CTRL-D

With the subprocess module, you might do something like this (untested) ...

from subprocess import Popen, PIPE
console = Popen(
cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True
)
console.stdin.write('This is line 1.\n')
console.stdin.close()
# ... or just ...
# console.communicate('This is line 1.\n')

But, I can't figure out why you would want to do this. Or I should say
it's not at all clear to me what you are trying to accomplish --
although I suspect you are making it difficult for yourself.

Can you provide a brief explanation of what you are trying to achieve?

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] finding difference in time

[Martin Walsh]

Kent Johnson wrote:
> On Fri, May 15, 2009 at 12:46 AM, R K  wrote:
>> Gurus,
>>
>> I'm trying to write a fairly simple script that finds the number of hours /
>> minutes / seconds between now and the next Friday at 1:30AM.
>>
>> I have a few little chunks of code but I can't seem to get everything to
>> piece together nicely.
>>
>> import datetime,time
>> now = datetime.datetime.now()
>>
>> i = 0
>> dayOfWeek = datetime.datetime.now().strftime( '%a' )
>> while dayOfWeek != 'Fri':
>> delta = datetime.timedelta( days = i )
>> tom = ( now + delta ).strftime( '%a' )
>> if tom != 'Fri':
>> i = i + 1
>> else:
>> print i
>> print tom
>> break
>>
>> So with this code I can determine the number of days until the next Friday
>> (if it's not Friday already).
> 
> This could be simpler. I would write
> nextFriday = datetime.datetime(now.year, now.month, now.day, 1, 30, 0)
>  while nextFriday.weekday() != 4:
>   nextFriday += datetime.timedelta(days=1)
> 
> Note the use of datetime attributes instead of relying on strftime().
> 
> What do you want the answer to be if you run the script at 1am Friday?
> at 2am Friday? If you want the next Friday in both cases, you could
> write this as
> nextFriday = datetime.datetime(now.year, now.month, now.day, 1, 30, 0)
> nextFriday += datetime.timedelta(days=1) # Make sure to get a Friday
> in the future
>  while nextFriday.weekday() != 4:
>   nextFriday += datetime.timedelta(days=1)

I don't believe you'll get much better than that, IMHO.

But, dateutil (3rd party) probably deserves a mention, at least, for
this kind of job. I'm pretty sure the dateutil equivalent of Kent's
second approach would look something like this, but I haven't thoroughly
tested it, YMMV ...

from datetime import datetime
from dateutil.relativedelta import relativedelta, FR

now = datetime.now()
delta = relativedelta(
days=1, weekday=FR, hour=1,
minute=30, second=0, microsecond=0
)
nextFriday = now + delta

print nextFriday

...

dateutil totally rocks, and I hope someday soon it will be included in
the standard library.

http://labix.org/python-dateutil

HTH,
Marty





___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] sqlite3 format data from tables

[Martin Walsh]

David wrote:
> David wrote:
>> I have a budget program I am using to learn from.
>> http://linuxcrazy.pastebin.com/f3b301daf
>>
>> I can not figure out how to get the transaction details to return so
>> that it looks nice. It returns like this now.
>>
>> Your transaction History is: [(1, u'Food', -100), (2, u'Deposit',
>> -200), (3, u'Deposit', 500), (4, u'Python Book', -50)]
>>
>> Here is how it gets to that point;
>>
>> def get_transactions(self):
>> ct=self.connection.cursor()
>> ct.execute("select * from transact;")
>> return ct.fetchall()
>>
>> def report_transactions(self):
>> return self.database.get_transactions()
>>
>> def display_transactions(self):
>> print "Your transaction History is:",self.bl.report_transactions()
>>
>> self.menu[5]=("Transaction History",self.display_transactions)
>>
>> Can I use the numbers 1,2,3,4 as a way to return the history?
>>
>> Thanks, I hope I explained it correctly :)
>> -david
> Ok this seems to work OK;
> def display_transactions(self):
> transactions = self.bl.report_transactions()
> for data in transactions:
> print "Transaction History: ", data[-2], data[-1]
> 
> 


It would be helpful if you showed the output you expect. Here is another
option ...

def display_transactions(self):
transactions = self.bl.report_transactions()
print "Your transaction History is:"
for n, k, v in transactions:
print ' %-15s %10.2f' % (k, v)

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How o convert spaces into tabs??

[Martin Walsh]

vince spicer wrote:
> 
> regex will do it
> 
> 
> import re
> 
> line = re.sub(r"\s+", "\t", line)
> 
> print line

The above replaces the newline, which reminds me that even seemingly
trivial uses of 're' can become not-so-trivial in a hurry.

In [1]: import re
In [2]: line = '1  2  3  4  5\n'
In [3]: re.sub('\s+', '\t', line)
Out[3]: '1\t2\t3\t4\t5\t'

Maybe this is closer to your intent, but I refuse to guarantee it ;)
Better to stick with str methods whenever possible.

In [4]: re.sub('[ ]+', '\t', line)
Out[4]: '1\t2\t3\t4\t5\n'

HTH,
Marty







___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] recursive glob -- recursive dir walk

[Martin Walsh]

spir wrote:
> Hello,
> 
> A foolow-up ;-) from previous question about glob.glob().
> 
> I need to 'glob' files recursively from a top dir (parameter). Tried to use 
> os.walk, but the structure of its return value is really unhandy for such a 
> use (strange, because it seems to me this precise use is typical). On the 
> other hand, os.path.walk seemed to meet my needs, but it is deprecated.

I often use Fredrik Lundh's implementation, when I need a recursive
'glob'. And even though it was contributed some time ago, it appears to
be 3.x compatible.

http://mail.python.org/pipermail/python-list/2001-February/069987.html

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help..Concatenaton Error

[Martin Walsh]

ayyaz wrote:
> Randy Trahan wrote:
>> Attached is an error I cannot get to work, I was doing a print
>> concatenation but it won't let me get past "+ "ibly impressive. " \
>> (then to next line)
>>  
>> Also Programming Lanquage Question:
>> I have studied and an fairly proficient at XHTML and CSS, I tried
>> Javascript but just didn't like it for some reason..so I am trying
>> Python which so far fits my personality, needs, whatever that part is
>> that makes you choose a lanquage. Will I be able to use Python in web
>> pages as I would of used Javascript?  From what I have read there are
>> Python to Javascript converters?...

The only python to javascript project I am aware of is pyjamas
(http://pyjs.org/), which is a python port of the Google Web Toolkit.
Not sure if that's exactly what you're looking for. There are several
javascript libraries available now-a-days that help make programming
javascript more like programming in python. Mochikit comes to mind
(http://www.mochikit.com/). But, I'm not a web developer.

>>  
>> Thanks in advance..
>> -- 
>> Randy Trahan
>> Owner, BullDog Computer Services, LLC
>> 478-396-2516
>
> Hello Randy,
> 
> You have two plus signs after "+ "ibly impressive. " \.
> 
> That might be the problem.

And, it appears you've missed the quote before "\nThis string ", as well.

Perhaps also important to note, the line continuation character '\' may
not be followed by any character on the same line, including whitespace,
so ...

print "\nThis string " + \
"may not"

... where  indicates the presence of ... er, well ... a
whitespace character, would be invalid (a SyntaxError).

PEP8 (http://www.python.org/dev/peps/pep-0008/) outlines some
alternatives to the line continuation character. For example,

 >>> s = ("This is a " +
 ... "long line" +
 ... ".")
 >>> s
 'This is a long line.'

Or, you can leave out the +, as string concatenation is implicit in the
following ...

 >>> s = ("This is a "
 ... "long line"
 ... ".")
 >>> s
 'This is a long line.'

... but, beware -- it's easy to overlook when constructing a list (or
tuple).


 >>> lst = ['This is a',
 ... 'long' # <- missing comma
 ... 'line',
 ... '.']
 >>> lst
 ['This is a', 'longline', '.']

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to change the working directory in IDLE

[Martin Walsh]

Elisha Rosensweig wrote:
> Hi Tutors,
> 
> I"m using Python 2.6.2 and the IDLE tool (also v. 2.6.2). However, when
> I open the editor I cannot seem to change the directory so as to allow
> for easy access to my modules. So, for example, the following occurs:
> 

> os.chdir('/Users/elisha/Documents/workspace/CacheNetFramework/src/Tests')
 os.getcwd()
> '/Users/elisha/Documents/workspace/CacheNetFramework/src/Tests'
 import CSITest
> 
> Traceback (most recent call last):
>   File "", line 1, in 
> import CSITest
> ImportError: No module named CSITest
> 
> 
> What am I doing wrong?

http://docs.python.org/tutorial/modules.html#the-module-search-path

You probably want to append to sys.path in your script, or to the
PYTHONPATH environment variable, instead of using os.chdir.

import sys
sys.path.append('/Users/elisha/Documents/workspace/CacheNetFramework/src/Tests')

import CSITest

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to change the working directory in IDLE

[Martin Walsh]

Martin Walsh wrote:
> Elisha Rosensweig wrote:
>> Hi Tutors,
>>
>> I"m using Python 2.6.2 and the IDLE tool (also v. 2.6.2). However, when
>> I open the editor I cannot seem to change the directory so as to allow
>> for easy access to my modules. So, for example, the following occurs:
>>
>> os.chdir('/Users/elisha/Documents/workspace/CacheNetFramework/src/Tests')
>>>>> os.getcwd()
>> '/Users/elisha/Documents/workspace/CacheNetFramework/src/Tests'
>>>>> import CSITest
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> import CSITest
>> ImportError: No module named CSITest
>>
>>
>> What am I doing wrong?
> 
> http://docs.python.org/tutorial/modules.html#the-module-search-path
> 
> You probably want to append to sys.path in your script, or to the
> PYTHONPATH environment variable, instead of using os.chdir.

Sorry, 'in your script' should read something like 'in Idle'. Here's
another doc reference.

http://docs.python.org/tutorial/modules.html#tut-standardmodules

> 
> import sys
> sys.path.append('/Users/elisha/Documents/workspace/CacheNetFramework/src/Tests')
> 
> import CSITest
> 
> HTH,
> Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Configuaration files and paths?

[Martin Walsh]

Allen Fowler wrote:
> 
> 
> 
>>> What is the recommended way to configure my application find the various 
>> database and/or configuration files it needs?
>>
>> Recommemded by whom? A lot depends on the OS. Apple for example have one set 
>> of 
>> recommendations for MacOS, Windows has another and Linux has several to 
>> choose 
>> from!
>>
> 
> 
> Thank you good point.
> 
> Planning on using a ConfigParser based .ini file located in the ./config 
> folder.
> 
>>> For instance my folder layout:
>>>
>>> /path_to_app/app.py
>>> /path_to_app/lib/
>>> /path_to_app/database/
>>> /path_to_app/config/
>>> /path_to_app/photos
>>>
>>>  and so on.  I would like to make an .ini in the config folder 
>> Seems fair enough, however on a Unix system you should also consider 
>> allowing 
>> the user to have their own personalised version in their home directory. 
>> Thus at 
>> startup you get the current user ID / home directory and look for a suitable 
>> config file. If it exists read it, if not read the default one in your 
>> config 
>> directory.
>>
>>> 1) How does my main app file find the config file in the first place?
>> Generally use a relative path so normally your app will run from its home 
>> folder 
>> so you can look in ./config. You might also set a system environment 
>> variable - 
>> for example the CLASSPATH or PYTHONPATH variables, or the ORACLE_HOME used 
>> by 
>> Oracle for their database. If the environment var is not set then look in 
>> ./config
>>
> 
> Assuming the application could be invoked in odd ways that may alter the 
> notion of the current working directory, how do I unambiguously find the 
> absolute path to the current python source file?  (So I can load the nearby 
> .ini)

I use a helper function that calculates the absolute path of my app
script, then I am able to os.path.join relative elements to my heart's
content. I haven't had a problem with this approach so far, which of
course doesn't mean it's the right way, or even a correct implementation
for that matter.

Something like this ...

# lib/mypaths.py
# --

import os

def script_path(base):
return os.path.realpath(os.path.abspath(base))

def script_dir(base):
return os.path.dirname(script_path(base))

def join_relative(base, path):
return os.path.join(script_dir(base), path)

# app.py
# --

...

from lib.mypaths import join_relative
print join_relative(__file__, 'config/prefs.ini')

# this may work when calling from
# other modules, I suppose (untested)

import sys
print join_relative(sys.argv[0], 'config/prefs.ini')

...


HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Configuaration files and paths?

[Martin Walsh]

Allen Fowler wrote:
> 

> 
> As a follow-up question, how do give my modules stored under ./lib access to 
> the data in my ConfigParser object?  (For instance, database connection 
> string, storage path, etc.)
> 
> I guess a global ConfigParser object would work, but that seems wrong.
> 

And yet, to me it seems wrong to have more than one instance of config
data floating around. Instead of using a global you can pass the config
object, or just the appropriate attributes, around to your lib
functions/classes, as necessary -- and keep the flow in your main
script. For example (highly speculative, FWIW),

# app.py
from database.connection import getcursor
from lib.persist import Storage

def main():
confp = ConfigParser()
confp.read(join_relative(__file__, 'config/prefs.ini'))
config = confp.defaults()

curs = getcursor(config[db_connection_string])

...

storage = Storage(config[storage_path])

...

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Configuaration files and paths?

[Martin Walsh]

Allen Fowler wrote:
> 
> 
> 
>> Something like this ...
>>
>> # lib/mypaths.py
>> # --
>>
>> import os
>>
>> def script_path(base):
>> return os.path.realpath(os.path.abspath(base))
>>
>> def script_dir(base):
>> return os.path.dirname(script_path(base))
>>
>> def join_relative(base, path):
>> return os.path.join(script_dir(base), path)
>>
>> # app.py
>> # --
>>
>> ...
>>
>> from lib.mypaths import join_relative
>> print join_relative(__file__, 'config/prefs.ini')
>>
>> # this may work when calling from
>> # other modules, I suppose (untested)
>>
>> import sys
>> print join_relative(sys.argv[0], 'config/prefs.ini')
>>
>> ...
> 
> 
> FWIW:
> 
> When using relative paths I got extra ../../ terms, so I changed  
> join_relative() to:
> 
> def join_relative(base, path):
> return os.path.normpath(os.path.join(script_dir(base), path))
> 
> 
> Seems to work...


Yeah, good catch ... looks great, and thanks for sharing your mod.

Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Configuaration files and paths?

[Martin Walsh]

Allen Fowler wrote:
> 
> 
>> Martin Walsh 
>>
> 
>> Allen Fowler wrote:
>>> As a follow-up question, how do give my modules stored under ./lib access 
>>> to 
>> the data in my ConfigParser object?  (For instance, database connection 
>> string, 
>> storage path, etc.)
>>> I guess a global ConfigParser object would work, but that seems wrong.
>>>
>> And yet, to me it seems wrong to have more than one instance of config
>> data floating around. Instead of using a global you can pass the config
>> object, or just the appropriate attributes, around to your lib
>> functions/classes, as necessary -- and keep the flow in your main
>> script. For example (highly speculative, FWIW),
>>
>> # app.py
>> from database.connection import getcursor
>> from lib.persist import Storage
>>
>> def main():
>> confp = ConfigParser()
>> confp.read(join_relative(__file__, 'config/prefs.ini'))
>> config = confp.defaults()
>>
>> curs = getcursor(config[db_connection_string])
>>
>> ...
>>
>> storage = Storage(config[storage_path])
>>
>>
> 
> 
> I hear your point.  It makes sense.
> 
> 
> The above sample code seems like the way to go... if I can structure my 
> modules like that.
> 
> For an object that needs many settings, what about passing in an instance of 
> ConfigParser?  (Or should I extract the settings to a dict, first?)
> 
> Thank you again,
> :)
> 

I don't see a problem with either of those approaches -- I suppose it
depends a little on the kind of config data you're working with, the
overall design of your app, which approach would be easiest to maintain.

If your intent is to package software that the general public will use,
then I believe Alan and Dave have correctly advised to follow the
packaging conventions for your target platform(s), and to be consistent
with user expectations (ie. how do popular apps typically handle config?)

I tend to write a lot of system automation scripts (sysadmin-style),
where I'm one of the few internal end users, and my toolbox module
contains wrappers for the various sources of config data (configparser,
optparse, os.environ, etc) each providing a common interface that I like
to use (and remember easily!) for dealing with config data in new
function and class definitions. But I use these a lot, and wouldn't
necessarily recommend the approach if you only need to consume config
data now and again. I think the point I may be trying to make is that it
took me a while to settle, after much trial and error, to find the
approach that works best for me -- others will have a different story to
tell, and I hope they do! :)

Good luck!

Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Configuaration files and paths?

[Martin Walsh]

Allen Fowler wrote:
>>> FWIW:
>>>
>>> When using relative paths I got extra ../../ terms, so I changed  
>> join_relative() to:
>>> def join_relative(base, path):
>>> return os.path.normpath(os.path.join(script_dir(base), path))
>>>
>>>
>>> Seems to work...
>>
>> Yeah, good catch ... looks great, and thanks for sharing your mod.
>>
> 
> 
> Glad I could return the favor.  :)
> 
> Since then, I took it a bit further for use in my code...  I am not finished 
> with it yet, but am curios what you think.  (I am not sure named things right 
> or structured it logically)
> 
> 
> ###
> 
> import os
> class HomePath(object):
> """For finding paths based on a home/install directory.
> 
> """
>  
> def __init__(self, home_dir = None, home_file = None):
> """Must be called with either a path to a directory or, as a 
> shortcut, a file in that directory.
> 
> """
> 
> if home_file != None:
> # Set home based on a path to a file in its directory
> self.home = os.path.normpath(self.fix_path(home_file))
> 
> elif home_dir != None:
> # Set home based on its path
> self.home = os.path.normpath(self.get_dir(home_dir))
> 
> else:
> raise Exception("Must call with either a path to a directory 
> or, as a shortcut, a file in that directory.")
> 
> def abs(self, rel_from_home):
> """Return an absolute path when passed a path relative to home.
> 
> """
> 
> return self.join_relative(self.home, rel_from_home)
> 
> def fix_path(self, base):
> return os.path.realpath(os.path.abspath(base))
> 
> def get_dir(self, base):
> return os.path.dirname(self.fix_path(base))
> 
> def join_relative(self, base, path):
> return os.path.normpath(self.fix_path(os.path.join(self.home, 
> path)))
> #

I'm not sure I understand your end goal, given the args to __init__. I'd
guess, perhaps incorrectly, that you may be trying to shoehorn
conflicting purposes.

If your only aim is to calculate an absolute path to a resource's
parent, or home, folder then your class can be simplified a lot, IMHO.
Specifically, I don't see a reason to differentiate between home_dir and
home_file. Please feel free to clarify if I'm missing the point.

import os

class HomePath(object):
def __init__(self, path):
# fix_path and get_dir on base path
self.home = os.path.dirname(
os.path.realpath(
os.path.abspath(path)
)
)

def join(self, relative_path):
return os.path.normpath(os.path.join(self.home, relative_path))


Also, have a look at
http://docs.python.org/library/os.path.html#os.path.abspath

... the main point is that your initial path element must have enough
information in it to construct an abspath, otherwise os.getcwd() can
pollute the calculation. I believe if you use __file__ you're good, and
sys.argv[0], while potentially relative in nature should always be
relative to os.getcwd() AFAICT, so you're good there too. Otherwise
avoid constructing a HomePath object with a relative start path.

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Confusion with $PATH

[Martin Walsh]

Wayne wrote:
> On Fri, Sep 25, 2009 at 11:32 AM, David Eric  > wrote:
> 
> ok this is really weird . . . 
> i tried running the helloworld again, and it doesnt work :/
> 
> DTm:~ davidteboul$ echo $PATH
> PATH$/Library/Frameworks/
> 
> Python.framework/Versions/3.1/bin:/Library/Frameworks/Python.framework/Versions/3.0/bin:/Library/Frameworks/Python.framework/Versions/Current/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin/Users/davidteboul/bin/pythonStuff

Also, it looks like you missed a colon, to separate the newly added path
from the last one in the list.

{...}/sbin:/usr/local/bin:/usr/X11/bin:/Users/davidteboul/bin/pythonStuff
  ^

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Not workin!

[Martin Walsh]

Luke Paireepinart wrote:
> In this case you are saying "is their input equal to this list with many
> elements?" and the answer is always going to be No because a string
> won't be equal to a list unless both are empty.

I know you probably didn't mean this as it reads, or as I'm reading it,
but an empty string and an empty list aren't 'equal' either.

In [1]: '' == []
Out[1]: False

HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Iterable Understanding

[Martin Walsh]

Stephen Nelson-Smith wrote:
> I think I'm having a major understanding failure.

Perhaps this will help ...
http://www.learningpython.com/2009/02/23/iterators-iterables-and-generators-oh-my/


> So in essence this:
> 
> logs = [ LogFile( "/home/stephen/qa/ded1353/quick_log.gz", "04/Nov/2009" ),
>  LogFile( "/home/stephen/qa/ded1408/quick_log.gz", "04/Nov/2009" ),
>  LogFile( "/home/stephen/qa/ded1409/quick_log.gz", "04/Nov/2009" ) ]
> 
> Gives me a list of LogFiles - each of which has a getline() method,
> which returns a tuple.
> 
> I thought I could merge iterables using Kent's recipe, or just with
> heapq.merge()

But, at this point are your LogFile instances even iterable? AFAICT, the
answer is no, and I think you should want them to be in order to use
heapq.merge. Have a look at the documentation
(http://docs.python.org/library/stdtypes.html#iterator-types) and then
re-read Kent's advice, in your previous thread ('Logfile multiplexing'),
about "using the iterator protocol" (__iter__).

And, judging by the heapq docs
(http://docs.python.org/library/heapq.html#heapq.merge) ...

"""
Merge multiple sorted inputs into a single sorted output (for example,
merge timestamped entries from multiple log files). Returns an iterator
over the sorted values.
"""

... using heapq.merge appears to be a reasonable approach.

You might also be interested to know, that while heapq.merge is(was) new
in 2.6, it's implementation is very similar (read: nearly identical) to
the one of the cookbook recipes referenced by Kent.

It's unclear from your previous posts (to me at least) -- are the
individual log files already sorted, in chronological order? I'd imagine
they are, being log files. But, let's say you were to run your
hypothetical merge script against only one file -- would the output to
be identical to the input? If not, then you'll want to sort the inputs
first.

> 
> But how do I get from a method that can produce a tuple, to some
> mergable iterables?
> 

I'm going to re-word this question slightly to "How can I modify the
LogFile class, for instances to be usable by heapq.merge?" and make an
attempt to answer. The following borrows heavily from Kent's iterator
example, but removes your additional line filtering (if
self.stamp.startswith(date), etc) to, hopefully, make it clearer.

import time, gzip, heapq

def timestamp(line):
# replace with your own timestamp function
# this appears to work with the sample logs I chose
stamp = ' '.join(line.split(' ', 3)[:-1])
return time.strptime(stamp, '%b %d %H:%M:%S')

class LogFile(object):
def __init__(self, filename):
self.logfile = gzip.open(filename, 'r')

def __iter__(self):
for logline in self.logfile:
yield (timestamp(logline), logline)

logs = [
LogFile("/home/stephen/qa/ded1353/quick_log.gz"),
LogFile("/home/stephen/qa/ded1408/quick_log.gz"),
LogFile("/home/stephen/qa/ded1409/quick_log.gz")
]

merged = heapq.merge(*logs)
with open('/tmp/merged_log', 'w') as output:
for stamp, line in merged:
output.write(line)

Will it be fast enough? I have no clue.

Good luck!
Marty



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Iterable Understanding

[Martin Walsh]

Stephen Nelson-Smith wrote:
>> It's unclear from your previous posts (to me at least) -- are the
>> individual log files already sorted, in chronological order?
> 
> Sorry if I didn't make this clear.  No they're not.  They are *nearly*
> sorted - ie they're out by a few seconds, every so often, but they are
> in order at the level of minutes, or even in the order of a few
> seconds.
> 
> It was precisely because of this that I decided, following Alan's
> advice, to pre-filter the data.  I compiled a unix sort command to do
> this, and had a solution I was happy with, based on Kent's iterator
> example, fed into heapq.merge.
> 
> However, I've since discovered that the unix sort isn't reliable on
> the last and first day of the month.  So, I decided I'd need to sort
> each logfile first.  The code at the start of *this* thread does this
> - it uses a heapq per logfile and is able to produce a tuple of
> timestamp, logline, which will be in exact chronological order.  What
> I want to do is merge this output into a file.

Well, you haven't described the unreliable behavior of unix sort so I
can only guess, but I assume you know about the --month-sort (-M) flag?

I did misunderstand your intent for this thread, so thanks for
clarifying. The fact remains that if you are interested in using
heapq.merge, then you need to pass it iterable objects. And, I don't see
any reason to avoid adapting your approach to fit heapq.merge. How about
something like the following (completely untested) ...

import gzip
from heapq import heappush, heappop, merge

def timestamp(line):
# replace with your own timestamp function
# this appears to work with the sample logs I chose
stamp = ' '.join(line.split(' ', 3)[:-1])
return time.strptime(stamp, '%b %d %H:%M:%S')

class LogFile(object):
def __init__(self, filename, jitter=10):
self.logfile = gzip.open(filename, 'r')
self.heap = []
self.jitter = jitter

def __iter__(self):
while True:
for logline in self.logfile:
heappush(self.heap, (timestamp(logline), logline))
if len(self.heap) >= self.jitter:
break
try:
yield heappop(self.heap)
except IndexError:
raise StopIteration

logs = [
LogFile("/home/stephen/qa/ded1353/quick_log.gz"),
LogFile("/home/stephen/qa/ded1408/quick_log.gz"),
LogFile("/home/stephen/qa/ded1409/quick_log.gz")
]

merged_log = merge(*logs)
with open('/tmp/merged_log', 'w') as output:
for stamp, line in merged_log:
output.write(line)


... which probably won't preserve the order of log entries that have the
same timestamp, but if you need it to -- should be easy to accommodate.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Iterable Understanding

[Martin Walsh]

Stephen Nelson-Smith wrote:
> Nope - but I can look it up.  The problem I have is that the source
> logs are rotated at 0400 hrs, so I need two days of logs in order to
> extract 24 hrs from  to 2359 (which is the requirement).  At
> present, I preprocess using sort, which works fine as long as the
> month doesn't change.

Still not sure without more detail, but IIRC from your previous posts,
your log entry timestamps are formatted with the abbreviated month name
instead of month number. Without the -M flag, the sort command will ...
well, erm ... sort the month names alphabetically. With the -M
(--month-sort) flag, they are sorted chronologically.

Just a guess, of course. I suppose this is drifting a bit off topic, in
any case, but it may still serve to demonstrate the importance of
converting your string based timestamps into something that can be
sorted accurately by your python code -- the most obvious being time or
datetime objects, IMHO.


>> class LogFile(object):
>>def __init__(self, filename, jitter=10):
>>self.logfile = gzip.open(filename, 'r')
>>self.heap = []
>>self.jitter = jitter
>>
>>def __iter__(self):
>>while True:
>>for logline in self.logfile:
>>heappush(self.heap, (timestamp(logline), logline))
>>if len(self.heap) >= self.jitter:
>>break
> 
> Really nice way to handle the batching of the initial heap - thank you!
> 
>>try:
>>yield heappop(self.heap)
>>except IndexError:
>>raise StopIteration

>> ... which probably won't preserve the order of log entries that have the
>> same timestamp, but if you need it to -- should be easy to accommodate.
> 
> I don't think  that is necessary, but I'm curious to know how...

I'd imagine something like this might work ...

class LogFile(object):
def __init__(self, filename, jitter=10):
self.logfile = open(filename, 'r')
self.heap = []
self.jitter = jitter

def __iter__(self):
line_count = 0
while True:
for logline in self.logfile:
line_count += 1
heappush(self.heap,
   ((timestamp(logline), line_count), logline))
if len(self.heap) >= self.jitter:
break
try:
yield heappop(self.heap)
except IndexError:
raise StopIteration

The key concept is to pass additional unique data to heappush, something
related to the order of lines from input. So, you could probably do
something with file.tell() also. But beware, it seems you can't reliably
tell() a file object opened in 'r' mode, used as an iterator[1] -- and
in python 3.x attempting to do so raises an IOError.

[1] http://mail.python.org/pipermail/python-list/2008-November/156865.html

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Iterable Understanding

[Martin Walsh]

Stephen Nelson-Smith wrote:
> Martin,
> 
>>def __iter__(self):
>>while True:
>>for logline in self.logfile:
>>heappush(self.heap, (timestamp(logline), logline))
>>if len(self.heap) >= self.jitter:
>>break
>>try:
>>yield heappop(self.heap)
>>except IndexError:
>>raise StopIteration
> 
> In this __iter__ method, why are we wrapping a for loop in a while True?
> 
> S.

Are you more interested in one of the loops over the other? Do you have
a more specific question?

The for loop is a means to populate the heap to a predefined size before
this iterator yields any values. You may have noticed that the first
time through the heap is empty so it will loop a number of times, as
defined by self.jitter, while subsequent entries into the loop will
break after only one execution. Perhaps not the most efficient approach
-- not sure either way, you'll have to test.

The while True loop is where values are yielded, until the heap is empty
again (heappop raises an IndexError) -- a fairly common pattern for
generators in general, I think.

The way I see it (not a programmer by trade or training, FWIW), both
loops have something like a do-until style.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Example of use of (?P) and (?P=name) in Python regular expressions?

[Martin Walsh]

Michael Hannon wrote:
> Greetings.  While looking into the use of regular expressions in Python, I 
> saw that it's possible to name match groups using:
> 
> (?P...)
> 
> and then refer to them using:
> 
> (?P=name)

I'm not sure you've got that quite right. IIUC, the (?P=name) syntax is
used to match a previously defined group, "in the regular expression
itself." (http://docs.python.org/library/re.html)



x = 'Free Fri Fro Fro From'
y = re.sub(
r'(?PFro) (?P=test)',
r'Frodo (--matched from \g, twice in a row)', x
)
# y == 'Free Fri Frodo (--matched from Fro, twice in a row) From'

> But, as you can see, to refer to the match I used the "\g" notation (that I 
> found some place on the web).

The \g notation is appropriate for re.sub.

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Attaching an uploaded file to an email

[Martin Walsh]

Tim Wilson wrote:
Hi everyone,
 

Hi Tim,
I'm a newb, first time posting, so please take any of the following 
advice at face value

# Collect form information
form = cgi.FieldStorage()
requestername = form["requestername"].value
fromaddr = form["email"].value
itemname = form["itemname"].value
description = form["description"].value
buildings = form.getlist("building")
room = form["room"].value
dateneeded = form["dateneeded"].value
po = form["po"].value
budgetcode = form["budgetcode"].value
attachment = form["attachment"].value
 

based on this cookbook recipe 
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/273844, it looks 
like cgi.FieldStorage() returns a file-like object for file input 
fields, having  .filename and .file attributes. the file attribute has a 
read() method which may be useful. having never used cgi I'm not sure 
what .value returns for file input fields, don't know if this is of any 
consequence.

buildinglist = ", ".join(buildings)
**[ misc code snipped ]**
# Set some email headers
#msg = MIMEText(msgtext)
msg = MIMEMultipart()
msg['Subject'] = itemname
msg['From'] = "%s <%s>" % (requestername, fromaddr)
msg['To'] = toaddr
if len(buildings) != 0:
   for building in buildings:
   msg['X-HRT-Building'] = building
if po != "": msg['X-HRT-PO'] = po
if dateneeded != "":
   try:
   duedate = time.asctime(time.strptime(dateneeded, "%m/%d/%Y"))
   msg['X-HRT-Due-Date'] = duedate
   except ValueError:
   pass
msg.preamble = "Tech order request"
msg.epilogue = ""
 

if you know that the attachment will always be a text file and your 
assignment of 'attachment' looks like this:

   attachment = form["attachment"]
then you might try the following (untested):
   part = MIMEText(attachment.file.read())
   # if I understand correctly, the 'Content-Disposition' header is 
necessary to make the file
   # appear in the message as an attachment, otherwise it may occupy 
the msg body.
   part.add_header('Content-Disposition', 'attachment', 
filename=attachment.filename)
   msg.attach(part)

# Send the message
server = smtplib.SMTP('localhost')
server.sendmail(fromaddr, toaddr, msg.as_string(0))
server.quit()
 

there's a great example in the email module docs if you're dealing with 
more than just text files :
http://docs.python.org/lib/node578.html (3rd example, using the 
mimetypes module)

I have collected code snippets from various sources (python docs, ASPNs 
python cookbook) into a basic MIMEMailer class, that I use fairly 
regularly in hobby projects. If anyone is interested, I'd be happy to 
share, or post it here.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help debuging a small program

[Martin Walsh]

Mark Kels wrote:
On Mon, 21 Feb 2005 13:21:35 -0500, Kent Johnson <[EMAIL PROTECTED]> wrote:
 

How far does it get? How do you know?
I would put some debug print statements in. Also you should call sk.close() in the else clause of
scan(). Finally, I don't think you will see any output in the result window until get_info()
returns; you have to give some time to mainloop() for it to be able to trigger the updates.
   

I don't understand how to prevent the program from crashing right now...
I'll deal with other bugs when I'll finish this one :)
So, why does the program crashes and how can I prevent it ??
Thanks !!
 

Hi Mark,
I believe Kent has given you the solution, it's perhaps just not what 
you expected it to be. Adding debug print statements might have helped 
you to see. To demonstrate further try scanning only 1 or 2 ports.

When your app appears to be frozen or crashing, I suspect it is happily 
scanning away. Looks like you have an overly ambitious while loop in the 
scan function, which isn't giving the Tk Text widget (result_t?) time to 
update.

Someone more experienced that I will certainly have a better way, but 
you might try adding root.update() at the end (but inside) the while 
loop to force an update.

# + ---The Scan---
def scan(host,start_port,end_port):
   sk=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   while start_port<=end_port:
   try:
   sk.connect((host,start_port))
   except:
   result_t.insert(END,"Port",start_port,"is CLOSED on",host,"\n")
   else:
   result_t.insert(END,"Port",start_port,"is OPENED on",host,"\n")
   start_port=start_port+1
   root.update() #<<-- add this
and have a look here 
http://www.pythonware.com/library/tkinter/introduction/x9374-event-processing.htm

>>> import Tkinter
>>> help(Tkinter.Tk)
Help on class Tk in module Tkinter:
...
|
|  update(self)
|  Enter event loop until all pending events have been processed by 
Tcl.
|
|  update_idletasks(self)
|  Enter event loop until all idle callbacks have been called. This
|  will update the display of windows but not process events caused by
|  the user.
...
HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] 3 questions for my port scanner project

[Martin Walsh]

Mark Kels wrote:
Hi list.
 

Hi Mark,
2. I got a while loop which does the port scan itself. How can I end
it while its working ?
 

previous message with code: *http://tinyurl.com/3lobo*
**I'm not totally sure of this, but you might try adding another 
conditional to your while loop that can be toggled by a Scan and Stop 
button in your Tkinter app.

global ok_to_scan
ok_to_scan = 1 # or True
while (start_port <= end_port) and ok_to_scan:
   # ... do scan ...
   root.update()
then bind a function to a Stop button which toggles 'ok_to_scan' to 0 
(or False), also as previously suggested you should close the sockets 
that find an open port (sk.close()) inside your while loop.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] variable name based on variables (expansion?)

[Martin Walsh]

John Christian wrote:
# But I want the assignment
# to be based on variables
LIST=1
POSITION=2
GameLogic.varList$LIST[$POSITION]=0
 

>>> help(getattr)
Help on built-in function getattr:
getattr(...)
   getattr(object, name[, default]) -> value
   Get a named attribute from an object; getattr(x, 'y') is equivalent 
to x.y.
   When a default argument is given, it is returned when the attribute 
doesn't
   exist; without it, an exception is raised in that case.

from http://docs.python.org/lib/built-in-funcs.html
*getattr*(  object, name[, default])
   Return the value of the named attributed of object. name must be a
   string. If the string is the name of one of the object's attributes,
   the result is the value of that attribute. For example, |getattr(x,
   'foobar')| is equivalent to |x.foobar|. If the named attribute does
   not exist, default is returned if provided, otherwise AttributeError
   is raised. 

#---
LIST = 1
POSITION = 2
getattr(GameLogic, 'varList'+str(LIST))[POSITION] = 0

__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python newbie..system call help

[Martin Walsh]

Mike Pindzola wrote:
> I have figured many things out. system works, i just forgot to type 
> os.system(). I have been looking into the os module and am finding alot 
> of useful stuff. I still need to workout the best way to ask a user for 
> a root password, show  when typed and then pass it to the system...

Since you're using linux -- you might also try the pexpect module found 
here:

http://pexpect.sourceforge.net

I've found it very useful for interacting with the shell. I don't 
believe a windows version is available, however.

Here's a very basic and untested example:

import pexpect # you'll have to install pexpect first, of course
import getpass

pexpect.spawn('sudo -k') # timeout any existing sudo session
child = pexpect.spawn('sudo mount /mnt/foo')
child.expect('Password:')
child.sendline(getpass.getpass())
mount_results = child.read()
print mount_results

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Send attachment

[Martin Walsh]

Jorge Louis De Castro wrote:
> Hello,
>  
> Any ideas how I can use Python and the Windows API to open a PC's mail 
> client and send an attachment?
> The idea is saving some data onto a file and then invoke the email 
> client (OE or Outlook or whatever is the default on the machine) with 
> the recipient's address filled in and the file ready to be sent as an 
> attachment.

Not sure if it would work for your particular situation/environment, but 
you might try to write out a mime encoded text file with an .eml 
extension, and then use...

import os
os.startfile('foo.eml') # I think os.startfile is windows only


> Kinda like when we right-click on a file and select the "send to mail 
> recipient" behavior.
> Been reading docs, trying out samples, and googling this for a few days 
> to no avail.
>  
> Cheers
> jorge

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Quickest way to find a folder on windows

[Martin Walsh]

Jorge Louis De Castro wrote:
> Hi all,
>  
> What is the quickest way to find/search a folder on windows? 
> Specifically, I want to find the 'My Documents' folder on different 
> machines but since this one lies in different user-specific folders (the 
> user account on XP) I'm being forced to traverse the whole drive looking 
> for it (though I can start from 'Documents and Settings' to prune the 
> search space)

While this doesn't go directly toward answering your fast search a 
folder question, it might get you closer for finding special folders -- 
and others on the list will certainly have a more concise solution

take a look at os.path.expanduser() and os.environ, which are available 
cross-platform

.>>> # on my setup (non-standard %HOME%), this works
.>>> os.path.join(os.path.expanduser('~'), 'My Documents')
'H:\\My Documents'

.>>> os.path.join(os.environ['USERPROFILE'], 'My Documents')
'C:\\Documents and Settings\\martin walsh\\My Documents'

as you can see, because of my non-standard setup, I get 2 different results

in this instance, Mark Hammond's win32 extensions can possibly help. Tim 
Golden has written the winshell module, uses win32 -- and includes a 
helper for "finding" the 'My Documents' folder -- it's available here:

http://tgolden.sc.sabren.com/python/winshell.html

 >>> import winshell # must have pywin32 & winshell installed
 >>> winshell.my_documents()
'H:\\my documents'


HTH,
Marty

>  
> Is there a faster way? I know this is very reliant on the windows search 
> API so maybe I'm stuck with my solution
> Any help is appreciated
>  
> cheers
> j
> 
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Help with pysvn

[Martin Walsh]

Hi Nam,

Pavar, Namratha wrote:
> Hi,
> 
>  
> 
> I am getting the error "pysvn._pysvn.ClientError: callback_get_login
> required" when I try to do a simple exercise using pysvn. I have made
> sure the subversion config file doesn't have the "store-auth-creds = no"
> .
> 
> I would appreciate any help. 
> 
> Following is the code and error message. 
> 
 import pysvn
> 
 client = pysvn.Client()
> 
> client.checkout('http://arcsource.nam.coair.com/arc/svn/devref/trunk/mak
> o/',
> 
> 'D:\cpy')
> 
> Traceback (most recent call last):
> 
>   File "", line 1, in ?
> 
> pysvn._pysvn.ClientError: callback_get_login required
> 
>  

In this case, the truth is in the traceback.

Here is the relevant part of the pysvn documentation:
http://pysvn.tigris.org/docs/pysvn_prog_ref.html#pysvn_client_callback_get_login

pysvn.Client seems to require that you declare a callback function to
provide login credentials when needed to access a repository/project, if
not previously cached (or if caching is disabled). A simple example
providing a callback function might look like this:

def login(*args):
return True, 'myuser', 'mypass', False

import pysvn
client = pysvn.Client()
client.callback_get_login = login
client.checkout('http://subversion.example.com/someproject/trunk')

But, the above might not be acceptable -- depending on security
concerns, as your repository password is presented in plain text.

HTH,
Marty

> Thanks,
> 
>  
> 
> Nam
> 
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] IDE / development environment

[Martin Walsh]

Eike Welk wrote:
> On Sunday 22 April 2007 10:51, Rohan Deshpande wrote:
>> What does everyone use as their development environment for Python,
>> particularly web dev?
> 
> I use PyDev an extension for Eclipse:
> http://pydev.sourceforge.net/

+1 for pydev. The additional plugins (wtp, aptana, and subclipse, etc)
make it particularly nice for web development -- however, I agree, it is
a bit heavy.

> 
> It is good at syntax completion but it is quite slow.
> PyDev also lacks an interactive console.

The 'pydev extensions' plugin (which is a $42.50 add on to the open
source pydev) appears to have an interactive console view. I can't say
that I've ever used it -- since you can make your own as an eclipse
external tool. I must admit it is far from a command line replacement,
but it suits my needs. YMMV.

Choose 'External Tools', from the Run >> External Tools menu, while in
the PyDev perspective (or any other where the Run menu is available).
Select 'Program' in the left-hand pane, and click the new configuration
button. Change the Name field to something appropriate
('python-interactive'), and in the location field specify the path to
the python binary (ex. '/usr/bin/python' or 'C:\Python25\python'). I use
'${container_loc}' in the working directory field, and you must specify
at least '-i' in arguments. Click apply to save, and Run to test it out.

If you want to run your module and drop into an interactive interpreter
after, you can do something very similar under the Run >> Run >> Python
Run menu item -- specifying '-i' in the VM arguments field on the
Arguments tab.

HTH,
Marty

> 
> Regards Eike.
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] screen scraping without the request

[Martin Walsh]

Hi Rohan,

You might also try the LiveHTTPHeaders firefox extension, it is also
very good for this type of reverse engineering.

http://livehttpheaders.mozdev.org/index.html

HTH,
Marty

Luke Paireepinart wrote:
> Kent Johnson wrote:
>> Rohan Deshpande wrote:
>>   
>>> Hi All,
>>>
>>> the previous thread on screen scraping got me thinking of starting a 
>>> similar project.  However, the problem is I have no idea what the POST 
>>> request is as there is no escape string after the URL when the resulting 
>>> page comes up.  I essentially need to pull the HTML from a page that is 
>>> generated on a users machine and pipe it into a python script.  How 
>>> should I go about doing this?  Is it possible/feasible to decipher the 
>>> POST request and get the HTML, or use some screen scraping python libs a 
>>> la the javascript DOM hacks? I was thinking of the possibilities of the 
>>> former, but the interaction on the site is such that the user enters a 
>>> username/password and goes through a couple links before getting to the 
>>> page I need.  Perhaps Python can use the session cookie and then pull 
>>> the right page?
>>> 
> Have you tried using Firebug?  It's an extension for Firefox.
> You might be able to run it while you're navigating the site, and see 
> the communciation between you and the server and get the POST that way,
> but I'm not completely certain about that.
> -Luke
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] + converted to 25 in http string

[Martin Walsh]

Hi,

govind goyal wrote:
> Hi,
> 
> I am using http to automate my Access point(AP) configuration where I sent
> following strings to AP server through script.
> 
> params = urllib.urlencode
> ({'WRegion':"USA",'ssid':"wifi",'ap':"ap",'ssid_enable':"ssid_enable",*
> 'wire_mode':"b+only",*
> 'w_channel':6,'lan_ac':"everyone",'int_ac':"everyone"})

You might consider having a look at the urllib quote_plus and
unquote_plus methods for clarification. I suspect the value you have
captured with ethereal is already quoted ('+' signs for spaces, etc) --
so you would have to unquote_plus it to find the correct value for use
in your script.

.>> urllib.unquote_plus('b+only')
'b only'

And, it appears that urllib.urlencode quotes the parameters while
constructing the query-string:

.>> params = urllib.urlencode({'write_mode': 'b only'})
.>> params
'write_mode=b+only'

HTH,
Marty

> 
> Above string I captured using a tool ethereal and then implementing this in
> my script.
> But the problem is that when I run this script all configurations in AP are
> OK except the parameter marked as bold in above string.
> 
> I used the same tool ethereal to see what string is actually passing to
> Access point while I run my script and I got following:
> 
> params = urllib.urlencode
> ({'WRegion':"USA",'ssid':"wifi",'ap':"ap",'ssid_enable':"ssid_enable",*
> 'wire_mode':"b25only",*
> 'w_channel':6,'lan_ac':"everyone",'int_ac':"everyone"})
> 
> In conclusion,the character "+" is getting converted into"25" whenever I
> run
> script.Thats why all other configuartions are OK except above mentioned
> case.
> 
> Can anybody help to resolve this issue?
> 
> Best Regards,
> 
> Govind Goyal
> 
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Socket question.

[Martin Walsh]

Hey Dude :)

Dude, WHOA! wrote:
> kinda thing. The problem is that the client I wrote doesn't receive
> data and display it, and it also only executes single word commands.

> Server side:
> #!/usr/bin/env python



>   try:
>   echo = Popen(command, stdout=PIPE).stdout.read()

On a linux system (and perhaps Windows as well), the type of 'command'
seems to be important and changes the behavior of Popen -- whether it be
string or sequence. If you pass a 'command' as a string that includes
arguments (ex. 'ls -l'), the above will raise an exception. I'm not sure
if the same applies to Windows. You could try passing command as a list
or tuple (ex command.split(), ['ls', '-l'], or similar), or add
'shell=True' to the Popen call.

> Client:
> #!/usr/bin/env python



>   send = raw_input('Send: ')
>   sock.send(send)
>   sock.recv(2048)

Try 'print sock.recv(2048)'.

>   sock.close()

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Continue Matching after First Match

[Martin Walsh]

Hi Tom,

Tom Tucker wrote:
 > Why the cStringIO stuff?  The input data shown below is collected from
> os.popen.  I was trying to find an easy way of matching my regex. 

Ah, ldap...

> Matching with a string seemed easier than looping through the ouput
> collected.  Hmm.  Come to think of it, I guess I could match on the
> first "^dn" catpure that output and then keep looping until "^cn:" is
> seen. Then repeat. 

Honestly, I'm not very good with regular expressions -- and try to avoid
them when possible. But in cases where they seem to be the best option,
I have formed a heavy dependence on regex debuggers like kodos.
http://kodos.sourceforge.net/

> Anyways, any suggestions to fix the below code?


Have you had a look at the python-ldap package?

http://python-ldap.sourceforge.net/

You could probably access ldap directly with python, if that's an
option. Or, you could roll your own ldif parser (but make sure your data
contains a newline between each dn, or the parser will choke with a
'ValueError: Two lines starting with dn: in one record.'):

import ldif
from cStringIO import StringIO

class MyLDIF(ldif.LDIFParser):
def __init__(self, inputfile):
ldif.LDIFParser.__init__(self, inputfile)
self.users = []

def handle(self, dn, entry):
self.users.append((entry['uid'], entry['cn']))

raw = """\

"""

if __name__ == '__main__':
io = StringIO(raw)
lp = MyLDIF(io)
lp.parse()
for user in lp.users:
uid = user[0][0]
cn = user[1][0]
print uid
print cn

... or ...

You could also use ldif.LDIFRecordList directly without creating a
custom parser class which would return a list of (dn, entry) tuples. The
module author warns that 'It can be a memory hog!', and I can imagine
this is true if you are working with a particularly large ldap directory.

io = StringIO(raw)
directory = ldif.LDIFRecordList(io)
directory.parse()
for dn, entry in directory.all_records:
print entry['uid'][0]
print entry['cn'][0]

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] question re: executing exe files with arguments

[Martin Walsh]

Hey Janani,

Janani Krishnaswamy wrote:
> Hi!
> I am having trouble executing an exe file with 3 arguments within a
> python script.  Right now I have something like this:
> 
> os.system(r'"1/2/3/program 1/2/3/argument1 1/2/3/argument2"')

Without an error message, or traceback, this is difficult to diagnose.
But, my guess is your use of quotes is causing the issue. Can you
successfully run the hypothetical command as used above (with the same
double-quote placement) in a shell? as in:

[EMAIL PROTECTED]:~$ "1/2/3/program 1/2/3/argument1 1/2/3/argument2"
... or, if you are a windows user ...
c:\> "1/2/3/program 1/2/3/argument1 1/2/3/argument2"

I think the command string would be treated as one long path, in this
case. If your paths contain whitespace or other special characters, you
probably want something like this:

"1/2/3/program" "1/2/3/argument1" "1/2/3/argument2"

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ActivePython and CPython

[Martin Walsh]

Alan Gauld wrote:
> <[EMAIL PROTECTED]> wrote 
>> Answer: PyDev for Eclipse turns Eclipse into a pretty good IDE for
>> Python. 
> 
> Agreed, I downloaded pydev over the weekend and have been 
> playing. The editor isn't exactly vim/emacs but its as good as 
> IDLE/Pythonwin
> 
> However I think the OP wants an HTML preview and I don't see that 
> facility in Eclipse anywhere.

Not with a default install, but there are a few html editor plugins with
preview mode that can be installed separately. Aptana
(http://www.aptana.com) is one such, that seems promising.

The easiest way to get aptana would be through the eclipse update
mechanism, described here:
http://www.aptana.com/docs/index.php/Plugging_Aptana_into_an_existing_Eclipse_configuration

I have not used Aptana regularly for editing html, but I really
appreciate it's code-completion for css and js. I tend to prefer the
eclipse web tools platform html editor, but I don't think any recent
release has html preview, or if it's even planned -- I'd be happy to be
proven wrong.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] MS SQL Connection

[Martin Walsh]

Kent Johnson wrote:
> Leon Keylin wrote:
>> Thanks Mike, unfortunately that's the mod I had problems with before. 
>> The one that
>> can't do truncate. I've written about it before here and it was easier 
>> to look for
>> another solution.
> 
> You could try adodbapi
> http://adodbapi.sourceforge.net/
> 
> Kent

Could be a long shot -- but you might also have a look at python-sybase.
A few years ago I used Sybase (with freetds) to access ms sql server
2000 from linux -- which I found very satisfying. However, I suspect
adodbapi is a lot easier to install on a windows system.

http://python-sybase.sourceforge.net/

Good luck!
Marty

> 
>> Is there any other way? Should I be looking for a different language to 
>> do it in?
>>
>>
>>  > -Original Message-
>>  > From: [EMAIL PROTECTED] 
>>  > [mailto: [EMAIL PROTECTED] ] 
>> On Behalf Of Leon Keylin
>>  > Sent: Monday, May 21, 2007 3:52 PM
>>  > To: tutor@python.org 
>>  > Subject: [Tutor] MS SQL Connection
>>  >
>>  > Been trying to do a very small and simple select via Python.
>>  > First I tried a module that looked clean but it couldn't
>>  > truncate tables.
>>  >
>>  > Then I started sifting through docs at python.org 
>>  and
>>  > couldn't find a simple
>>  > way of communicating with a MS SQL via Python program.
>>  >
>>  > Do I have to use ODBC?
>>  >
>>  > Any examples would be awesome. Thanks!
>>  >
>>
>> I did a quick search on Google for MS SQL Python, and this came up. It
>> looks like it will do the trick.
>>
>> http://pymssql.sourceforge.net/
>>
>> Mike
>>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] example of spawn

[Martin Walsh]

Jason Coggins wrote:
> In the example listed below, reproduced from the web page you recommended, 
> what does `cp`, `cp` and `/dev/null` do?  I am assuming `index.html` is the 
> name of the program that will be launched.
> 
> example:--
> os.spawnlp(os.P_WAIT, 'cp', 'cp', 'index.html', '/dev/null')
> 

I don't fully understand the workings of os.spawnlp myself, but it seems
clear that the author of the example was writing it for use in a
unix/linux environment. 'cp' is a program (for copying files and
folders). And the second 'cp', 'index.html' & '/dev/null' -- the
argument list. So, the example above would seem somewhat equivalent to
the following linux shell command:

  host:~$ cp index.html /dev/null

which would copy the file 'index.html' (from the current directory) to
'/dev/null'.

'/dev/null' is a special file in a *nix environment that discards any
data written to it: http://en.wikipedia.org/wiki//dev/null
I can't think of a windows parallel to /dev/null.

I may be completely wrong, but I don't beleive os.spawnlp is available
on windows. But if you are a windows user, don't despair -- the
subprocess module equivalent should work in it's absence:
http://docs.python.org/lib/node538.html

import subprocess
# replacement for os.spawnlp(os.P_WAIT, ...)
retcode = subprocess.call(['notepad.exe', 'myfile.txt'])

In general, I believe use of the subprocess module is recommended over
os.spawn*, os.popen*, etc. -- regardless of the platform.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] CGI error

[Martin Walsh]

Hi Darren,

Darren Williams wrote:
>  Hi all Smile
> 
> I have a Windows hosting account with lunarpages.com and am having a bit
> of trouble with CGI. I have enabled both CGI and Python in the control
> panel and made sure the permissions for all my CGI scripts are set to
> both readable and executable for all users but I keep getting the same
> 'The specified CGI application misbehaved by not returning a complete
> set of HTTP headers' error.

Have you tried to run the script from the command line? If the code
provided below is accurate and has not been changed by your mail client,
then it will raise an IndentationError, and the script will exit before
printing the Content-type header.

> The script that i'm trying to execute (example_7.1.py) -

>   32. # Define function display data.
>   33. def display_data(name, age):

Here is an improper indent ...

>   34. print "\n"
>   35. print "\n"
>   36. print "\tInfo Form\n"
>   37. print "\n"
>   38. print "\n"
>   39. print name, ", you are", age, "years old."
>   40. print "\n"
>   41. print "\n"
>   42.
>   43. # Define main function.
>   44. def main():

... and here ...

>   45. form = cgi.FieldStorage()
>   46. if (form.has_key("action") and form.has_key("name") \
>   47. and form.has_key("age")):
>   48.  if (form["action"].value == "display"):
>   49. display_data(form["name"].value, form["age"].value)
>   50. else:
>   51.  generate_form()
>   52.
>   53. # Call main function.
>   54. main()

... and here ...

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Getting Date/Time from a file and using it in calculations

[Martin Walsh]

[EMAIL PROTECTED] wrote:
> Hello there
> 
> Messing around with certain time and datetime objects, I have managed to 
> subtract a date/time from the present time thusly:
> 
> from time import *
> import datetime
> 
> one = datetime.datetime.now()
> two  = datetime.datetime(2007, 8, 29, 11, 15, 00)
> 
> difference = one - two
> 
> print difference
> 
> However, I have to take a date from a file and then insert it to where two 
> should be, however to no success. I have managed to get a string containing 
> the above date/time, but that is as far as I've gotten without it not 
> working.
> 
> Does anyone have any ideas on how to do this properly?  
> 
> Also I wish to display the result only in seconds left, rather than the 
> current result, which just displays days/hours/minutes/seconds/milliseconds 
> left, but am again struggling to progress.

Have a look at time.strptime (and time.mktime) -- you may be able to
drop datetime entirely. Here is my attempt (not tested):

import time

time_string = '2007/08/15 22:10:21'
one = time.time() # now
two = time.mktime(time.strptime(time_string, '%Y/%m/%d %H:%M:%S'))

# difference in seconds (float)
difference = one - two

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] CGI File Woes

[Martin Walsh]

wormwood_3 wrote:
>>> I've never used cgitb (and until now didn't know it existed!)
>>> so can't comment.
> 
> I had not heard of it until this week when I started working on CGI stuff, 
> but I have found it super handy! All you have to do it "import cgitb; 
> cgitb.enable()" and all tracebacks will get printed to nicely formatted HTML 
> output on the page you were trying to load.
> 

No doubt cgitb is a great tool for debugging cgi, but IIUC there are at
least two instances when you will not get the pretty printed tracebacks
in the browser when using cgitb. One is after, what I would call, a
'compile time' exception such as SyntaxError, in your python code. The
other is when the python code runs without exception, but you have not
separated the header and the document content with a newline. At least,
I have found these to be true when using apache-cgi.

Consider the following examples:

#!/usr/bin/env python
# raises a SyntaxError

import cgi
import cgitb; cgitb.enable()

print "Content-type: text/html\n\n"
# NOTE: purposeful misspelling of the print statement
prin "Hello, world!"

# the code above will produce a server 500, with apache
# complaining about "premature end of script headers"

...

#!/usr/bin/env python
# good python, bad data

import cgi
import cgitb; cgitb.enable()

print "Content-type: text/html"
print "Hello, world!"

# this is syntactically correct python, and will
# run from the command line, but the html header
# and html content have no separation, so apache
# will consider all of it to be header info, resulting
# in another server 500, "malformed header"

As others have advised, under these circumstances you can review the
apache error log (if your webhost allows it). Or roll-your-own logging
equivalent, and run your cgi from a terminal to catch SyntaxErrors and
the like.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] CGI File Woes

[Martin Walsh]

wormwood_3 wrote:
> Those examples were a lot of help Martin. Turned out the only issue was that 
> I did not have this line right:
> 
> print "Content-type: text/html\n\n"

Sam,

Glad the examples helped. I should probably fess up and point out that I
mistakenly added an additional newline to that print statement. You only
need one, as the print itself will append an extra. So by adding the
addition \n, a blank line will be sent to the browser as part of the
page content, which shouldn't be a problem for text/html content-types.
But as soon as you want to serve up another content-type (eg. image/png)
you may run into odd problems.

So, the corrected print statement should look like this:

print "Content-type: text/html\n"

or as described in the docs, http://docs.python.org/lib/cgi-intro.html

print "Content-type: text/html"
print

HTH,
Marty

> 
> With that form, it loaded just fine. It had been running fine from the 
> terminal, but without this line being right, Apache did not know what to do 
> with it.
> 
> I had spoken with my web-hosting provider, but since I had a shared account I 
> was unable to view the server logs. And the person helping me knew nothing 
> about Python (he kept slipping and calling it PHP actually, to my dismay and 
> chagrin:-) ).
> 
> Thanks for all the help, Alan and Martin.
> 
> -Sam
> 
> _
> - Original Message 
> From: Martin Walsh <[EMAIL PROTECTED]>
> To: tutor@python.org
> Sent: Sunday, September 30, 2007 1:07:02 PM
> Subject: Re: [Tutor] CGI File Woes
> 
> No doubt cgitb is a great tool for debugging cgi, but IIUC there are at
> least two instances when you will not get the pretty printed tracebacks
> in the browser when using cgitb. One is after, what I would call, a
> 'compile time' exception such as SyntaxError, in your python code. The
> other is when the python code runs without exception, but you have not
> separated the header and the document content with a newline. At least,
> I have found these to be true when using apache-cgi.
> 
> Consider the following examples:
> 
> #!/usr/bin/env python
> # raises a SyntaxError
> 
> import cgi
> import cgitb; cgitb.enable()
> 
> print "Content-type: text/html\n\n"
> # NOTE: purposeful misspelling of the print statement
> prin "Hello, world!"
> 
> # the code above will produce a server 500, with apache
> # complaining about "premature end of script headers"

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Really basic web templating

[Martin Walsh]

wormwood_3 wrote:
> Well yes and no:-) This sort of application would fall under the
> sprawling category of CGI, yes, and I can use Python scripts on my web
> server, so it is supported. But nearly every tutorial I have seen
> regarding Python and CGI only have to do with form submissions, doing
> calculations and other things with data sent from webpages to Python
> scripts. But that is not really what I want to do. I am wondering what a
> script would need to do to take requests for pages on my site, and
> generate them from templates. I am not sure if I can do this without
> having access to Apache rewrite rules, etc.

I am going to assume from your description that you are looking for a
way to use a *single* cgi script to handle all page requests to your
site, and respond with the appropriate template based on the requested
url. If I have assumed incorrectly then you can safely ignore the rest
of this message. :)

Some mod_rewrite directives are available to .htaccess, IIRC -- so that
may be a valid option, only if your webhost has enabled mod_rewrite of
course, and they allow overrides with htaccess.

Otherwise, this is a fairly simple task if you are content with "ugly"
urls. For example, given a url something like
http://mysite.com/mycgi.py?page=index.html, a very basic cgi
implementation might look like this:

#!/usr/bin/env python
import cgi

form = cgi.FieldStorage()

print "Content-type: text/html\n"
try:
page = form["page"]
except KeyError:
print file('custom404error.html').read()
else:
content = file(page.value).read()
# ... do something with content ...
print content

---

If your webhost has enabled mod_actions (I suppose this is unlikely),
you can also do the following in .htaccess (again, only if they allow
override):

AddHandler my-python-handler .html
Action my-python-handler /cgi-bin/handler.py

which should force apache to set an environment variable named
'PATH_TRANSLATED' with the value of the requested url, and call
'handler.py' for any requested file with an .html extension (that
exists) in that directory -- honestly, I've never attempted this myself
-- and there is probably a better way, but it's all I can think of ATM.
If the file doesn't exist apache returns a 404, which is only desirable
if you plan to store your templates on the filesystem. And you can use
the os module to access the cgi environment variables:

#!/usr/bin/env python
import os

print "Content-type: text/plain\n"

try:
path = os.environ["PATH_TRANSLATED"]
except:
path = 'PATH_TRANSLATED is not defined.'

print "The requested url is:", path

---

Perhaps you could convince your webhost to install mod_python, if they
haven't already. It's fairly trivial to write a simple mod_python
handler to accomplish exactly what you are trying to do.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] OT: webhosting

[Martin Walsh]

wormwood_3 wrote:
> There was another host that I wanted to mention along these lines (for Python 
> sites) that I think is even better: VPSLink (http://www.vpslink.com). They 
> allow root SSH access, and can install your choice of OS (lots of linux 
> flavors, ubuntu, SUSE, CentOS, etc) from a control panel. Aside from that 
> they are similar to WebFaction, but having that much flexibility, with the 
> base plan only ~$8/month, is pretty awesome.
> 
> -Sam

Running the risk of taking this tread in an inappropriate direction, I
feel compelled to mention http://openhosting.com, founded by Gregory
Trubetskoy the author of mod_python. Aside from the pleasant opportunity
 to name-drop :) I have found their service, up-time, and support to be
phenomenal -- while not the cheapest vps on the block. I think their
service plans start at around $20 US a month.

> 
> - Original Message 
> From: Kent Johnson <[EMAIL PROTECTED]>
> 
> Webfaction gives you both - you can install anything you like as long as 
> it doesn't require superuser privileges and you have your own Apache 
> instance that you can tweak any way you like.
> 
> Kent

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] "standard output: Broken pipe"

[Martin Walsh]

James wrote:
> Hi,
> 
> I have a snippet of code in a Python script I'm whipping up that's  
> causing a not-so-pretty output.  Here's the code:
> 
> subprocess.call( "yes '' | make oldconfig" , shell=True )
> 
> When I run this code, Python loyally executes the command, and then I  
> see the following error on my console:
> 
> -
> 
> yes: standard output: Broken pipe
> yes: write error
> 
> Thoughts / ideas?

File this one under 'wicked scary hack', but my first thought would be
to redirect stderr of 'yes' to /dev/null, while preserving stderr of
'make oldconfig'. Something like this:

import subprocess as sp
sp.call("yes '' 2> /dev/null | make oldconfig", shell=True)

... or ...

import subprocess as sp
p1 = sp.Popen("yes ''", shell=True, stdout=sp.PIPE,
 stderr=file('/dev/null', 'w'))
p2 = sp.Popen('make oldconfig', shell=True,
 stdin=p1.stdout)

However, since the 'equivalent' shell command doesn't seem to result in
a broken pipe error there must be a better way. Upon further inspection:

  $ strace yes 'Spam' | head -n 10
  <...>
  --- SIGPIPE (Broken pipe) @ 0 (0) ---
  +++ killed by SIGPIPE +++
  <...>

We may need to handle SIGPIPE. A quick search leads here:
http://article.gmane.org/gmane.comp.python.devel/88798/

... so, thanks to Mr. Woodcraft ...

import subprocess as sp
import signal
# warning: uses the system default action
# for SIGPIPE for all children, use subprocess
# preexec_fn arg if this is not desirable
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
sp.call("yes 'Spam' | head -n 10", shell=True)

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] "standard output: Broken pipe"

[Martin Walsh]

James wrote:
> Given this explanation, it's probably not a great idea to use  
> signal.signal() then.  ;)

In this case, it seems to me that setting the SIG_DFL action for SIGPIPE
is precisely what one would want to do.

This is pure speculation, but if you were to re-implement some of the
subprocess module functionality for a similar task (using the os module
methods for processes and pipes), you could wrap the appropriate calls
in a try/except to trap a broken pipe. IIUC, this is exactly the reason
python sets SIGPIPE signals to SIG_IGN (see the signal module docs:
http://docs.python.org/lib/module-signal.html). A casual attempt appears
to reveal that the subprocess module swallows these exceptions. I hope
I'm wrong, because I tend to think that trapping with a try/except would
be the preferred approach to this problem. Perhaps someone can confirm,
or help to clarify?

My previous signal example was a bit careless. I agree with Alan, it's
seems best to exercise some caution when messing with signals. It may be
safer to restrict the sigpipe signal changes to processes only when you
need to.

Fortunately, you can accomplish this by passing a function as the
preexec_fn argument to subprocess 'call', 'Popen', and perhaps others.

Consider the following (credit to Matthew Woodcraft, posted to the
python-dev list in Jul 2007):

import signal
import subprocess as sp

def permit_sigpipe():
signal.signal(signal.SIGPIPE, signal.SIG_DFL)

# no broken pipe, preexec_fn is called inside the child process
# thus setting sigpipe/sig_dfl for this process only
sp.call("yes 'Spam' | head -n 10", shell=True,
preexec_fn=permit_sigpipe)

# oops, we should get a broken pipe message
sp.call("yes 'Spam' | head -n 10", shell=True)

> 
> Any other thoughts?  I've tried about redirecting my output using  
> subprocess.Popen( ... , stdout=*something* ), but I haven't had much  
> luck.  The error still appears.  Or should I be redirecting the  
> "stderr" instead of "stdout"?

yes :) but only stderr of the 'yes' command. If the 'make oldconfig'
command writes to stderr I presume you want to see it.

>
> Any ideas on how to use subprocess.Popen() to make this lovely little
> error disappear?
>

Use two Popen objects, redirect stderr from the left-side process
('yes') to /dev/null (or a log file, if you want to inspect), and
connect stdin of right-side process ('make oldconfig') to stdout of the
left-side by way of a subprocess.PIPE. Here's an example:

import subprocess as sp
# replace /dev/null below to a path of your choosing
# to send stderr to a file for later review
leftside = sp.Popen("yes ''", shell=True, stdout=sp.PIPE,
 stderr=file('/dev/null', 'w'))
rightside = sp.Popen('make oldconfig', shell=True,
 stdin=leftside.stdout)


IMHO, I think this approach is 'ok' for your purposes, since you are
unlikely to care about most stderr messages from 'yes'. However, I
wouldn't recommend it as a general practice.

Also beware, if you do this ...

import subprocess as sp
sp.call("yes '' | make oldconfig", shell=True,
 stderr=file('/dev/null', 'w'))


... stderr from 'make oldconfig' will be sent to /dev/null as well.

HTH,
Marty


> Thanks!
> .james
> 
> On Oct 20, 2007, at 7:34 PM, Alan Gauld wrote:
> 
>> "James" <[EMAIL PROTECTED]> wrote
>>
>>> signal.signal(signal.SIGPIPE, signal.SIG_DFL)
>>>
>>> Before the snippet of code I included in my original e-mail
>>> (subprocess.call( "yes '' | make oldconfig" , shell=True )) got rid
>>> of the error.  I can't seem to figure out precisely what the
>>> signal.signal() function does (as shown above).
>> A signal is basically an interrupt. Its raised by the OS.
>> signal.signal catches the named signal(s) and handles it.
>> The default handler appears to just swallow the signal
>> silently.
>>
>> Its a bit like, in Python, doing
>>
>> try: someFunc()
>> except SomeError: pass
>>
>> The except clause catches the error but ignores it.
>>
>> signal.signal is doing much the same but at the OS level.
>>
>> My personal approach is to avoid using signal to trap
>> and ignore signals since they can be needed by other
>> processes running concurrently on the machine,
>> but in this case it appears harmless. But in general,
>> any approach which could interfere in unpredicable
>> ways with other processes on the computer is a
>> risky strategy IMHO.
>>
>> HTH,
>>
>> Alan G.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] "standard output: Broken pipe"

[Martin Walsh]

Michael Langford wrote:
> 
> This signal is not something you care about. All SIGPIPE means is that
> the source of the signal found itself writing to a pipe with no sender.
> Your line  "signal.signal(signal.SIGPIPE, signal.SIG_DFL)" means use the
> default signal handler for SIGPIPE. While this works (as the default
> signal handler is the ignore handler, you actually want to explicitly
> use the IGN handler, which will just ignore the signal.  To do this use
> " signal.signal(signal.SIGPIPE, signal.SIG_IGN)"

I don't think this is quite right, but please correct me if I'm
misinformed, or just plain wrong. :)

Using "signal.signal(signal.SIGPIPE, signal.SIG_IGN)" (kernel 2.6.20,
bash 3.2, python2.5) still produces a broken pipe with the following code:

import signal
import subprocess as sp
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
sp.call("yes 'Spam' | head -n 10", shell=True)

My understanding is that python is *already* overriding with a SIG_IGN
for SIGPIPE, and child processes inherit, which explains the difference
in behavior when running the command from a shell terminal (on my system
at least) vs. a python subprocess object. This is referenced in the
signal module docs (http://docs.python.org/lib/module-signal.html), and
appears to be confirmed in the python2.5 source (pythonrun.c).

Also, while it is unclear in the signal module docs (to me at least), my
take on the use of SIG_DFL is to revert (if necessary) to the *system*
default action for a signal, not the python default (which is SIG_IGN).
Again, this is my interpretation based on a very minimal understanding
-- please correct me if I'm wrong.

> 
> The reason you don't see the error on the shell is that the bash shell
> does not print notifications of SIGPIPE when running interactively. If
> you instead copied your snippit of scripting into a shell script then
> called that with "bash foo.sh", you'd see the exact same broken pipe.

Ok, this isn't true on my system either -- I don't get a broken pipe
from a bash script. So, perhaps this is due to the differences in our
kernel or bash versions. The version of bash on my system (3.2) has a
DONT_REPORT_SIGPIPE define that is honored in both interactive and
non-interactive shells. Presumably, this is how the ubuntu binary was
compiled.

> 
> Here is what all the signals available do:
> http://www.delorie.com/gnu/docs/glibc/libc_471.html
> 

Excellent link, thanks! IMO, the signal manpage ('man signal') is also a
good resource. On my system, it contains a table of default actions.
Here's what it has to say about SIGPIPE:

"""
Signal Value Action   Comment
-
SIGPIPE  13   TermBroken pipe: write to pipe with no readers
"""

and, the 'Term' action is defined as follows:

"""
Term   Default action is to terminate the process.
"""

Thanks!
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Looping + Variables

[Martin Walsh]

James wrote:
> Hi.  :)
>
> I'm trying to write a loop to simplify my life (and code :)).  The
> loop is going to iterate over a list of values that I have to change
> in a file.  I think my problem is better described with some code.  :)

Use a dictionary instead of a tuple ...

# variables
editValues = {
"interface": "eth0",
"address": "192.168.1.5",
"mask": "255.255.255.0",
"gateway": "192.168.1.1"
}

> def replaceText( old , new , file ):
>   for line in fileinput.FileInput( file , inplace = 1 ):
>   line = line.replace( old , new )
>   sys.stdout.write( line )
> 

for key in editValues.keys():
replaceText('$' + key + '$' ,
editValues[key] , '/etc/conf.d/net' )

> 
> config_$interface$=( "$address$ netmask $mask$" )
> routes_$interface$=( "default via $gateway$" )

Are you locked into this substitution format? If not, then how about
something like this:

# /etc/conf.d/net
config_%(interface)s=( "%(address)s netmask %(mask)s" )
routes_%(interface)s=( "default via %(gateway)s" )

You can then use python string formating (described here:
http://docs.python.org/lib/typesseq-strings.html), and pass the dict
directly. Note: if your file already contains '%' symbols that won't be
substituted, you must escape (prefix) them with an additional '%'.

valuedict = {
"interface": "eth0",
"address": "192.168.1.5",
"mask": "255.255.255.0",
"gateway": "192.168.1.1"
}

template = """\
# /etc/conf.d/net
config_%(interface)s=( "%(address)s netmask %(mask)s" )
routes_%(interface)s=( "default via %(gateway)s" )
"""

print template % valuedict

... prints ...

# /etc/conf.d/net
config_eth0=( "192.168.1.5 netmask 255.255.255.0" )
routes_eth0=( "default via 192.168.1.1" )

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] "standard output: Broken pipe"

[Martin Walsh]

Eric Brunson wrote:
> I'm coming in late to the discussion and thought that someone would 
> explain it succinctly, but there have been so many correct statements 
> which I feel fail to nail down the problem that I thought I'd chime in.
> 

Hi Eric,

Thank you for your considerate response. Your analysis is spot on --
speaking for myself, of course -- I'm mostly just flailing here; not
having a very strong grasp of how all the components work
together (kernel, shell, python, and the subprocess module). But I'm
really enjoying this thread and learning a lot in the 'process' ...
ahem, sorry for the pun ... painful :)

> Here's the important concepts to understand.  The pipe is a construct of 
> the shell, you are spawning a shell to run your entire command line.  
> When the "make" finishes and stops accepting the input from "yes", the 
> kernel sends a SIGPIPE to the parent process, i.e. the shell.  The 
> shell's default behavior when receiving the SIGPIPE is to print an error 
> message.

You're right. Considering the shell is definitely important and I missed
it. Yet given the following, slightly adapted from the subprocess module
docs (your reference link below) ...

from subprocess import Popen, PIPE
p1 = Popen(["yes", "Hello"], stdout=PIPE)
p2 = Popen(["head", "-n", "10"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

... still produces ...

yes: standard output: Broken pipe
yes: write error

If run as a script, the message above shows up in the terminal after the
python script terminates (or apparently so; shell prompt first, then
error). And when using the interactive interpreter... nothing until
exiting the interpreter, when the message again appears in the terminal.
Perhaps this is some peculiarity of my environment, or more likely I am
still missing something.

> Your problem, as I interpret it, is that you don't like seeing the error 
> message.  So you have two choices:  1) find some way to filter it, 
> because all it is is a message, or 2) get rid of the shell and handle 
> the management of the subprocesses yourself.  It's been discussed how to 
> suppress the output and it's been discussed how to intercept the signal, 
> but I don't thing that anyone has pointed to the definitive python 
> construct to handle it properly. 
> 
> You've had several responses that correctly tell you how to handle a 
> signal, but without any testing I'm pretty sure that none of that will 
> do you any good with the subprocess.call() construct you're using.  The 
> shell is your subprocess and it is already handling the signal, so you 
> should never see it.  The trick is to spawn both processes via python 
> *without* an intervening shell to interfere.  Here's the canonical 
> example: http://docs.python.org/lib/node536.html.  You can spawn each 
> subprocess, 'yes' and 'make' without the shell being required to pipe 
> the output of one to the other, you can do it all completely in python.

Based on my own tests, I don't think it makes a difference. I suspect
the python override for sigpipe is inherited by all child processes,
shell or otherwise, and the SIG_IGN is allowing 'yes' to see the broken
pipe, report it and terminate -- where the default action would normally
be to terminate only. This is, of course, not that far from wild
speculation. Though, I am curious if others observe the same behavior.

Then again, the error is *not* displayed for any of tests where
signal.signal(signal.SIGPIPE, signal.SIG_DFL) is added, in either the
global scope or in the form of a preexec_fn argument passed to
subprocess, with shell=True or without.

IMHO, this is certainly a tripping point if true. Granted, probably a
fringe case and relatively rare since it requires spawning a subprocess,
which writes to a pipe that is broken before the subprocess ends, and is
not implemented to handle the event gracefully. But I still find it
weirdly intriguing.

> 
> Again, sorry to come in late, but while reading many absolutely factual 


As far as I'm concerned, no apologies are necessary -- not that you are
apologizing to me necessarily :) But nonetheless, thank you for helping.

In fact, a big thank you to all the tutors. I've been a member of the
list for quite a few years now, reaping the benefits of your collective
experience (like many others I would imagine), mostly by lurking. You
provide a tremendous service to the python community, especially those
new to python or programming in general, with near super-human patience
and skill. Quite a feat! FWIW, thanks.


Marty

> an correct responses, they all seemed to be either handling the output 
> or discussing signal handling without, in my mind, pointing out that 
> it's the shell that's giving you the problems.  I think it's Python's 
> default behavior to ignore SIGPIPE, as Martin comments on in his latest 
> reply.  It's just that the shell has already accepted and handled the 
> signal.
> 
> Interestingly, when you eliminate the shell and use something simila

Re: [Tutor] "standard output: Broken pipe"

[Martin Walsh]

Eric Brunson wrote:
> Martin Walsh wrote:
>> Michael Langford wrote:
>> I don't think this is quite right, but please correct me if I'm
>> misinformed, or just plain wrong. :)
>>
>> Using "signal.signal(signal.SIGPIPE, signal.SIG_IGN)" (kernel 2.6.20,
>> bash 3.2, python2.5) 
> 
> I just noticed something, Martin.  subprocess.call() on my machine is
> calling the shell as /bin/sh, not /bin/bash.  This changes the behavior
> of bash, possibly causing it to print the message.  (Anyone else think
> that subprocess should honor the SHELL environment variable?)

Yeah, I thought that was strange too, and I completely agree that the
SHELL env var would seem the obvious choice -- or perhaps, a mechanism
to change the shell command. Now that you mention the behavior of
/bin/bash when called as /bin/sh, I guess it could have something to do
with posix compliance, but I don't really understand the implications.

Maybe a quick modification of the subprocess source will shed some
light; setting it to /bin/bash, instead of /bin/sh (or '/bin/bash --posix'):

"""
def _execute_child( ... ):

if shell:
args = ["/bin/bash", "-c"] + args

"""

Unfortunately, no change in the odd SIGPIPE behavior.

Another factoid -- Ubuntu /bin/sh is a symlink to dash. After your
advice to pay closer attention to the shell sub-process, I thought dash
might be the culprit. But alas, it produced the same behavior. And, of
course, changing the symlink to point to bash didn't make a difference
either.

> 
> I can't get any invocation of the commands to generate the error
> message, only when it's called from python does the ouput get
> generated.  Quite interesting, I'd love to know the explanation for that.

I'm seeing the same thing, error message from python and not from shell
or shell scripts. I am still leaning toward signal inheritance as a
likely contributor, which I think would also indicate that the
subprocess module is rolling up an exception and not passing it along.
As I mentioned it's almost all gut-reaction at this point, so far I
haven't found any definitive proof to substantiate. Slow going for me
I'm afraid, to many tangential paths to follow to fill gaps in my
understanding. But it sure is fun.

Again, thanks for you insight Eric!

Marty

> 
> Still poking at it, though.
> 
>> still produces a broken pipe with the following code:
>>
>> import signal
>> import subprocess as sp
>> signal.signal(signal.SIGPIPE, signal.SIG_IGN)
>> sp.call("yes 'Spam' | head -n 10", shell=True)
>>
>> My understanding is that python is *already* overriding with a SIG_IGN
>> for SIGPIPE, and child processes inherit, which explains the difference
>> in behavior when running the command from a shell terminal (on my system
>> at least) vs. a python subprocess object. This is referenced in the
>> signal module docs (http://docs.python.org/lib/module-signal.html), and
>> appears to be confirmed in the python2.5 source (pythonrun.c).
>>
>> Also, while it is unclear in the signal module docs (to me at least), my
>> take on the use of SIG_DFL is to revert (if necessary) to the *system*
>> default action for a signal, not the python default (which is SIG_IGN).
>> Again, this is my interpretation based on a very minimal understanding
>> -- please correct me if I'm wrong.
>>
>>  
>>> The reason you don't see the error on the shell is that the bash shell
>>> does not print notifications of SIGPIPE when running interactively. If
>>> you instead copied your snippit of scripting into a shell script then
>>> called that with "bash foo.sh", you'd see the exact same broken pipe.
>>> 
>>
>> Ok, this isn't true on my system either -- I don't get a broken pipe
>> from a bash script. So, perhaps this is due to the differences in our
>> kernel or bash versions. The version of bash on my system (3.2) has a
>> DONT_REPORT_SIGPIPE define that is honored in both interactive and
>> non-interactive shells. Presumably, this is how the ubuntu binary was
>> compiled.
>>
>>  
>>> Here is what all the signals available do:
>>> http://www.delorie.com/gnu/docs/glibc/libc_471.html
>>>
>>> 
>>
>> Excellent link, thanks! IMO, the signal manpage ('man signal') is also a
>> good resource. On my system, it contains a table of default actions.
>> Here's what it has to say about SIGPIPE:
>>
>> """
>> Signal Value Action   Comment
>> -
>> SIGPIPE  13   TermBroken pipe: write to pipe with no readers
>> """
>>
>> and, the 'Term' action is defined as follows:
>>
>> """
>> Term   Default action is to terminate the process.
>> """
>>
>> Thanks!
>> Marty
>> ___
>> Tutor maillist  -  Tutor@python.org
>> http://mail.python.org/mailman/listinfo/tutor
>>   

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Wrong version of Python being executed

[Martin Walsh]

Tony Cappellini wrote:
>>> What do you get if you print sys.path from
>> the interpreter?
> 
> I've printed out sys.path from inside the script as well,
> and all references to Python25 are replaced with Python23
> 
> 
> FWIW- This isn't a problem unique to this script.
> I've just printed out sys.path from another script in another
> directory, and Python2.3 is referenced.
> So, it's a system wide issue- but I still don't know how or why it's 
> happening.

That is odd.

Try using the full path to python, just to be sure: c:\python25\python
script.py -- do you get the same behavior?

Also, if you haven't already, you can run python with the -E and/or -S
flags (ex. 'c:\python25\python -E -S script.py'). The -E flag will cause
the PYTHONPATH and PYTHONHOME environment variables to be ignored. And
the -S flag prevents 'import site' at python startup. Regardless, even
if these are configured incorrectly I would expect the same behavior if
running a script or using the interactive interpreter.

You mentioned that python 2.4 is installed also... does it have the same
sys.path problem?

HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Wrong version of Python being executed

[Martin Walsh]

Tony Cappellini wrote:
> Martin Walsh mwalsh at groktech.org
> Sun Nov 11 06:13:10 CET 2007
> 
>>> That is odd.
> 
>>> Try using the full path to python, just to be sure: c:\python25\python
>>> script.py -- do you get the same behavior?
> This works just fine- I would expect it to.

Actually, I would have expected the opposite. My initial thought based
on your description was that python2.5 is being invoked with PYTHON* env
vars from a previous install, or some site module weirdness. But, the
fact that running python.exe with it's full path corrects the issue,
seems to indicate a problem with your PATH, rather than any python
specific environment setting. What does 'set PATH' report?

Though this still doesn't explain why you get python2.5 interactively,
and python2.3 when running a script -- perhaps I'm still unclear what
you are seeing. Would the following be an accurate description of the
behavior?

assuming:
- you run inside a fresh 'cmd' console each time (typing 'cmd' at the
run dialog, or similar), to be sure there is no app environment kruft

- the current working directory doesn't contain any programs, scripts or
possibly links that could interfere (preferably an empty path)

- you don't have any PYTHON* environment vars set (including PYTHONSTARTUP)

you observe:
- when you type 'python' (only 'python') at the prompt, you get
python2.5 interactively

- when you use the form 'python script.py', the script is run with
python2.3 (can you verify with sys.version?) with sys.path appropriate
for 2.3

- when you use the form 'c:\python25\python.exe script.py', the script
is executed with python2.5 and you have the correct sys.path (for 2.5)

> 
>>> Also, if you haven't already, you can run python with the -E and/or -S
>>> flags (ex. 'c:\python25\python -E -S script.py'). The -E flag will cause
>>> the PYTHONPATH and PYTHONHOME environment variables to be ignored. And
> 
> This also works just fine. I've tried both switches independently, and
> the scrip runs normally when I use either and both at the same time.
> If I don't use them, then Python2.3 is being invoked somehow.

Yeah, very odd indeed.

 > Very strange indeed. It's starting to remind me of an episode from The
> Twilight Zone ;-)
> 
> Is ti possible that my registry is corrupted?

I wouldn't think so, but I suppose it is possible. I believe all the
pertinent registry keys are store under
"HKLM\Software\Python\Pythoncore\", so you could have a look.
There are settings stored elsewhere, but I think they are all related to
file associations, and enabling double-click launching etc. I hope
someone will correct or clarify, if I'm wrong.

If it is a registry issue, re-installing python2.5 *may* provide a quick
fix. BTW, are you using an alternate distribution of python (ex.
ActiveState), or the standard python.org version?

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Wrong version of Python being executed

[Martin Walsh]

Alan Gauld wrote:
> "Tony Cappellini" <[EMAIL PROTECTED]> wrote
> 
>> I have to switch between 2.3 and 2.5, so to make it easy, I use an
>> environment variable called CURRENT_PYTHON.
>> (someone on this list or the wxPython list told me I should NOT use
>> PYTHONPATH and modify it the way I am using CURRENT_PYTHON)
>>
>> CURRENT_PYTHON=C:\PYTHON2X
>> path=%CURRENT_PYTHON%
>> (The existing path isn't shown, only for brevity)
> 
>> Note, these are entered in Ctrl Panel, System environment variables,
>> NOT at the command line.

That seems like a perfectly rational approach, and good advice.

Unfortunately I don't have a windows machine to test/confirm at the
moment, but IIRC you can set both user-space and system-wide environment
variables this way. I think PATH is special, in that user-space PATH is
appended to the *end* of the system-wide PATH, where other vars set in
user-space override the system defaults? Is it possible you have
python23 in your system-wide path, and you've edited user-space path to
include %CURRENT_PYTHON%?

Regardless, as a test you might try to add C:\Python25 to the front of
your system-wide PATH temporarily. This should aid in diagnosing a
simple PATH lookup problem. But, I'm stumped also. I don't know of any
reason why you would be seeing this behavior if you are invoking python
in a consistent way.

> 
> Umm, have you rebooted? Probably an obvious step but I don't
> think environment vars get reset in real time. They didn't used
> to on NT but that may have changed in W2K or XP... I haven't
> checked in a while.

I believe that this has changed somewhat, where the environment is read
at program startup. So the env is not refreshed until you close and
re-launch the program in question (in this case 'cmd'). Not sure how
windows explorer is effected, which may require a logoff/logon.  But,
again I don't have a windows machine available to confirm, so take that
for what it's worth. It's not going to hurt to reboot, that's for certain.

> 
> Othewise I'm as stumped as you.
> 
> Alan G 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] subprocess help, nohup

[Martin Walsh]

John wrote:
> Hello,

Hi John,

I didn't see a response to your question, so I'll make an attempt ...

>  
> I've written a script which conducts several subprocess calls and then
> ultimately calls a shell script which runs even more programs... my
> script is using subprocess to execute a few sed calls, and then execute
> the script. I'm getting strange behavior:

Just a suggestion, if you find you are preforming sed replace operations
regularly, you might consider writing your own sed-like replace
function. This way you avoid spawning a subprocess altogether for
search/replace functionality, and as an added bonus you can re-use it in
future scripts. The following (untested) code should get you started, if
you choose to go this route:

import re

def sed_replace(search, replace, text):
pattern = re.compile(search)
return pattern.sub(replace, text)

def infile_sed_replace(search, replace, path):
text = file(path).read()
newtext = sed_replace(search, replace, text)
file(path, 'w').write(newtext)

...
infile_sed_replace('RunMin=[0-9][0-9]*', 'RunMin=%s' % k, runFile)
...

> 
> You'll notice, the last subprocess call is commented out. Right now I'm
> just getting to that point to make sure everything is working. So, it
> seems to work, but I'm not sure how to get it to work if I change the
> command to nohup. I still want python to wait for it to return, in fact,
> I would like to set the python job running in the background as well...
> so what I'm looking at doing is:
>  
> % nohup myControl.py
>---> which will make several subprocess.call(s) including some that
> should be 'nohupped' as well...

This strikes me as nohup abuse, though I'm not entirely certain what you
are trying to accomplish. Since you plan to nohup *and* background your
script anyway, you might be better served by preemptively detaching from
the controlling terminal (or daemonizing) instead. If so, you may find
one or both of the following recipes useful:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012

Of course, you may need to handle logging a little differently.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python on T-Mobile Wing???

[Martin Walsh]

Trey Keown wrote:
> Hey all,
> I just got a brand new T-Mobile Wing, and, as you might guess, I want to
> install python on it. Well, I tried both the pythonce main build (with the
> .exe), and the build for the smartphone (also used .exe), but once I
> downloaded them, when I tried to run them, a dialog comes up saying that
> this is not a valid pocket pc app. Any ideas?
> 

I think the exe installer is for running on a windows pc with activesync
-- you would need the cab version to install directly on a windows
mobile device.

>From the pythonce wiki:
http://pythonce.sourceforge.net/Wikka/Installing
http://pythonce.sourceforge.net/Wikka/Downloading

> Note- the wing uses windows mobile 6. The specs can be found at-
> http://www.t-mobile.com/shop/phones/Detail.aspx?device=acc8102d-4506-4eaa-bc2f-9c7b8ec1b1e0
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Selecting a browser

[Martin Walsh]

Ricardo Aráoz wrote:
> Hi, I've checked webbrowser module and so far I find no way of selecting
> a browser other than the default one. Say I want certain places opened
> with IE and others with Mozilla, and I don't want to mess with the
> user's setting of the default browser. Any tips?
> TIA

I think one would normally use the form webbrowser.get('firefox'), on
unix systems. But if I understand correctly, the "problem" with the
webbrowser module on windows (and perhaps it is similar on a mac) is
that unless the program can be found on your system PATH, only a generic
'windows-default' browser class is registered, which uses os.startfile,
releasing control to the os, and serves to open the url in the user's
default browser.

If you're determined to use the webbrowser module on windows, you might
be able to do something like this:

import webbrowser

ffcommand = "c:/program files/mozilla firefox/firefox.exe %s &"
ff = webbrowser.get(ffcommand)
ff.open("http://www.example.com";)

iecommand = "c:/program files/internet explorer/iexplore.exe %s &"
ie = webbrowser.get(iecommand)
ie.open("http://www.example.com";)

I suppose you could also register them manually for later use with the
webbrowser.get(browser_name) form.

webbrowser.register('firefox', None, ff)
webbrowser.get('firefox').open('http://example.com')

Personally, I would probably just cut out the middle module and use
subprocess.Popen to start the browser, after checking if it is installed
(with os.path.isfile, or similar) -- which seems to be, more or less,
what the webbrowser module does if it finds one of the predefined
browsers on your system PATH. Something like this (pseudo-code):

browser = 'c:/program files/mozilla firefox/firefox.exe'

if os.path.isfile(browser):
p = subprocess.Popen([browser, 'http://www.example.com'])
# if you want to wait for the browser to
# close before continuing use p.wait() here
else:
... web browser not found ...

For dispatching based on site (url, or some other criteria), one idea
would be to wrap something like the above in a function which accepts
the web browser program path as an argument, and then pass the function
a path appropriate for the given criteria. Here is another (untested)
example to demonstrate:

import subprocess
import urlparse
import sys, os

FFPATH = 'c:/program files/mozilla firefox/firefox.exe'
IEPATH = 'c:/program files/internet explorer/iexplore.exe'

IESITES = ['microsoft.com', 'www.microsoft.com']

def launch(url, browser, wait=False):
if os.path.isfile(browser):
p = subprocess.Popen([browser, url])
if wait:
p.wait()
else:
print 'Invalid browser.'

def main(url):
# pick browser path by domain name
netloc = urlparse.urlparse(url)[1]
if netloc in IESITES:
launch(url, IEPATH)
else:
launch(url, FFPATH)

if __name__ == '__main__':
if sys.argv[1:]:
main(sys.argv[1])
else:
print 'Not enough arguments.'

In theory, if you run this script from a console on windows with any
microsoft.com url as an argument, it should open in IE -- where all
others open in firefox. Really rough, but I hope it helps.

Regards,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Selecting a browser

[Martin Walsh]

Ricardo Aráoz wrote:
> Martin Walsh wrote:
> Hi Marty, thanks for your help.
> I've tried your suggestions but they don't seem to work for me. In W's
> system window I can do :
> C:/> S:\FirefoxPortable\FirefoxPortable.exe http://www.google.com
> and it will open my browser ok. But no matter what I try :
> "c:/program files/mozilla firefox/firefox.exe %s &" or "c:/program
> files/mozilla firefox/firefox.exe %s" as input to webbrowser.get() it
> won't work.

Hi Ricardo,

Never would have guessed that you were using a portable browser :) But
it really shouldn't matter. And by the way, the '&' has special meaning
to the webbrowser.get method -- it determines whether a
BackgroundBrowser or GenericBrowser object is returned.

> Here's my session :
> 
>>>> import webbrowser
>>>> ff = webbrowser.get("S:\FirefoxPortable\FirefoxPortable.exe %s &")
>>>> ff.open('http://www.google.com')
> False

I suspect (with no way to confirm at the moment) that something in the
webbrowser module is confused by the backslashes. As you may know, the
backslash has special meaning in python strings, used as an escape
character to denote newlines (\n), tabs (\t), among others. I believe it
is helpful to be aware of this when using subprocess.Popen also. And, I
think you have more than one option for dealing with slashes in windows
paths, but I typically just replace the backslashes with forward slashes:

ff = webbrowser.get("S:/FirefoxPortable/FirefoxPortable.exe %s &")

> Besides looking at the Python 2.5 documentation gave me no clues. It is
> very poorly documented e.g. :
> 
> register( name, constructor[, instance])
>   Register the browser type name. Once a browser type is registered, the
> get() function can return a controller for that browser type. If
> instance is not provided, or is None, constructor will be called without
> parameters to create an instance when needed. If instance is provided,
> constructor will never be called, and may be None.
>   This entry point is only useful if you plan to either set the BROWSER
> variable or call get with a nonempty argument matching the name of a
> handler you declare.
> 
> But it does not define what a 'constructor' is (I guess a function, but
> no clue about what it's signature or functionality should be) or what an
> "instance" is (but I can guess). Trouble is when you guess and things
> don't work you don't have a clue if the fault lies in your 'guess' or in
> your code. Note that it mentions 'the BROWSER variable' but does not say

Based only on my own review of the webbrowser module source, the
constructor is the '*Browser' class object [ in other words
BackgroundBrowser and not BackgroundBrowser() ], presumably it should be
a subclass of webbrowser.GenericBrowser. And instance is what you would
think, an instance of a '*Browser' class.

> what it is, what it's value should be, nor any clue about it's use.
> Probably I lack knowledge in the subject, but hey! the whole purpose of
> this module is to be able to call the browser WITHOUT any knowledge of
> the subject.
> I guess I'll have to look at the code when I have a couple of free hours.

I would highly recommend it. It's straight forward, and very readable.

A helpful tip, perhaps: one of my favorite features of ipython, allows
fairly trivial access to a module's python source (not that it's
difficult by other means, really):

In [1]: import webbrowser

In [2]: webbrowser??

A single question mark prints the doc string if available, where the ??
pages the source. On linux, not sure about windows, I can move around
and search for text with all the same familiar keyboard commands (less
style).

> I'd rather go with the 'webbrowser' module than the subprocess.Popen
> option, as it 'should' abstract me from browser management. Anyway if I

I think that's the intent of the module, and a worthwhile pursuit. But,
IMHO you may find it's more work, and less flexible, than keeping track
of the browser paths yourself and calling subprocess.Popen -- on Windows
at least. YMMV. And it is certainly possible that you'll discover some
feature of the webbrowser module that I've overlooked. So, I hope you'll
report back to the list with your progress.

Good luck!
Marty

> can't advance through webbrowser I'll give your examples a try.
> Thanks a lot for your help.
> 
> Ricardo
> 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Output of list

[Martin Walsh]

Ricardo Aráoz wrote:
> Emil wrote:
>> hey
>>
>> I want to be capable of converting a string into a list where all the items, 
>> in  the list, have a fixed length not equal to 1 e.g i have k = 'abcdefgh' 
>> and I want the fixed length for all the the items to be 2 then the list 
>> would look like ['ab', 'cd', 'ef, 'gh']. How do i do this?
>>
>>
> 
> Also : [''.join(i) for i in zip(k[::2], k[1::2])]

Cool use of 'zip' and extended slicing!

Just thought I would add that 'zip' truncates after the shortest
sequence, which would cause data loss for strings of odd length -- of
course, the OP may not consider this a problem.

In [1]: k = 'abcdefghi' # <- note the 'i'

In [2]: len(k)
Out[2]: 9

In [3]: [''.join(i) for i in zip(k[::2], k[1::2])]
Out[3]: ['ab', 'cd', 'ef', 'gh'] # <- 'i' is gone


HTH,
Marty





___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Output of list

[Martin Walsh]

János Juhász wrote:
 > It is nice place to use a generator:
> 
> def pairs(sliceit):
> streamlist = list(sliceit)
> streamlist.reverse()
> while streamlist:
> pair = streamlist.pop() 
> try:pair += streamlist.pop()
> except: pass
> yield pair
> 
> ## Probably it is easier to understand
> def pairs2(sliceit):
> try:
> while sliceit:
> yield sliceit[:2]
> sliceit = sliceit[2:]
> except: # oops, it was odd length
> yield sliceit
> 

... Or, by extending Alan's solution ...

def splitStringByN(s, n):
for m in range(0, len(s), n):
yield s[m:m+n]

k = 'abcdefghi'
list(splitStringByN(k, 2))

As it turns out, this is similar to an ASPN Cookbook recipe contributed
by Dmitry Vasiliev:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302069

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Output of list

[Martin Walsh]

János Juhász wrote:
> Dear Marty,

Hi Janos,

>> ... Or, by extending Alan's solution ...
>>
>> def splitStringByN(s, n):
>>for m in range(0, len(s), n):
>>yield s[m:m+n]
>>
>> k = 'abcdefghi'
>> list(splitStringByN(k, 2))
> 
> It seems to be the most readable solution for me.

For completeness, one could also pull it out of the function def and use
a generator expression directly. If I'm not mistaken, this would be more
or less equivalent:

k = 'abcdefghi'
list((k[m:m+2] for m in range(0, len(k), 2)))

> 
> 
>> As it turns out, this is similar to an ASPN Cookbook recipe contributed
>> by Dmitry Vasiliev:
>> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302069
>>
>> HTH,
>> Marty
> 
> Thanks for the link.

No problem.

Happy Holidays!
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Processing unix style timestamp

[Martin Walsh]

Ravi Kondamuru wrote:
> Hi,
> 
> I have a log file that prints the date and time in the following format:
> Mon Feb 11 01:34:52 CST 2008
> I am expecting multiple timezone entries (eg: PST, PDT and GMT) on the
> system running in America/Los Angeles time zone.
> I am looking for a way to internally store all the different timezone
> entries in GMT.
> I looked at datetime, but it seems slightly complex to work with non GMT
> timestamps.
>
> Any pointers?
>

If you are not offended by a 3rd-party module, then the string parser
included in the egenix mxDateTime module[1] is hard to beat. You may
even have it installed already, as it appears to be a popular dependency
of other 3rd-party modules, especially db adapters.

In [1]: import mx.DateTime

In [2]: d = mx.DateTime.DateFrom('Mon Feb 11 01:34:52 CST 2008')

In [3]: d
Out[3]: 

In [4]: d.strftime('%a %b %d %H:%M:%S %Y')
Out[4]: 'Mon Feb 11 07:34:52 2008'

In [5]: d = mx.DateTime.DateFrom('Mon Feb 11 01:34:52 EST 2008')

In [6]: d.strftime('%a %b %d %H:%M:%S %Y')
Out[6]: 'Mon Feb 11 06:34:52 2008'

In [7]: d = mx.DateTime.DateFrom('Mon Feb 11 01:34:52 UTC 2008')

In [8]: d.strftime('%a %b %d %H:%M:%S %Y')
Out[8]: 'Mon Feb 11 01:34:52 2008'

HTH,
Marty

[1] http://www.egenix.com/products/python/mxBase/mxDateTime/

> thanks,
> Ravi.

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SSH commands in Python on Linux

[Martin Walsh]

Bernard Lebel wrote:
> Hello,

Hi Bernard,

> I'm trying to make a script to send a SSH command from a Linux
> computer to another Linux compter.
> 
> The Python syntax I'm using...
> 
> 
> import os
> os.system( 'ssh [EMAIL PROTECTED] "ls"' )
> 
> 
> Now the problem is that I'm always asked for the password. So my
> question is two-fold:
> 
> 1- Is there a way to not be asked for the root password?
> 2- If not, then is it possible to feed the SSH password input with my
> Python script? I have about stdin redirection, but I have to admit
> that I'm a bit lost and I don't know if it applies to SSH input as
> well.

you might also have a look at the pexpect module, found here:
http://pexpect.sourceforge.net/, I've found it to be a very useful tool
for working with interactive shell apps. There are a few ssh examples
provided in a separate download (pexpect-examples.tgz
http://sourceforge.net/project/showfiles.php?group_id=59762)

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to get response from os.system()

[Martin Walsh]

Hi Nathan,

Nathan McBride wrote:
> Yup I use the pexpect module for a lot however couldn't get 'pexpect.run' to 
> work with mysqldump piping to gzip 
> 

Just to hazard a guess -- when you want to pipe commands with pexpect
you have to spawn ('run', it seems, would work the same way) the shell
command as an argument to bash (or similar) since pexpect does not
natively interpret shell operators or wildcards, like redirect, pipe, etc...

from http://pexpect.sourceforge.net/pexpect.html
"""
Remember that Pexpect does NOT interpret shell meta characters such as
redirect, pipe, or wild cards (>, |, or *). This is a common mistake.
If you want to run a command and pipe it through another command then
you must also start a shell. For example::

child = pexpect.spawn('/bin/bash -c "ls -l | grep LOG > log_list.txt"')
child.expect(pexpect.EOF)
"""

HTH,
Marty

> -Original Message-
> From: Jeff Younker <[EMAIL PROTECTED]>
> Sent: Sunday, March 16, 2008 6:59 PM
> To: Nathan McBride <[EMAIL PROTECTED]>
> Cc: tutor@python.org
> Subject: Re: [Tutor] how to get response from os.system()
> 
> 
>> Would you mind perhaps show an example running an interactive  
>> command like su and show how to send input to the commands waiting  
>> propmts?
> 
> If you're doing that then you *really* want to be using the pexpect
> module.
> 
> cmd = pexpect.spawn('su - SOMEINTERACTIVECOMMAND')
> cmd.expect('# ')   # the prompt
> cmd.sendline('A COMMAND')
> cmd.expect('# ')   # wait for the prompt again
> output = cmd.before  # the stuff before the prompt
> cmd.sendline('exit')
> cmd.close()
> 
> 
> - Jeff Younker - [EMAIL PROTECTED] -
> 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] PyMOTW: difflib

[Martin Walsh]

Dick Moores wrote:
> See < http://blog.doughellmann.com/2007/10/pymotw-difflib.html>
> 
> And my try with the Differ example, <
> http://py77.python.pastebin.com/f41ec1ae8>, which also shows the error,
> 
> "E:\Python25\pythonw.exe" -u "E:\PythonWork\demo_pymotw-difflib.py"
> Traceback (most recent call last):
>  File "E:\PythonWork\demo_pymotw-difflib.py", line 12, in 
>from difflib_data import *
> ImportError: No module named difflib_data
> 
> What is difflib_data ?

It is the example data provided with the PyMOTW tutorial.

Near the top of the article (from the link you provided) you'll see the
heading "Test Data". I assume the author wants you to copy and paste the
source into a new file named difflib_data.py in your working dir.
Alternatively, it looks like you can download all the source and example
data for all PyMOTWs in one compressed file:
http://www.doughellmann.com/projects/PyMOTW/

PyMOTW has a cheese shop entry also (http://pypi.python.org/), so one
would assume you could get the source with easy_install as well, but
I've never tried it.

HTH,
Marty


> 
> Thanks,
> 
> Dick Moores

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] urllib2.urlopen(url)

[Martin Walsh]

Monika Jisswel wrote:
> Hi,
> 
> can i stop urllib2.urlopen() from  following redirects automatically ?

It doesn't answer your question directly, but if you care more about the
initial request/response than the content at the other end of a redirect
-- you can use httplib. It might look something like this:

"""
import urlparse
import httplib

url = urlparse.urlsplit('http://google.com/search?q=python')
host = url[1]
path = urlparse.urlunsplit(('', '')+url[2:])

con = httplib.HTTPConnection(host)
con.request('GET', path)
response = con.getresponse()

print 'status:', response.status
print 'reason:', response.reason
print 'document:', response.read()
"""

You lose many advantages of the higher-level urllib2, as it does much of
the mundane work for you -- parsing urls, choosing http or https
transparently, etc -- but I think httplib is still appropriate if your
needs are simple.

More information here:
http://docs.python.org/lib/module-httplib.html

HTH,
Marty

> thanks in advance
> 
> Monika Jissvel
> 


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] seeking help to a problem w/ sockets

[Martin Walsh]

James Duffy wrote:
> I have a problem w/ a file transfer receiver. They way it works is it
> binds a port for incoming transfer , when the file transfer is complete.
> It closes the connection and the socket, then loops back and restarts
> the bind and listen. I have it set so that the socket is reuseable,
> which is why this works. However, if the program that is using this
> function is closed while listening, it appears that it does not
> ”un-bind” because when the program is reopened and a listen attepted to
> start I get a “port already in use” error. Only a reboot fixes this
> issue. This code is imported into a main GUI script. We have it set to
> execute some cleanup functions on exit, I need a function that can dig

It should be noted that my socket, threading, and related gui skillz are
lacking, so beware.

IIUC, the usual cause for an 'address already in use' error is when the
server closes it's end of the socket first, leaving it in a TIME_WAIT
state. Presumably, this would be a feature of tcp, and the socket is
released after some timeout period to be sure the client is no longer
communicating. However setting the socket options as you have, with
setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1), is the typically
suggested workaround in this situation.

Also, given that you need to reboot to clear the error, I suspect that
your gui program isn't actually terminating -- but instead hanging on
the Receiver thread, and thus holding on to the socket. You can, of
course, confirm with netstat and/or by checking the process list. If you
just want the thread to die when you close the gui app, then you could
also try to setDaemon(True) on your Receiver class -- which should allow
the program to exit, that is -- if there are no non-daemon threads
remaining.

HTH,
Marty
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] free loop device

[Martin Walsh]

Nathan McBride wrote:
> Yup, I got some help in IRC.  What I ended up doing was using regex to
> pull out each "/dev/loopX".  Then
> took the X and fed it to max which in turn gave me the highest numbered
> loop device in use.  After which I
> then just added 1 to X and added it to the end of "/dev/loop_".  The
> only other thing I had to do was put a
> check in incase there were no used loop devices in which case then it
> defaults to "/dev/loop0".

One might also consider reimplementing 'losetup -f' in python, but after
my own attempt I realize it might not be that practical, and is
potentially dangerous I suppose -- if, like me, you don't fully
understand the underlying system calls. I've attached my attempt for the
sake of discussion only, and not as a solution -- perhaps someone with
interest will correct any errors and make it usable. I would definitely
appreciate it.

Drifting off topic now... I copied most of the find_unused_loop_device
implementation in util-linux/lomount.c[1]. The main points of interest,
and potential doom due to my ignorance, are related to the fcntl.ioctl call:

1) the LOOP_GET_STATUS ioctl op const, isn't exposed in any of the
typical python modules that I can find, and as such I'm worried that
it's value is somehow platform dependent.

2) for a similar reason, I am passing a string of the largest allowed
length as the 3rd arg to the fcntl.ioctl call on line 33, since the size
of the returned data seems to be governed by a struct defined in loop.h,
which needs dev_t from a kernel header. Whew. This seems to work fine on
my ubuntu system, if sloppy. But, since I don't know, I tend to assume
it could cause problems with stability or security.

Anyway, thanks for the interesting question Nathan. Now I have some
reading to do. :)

Marty

[1]
http://www.google.com/codesearch?hl=en&q=show:3q3vE6vLdaY:0lRVP2J7BtU:j-QqODsRp3s&sa=N&ct=rd&cs_p=ftp://ftp.kernel.org/pub/linux/utils/util-linux/testing/util-linux-2.13-pre7.tar.gz&cs_f=util-linux-2.13-pre7/mount/lomount.c&start=1
#!/usr/bin/env python2.5
import os
import stat
import errno
import fcntl

if os.uname()[4] == 'x86_64':
LOOP_GET_STATUS = 0x4C05
else:
LOOP_GET_STATUS = 0x4C03

def find_unused_loop_device():
"""
Return the next unused loop device node, returns None if the 
next device cannot be determined (and swallows exceptions 
encountered along the way: permission denied, no such file, etc). 
"""
for loop_format in ['/dev/loop%d', '/dev/loop/%d']:
for i in range(256):
dev = loop_format % i
try:
st = os.stat(dev)
except OSError:
break # assume invalid loop_format

if stat.S_ISBLK(st.st_mode):
try:
fd = os.open(dev, os.O_RDONLY)
except OSError:
pass # assume permission denied
else:
try:
fcntl.ioctl(fd, LOOP_GET_STATUS, 1024*'\x00')
except IOError, e:
if e.errno == errno.ENXIO:
os.close(fd)
return dev
os.close(fd)

if __name__ == '__main__':
print find_unused_loop_device()






___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Logging module

[Martin Walsh]

Dick Moores wrote:
> But how to use the logging module to log the report of the screw up?
> Right now I don't care about the various levels. I just want to get
> something into a log file.
> 
> Hellmann suggest this:
> 
> import logging
> LOG_FILENAME = '/tmp/logging_example.out'
> logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG,)
> logging.debug('This message should go to the log file')
> 
> But where to put it? In my above script?

I usually wrap up code in a try except and use the logging.exception
method, which is shorthand for logging.error(msg, exc_info=1), to
include the traceback. Something like this:

import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG)

def main():
a = "qwerty"
b = a * b

try:
main()
except:
logging.exception('unknown')
raise

HTH,
Marty

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to get a string from a DOM Text node ?

[Martin Walsh]

Zameer Manji wrote:
> I'm trying to extract some information from the following xml file:
> http://pastebin.ca/1029125
> 

> file = "Music.xml"

> dict = plist.childNodes[1] #dict contains App version and stuff, as well

You really should try to avoid rebinding built-in names, like dict and
file, this is just asking for trouble.

> The problem is that the Text node that is printed out is blank, when it
> should be 42. What do I need to do so I can get tdict.childNodes[2] to
> become a string being "42" ?

I have not used minidom at all really, but here is an example using
BeautifulSoup, perhaps you'll find it helpful:

from BeautifulSoup import BeautifulStoneSoup

class MusicSoup(BeautifulStoneSoup):
NESTABLE_TAGS = {
'dict': [], 'array': []
}


bsoup = MusicSoup(file('Music.xml'))
print bsoup.dict.dict.key.string
# 42
print bsoup.dict.dict.key.findNextSibling('key').string
# 183

... or ...

tracks = bsoup.dict.dict.findChildren('key', recursive=False)
print tracks

for track in tracks:
print track
details = track.findNextSibling('dict')


HTH,
Marty


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Modifying files in a directory.

[Martin Walsh]

[EMAIL PROTECTED] wrote:
>.>> soup = BeautifulSoup(html)
>.>> ord_tbl_price = soup.find('td', {'class': 'order_tbl_price'})
>.>> ord_tbl_price
>   $32.66
> 
> So now, how do I reduce the price by 15% and write it back to the 
> document?
Not sure if this is the right way, but it seems to work for me, YMMV:

html = """\
From $32.66"""

soup = BeautifulSoup(html)
otp = soup.find('td', {'class': 'order_tbl_price'})
price = float(otp.contents[1].lstrip(' $'))
otp.contents[1].replaceWith('$%0.2f' % (price * 0.85))

print soup.renderContents()

"""
From$27.76
"""

I'll second Kent's suggestion to experiment on the command line, and go
one further -- download and install the ipython interpreter[1]. It
offers tab-completion of methods and attributes of an object (for
example, type yourobject.), easy access to doc-strings with a
single ?, the source code (when available) with ??, and much much
more[2]. It's a real time saver, even if you don't use the fancier
features it beats dir()/help() hands down.

[1] http://ipython.scipy.org/moin/
[2] http://ipython.scipy.org/doc/manual/node4.html

HTH,
Marty






___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor