Re: [Tutor] Python fast enough for ad server?

[Eric Walstad]

Hey OkaMthenbo,

OkaMthembo wrote:
> Hi guys,
> 
> I need to write an ad-serving application and i'm using Win XP as my dev
> platform. Naturally, i want it to be as painless as possible and i was
> thinking of writing it 100% in Python. However, i have not written any
> big apps in the language and i wonder if Python would have the
> performance or scale fast enough to a large user base.
Most certainly for some definitions of 'large'  :)

Most web apps these days are not written in a single language/technology
and are often not running on a single piece of hardware.  If you search
the archives of your favorite Python web application framework I'm
pretty sure you'll find a discussion on how to scale your app to handle
a 'large' user base.  At the risk of oversimplification, but in hopes of
avoiding premature optimization, I'd focus first on achieving working
code, then benchmark it, then optimize if optimization is still needed.

Many others have achieved high volume Python web apps using a mix of all
the wonderful open source tools available.  If your content doesn't
change quickly and the ratio of GETs/POSTs is high, a caching server in
front of your python app might be just the trick (memcached, squid,etc).
 But don't waste your time if you don't need to.  Define 'too slow' and
then prove to yourself that your app passes that threshold.  If so, then
figure out why it's slow and optimize the slow parts.

Good luck,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Web GUI for a network management tool

[Eric Walstad]

Hi Thanos,

Alan Gauld wrote:
> "Thanos Panousis" <[EMAIL PROTECTED]> wrote
> 
>> The time has come to write some kind of gui for it, so that graphs,
>> visualizations and configuration options are exposed to non
>> developers. Do you think that a web app frame work like turbogears 
>> is
>> appropriate in my case?
> 
> Graphs etc on web apps can be problematic, but thats true of
> all web apps regardless of framework. You may need to hunt
> for some graphing code to suit the framework you choose
> - or maybe a Java applet.

I'm not sure how complex your charts/graphs need to be.  Here's a simple
Django solution for simple HTML Bar Charts that may give you some ideas
for solving the problem:

and an example of what it looks like:


I'd be interested to hear about how you ultimately solve the problem
when you are done.

Best,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python fast enough for ad server?

[Eric Walstad]

Hi OkaMthembo,

Again, are you sure you need it?  Have you done any benchmarking yet?
It looks like you have to run Psyco on 32 bit x86 architecture.  That
may limit other parts of your system design that you want to speed up by
running on x86-64, for example.  OTOH, it's easy to add/remove psyco (at
least it was easy to add it to one of my Django projects in the past).

I think it's good to consider possible bottle necks at design time.
Just remember that your assumptions may be wrong yielding your premature
optimizations useless.  Psyco may help, it may not.  I suspect that you
won't need it because Python is pretty darned fast already and the
Python web-framework guys are *really smart* and have done the much of
the optimization for you already.

Best regards,

Eric.

OkaMthembo wrote:
> Hi guys,
> 
> I stumbled upon a tool called Psyco (http://psyco.sourceforge.net/) sounds
> like what i need.
> 
> Thanks again,
> 
> Lloyd
> 
> On 5/10/07, OkaMthembo <[EMAIL PROTECTED]> wrote:
>>
>> Thanks for all your contributions. i think i will do it all in Python, it
>> seems to me that the advantages far outweigh any negatives.
>>
>> Maybe once its a working project, we can then benchmark the code and see
>> what gives.
>>
>> Thanks again,
>>
>> Lloyd
>>
>> On 5/9/07, Eric Walstad <[EMAIL PROTECTED]> wrote:
>> >
>> > Hey OkaMthenbo,
>> >
>> > OkaMthembo wrote:
>> > > Hi guys,
>> > >
>> > > I need to write an ad-serving application and i'm using Win XP as my
>> > dev
>> > > platform. Naturally, i want it to be as painless as possible and i
>> was
>> > > thinking of writing it 100% in Python. However, i have not written
>> any
>> >
>> > > big apps in the language and i wonder if Python would have the
>> > > performance or scale fast enough to a large user base.
>> > Most certainly for some definitions of 'large'  :)
>> >
>> > Most web apps these days are not written in a single
>> language/technology
>> >
>> > and are often not running on a single piece of hardware.  If you search
>> > the archives of your favorite Python web application framework I'm
>> > pretty sure you'll find a discussion on how to scale your app to handle
>> > a 'large' user base.  At the risk of oversimplification, but in
>> hopes of
>> > avoiding premature optimization, I'd focus first on achieving working
>> > code, then benchmark it, then optimize if optimization is still needed.
>> >
>> > Many others have achieved high volume Python web apps using a mix of
>> all
>> > the wonderful open source tools available.  If your content doesn't
>> > change quickly and the ratio of GETs/POSTs is high, a caching server in
>> > front of your python app might be just the trick (memcached,
>> squid,etc).
>> > But don't waste your time if you don't need to.  Define 'too slow' and
>> > then prove to yourself that your app passes that threshold.  If so,
>> then
>> >
>> > figure out why it's slow and optimize the slow parts.
>> >
>> > Good luck,
>> >
>> > Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to test for a remainder from division

[Eric Walstad]

Hey Matt,

Skirting your question that looks like a modulo issue[1]...

Maybe isleapyear[2] will work for you:

import calendar

calendar.isleap(2007)  -> False
calendar.isleap(2008)  -> True


I hope that helps,

Eric.
[1] http://docs.python.org/ref/binary.html
[2] http://docs.python.org/lib/module-calendar.html

Matt Smith wrote:
> Hi there,
> 
> I'm trying to write a short function to test whether a year is a leap
> year or not. To do this I need to check whether the year divides exactly
> by 4, 100 and 400. I can't think of an easy way to test whether there is
> a remainder or not. The best I can come up with so far is:
> 
> if (year / 4.0) - (year // 4.0) <> 0:
> 
> This doesn't seem to work, it is always True, is there a problem with
> the comparison? The arithmetic seems to be the correct way to isolate
> the remainder of the division.
> 
> Can anyone suggest a better way of performing this test or alternately,
> how can I get the line above to work.
> 
> Thanks,
> 
> Matt
> 
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

-- 
--
_

Eric Walstad
2856 Diamond Street
San Francisco, CA 94131
[EMAIL PROTECTED]
415-864-4224
_
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to test for a remainder from division

[Eric Walstad]

Hey Matt,

Matt Smith wrote:
> I guessed that there would be a module out
> there providing a function to do this but wanted to go through the
> process as a learning exercise.
Good form, old boy.


> Here is the final code I have come up with, any comments?
I think your code looks fine.  I like to do the following, just to be a
little more concise:

def is_leap_year(year):
return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0)

That is, the 'if' in your code is evaluating the statement into a
boolean.  Why not just return the results of that evaluation?
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ImportError: No module named _apache

[Eric Walstad]

Hey ShivKumar,

ShivKumar Anand wrote:
> *I am getting this error message:*
>  
> Traceback (most recent call last):
>   File "C:\Python24\exp\Web\SoapTest.py", line 1, in ?
> from mod_python import apache
>   File "C:\Python24\lib\site-packages\mod_python\apache.py", line 28, in ?
> import _apache
> ImportError: No module named _apache

Are you trying to call your mod_python code from the command line?  If so:
"you can't import mod_python into scripts that are run from the command
line or any other context outside a running apache server."



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] trouble with "if"

[Eric Walstad]

adam urbas wrote:
> Hi all,
> 
> I've been working with this new program that I wrote.
...
> #"User's Choice:"
> shape=raw_input("> ")
>
> #"Select Given:"
> if shape == 1:
...


[EMAIL PROTECTED]:~$ python
Python 2.5.1 (r251:54863, May  2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> shape = raw_input('>')
>1
>>> type(shape)

>>> shape == 1
False
>>>


HTH,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] trouble with "if"

[Eric Walstad]

Hi Adam,

adam urbas wrote:
> when I input a radius, it says:
> 
> can't multiply sequence by non-int of type 'float'
...
> > radius=raw_input("Enter Radius:")
> > diameter=(radius*2)


After you collect the raw_input for the radius, the radius variable
contains a string, not a number (that's what '' means).
Python is calling the string a sequence in your error message.

Try converting your radius to a float type first:

radius=float(raw_input("Enter Radius:"))


As side notes: those '>>>' characters in previous responses are what the
python interactive terminal displays as its command prompt.  The
'type()' function tells you the data type of a variable.

Here's an example of using the Python interactive terminal to debug your
issue (give it a try yourself, but don't enter the '>>>' in the terminal):

[EMAIL PROTECTED]:~$ python
Python 2.5.1 (r251:54863, May  3 2007, 12:27:48)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> radius=raw_input("Enter Radius:")
Enter Radius:5
>>> radius
'5'
>>> type(radius)

>>> diameter=(radius*2)
>>> diameter
'55'   # That is, it is giving you the string '5', two times
>>> type(diameter)

>>>
>>>
>>> radius=float(raw_input("Enter Radius:"))
Enter Radius:5
>>> radius  # radius now contains a float type
5.0
>>> type(radius)

>>> diameter=(radius*2)
>>> diameter
10.0
>>> type(diameter)

>>>

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] write a movie file thumbnail in Linux...

[Eric Walstad]

Hi all,

Nautilus does it when I browse to a directory that contains images and
movie files created with my digital camera.  I see thumbnails of the
image (jpeg) and the movie (mov) files.

I use python to process (rename, make reduced resolution copies (PIL))
of my image files.  I'd like to also have my python script make those
cool thumbnails of the movie files, too.

Anybody have recommendations for how to do this with Python or other
Linux command line tools?

Thanks in advance,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] sorting objects in lists by 2 attr

[Eric Walstad]

Philippe Niquille wrote:
> Using django I get a QuerySet of Score objects which are sorted by the
> actual score, the actual score divided by the max. possible score (so
> sorting by two db fields).
> 
> I then need to loop through that queryset and sum up all the score
> objects which belong to the same user into one Score objects. This works
> (see code below).


I'm with Eric and Alan here; do as much in the DB as possible/practical
first, then work with the results in Python.  Django's ORM may be able
to do your aggregating for you.  If not and if your DB has something
like PostgreSQL's Views (stored queries) then you can create a database
view that filters/sorts/aggregates your data as needed.  From there you
can create a Django Model associated with the DB View, just like you
normally create a Django Model on a DB table.  The main difference is
that objects returned by the QuerySet on the DB View will be read-only
because the DB View is read-only.  On the other hand you'll still be
able to python-ize your aggregated db data using Django's ORM in a
manner you are used to, something like:

scores = ModelOnMyDBView.objects.order_by('max_score','owner')

I hope that is helpful,

Eric.

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python / CGI

[Eric Walstad]

Michael Connors wrote:
> Hi,
> If you have your own server to run it on, I think it would make sense to
> use one of the Python web frameworks that are out there. I used cherrypy
> for my first web-based python project and I found it very easy to learn
> and develop in quickly.
> Regards,
> Michael

I'd argue that sticking with a Python CGI would be a great way to start
because the OP can focus on the basics rather than on the framework API.
 The Python CGI stuff I've written in the past was helpful in my
understanding of how and why Django works the way it does.

After his Web Calendar is functioning as a CGI he'll have lots of ideas
on how to improve it.  A framework might help at that point.

This article looks like it might have some useful information:
[http://python.about.com/od/advancedpython/ss/howtocal.htm]

Sounds like a fun project.

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Apache, CGI-BIN, Python

[Eric Walstad]

Ryan wrote:
> Jan Erik Moström wrote:
>> Ryan <[EMAIL PROTECTED]> 07-09-10 15:35
>>
>>> Once I edit the config files for cgi scripts the browser says I don't 
>>> have permission to access the file.
>> You must make sure that the script file has execute/read permissions 
>> for the process/user who runs the apache server. On my ubuntu machine 
>> that is 'www-data'
>>
>> jem
> I found the lines with 'www-data' but I am not sure what exactly I have 
> to do. I have User and Group-www-data.


Hi Ryan,

Have you RTFM?  Googled for something like "apache file permission", etc?

http://httpd.apache.org/docs/2.0/
http://httpd.apache.org/docs/2.0/howto/cgi.html
http://www.google.com/search?q=apache+cgi+file+permission
http://catb.org/~esr/faqs/smart-questions.html#before

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Updating MySQL Database

[Eric Walstad]

Hey Sam,
wormwood_3 wrote:
> Hello all,
> 
> I have a script which takes data from a file or MySQL DB, looks up some 
> stuff, then can print results to console or file. I would also like it to be 
> able to update a MySQL database with the results. Does anyone have any ideas 
> on how to do this? I can update records just fine, but what is the best way 
> to do LOTS of updates aside from running an update statement per record?

I recommend not worrying about speed on your first iteration of your
script.  I'd first define 'too slow', then write the script the way that
feels intuitive to you.  If your script passes your threshold of 'too
slow', then look at optimizing it.

If optimization is really necessary, I'd look into .executemany().  If
that is still too slow for you then I'd consider writing the update SQL
to a file and then calling mysql, passing it the sql file your script
created.  I found this last approach the fastest for a data import
script I once wrote for importing millions of records to a PostgreSQL
database.

-E


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] An idea for a script

[Eric Walstad]

Dick Moores wrote:
> I think I could learn a lot about the use of Python with the web by 
> writing a script that would look at 
>  and find all the links 
> to more that just the default shown by this one: 
> . I think there should be 
> about 20 URLs in the list. But I need a start. So give me one?

BeautifulSoup?

# untested...

import urllib
from BeautifulSoup import BeautifulSoup
starship_crew_list_url = 'http://starship.python.net/crew/index.html'
starship_crew_list_html = urllib.urlopen(starship_crew_list_url).read()
soup = BeautifulSoup(starship_crew_list_html)
for anchor in soup.fetch('a'):
print anchor

# / untested

-Eric

should be one of the results you see :)

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to read from a csv file?

[Eric Walstad]

Tim Golden wrote:
> You really just want to rename __init__ to rawdata
> and drop the redundant "return rawdata" function.

But remember to include the line:
return rawdata

at the end of your function or else nothing will be printed when
print csvdatareader.rawdata()
is called.

Also, I'd consider using different names for the function and list variable.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] A new Kent essay: A Brief Introduction to Beautiful Soup

[Eric Walstad]

Dick Moores wrote:
> 

That looks like a very nice, concise tutorial that should help soup
noobs like me to get up and running quickly.  I like the added value of
comments like: "Under the hood, attribute access is actually a search
for the first tag of the correct type, so soup.title will also access
the first  tag."

I haven't needed BeautifulSoup yet, but I know I will some day.

Thanks for sharing, Kent.

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Turning multiple Popen calls into a function

[Eric Walstad]

Hi Jay...
jay wrote:
...
> I would be sending an arbitrary number of PIPES with each function call.
> 
> I'm a little stumped as to how to handle the variables.  If I have an 
> arbitrary number of PIPES, how do I declare my variables (p1, p2, p3, 
> etc...) ahead of time in the function??
> 
> Thanks for any suggestions!


 >>> def foo(bar, *fizz, **bang):
... print "bar:", bar
... print "fizz:", fizz
... print "bang:", bang
...

 >>> foo('and now for something completely different')
bar: and now for something completely different
fizz: ()
bang: {}

 >>> foo('hello', 'and', 'cows', 'eat', 'grass')
bar: hello
fizz: ('and', 'cows', 'eat', 'grass')
bang: {}

 >>> foo('hello', 'and', 'cows', 'eat', 'grass', greeting='hello', 
location='world')
bar: hello
fizz: ('and', 'cows', 'eat', 'grass')
bang: {'greeting': 'hello', 'location': 'world'}

 >>> a_tuple = ('and', 'cows', 'eat', 'grass')
 >>> a_dict = dict(greeting='hello', location='world')
 >>> foo('hello', *a_tuple, **a_dict)
bar: hello
fizz: ('and', 'cows', 'eat', 'grass')
bang: {'location': 'world', 'greeting': 'hello'}


Or, just pass the pipes in as an iterable to your function:
def pipe_handler(pipes):
 for n, pipe in enumerate(pipes):
 print "Pipe %d: '%s'" % (n, pipe)

pipes = []
pipes.append(some_pipe)
...
pipe_handler(pipes)

See also:


I hope that helps,

Eric
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Noob question

[Eric Walstad]

Eric Brunson wrote:
> quantrum75 wrote:
>> Hi there,
>> I am a newbie trying to actively learn python.
>> My question is,
>> Suppose I have a list
>> a=["apple","orange","banana"]
>>
>> How do I convert this list into a string which is
>>
>> b="appleorangebanana"
>> Sorry for my ignorance, 
> 
> No worries, every new language has its idioms that only come from 
> experience and exposure.
> 
> Try this:
> 
> In [1]: a=["apple","orange","banana"]
> 
> In [2]: print "".join( a )
> appleorangebanana
> 
> And just for clarity:
> 
> In [3]: print "|".join( a )
> apple|orange|banana

And another good reference for you to know about is the built-in help
system that comes with Python.  For example, to learn a bit about why
Eric's code works the way it does:
>>> help("".join)

Help on built-in function join:

join(...)
S.join(sequence) -> string

Return a string which is the concatenation of the strings in the
sequence.  The separator between elements is S.


In Eric's example, the variable 'a' was a type of Python sequence,
specifically, a list.


You could also achieve the same result of concatenating a list of
strings by looping over the list items like so:

b = ''
for fruit in a:
b += fruit

print b

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] XML data reading [slightly off topic/silly]

[Eric Walstad]

Lockhart, Luke wrote:
> ...now I'm a
> Linux man and trying to realize my overly ambitious programming dreams 
> with Python
Welcome to the real world, Neo.  I'm not saying that you'll be able to 
work with XML via DOM or SAX in Python.  I'm saying that when you are 
ready, you won't have to.  :)
(with apologies to the Wachowski brothers)


 > But I just can't figure out the various
 > libraries that Python uses to read XML, and my friend's code doesn't
 > help much.
I think Google will help with many examples.  Also have a look at the 
Python Cookbook:
http://aspn.activestate.com/ASPN/Python/Cookbook/
for examples.


/me climbs onto soap box (no pun intended)
I think this might be a little off topic but my advice as you learn 
Python, considering you are parsing data and might have OOP experience, 
is to consider that there is value in simple code.  Try to keep an open 
mind and try not to use too many dots, especially if you have control 
over both ends of your data transfer.

Here's what I think is a nice example of roughly 17 lines of python code 
that elegantly parse and process a 5M record file with an impressive 
speed gain, too:
http://groups.google.com/group/comp.lang.python/msg/38a13587b6b298a2

Welcome, have fun and do post to the list when you have specific questions!

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ssh script

[Eric Walstad]

I've done similar with pexpect:
http://pexpect.sourceforge.net/


washakie wrote:
> Hello, I'm trying to write a script which will allow me to create a reverse
> tunnel from a remote machine using SSH -R
> 
> I know the reverse tunnel script works on it's own run from the shell, but I
> don't want to leave it open always... does anyone have any ideas on how to
> make this work? There seems to be something wrong with the subprocess call
> such that it just hangs on the remote side...
> 
> #!/usr/bin/python
> 
> import os, time, subprocess
> REMOTE_HOME='../'  #mounted drive to REMOTE_HOME from LOCAL_MACHINE
> 
> cmd = 'while true; do ssh -R 8022:localhost:22 [EMAIL PROTECTED] ; sleep
> 60; done'
> while 1:
>   while os.path.exists(os.path.join(REMOTE_HOME,'mySecretFile'):
>   proc=subprocess.call(cmd,shell='True')
> 
>   if proc: os.kill(proc.pid)
> 
>   if os.path.exists(os.path.join(REMOTE_HOME,'KillScript'):
>   break

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] ssh script

[Eric Walstad]

Hi Washake,

Here's a sample I just whipped up from the example I found here:

Note that I had to tweak the 'tunnel_command' line a bit as it was 
missing the conversion type.  This sets up a tunnel from a PostgreSQL 
server I have to my workstation.  I then use the tunnel on my 
workstation to connect to the database.

# On the PostgreSQL server machine ('a_server') #
# Setup a reverse forward
[EMAIL PROTECTED]:~$ python
Python 2.5.1 (r251:54863, Oct  5 2007, 13:36:32)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import pexpect
 >>> import getpass
 >>> import time
 >>> tunnel_command = """ssh -R5432:localhost:5432 %(user)[EMAIL 
 >>> PROTECTED](host)s"""
 >>> host = raw_input('Hostname: ')
Hostname: 192.168.1.1
 >>> user = raw_input('Username: ')
Username: username
 >>> X = getpass.getpass('Password: ')
Password:
 >>> ssh_tunnel = pexpect.spawn(tunnel_command % globals())
 >>> ssh_tunnel.expect('password:')
 >>> time.sleep(0.1)
 >>> ssh_tunnel.sendline(X)
 >>> ssh_tunnel.expect (pexpect.EOF)
 >>>

# Now the reverse forward is set up and I can connect to the PostgreSQL
# instance running on 'a_server'
# On the client machine: 192.168.1.1 #
[EMAIL PROTECTED]:~$ python
Python 2.5.1 (r251:54863, Oct  5 2007, 13:36:32)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import psycopg2 as Database
 >>> conn_parts = {
... 'user':'dbuser',
... 'dbname': 'mydatabase',
... 'password':'',
... 'host':'127.0.0.1',
... 'port':'5432',
... }
 >>> cs = ' '.join(["%s=%s" % (k, v) for k, v in conn_parts.items() if v])
 >>> print cs
port=5432 host=127.0.0.1 user=dbuser dbname=mydatabase
 >>> conn = Database.connect(cs)
 >>> cur = conn.cursor()
 >>> sql = 'select count(*) from mytable'
 >>> cur.execute(sql)
 >>> records = cur.fetchall()
 >>> conn.close()
 >>>
 >>> print records
[(557L,)]
 >>>


washakie wrote:
> Thanks everyone these seem like promising directions to go... Eric, any
> chance you could share your 'similar' code? The problem it seems with
> paramiko is that it has little 'newbie' documentation, and I'm not clear how
> to set up the reverse port forwarding on it. Thus I may need to use the
> pexpect solution. 
> 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SSH with Python

[Eric Walstad]

Eric Brunson wrote:
> Tom wrote:
>> I have a webfaction server (highly recommended btw) and I'm trying to
>> use automate some tasks over ssh but I'm not having much luck with
>> pyssh http://pyssh.sourceforge.net/ .
>>
>> Some of the docs mention methods that don't exist in the version I
>> downloaded, such as pyssh.run.
>>
>> I'm using windows vista (work pc). Any ideas?
>>   
> I've never used pyssh, so I can't comment directly on your problems.  We 
> use paramiko very successfully.  It's well documented and has excellent 
> example code included with the source.
> 
> http://www.lag.net/paramiko/

Also have a look at pexpect which essentially wraps command line calls. 
  I use it to automate mysql backups on my webfaction account.  Here's 
the meat of the code:

import pexpect
cmd = "ssh [EMAIL PROTECTED] 'mysqldump --opt -p foodb > foodb.sql'"
child = pexpect.spawn(cmd)

# expect mysql to prompt for a db password
child.expect('Enter password: ')
# send the password
child.sendline("nottherealpassword")

Homepage:
http://www.noah.org/wiki/Pexpect

It looks like there is now a pexpect SSH module.  I've not used it, just 
FYI:
http://pexpect.sourceforge.net/pxssh.html

I hope that helps.

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SSH with Python

[Eric Walstad]

Hey Eric,
Eric Brunson wrote:
> Eric Walstad wrote:
>> Eric Brunson wrote:
>>   
>> import pexpect
...
>> child.sendline("nottherealpassword")
...
>>   
> 
> What's the path to ssh under windows vista?
I hope I never have to find out :)


> Paramiko is a pure python 
> implementation of the SSH protocol, so it should run on any platform. 
Cool!  I'd like to give Paramiko a try some day when I don't need to be 
concerned with prompts from the remote command I am running.  Yes, 
Paramiko looks interesting.


> Besides, IMHO, expect, pexpect, Perl::Expect... the entire concept is a 
> horrible, horrible kludge and should only be used as a last resort when 
> no other interface is possible.
I mostly agree, except when other priorities win, when time is of the 
essence, when I don't have root or when I'm just feeling nervous about 
hobgoblins.  Yeah, it's ugly but it works for me and my needs and has 
been surprisingly stable over the years.  That said, I admit that the OP 
might be better served by Paramiko, assuming it does what they need.

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] database web app, what tool?

[Eric Walstad]

Hey Paulino,

I'm a Django fan but from your description, a simple CGI app might do 
nicely.  Heck, you've already got the code written, just wrap it in a 
CGI script that formats your list of records in HTML and serve the 
script from your (already setup?) apache or even from your workstation 
with Python's CGI server.

Best,

Eric.


Liam Clarke wrote:
> Paulino wrote:
> 
> Hi Paulino,
> 
> Tough question that. What is your existing web server? That'd be a good 
> starting point. Zope/Plone are great, but they have a steep learning 
> curve that doesn't seem essential to what you're doing. Django is 
> simpler, but once again, doesn't immediately fit to what you're trying 
> to do, but it could do.
> 
> I'd say just a straight mod_python would be simplest, to be honest, but 
> it really depends on what you're doing.
> 
> Regards,
> 
> Liam Clarke
>> Hi!
>>
>> Im a very biginner in programming and I started with python, witch I 
>> found much easier tahn i thought.
>>
>> I want to build an intranet page in my organization that shows some data 
>> from our financial aplication's database.
>>
>> For the starting point I would like it to show invoices lists per 
>> supplier and link each line to a pdf image of the selected invoice that 
>> is stored in a determined folder.
>>
>> The aim is only to make querys and show the results. No updates, nor 
>> insert's
>>
>> The database is MS SQL 2005
>>
>>
>>
>> Right now I connect to the database via pymssql, and i can retrieve data 
>> and send it to a wx.Grid or to a reportlab pdf file, with platypus 
>> tables (a couple of months ago i would say i'd never be able to do it... 
>> and now i want to do more...)
>>
>> Now I want to send it to the browser so the administration can see it 
>> every time they need.
>>
>>
>> What tool is more suitable?
>>
>> There are so many options!
>>
>> Zope-Plone have it's own web server, so less config is required. But i 
>> can't find any recipe or tutorial usefull for this task...
>>
>> there is also django, turbogears, webware, apache and mod-python.
>>
>>
>> Thank you
>>
>>
>> Paulino
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python code to split and reassemble files

[Eric Walstad]

Andrew Robert wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Never mind.. :)
> 
> 
> found it at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/224800
> 
> However, if someone has something better, I would be very interested.

Hi Andrew,

If you are on a *nix machine you could use split and cat:

"""
SPLIT(1)

NAME
split - split a file into pieces

SYNOPSIS
split [OPTION] [INPUT [PREFIX]]

DESCRIPTION
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; 
default size is 1000 lines, and default PREFIX is ‘x’.
"""

"""
CAT(1)

NAME
cat - concatenate files and print on the standard output

SYNOPSIS
cat [OPTION] [FILE]...

DESCRIPTION
Concatenate FILE(s), or standard input, to standard output.
"""

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Random Variable at root

[Eric Walstad]

Hi Hugo,

halamillo wrote:
> I;m starting this
> code so I can assign either a 0 or a 1 at a root node that I will later 
> evolve throughout a tree, but for some reason it is not printing the 
> rootState value.

It's not clear to me which variable you are trying to assign a zero or 
one.  Here's my guess at how you are wanting to use your class:

 >>> import random
 >>> from random import random as rnd
 >>>
 >>>
 >>> class Root:
... """Single Node in a Tree"""
... def __init__(self, rootState):
... self._rootState = rootState
... # random choice for root state from a list
... self._rootState = random.choice([0, 1])
... def __str__(self):
... return str( self._rootState )
...
 >>>
 >>>
 >>> rs = 99
 >>> r = Root(rootState=rs)
 >>> print rs
99
 >>> print r
0
 >>> print "The character state at the root is %0.f" % float(r._rootState)
The character state at the root is 0


Note:
   1. I removed the loop in your __init__method because it was simply 
making two random choices when one should do.
   2. I assigned the random choice to the class' _rootState attribute, 
not the rootState argument to the __init__ method.  You can also make a 
"rootState" property of the class so that the last line, above, would 
look like:
 >>> print "The character state at the root is %0.f" % float(r.rootState)

I hope that helps.

Eric.


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Share variables between py scripts

[Eric Walstad]

Bennett, Joe wrote:
> Can anyone direct me to some documentation on how to take variables from 
> one py script, execute and pass them to another script? Then I need to 
> get data back from the executed script? I've been searching Google and I 
> see information, but I am in need of some examples...
>  
>  
> Thanks!
> 
> 
> -Joe

Hi Joe,

If possible, just "run" your script (B) from within the second script 
(A).  I'd rephrase that as "in module a, import and use variables, 
functions and/or classes from module b":

# Script B (script_b.py) #

my_b_variable = ''

def my_b_function(var):
 return "Hello World: %s" % var

class MyBClass:
 # an empty class
 pass

 End Script B #

# Script A (script_a.py) #
from script_b import my_b_variable, my_b_function, MyBClass

def my_a_function(var):
 my_a_variable = MyBClass()
 # or
 my_a_variable = my_b_variable
 # or
 my_a_variable = my_b_function(var)
 # etc
 return my_A_variable

if __name__ == '__main__':
 # this A script is being called from the command line
 # do whatever you like in here, like run a function:
 import sys
 command_line_arg = sys.argv[1]
 print my_A_function(command_line_arg)

 End Script A #

$ python script_a.py foo
Hello World: foo

I hope that wasn't too remedial for you and that it's helpful.  Search 
the docs for 'functions' and 'import' for more info.

Best,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] (no subject) (fwd)

[Eric Walstad]

Hey Max,
Danny Yoo wrote:
> [Forwarding to tutor.  Can someone answer Max?  Slightly busy at the
> moment.  Quick note: get him to realize that the list he was playing
> with originally had three elements.  The example in his book has two
> elements. Something has to change.  *grin*]
> 
> 
> -- Forwarded message --
> Date: Fri, 19 Jan 2007 10:10:04 -0800 (PST)
> From: Max Jameson <[EMAIL PROTECTED]>
> To: Danny Yoo <[EMAIL PROTECTED]>
> Subject: Re: [Tutor] (no subject)
> 
[...]
> This is exactly how the tutorial reads (I omited the definition of the
> aList and anoterh):
 aList.append(another)
 print aList
> [42, [1, 2, 7]]
I suspect I didn't see the definition of 'aList' on Alan's website, but
the values look different than what you posted originally.

Based on the output above, I'd say:
aList = [42,]
another = [1, 2, 7]

Whereas your definition, in your original email:
aList = [2,3]
bList = [5,7,9]

Do you see the difference between the two (aside from the different
variable names)?


> I expected to get the third element of the second list printed
In your example, that would be the 'second' item in your bList (because
lists are zero-based):
bList[2] == 9

After 'append'ing bList to aList, aList has three items.  Try this at
the python command line:
print aList[0]
print aList[1]
print aList[2]

Two of those will be integers, One will be a list.

I hope that helps.

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Optimal solution in dealing with huge databases in python

[Eric Walstad]

Alan Gauld wrote:
> "Shadab Sayani" <[EMAIL PROTECTED]> wrote 
> 
>> The data I need to deal with is  in 100s of GB.
>> I am using postgresql backend and SQLALCHEMY ORM.
> 
> All ORMs will introduce a significant performance hit.
> If you really need high speed, and populating a 100G+ database 
> probably is included in that, then you should look at raw SQL.
> In fact for really big data loads most databases provide a 
> C api that goes under the SQL, because even SQL is 
> relatively slow.

I've had this experience, too.  I routinely load a couple million
records from a tab delimited file into our PostgreSQL database with
Python.  My import script:
 1. iterates over the tab file a line/record at a time
 2. does some data validation, some data manipulation
 3. writes an SQL file with the massaged data
 4. calls psql  -f 

Generating the SQL file in python goes amazingly fast, considering the
amount of validation and manipulation (normalizing addresses, generating
hashes, etc) - about 1.5minutes/million records.

The SQL it generates does the usual DB stuff including dropping indexes
and constraints, COPY FROM stdin, regenerate indexes, replace
constraints.  In my experience, psql is MUCH faster at these kinds of
imports than even direct python interaction with psycopg.  Adding an ORM
on top of the database connector (psycopg in my case) slows things down
even more.

As a rough metric, the import I described takes a little over 2 hours on
my 2GHz/1GB laptop.

Good luck,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Optimal solution in dealing with huge databases inpython

[Eric Walstad]

Alan Gauld wrote:
> "Shadab Sayani" <[EMAIL PROTECTED]> wrote 
> 
>> Thank you very much for immediate response.I didnt get
>> the point of loading the data using SQL.What does that
>> mean?
> 
> It means writing a SQL file that can then be run from 
> the database SQL prompt. I don't know what that means 
> in PostGres terms

psql  -f 


> Loadfile.sql would in turn contain lots of SQL commands like:
> 
> INSERT into CUSTOMER 
> VALUES ( 'fred', 'bloggs', 45, '01773-987321');
I think PostgreSQl will wrap each of these INSERT statements into a
transaction, which will add a lot of overhead and slow the process.  I
find the following pattern to result in very fast loads with the psql
command line tool:
"""
ALTER TABLE   DROP CONSTRAINT ;
DROP INDEX 

COPY  ()
FROM stdin;

\.

CREATE INDEX  ON  ();
ALTER TABLE  ADD CONSTRAINT  FOREIGN KEY
() REFERENCES ();
"""
You can wrap parts/all of the above in BEGIN;/COMMIT; if you want them
done in a transaction.


> One thing to note is that due to caching issues you might find 
> it works better if you keep the individual loader files fairly small 
> - say 1000 or so records each. On other databases (DB2 for 
> example) very big data files seem to be faster, it just depends on 
> how the internal SQL engine works.
My SQL files are about 350MB, fwiw.  I haven't tried breaking them down
to smaller files because I haven't read in the PostgreSQL docs, or
forums, that doing so would be helpful.


>> Do have any idea about the C api for Postgresql and
>> some documentation to use it?
I've not used the C interface directly.  I think 'psql' will do what you
want:
man psql
or, just browse the docs online for your db version (psql --version)




Best regards,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Convert my .bat and .vbs to .py

[Eric Walstad]

Hey Mark, welcome aboard!

There are a few different approaches you could take to convert your
scripts.  If most of your scripts are related to copying/moving files,
I'd have a look at Python's shutil module.  I think it'll work in both
Windows and Linux but I don't have a Windows machine handy to test it.



Have fun,

Eric.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Convert my .bat and .vbs to .py

[Eric Walstad]

Hey Mark,
Mark Bystry wrote:
> sh.copy('C:\testing_it.txt', 'D:\')

Also have a look at the os.path module.  Because you want these scripts
to work on both Linux and Windows, using os.path will let you avoid
writing platform specific code to handle your slashes.

For example:
import os
path_parts_list = ['C:', 'testing_it.txt']

Now, os.path.join will join those path parts together with whatever is
appropriate for the OS you are running on:

os.path.join(path_parts_list)

This should give you:
'c:\testing_it.txt' on Windows and
'c:/testing_it.txt' on Linux (which doesn't make sense, but demonstrates
how os.path can handle your os-specific slashes for you).

There are lots of helpful path related built-ins:

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] help me to do my exercise

[Eric Walstad]

In addition to what the others have said, you have a syntax error on line 7 of 
your leap.py file.  The 'for' loop will want an iterable to loop over.  The 
syntax error relates to the list you are trying to use, a slice of sys.argv.  
Remember, lists are denoted by square brackets '[' and ']'.


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python, wxpython and postgresql

[Eric Walstad]

Hey Bill,

Bill Mais wrote:
> I'm a newbie to python and would like to build a database app with 
> wxpython and postgresql.

I would suggest you first write the basic app to work from the command
line.  It's easy to debug that way and enables you to focus on the
critical features of your app without wasting much, if any, of your
time.  Once you are able to do the basics from the command line it is
generally easy to wrap it in wxPython.

As Luke said, there are a lot of examples out there on how to read/write
to a postgresql db.  At some point, probably after you try a code
example or two, you may find the DB API pep helpful:
http://www.python.org/dev/peps/pep-0249/
It defines the interface the db adapters (psycopg, for example) are
supposed to provide you.

We are happy to help when you have specific questions, too.

Best,

Eric.

other keyword pointers include:
optparse (makes handling command line options a breeze)
raw_input (get user input from the command line)
psycopg2 (postgresql database adapter)
doctest (document your code with runnable tests)



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] c++::return

[Eric Walstad]

elis aeris wrote:
> what do I do if i want the program/function to end?
> 
> 
> in c++ of
> 
> int main(){}
> 
> I do a return 0;
What have you tried in Python?  How did it work and what didn't work the 
way you expected?  What Python documentation or tutorials have you read?

Maybe helpful:
http://www.google.com/search?q=python+function+return
http://www.google.com/search?hl=en&q=eric+smart+question


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] c++::return

[Eric Walstad]

elis aeris wrote:
> I seriously have done my homework on writing in python
I wonder if I am misunderstanding your question.  This is a simple 
question about how functions work in Python, right?  Something that most 
every beginning tutorial covers?  You wouldn't be asking here if you had 
done your homework.


> below is a 29 kb 
> .py file that I wrote to do something, although you may not care what it 
> does
Wow, that's one long function!  It looks like either you don't 
understand what you wrote:
 consective_zero = 0
 ## I don't know what this is
or maybe someone else has commented your code?


> please just tell me in plain words:  how do I break function?
And seriously, you'll likely get better answers, faster, when you ask 
better questions.  It's faster, more efficient and more rewarding.  A 
smart question also shows us that you will value the time we spend on 
the answer.  Taking the time to formulate a smart question will often 
help you answer the question on your own.  Actually you did find the 
answer on your own (assuming I understand the question and that you 
wrote the function yourself): 'return'.
http://docs.python.org/tut/node6.html
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] loading modules only when needed and PEP 008

[Eric Walstad]

Hi Tim,
Tim Michelsen wrote:
> Hello fellow Pythonistas,
> I have a question concerning import statements.
...
> it takes a lot 
> of time for a TKinter-GUI to start up. And this start-up time is even 
> longer when matplotlib is imported

> a optimized version would be:
> import sys
> 
> plot_data = 'yes' # option: yes/no
> 
> if plot_data == 'yes':
>   import matplotlib
> else:
>   pass
...
> How would you handle such a case?
> What is recommended in such a case?
> Does anyone have experience with this?
Beware that code later in your module that calls matplotlib.foo() may 
fail if plot_data is not 'yes'.  When I do this sort of thing I like to 
move my imports into my functions/methods.  The 'main' code then 
conditionally calls the function/method.  All the code in the function 
safely assumes that the import has been done but code in the calling 
function assumes the import hasn't been done.

-
if plot_data:
 show_plot_data(mydata)

-
def show_plot_data(data):
 "load matplotlib and show the user the data"
 import matplotlib
 ...do stuff with matplotlib...
-

And as we are talking about style, note that your
else:
 pass
isn't really necessary but it does make it explicitly clear that you are 
choosing not to do anything if the plot_data isn't 'yes'.  Are those 
tabs you're using?  Four spaces are preferred, according to the style guide.

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] loading modules only when needed and PEP 008

[Eric Walstad]

Hey Timmie,
Tim Michelsen wrote:
>> When I do this sort of thing I like
>> to move my imports into my functions/methods. 
> Would this then be compliant with the style recommendations?
Hm, I'm not sure.  I guess that if you consider that somewhere near the
top of the style guide it says something about 'foolish consistency'
than I suppose it does .  I think it's ok to deviate from the pep
when it makes sense to do so.  Define 'makes sense'.  I'm assuming that
importing the matplotlib at the top of your file is making your app
unusably slow; to me that is a great reason to break from the style guide.


>> And as we are talking about style, note that your else: pass isn't
>> really necessary but it does make it explicitly clear that you are 
>> choosing not to do anything if the plot_data isn't 'yes'. 
> Ok, so I already got some style here, at least...
I wouldn't include the 'else' bits, but that's my style.


>> Are those
>>  tabs you're using?  Four spaces are preferred, according to the
>> style guide.
> I always use 4 spaces. This sometimes leads to confusions when working 
> with the same editor on text files that rely tabs but this is another 
> issue...
Sorry.  In my mail reader the indentation looked like tabs.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] CSV file processing...

[Eric Walstad]

Hi Spencer
Spencer Parker wrote:
> I have a 
> piece of software that runs and each iteration is one like.
I don't understand what this means.


 >  It only
> runs once right now; there is only one line + the headers.  I use the 
> csv module to kill the headers and import the one line.
Does 'kill the headers' mean the same as ignoring them in your code?


> The problem 
> is...I need to have it split the csv file at some point. I need to first 
> 20 items taken off and then have the next 7 + the first 20 imported into 
> the database...then have it do this for the next 7 + the first 20...so 
> on and so forth until hits the end of the line.
Define 'item' here.  Is it a field, a character, is it a line?
I'm going to assume that you mean that you want to loop through the file 
line by line, inserting to the database the line's fields in this order:
fields[20], ..., fields[27], fields[0], fields[1], ..., fields[n-1]

This example:
http://dpaste.com/hold/40503/

shows how the csv.reader lets you loop over csv data line by line, each 
line is a list of fields.  Using Python's slicing operations you can 
easily rearrange the line's fields to suit your output needs.

See also:
http://www.diveintopython.org/native_data_types/lists.html

Eric.




___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] CSV file processing...

[Eric Walstad]

Hey Spencer,

Spencer Parker wrote:
> This is why I should not be allowed to write emails at the end of the 
> day.  At least ones that need deep explanation.  Let's try this a second 
> time.
Thanks, this looks better, but...


> This is what I have now...
*What isn't working* the way you want?  Traceback?  I'll guess that you 
don't want to have to hardcode the columns for a fixed number of virtual 
machine data into your sql string...


> try:
> co = db.cursor()
> csv_data = csv.reader(file('output.txt','r'))
> 
> for row in csv_data: print row
> co.execute("""
> INSERT INTO stats (Hostname, Time, Arch, PhysicalCPU, Count, 
> Running, Blocked, Paused, Shutdown, Shutoff, Crashed, Active, Inactive, 
> PCPU, TotalMemory, Totalguestmemory, TotalCPUtime, DomainID, Domainname, 
> CPUTime, PGCPU, RDRQ, WRRQ, RXBY, TXBY)
> VALUES 
> (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s);
> """,row)
> 
> co.close()
Huh?  I was expecting to see 27 fields (20 for the host machine and 7 
for the virtual machine), you have 25 above.  Yes, a traceback or 
explanation of what's going wrong would be helpful.

Other things I'd recommend are:

1. I'm pretty sure that your 'row' above is a list of values. 
Investigate Python's list slicing operator[1] that other responses to 
your first email have used.  Doing so will help you understand...

2. Kent's response to your first email looks great.  Reread it and I 
think you'll find it does what you want (handle a variable number of 
virtual machines and their data).  See also: the range function and it's 
'step' argument[2].

3. Nice that you are sending the 'row' list to the database adapter and 
letting it escape the values.  In your case I don't think you have to 
worry about SQL injection, but it's a good habit to get into if you ever 
need to stuff end-user data into a database.

hth,

Eric.
[1] 
http://www.diveintopython.org/native_data_types/lists.html#odbchelper.list.3.1
[2] http://docs.python.org/tut/node6.html#SECTION00630
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] rsync python script

[Eric Walstad]

Kaushal Shriyan wrote:
> Hi
> 
> [EMAIL PROTECTED] 
> 0 18 * * * rsync -av /var/lib/mysql [EMAIL PROTECTED]:/var/lib/
> 
> If i put this two lines in crontab it will run correctly,My requirement 
> was to create a python script, this python script should indicate 
> success or failures and the reason for failure
It looks like rsync can output to a text file.  Python makes text 
parsing a breeze.  Have you defined 'indicate', 'success' and 'failure' 
yet?  When will your python script run?  Who/What will run it?
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Saving an Image in a Given Format

[Eric Walstad]

Wayne Watson wrote:
> current_image ... convert() and 
> save() must be methods in that class. The question is what class/module? 
> PIL?
I don't know, but you could try the following from the python command 
line, which might give you some hints:

dir(current_image)
current_image.foobarfizzbang


 >>> import Image  # PIL Image class
 >>> im = Image.open('/path/to/foo.jpg')

 >>> dir(im)  # Gives you a list of the object's attribute names
['_Image__transformer', '__doc__', '__getattr__', '__init__', 
'__module__', '_copy', '_dump', '_expand', '_getexif', '_makeself', 
'_new', '_open', 'app', 'applist', 'bits', 'category', 'convert', 
'copy', 'crop', 'decoderconfig', 'decodermaxblock', 'draft', 'filename', 
'filter', 'format', 'format_description', 'fp', 'fromstring', 
'getbands', 'getbbox', 'getcolors', 'getdata', 'getextrema', 'getim', 
'getpalette', 'getpixel', 'getprojection', 'histogram', 'huffman_ac', 
'huffman_dc', 'im', 'info', 'layer', 'layers', 'load', 'load_djpeg', 
'load_end', 'load_prepare', 'mode', 'offset', 'palette', 'paste', 
'point', 'putalpha', 'putdata', 'putpalette', 'putpixel', 
'quantization', 'quantize', 'readonly', 'resize', 'rotate', 'save', 
'seek', 'show', 'size', 'split', 'tell', 'thumbnail', 'tile', 
'tobitmap', 'tostring', 'transform', 'transpose', 'verify']


The following error message tells me the object came from PIL.Image, a 
good hint.

 >>> im.just_throw_me_an_error
Traceback (most recent call last):
   File "/home/ewalstad/var/tmp/", line 1, in 
   File "/usr/lib/python2.5/site-packages/PIL/Image.py", line 493, in 
__getattr__
 raise AttributeError(name)
AttributeError: just_throw_me_an_error
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Newbie: Lost in Loop

[Eric Walstad]

Hi Yogi, welcome to Python!

yogi wrote:
...
 > if (str(gen) == str(row[0])):
 > print 'Match for 16 found
Is the conversion to string really necessary?  Even if it is, do it once 
for gen, when you define it:
gen = '16'
so you don't have to convert on every iteration of the loop.  I think 
you want gen to be a integer, though, because you later compare it to 
other integers in your range().


...
> for row[2] in range (rval0l,rval0u):
> print row

I don't think this bit is doing what you think it is.  For each 
iteration you are assigning a value to the third element in the row list:
   row[2] = 6009890
   row[2] = 6009891
   ...
   row[2] = 6009938
etc.
I don't think you want to loop over the range but instead want to check 
to see if row[2] is between rval0l and rval0u.  You can do that by using

if x in some_list

which will return true if x is a member of the list.  range() will give 
you the list of values to check against (but only create that list once 
unless it changes on each row you are processing and in your code 
example, it's not changing).  Remember, it's easy and enlightening to 
test code on the Python command line:
 >>> rval0l = 6009890
 >>> rval0u = 6009939
 >>> range(rval0l,rval0u)
[6009890, 6009891, 6009892, 6009893, 6009894, 6009895, 6009896, 6009897, 
6009898, 6009899, 6009900, 6009901, 6009902, 6009903, 6009904, 6009905, 
6009906, 6009907, 6009908, 6009909, 6009910, 6009911, 6009912, 6009913, 
6009914, 6009915, 6009916, 6009917, 6009918, 6009919, 6009920, 6009921, 
6009922, 6009923, 6009924, 6009925, 6009926, 6009927, 6009928, 6009929, 
6009930, 6009931, 6009932, 6009933, 6009934, 6009935, 6009936, 6009937, 
6009938]
 >>> my_range = range(rval0l,rval0u)
 >>> 6009893 in my_range
True
 >>> 5 in my_range
False


gen = 16
idx_snp = 2
my_range = range(rval0l,rval0u)
for row in file:
 snp = int(row[idx_snp])
 if snp == gen:
   print 'Found %s' % gen
   if snp in my_range:
   print "Found match:", row


Eric.
PS, if your input file is as simple as your example, the csv module 
isn't getting you much benefit/complexity:

for row in open("divs.map"):
 fields = row.split('\t')
 print fields


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Diff for Python

[Eric Walstad]

And my whoops, I should have sent my first one to the list, too.

I don't run Windows very often.  I think 'WinDiff' is what I used
there.  Have you tried that?

There's always cygwin, too.

Eric.
On Sat, Apr 5, 2008 at 12:28 PM, Wayne Watson
<[EMAIL PROTECTED]> wrote:
>
>  Whoop, I should have specified I'm on Win XP.
>
>
>  Eric Walstad wrote:
>  Hi Wayne,
>
> On Fri, Apr 4, 2008 at 8:37 PM, Wayne Watson
> <[EMAIL PROTECTED]> wrote:
>
>
>  Is there a Linux diff-like command for Python code? I'd like to see the
>  difference between two py files.
>
>  Why don't you just use diff?
> What OS are you on?
>
> diff -Bu fileone.py filezero.py
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Code optmisation

[Eric Walstad]

Hi Yogi

On Fri, Apr 4, 2008 at 10:05 PM, yogi <[EMAIL PROTECTED]> wrote:
> Hi ,
>Here is my first usable Python code.
>  The code works.
Woohoo! congratulations.


>  Here is what I'm trying to do.
>  I have two huge text files. After some processing, One is 12M  (file A) and 
> the other 1M (file B) .
>  The files have columns which are of interest to me.
...
>  Question1 : Is there a better way ?
I admit that I didn't spend too much time trying to understand your
code.  But at first glance your logic looks like it could be easily
represented in SQL.  I bet a relational database could do your lookup
faster than doing it in pure Python.  I do this kind of thing
frequently: use python to import delimited data into a relational
database like PostgreSQL, add indexes where they make sense, query the
database for the results.  It can all be done from inside Python but
it doesn't have to be.

SELECT a.*
FROM a INNER JOIN b ON a.field0=b.field0
WHERE
  b.field3=0
  AND
  a.field3 >= (b.field1-100) AND a.field3 <= (b.field2+101)
... etc.


>  Question2 : For now I'm using shells time  call  for calculating time 
> required. Does Python provide a more fine grained check.
I think so but I've not used it: timeit.  Search this mailing list's
archives for 'timeit' and/or at the Python command line:
import timeit
help(timeit)


>  Question 2: If I have convert this code into a function.
>  Should I ?
Yes, especially if it helps make your code easier to read and understand.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Doubts about Pylint

[Eric Walstad]

On Wed, Apr 9, 2008 at 9:43 AM, Dick Moores <[EMAIL PROTECTED]> wrote:
> I'd never used Pylint until yesterday
...
>  Since when is 'az' a bad variable name? And 'AZ' is OK?
...
>  Comments?
I understand that Pylint settings and output are *very* customizable.
I seem to remember talk about a PEP['Style Guilde'] config but I don't
know if that ever happened.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python, CGI and CSS

[Eric Walstad]

Hi Alex,
On Thu, Apr 10, 2008 at 1:22 PM, Alex Krycek <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I've looked all over the internet but have not found an answer to my
> question. How do I apply an external stylesheet to the XHTML in a Python
> script? I tried to include the standard " type='text/css' href='style1.css' />"" in with the rest of the printed
> XHTML.
Try writing your html file by hand, tweaking the href and/or server
config and/or file permissions by hand until it *does* work.  Once you
have a working reference point, then make your Python script duplicate
what you know works.


> I
> tried other cgi scripts (still residing in the cgi-bin) and they did work.
View the HTML source of these working CGI scripts to see what the css
link tag looks like.  How does it differ from the css link tag output
of your Python script?


> I
> checked the error log, and this is what I'm getting: (2)No such file or
> directory.
Paste the actual log file line here.


> When I took both files out of my cgi-bin, as a test, the CSS rules were
> implemented. I'm not sure why it stops working when located in the cgi-bin
> folder.
It feels like a permissions problem or a configuration problem with
your web server.


What webserver are you using?
Have you followed any of the CGI tutorials?
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python, CGI and CSS

[Eric Walstad]

Hey Alex,
On Thu, Apr 10, 2008 at 1:43 PM, Alex Krycek <[EMAIL PROTECTED]> wrote:
> Eric,
>
> I'm sorry, I never actually managed to get any stylesheets to work with my
> cgi scripts. The scripts that did work are just simple HTML. I'm using
> Apache 2.2.8 on a Mac OS X 10.4.11 system.
If I understand your problem, you are expecting to see your stylesheet
loaded but that is not happening.  If that's what you want to fix,
then I suggest you fix it in as simple a test case as possible.  Try
browsing directly to the css file with your web browser.  When you
have your web server configured correctly you should see the contents
of your css displayed in the web browser.  If you can't view your css
file in your browser, then send us the apache log file line that shows
the failed GET request and we can go from there.

Keep the scripting and cgi stuff out of the equation until you get this working.
Also, reread the apache FAQ.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Conventions for code snippets for debugging and testing?

[Eric Walstad]

Hi Robert

On Thu, Apr 10, 2008 at 1:34 PM, Robert Kirkpatrick <[EMAIL PROTECTED]> wrote:
> Hi All,
>
>  Just wondering if there are any basic conventions for including code
>  snippets that are for testing / debugging only?
>
>  For example, you could set a boolean variable called DEBUG, then have
>  snippets of code like:
>
>  if DEBUG:
> do stuff
>  else:
> do otherstuff
I'd do something like this:

from settings import DEBUG

def my_user_list()
# assumes long_list is already populated (db, file, etc)
if DEBUG:
return long_list[:5]
else:
return long_list

def process(user):
# do the work on the user here
pass

def main():
# ...
[process(user) for user in my_user_list()]

The idea is to encapsulate the retrieval of the user list and let that
encapsulation hide the DEBUG logic.  It could be done with a function,
like above, or with a MyUserList class.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor