[Tutor] authentication again a crm web application

[linuxian iandsd]

hi everyone,
I need to get pass throught an authentication form (username & password) to
access our client CRM web application automaticaly, which means at some
intervals of time i need to log in to thier web site download a specific
table parse it & then load it into my mysql database.
the parsing & loading part is already done, the part about authenticating is
still in progress, any help would be very appreciated.
the site uses https.

thanks in advance.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Correct way to call an outside program?

[linuxian iandsd]

simplest way to run external commands !

import os
cmd="/usr/bin/ssh 10.0.0.20 uptime"
os.popen(cmd)

my cmd is just an example, use any cmd you want & its output will be
displayed to you.
hope this helps




On Thu, Mar 13, 2008 at 12:05 PM, Kent Johnson <[EMAIL PROTECTED]> wrote:

> Allen Fowler wrote:
> > Hello,
> >
> > I need to call an external command line .exe utility from my Python
> script.
> >
> > What is the best way to capture the output (if any) and (optionally)
> direct it to my normal standard output?
>
> subprocess.Popen().communicate() will do it:
>
> In [1]: import subprocess
> In [7]: x=subprocess.Popen('ls', stdout=subprocess.PIPE,
> stderr=subprocess.PIPE).communicate()
> In [10]: print x[0]
> ...
>
> If you just want stdout and stderr of the subprocess to go to stdout and
> stderr of the calling process you can omit those arguments to Popen().
>
> Kent
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to get response from os.system()

[linuxian iandsd]

use os.popen("your cmd here")

On Sun, Mar 16, 2008 at 8:06 PM, Luke Paireepinart <[EMAIL PROTECTED]>
wrote:

> shawn bright wrote:
> > Lo there all,
> >
> >  I am needing to get a response back from a system command.
> >  i can do this:
> >
> >  os.system('mailq | wc -l")
> >
> >  if i do this in the terminal  mailq | wc -l   ,   it will spit out a
> number.
> > How do i get that number as a python variable ?
> You need to use the subprocess module, specifically subprocess.Popen, to
> open an input pipe from the command so you can read() the data in.
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to get response from os.system()

[linuxian iandsd]

i guess that was too short tutorial well, here is an example from a live
session on the interpreter.

[EMAIL PROTECTED] ~]# python
Python 2.5.1 (r251:54863, Nov 23 2007, 16:16:53)
[GCC 4.1.1 20070105 (Red Hat 4.1.1-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> cmd='ping -c 1 localhost'
>>> import os
>>> a=os.popen(cmd)
>>> print a

>>> print a.read()
PING serv.kontactel.loc (127.0.0.1) 56(84) bytes of data.
64 bytes from serv.kontactel.loc (127.0.0.1): icmp_seq=1 ttl=64 time=0.134ms

--- serv.kontactel.loc ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.134/0.134/0.134/0.000 ms

>>>

i hope you can understand that now u have the output in a variable called
"a" that you can parse & use as you wish.




On Sun, Mar 16, 2008 at 7:12 PM, linuxian iandsd <[EMAIL PROTECTED]>
wrote:

> use os.popen("your cmd here")
>
>
> On Sun, Mar 16, 2008 at 8:06 PM, Luke Paireepinart <[EMAIL PROTECTED]>
> wrote:
>
> > shawn bright wrote:
> > > Lo there all,
> > >
> > >  I am needing to get a response back from a system command.
> > >  i can do this:
> > >
> > >  os.system('mailq | wc -l")
> > >
> > >  if i do this in the terminal  mailq | wc -l   ,   it will spit out a
> > number.
> > > How do i get that number as a python variable ?
> > You need to use the subprocess module, specifically subprocess.Popen, to
> > open an input pipe from the command so you can read() the data in.
> >
> > ___
> > Tutor maillist  -  Tutor@python.org
> > http://mail.python.org/mailman/listinfo/tutor
> >
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] login

[linuxian iandsd]

well, you first get to the authenticating page, that is where you put your
username & pass ! & then you post them to be authenticated by server which
then sends you a session id, which is a cookie.

now to make the server think you're loged in to the site you have to send
him back the cookie.

now to send him the cookie you need the cookielib

so lets do it right now :


import urllib, urllib2, cookielib
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
url='http(s)://www.website_to_treat_nicely_and_not_abuse_using_python_scripts.com'
cj=cookielib.LWPCookieJar()
values={'username':'python_user', 'passwd':'elephant'}

# let me explain here what values stand for :
#  values will represent every one element of a form that is in the
authentication page.
#  yes you got it what is between the  and  & there might be
more
# so you ll have to add them.
#

headers={'User-Agent' : user_agent}
opener=urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)

# install_opener ?  what the hell is this ? ???
# well, now every request will take with it the cookie you got from server
# in other words you will be logged-in to this website from now on
#

data=urllib.urlencode(values)

# this is where actin starts (login page here)

req=urllib2.Request(url, data, headers)
response=urllib2.urlopen(req)


url2='http(s)://www.website_to_treat_nicely_and_not_abuse_using_python_scripts.com/second_page'
response2=urllib2.urlopen(url2)
response2.read()   #you can get the page because urlopen manages the
session/cookie for you for free.

url3='http(s)://www.website_to_treat_nicely_and_not_abuse_using_python_scripts.com/second_page'
response3=urllib2.urlopen(url3)
response3.read()


well, hope this helps
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] parse emails as they come in

[linuxian iandsd]

good morning everybody !

I have scripted a small program to parse a 5 lines email message as it comes
in to my inbox (this is handled by procmail & here is a wonderful intro to
it : http://linuxfocus.org/English/November1997/article8.html)

so every email is being parsed & information is extracted from it.

but sometimes two or more emails come in at once so the input file that my
python script has to parse is more than five lines !! my question is how do
i effeciently manage this from within my original script.

thanks
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] parse emails as they come in

[linuxian iandsd]

well, my script is so simple ! nothing complicated

#!/usr/bin/python
#

import re, sys

a=open('/home/john/data/file_input.tmp', 'r')
b=open('/home/john/data/file_output', 'w')

aa=a.readlines()
n=0
for L in aa:
# I split every line because i only need what's after the ":"
# the email comes in the form "field : value"  in  17 lines
 La=L.split(':')
 n=n+1
# 18 is the last line & it is an empty line that comes with every email
# so i quit there.
 if n==18:
  sys.exit()
# second line is a time value like this one "18:20:45"
# i don't need the ":" but i need the numbers
 elif n==2:
# as usual i remove the \n & put a ; in its place and that happens
# at the third element
  La3=re.sub('\n',';',La[3])
# i gather the time value also there is no need for : in between
  La123=La[1]+La[2]+La3
  b.write(La123)
# any other line is treated equaly like this
# i only replace \n by ;
 else:
  La1=re.sub('\n',';',La[1])
  b.write(La1)

# a little secret : this little script helps me load data from mail to a
mysql database by converting it into ; separated values :)




On Tue, Apr 1, 2008 at 2:09 AM, Chris Fuller <[EMAIL PROTECTED]>
wrote:

> Every five lines of the raw email, the headers, or the body?
>
> A text file is just data.  You can navigate however you like, but you need
> to
> choose a model, to give it some structure for you to work with,
>  Navigating
> around at the byte level is probably going to be tedious, error prone, and
> not very useful anyway.  Choosing five lines at a time is probably not
> going
> to be much better.  There's no particular reason it can't be ten lines, or
> two, unless you pick a model that
>
> Maybe we could help more if you showed us what this "original script" is.
>  We
> can help you pick a better model if the one implicit in your script isn't
> working for you.
>
> Also, you should probably reply to the mailing list.
> I'll be more careful about the reply-to field from now on.
>
> Cheers
>
>
> On Monday 31 March 2008 14:43, you wrote:
> > the mail module seems interesting.  but what  I was thinking of some way
> > that would work on only five lines then moves to the next five lines &
> so
> > on ... is that possible ? is there a way of navigating a text file ?
> > process the line that we want,  maybe delete it or maybe add text to it
> &
> > then save & close the file ?
> >
> >
> > On Fri, Mar 28, 2008 at 1:08 PM, Chris Fuller
> > <[EMAIL PROTECTED]>
> >
> > wrote:
> > > The email and mailbox modules might help you out.  Multiple email
> > > messages will probably parse as an mbox format mailbox.
> > >
> > > http://docs.python.org/lib/module-email.html
> > > http://docs.python.org/lib/module-mailbox.html
> > >
> > > Cheers
> > >
> > > On Friday 28 March 2008 03:14, linuxian iandsd wrote:
> > > > good morning everybody !
> > > >
> > > > I have scripted a small program to parse a 5 lines email message as
> it
> > > > comes in to my inbox (this is handled by procmail & here is a
> wonderful
> > > > intro to it :
> http://linuxfocus.org/English/November1997/article8.html)
> > > >
> > > > so every email is being parsed & information is extracted from it.
> > > >
> > > > but sometimes two or more emails come in at once so the input file
> that
> > >
> > > my
> > >
> > > > python script has to parse is more than five lines !! my question is
> > > > how
> > >
> > > do
> > >
> > > > i effeciently manage this from within my original script.
> > > >
> > > > thanks
> > >
> > > ___
> > > Tutor maillist  -  Tutor@python.org
> > > http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] parse emails as they come in

[linuxian iandsd]

ok - as i mentioned in my first email i use procmail to put THE BODY of all
incoming mail into a file (that is one per incoming email as i use the
variable $date-$time in the name).

now this file can contain only one email but it can also contain 2 or more
(this happens if for example there is a dns problem in the internet, so mail
can't make it, but once internet recovers from the dns problem mail rushes
in & we may have multiple messages per file. this is also true is i do this
cmd :
for i in 1 2 3 4 5 6 ; do echo $i | mail -s 'test mail'  john ; done

well, this file is processed by my script to extract data.

the problem is : i can parse 18 lines (that is one email per file) fine, but
i need suggestions in parsing it when it contains two emails (that is 18
lines + 18 lines )

i hope i have explained my problem well this time.

i will make the optimizations you told me (directly inserting data into
mysql & the lines loop as well)

thanks a lot.


On Tue, Apr 1, 2008 at 9:17 PM, Steve Willoughby <[EMAIL PROTECTED]> wrote:

> On Tue, Apr 01, 2008 at 09:07:04PM +, linuxian iandsd wrote:
> > a=open('/home/john/data/file_input.tmp', 'r')
> > b=open('/home/john/data/file_output', 'w')
>
> This is collecting mail as it comes in?  If you have a mail
> rule in place to dump mail into this file_input.tmp file,
> you could run into trouble if multiple messages arrive close
> enough together that you get a race condition.
>
> I'd suggest just using something like procmail to invoke
> your Python script directly on the incoming message, so
> you don't have to dump it to a temporary input file.
> You'll be guaranteed to see one and only one mail per
> invocation of your script (although it may invoke
> several copies of your script at the same time, so plan
> for that, e.g., don't write to the same output filename
> every time--or don't write to a file at all, just have
> your script put the data into MySQL or whatever directly).
>
> > aa=a.readlines()
> > n=0
> > for L in aa:
>
> Generally speaking, it's better to let Python iterate
> through the lines of a file.  The above code sucks in
> the entire (possibly huge) file into memory and then
> iterates over that list.  Better:
>
> for L in a:
>
> or better yet:
>
> for lines in input_file:
>
> > # a little secret : this little script helps me load data from mail to a
> > mysql database by converting it into ; separated values :)
>
> I'd look at just gathering the raw data into Python variables and then
> connecting to MySQL directly and executing a SQL statement to import the
> data straight in.  You'll avoid a host of problems with properly quoting
> data (what if a ';' is in one of the data fields?), as well as making it
> unnecessary to carry out another post-processing step of gathering this
> script's output and stuffing it into MySQL.
>
> --
> Steve Willoughby|  Using billion-dollar satellites
> [EMAIL PROTECTED]   |  to hunt for Tupperware.
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] parse emails as they come in

[linuxian iandsd]

well, here is a piece of final script :

#!/usr/bin/python
#

import sys

b=[]
while 1:
 data = sys.stdin.readline()
 if data != '\n':
  b.append(data)
 else:
  break

for i in (0,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16):
 b[i]=b[i].split(':')[1].strip()
 #print b[i]

b[1]=b[1].split(':')
b[1]=b[1][1]+b[1][2]+b[1][3].strip()
#print b[1][1]+b[1][2]+b[1][3].strip()

bb=",".join(b)
print bb

mysqlcmd='insert into webdata field1, field2, field3, field4, field5,
field6, field7 . field17 values (%s)' % bb

print mysqlcmd

END OF SCRIPT


the parsed file looks like this :

field1: x
field2: x

field17: x
empty line here for end of message


On Wed, Apr 2, 2008 at 9:50 AM, Steve Willoughby <[EMAIL PROTECTED]> wrote:

> linuxian iandsd wrote:
>
> > well, i don't know how to pipe the file to my script !!
> >
>
> It's how procmail works.  Presuming you looked up how to write
> a procmail rule to save the body of your mail into a file, you
> should also see right next to it the instructions for piping
> the message to a program.  (It also can be found by a quick
> Google search, for future reference if you want a quick answer.)
>
> You just put a "|" symbol in front of the script name.
>
> To save the message to a file, you'd say something like
>
> :0:
> *Subject:.*pattern to look for
> /home/me/temp_input_file$date$time
>
> To pipe it to your script, you'd say something like
>
> :0
> *Subject:.*pattern to look for
> |/home/me/scriptname
>
> For more information see procmailrc(5) and procmailex(5).
>
> Your Python script will see the message input on stdin.
>
>
>
> > On Wed, Apr 2, 2008 at 7:18 AM, Steve Willoughby <[EMAIL PROTECTED]>
> > wrote:
> >
> >  linuxian iandsd wrote:
> > >
> > > > ok - as i mentioned in my first email i use procmail to put THE BODY
> > > > of
> > > >
> > > all
> > >
> > > > incoming mail into a file (that is one per incoming email as i use
> > > > the
> > > > variable $date-$time in the name).
> > > >
> > > > now this file can contain only one email but it can also contain 2
> > > > or
> > > >
> > > more
> > >
> > > > (this happens if for example there is a dns problem in the internet,
> > > > so
> > > >
> > > mail
> > >
> > > > can't make it, but once internet recovers from the dns problem mail
> > > >
> > > rushes
> > >
> > > > in & we may have multiple messages per file. this is also true is i
> > > > do
> > > >
> > > this
> > >
> > > Using $date-$time is insufficient since I'll wager a dozen doughnuts
> > > that the resolution of $time isn't small enough compared to the speed
> > > messages can arrive.
> > >
> > > But as I tried to explain in my previous mail, this is a problem you
> > > don't have to solve.  By choosing to use procmail to dump a file with
> > > a non-unique name, you create a race condition you then have to deal
> > > with in your code.
> > >
> > > If, on the other hand, you use procmail to _filter_ the message
> > > through your script, this cannot possibly happen.  You'll get an
> > > invocation of your script per message every time.  If you have
> > > your script directly dump the data into MySQL you never need to
> > > write any disk files at all.
> > >
> > >
> > >
> >
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] socket / over network

[linuxian iandsd]

re-inventing the wheel ?

http://www.howtoforge.com/linux_backuppc

>
>
> On Thu, Apr 3, 2008 at 3:44 PM, Alan Gauld <[EMAIL PROTECTED]>
> wrote:
>
> > "Nathan McBride" <[EMAIL PROTECTED]> wrote
> >
> > Hi Nathan,
> >
> > Please don't reply to an existing message to start a new discussion.
> > It messes up those of us using threaded mail/news readers and
> > increases the likelihood that your message will be missed.
> >
> > > I'm pretty tired of the lame backup solution we have at work.
> > > Could anyone point me to a (more or less newbieish) example of how
> > > to
> > > have python open a socket on one box and get data from it, then have
> > > another
> > > box write to it over the network?
> >
> > For a very simple example of using a socket you could try the
> > Network Programming topic in my tutorial.
> >
> > There is also a HowTo or Topic guide on the Python web site
> > that gives a more detailed example.
> >
> > That having been said, backups are usually best done using
> > OS tools or if you must roll your own then using ftp or similar
> > as a file transfer mechanism rather than trying to send a
> > bytestream over a socket. ftp can handle broken connections
> > etc more easily. Detecting and fixing errors over a socket
> > stream is non trivial and for backups is pretty much essential!!
> >
> > --
> > Alan Gauld
> > Author of the Learn to Program web site
> > http://www.freenetpages.co.uk/hp/alan.gauld
> >
> >
> > ___
> > Tutor maillist  -  Tutor@python.org
> > http://mail.python.org/mailman/listinfo/tutor
> >
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Don't miss "Python-by-example - new online guide to Python Standard Library"

[linuxian iandsd]

well, good to see something like this comming ... i used to browse for hours
to find nothing but long long documentation & not a single real life
example... but i guess its because i m used to the old  'howto'  way of
ducumentation ...


On Fri, Apr 4, 2008 at 2:12 PM, Dick Moores <[EMAIL PROTECTED]> wrote:

>  A new one-man project, just getting started.
> < http://www.lightbird.net/py-by-example/>
>
> This guide aims to show examples of use of all Python Library Reference
> functions, methods and classes. At this point, only the more widely used
> modules were added and only functions use examples are given. Python version
> 2.5 was used for examples unless noted otherwise. Example of output of a
> function is shown in the form of function_call(args) # [result]:
>
> math.sqrt(9)   # 3.0
>
>
> See the thread in the python-list 
> archive:
>
> Dick Moores
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] socket / over network

[linuxian iandsd]

in case it helps here is a very basic  example:

import MySQLdb, glob, os, re, shutil
from ftplib import FTP

a=file_to_fetch
ftp=FTP('ftp_server')
  ftp.login('user_name','password')
  try:
   aa=ftp.nlst(a)
   b='/home/a'
   bb=os.path.basename(aa[0])
   e=os.path.basename(b)
   c=open(b, 'wb')
   ftp.retrbinary('RETR '+aa[0], c.write)
   c.close()

well u just copied some pieces of my own code to maybe help you get started
with ftp as you maybe don't know that you have to open a file for writing &
then write into it the stream from ftp retrieve cmd.


On Mon, Apr 7, 2008 at 7:16 AM, Alan Gauld <[EMAIL PROTECTED]>
wrote:

>
> "Nathan McBride" <[EMAIL PROTECTED]> wrote
>
> > Going off of wha tyou said, if I choose to use ftp, is there a way i
> > could do everything from within python including the server to get
> > the
> > files?  Is there like a ftp module for python to help in the passing
> > of
> > the files between the computers?
>
> Yes, there is an ftp module in the standard library.
>
> Alan G
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Tutor Digest, Vol 50, Issue 9

[linuxian iandsd]

On Wed, Apr 9, 2008 at 12:59 PM, rui <[EMAIL PROTECTED]> wrote:

> Hi Gloom,
>
>
>
> You should give a look at the method "split" (of the string objects) and
> int.
>
> The first is used do break a string into smaller pieces and the other to
> convert a string to an int object, raising an exception when it is not
> possible.
>
>
> On Wed, Apr 9, 2008 at 9:29 AM, Gloom Demon <[EMAIL PROTECTED]> wrote:
>
> > Hello :-)
> >
> > Can someone please explain to me ho can I find out how many elements are
> > there in one record of a list?
> >
> > The problem is as follows:
> >
> > I have a txt file from which I read data into Python.
> >
> > The file looks something like this:
> >
> > 01 bla bla bla 23,15 2345,67
> > 02 alb alb 2,4 890,1
> > 03 bal bla alb lab 567,12345 87,45
> > 
> >
> > I need to be able to discriminate the string
> > parts from the numeric ones.
> >
> > Since the number of words in the file can vary, I have to be able to find 
> > out when they are finished
> > and when the floats come in
> >
> > mystring[0]-> always integer
> > mystring[1]-> string (word)
> > mystring[1-X]-> last string (word)
> > mystring[X+1]-> always float
> > mystring[X+2]-> always float
> >
> > it would have been nice if I could find out the total number of the
> > fields in one list record so that I could then adress them via a variable.
> >
> > ___
> > Tutor maillist  -  Tutor@python.org
> > http://mail.python.org/mailman/listinfo/tutor
> >
> >
>
>
> --
> Ruivaldo Neto
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>

my guess would be something like this :

a=open('/home/some_file.txt')
for line in a:
  tmp_list_1=line.split()

  num_floats=0
  num_strings=0
   for i in tmp_list_1:
if type(i) == int:
 num_floats=num_floats+1
else:
 num_strings=num_strings+1

if you explain more you case maybe i can get more ideas - hope this helps
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List comprehensions

[linuxian iandsd]

On Wed, Apr 9, 2008 at 7:44 PM, Jerry Hill <[EMAIL PROTECTED]> wrote:

> On Wed, Apr 9, 2008 at 7:12 AM, Dinesh B Vadhia
> <[EMAIL PROTECTED]> wrote:
> > I want to replace the for loop with a List Comrehension (or whatever) to
> > improve performance (as the data list will be >10,000].  At each stage
> of
> > the for loop I want to print the result ie.
>
> List comprehensions are for building lists, not consuming them.  If
> you want to do something with every element in a list, other than
> building a new list with it, you should be using a for loop.
>
> > [print (item + "\n")  for item in data]
> >
> > But, this doesn't work as the inclusion of the print causes an invalid
> > syntax error.
> >
> > Any thoughts?
>
> Don't do this.
>
> Perhaps you could share a small piece of code that you think is too
> slow, and ask for advice in speeding it up?  If you're not sure which
> small pieces of code are too slow, you need to profile your
> application to find out.  See the documentation for python's profile
> module.  If you don't have enough code written to profile, then it's
> probably too early to be doing these optimizations.
>
> --
> Jerry
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>


if you explain the source of the list, what do you want to change in it,
what destination will it take, i m sure the guys here will help a lot.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List comprehensions

[linuxian iandsd]

i can't think of anything but a loop here UNLESS you take the list from its
source one element at a time, process it & then print the result.

example of this would be :

 list comes in from standard input.
 list comes from a database
 list is read from a file.

so again where the list comes from is important.

if its standard input then you program will be easy & won't use any memory i
guess.

import sys   # cgi & cgitb if going web
data = sys.stdin.readline()

print data




On Wed, Apr 9, 2008 at 8:15 PM, Dinesh B Vadhia <[EMAIL PROTECTED]>
wrote:

>  Sorry, let's start again.
>
>  Here is a for loop operating on a list of string items:
>
> data = ["string 1", "string 2", "string 3", "string 4", "string 5",
> "string 6", "string 7", "string 8", "string 9", "string 10", "string 11"]
>
> result = ""
> for item in data:
> result =  item
> print result
>
> I want to replace the for loop with another structure to improve
> performance (as the data list will contain >10,000 string items].  At each
> iteration of the for loop the result is printed (in fact, the result is sent
> from the server to a browser one result line at a time)
>
> The for loop will be called continuously and this is another reason to
> look for a potentially better structure preferably a built-in.
>
> Hope this makes sense!  Thank-you.
>
> Dinesh
>
>
>
> - Original Message - *From:* Kent Johnson <[EMAIL PROTECTED]>
> *To:* Dinesh B Vadhia <[EMAIL PROTECTED]>
> *Cc:* tutor@python.org
> *Sent:* Wednesday, April 09, 2008 12:40 PM
> *Subject:* Re: [Tutor] List comprehensions
>
> Dinesh B Vadhia wrote:
> > Here is a for loop operating on a list of string items:
> >
> > data = ["string 1", "string 2", "string 3", "string 4", "string 5",
> > "string 6", "string 7", "string 8", "string 9", "string 10", "string
> 11"]
> >
> > result = ""
> > for item in data:
> > result = item + "\n"
> > print result
>
> I'm not sure what your goal is here. Do you mean to be accumulating all
> the values in data into result? Your sample code does not do that.
>
> > I want to replace the for loop with a List Comrehension (or whatever) to
>
> > improve performance (as the data list will be >10,000].  At each stage
> > of the for loop I want to print the result ie.
> >
> > [print (item + "\n")  for item in data]
> >
> > But, this doesn't work as the inclusion of the print causes an invalid
> > syntax error.
>
> You can't include a statement in a list comprehension. Anyway the time
> taken to print will swamp any advantage you get from the list comp.
>
> If you just want to print the items, a simple loop will do it:
>
> for item in data:
>print item + '\n'
>
> Note this will double-space the output since print already adds a newline.
>
> If you want to create a string with all the items with following
> newlines, the classic way to do this is to build a list and then join
> it. To do it with the print included, try
>
> result = []
> for item in data:
>newItem = item + '\n'
>print newItem
>result.append(newItem)
> result = ''.join(result)
>
> Kent
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List comprehensions

[linuxian iandsd]

also if you need to go for 2 results I propose you use filters &
interactive menus which will help you tailor the query to the users desires
& thus limit the query results.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Copy script

[linuxian iandsd]

could you do a : "dir /b" inside this directory just so that we can know the
real file names.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List comprehensions

[linuxian iandsd]

i think you are using ajax ... which undoubdetly uses an sql database since
its based on queries run from whithin the application in the browser
whithout the need for refreshing the page ... i would suggest you try
serching internet for something like  "google autocomplete feature" & i
guess the queries are also no that long they have a limit of the results ...
for example not more than 20 results per query.

than said there would be no loop. just a query (with a limit of 20 rersults)
each time a use inputs a character.

hope this helps.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] queries from the web & random number

[linuxian iandsd]

I have been used to making queries to websites automatically & analizing the
results. a few minutes ago ... i can no longer do that
now a website is protected by a ramdom number that i have to enter before i
can login.
its a picture with numbers on it that i have to type in in the
authentication form before login in.

i use urllib2.install_opener(opener) to keep the kookies & work on other
pages of the website using response=urllib2.urlopen(req)

now i need to at least be able to download this very first picture that has
the random number on it to my hard disk so that i can look at it & feed my
script with that number automatically. is that possible ? its a picture
thats generated on the fly  i guess & it comes only at the login page.

i mean launch script, look at the picture, feed taht number to my script
from the prompt & hit enter & it would go on as usual.

thanks
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] queries from the web & random number

[linuxian iandsd]

I modified old script - added

get_img=urllib2.urlopen('http://website.com/jcaptcha.jpg')
> secret_img=open('/home/iandsd/secret_img.jpg','wb')
> secret_img.write(get_img.read())
> secret_img.close()
>

and then to add the value to the dictionary :

values['captcha']=raw_input("Enter the number you see on picture  :")
>

data=urllib.urlencode(values)
> req=urllib2.Request(url, data, headers)
>
>
& i m on the road again.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] queries from the web & random number

[linuxian iandsd]

i was wondering if there was there any means of sending this number to the
script thru the web.
I mean have the script run from cron & i can remotely access the image read
the code & enter the code in a form to be "submitted to my script" or
another script i don't mind as long as it gets to final destination.

I have already setup webpage with form & i have the image on it.

i guess i have to setup some sort of server /client application to do this
...
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SQLite LIKE question

[linuxian iandsd]

i forgot to mention that you need to try your sql commands out of your
script before trying them inside,
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SQLite LIKE question

[linuxian iandsd]

I m not sure this is your case but i believe you are missing the
"cur.fetchall()" command which does fetch data from sql db 
i suggest you put a print statement for every data you use in your program
that way you know whats empty & whats not...
here is example of MySQLdb process:

con=MySQLdb.connect(host='x', user='x', passwd='x',
db='' )
cur)

cur=con.cursor()

my_cmd="select %s from  where fieldx= '%s' ; " % (var1, var2)
cur.execute(my_cmd)

#now the cmd you are missing :

data=cur.fetchall()

# & then comes the :

for i in data:
 print i


hope this helps
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] queries from the web & random number

[linuxian iandsd]

i see no body has replied to my question,
probably you are thinking this guy is hijicking someones website ?

well, i can assure you i am not. simply because i have a username/password
for this site, plus, I have thier agreement to automate the process of
utilizing thier database all i want from midnight to 6 am.

to axplain to you my case, we sell products, & we need to access extranet of
our provider, & my job is to prepare an internal database for my company
every night so that staff uses it during the next day, this is very
important to our business because it gives us a step ahead of others who
have to work on database live during the day manualy & thus wasting
preciouse time of employees, making the surches themselves, waiting for
queries to complete, making some decisions based on that,  computers do
this better ... & our provider is happy to see that we have this technology.
now why would he add CAPTCHA to his extranet ? well, i don't know & i
believe thier IT stuff are trying to improve thier extranet for the better,
& they have all the right to doing that, as i myself do shut our website &
sql db server every night at 10pm, or use another port other than 80, i am
free to protect my stuff as well as they are free too.

this said, i believe there are other means to protecting content of websites
other than CAPCHA simply by ip or by number of queries ... etc ... so don't
make me look like tryin to hijack websites please.

i hope you find some truth in this.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Simple copy script

[linuxian iandsd]

right ! show us what you've done so far.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] input and output files from terminal

[linuxian iandsd]

i think you need to try :

cat input.txt | /usr/bin/python test.py >output.txt
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Font capture from webpages

[linuxian iandsd]

> I want to find the font style,Font size written in webpage without looking
> > into source code.
> >
>


your best bet would be your eyes. otherwise python will need to parse the
source code to tell.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Hoping to benefit from someone's experience...

[linuxian iandsd]

just a thought ...
 there must be some way of using OpenOffice to convert your ORIGINAL word
documents into HTML files ... then as html is a very nice & structured
language that you can manipulate & modify with ease just by adding some tags
inside where you want or wish 
this is also only if you have some rich content like tables inside your
documents ... if thats not your case, you can use OpenOffice tou extract
just text ...

hope this helps
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python assignments

[linuxian iandsd]

http://www.pythonchallenge.com/
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Error with incorrect encoding

[linuxian iandsd]

Kent was right,

>>> print u'\xae'.encode('utf-8')
> (R)
>


but i think you are using the wrong source file, i mean don't copy & paste
it from your browsers 'VIEW SOURCE' button. use python native urllib to get
the file.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Reading Multiple Files in Sequence

[linuxian iandsd]

if you only want to start a loop to parse all files in a directory try
something like this:

import glob
>
> n=0
> for file in
> glob.glob('/Users/Kelvin/TEMPDATASET/CLEANEDFIELDDATA/WEST/*'):
>  n=n+1
>  new_output_file='/Users/Kelvin/TEMPDATASET/PCA/'+ str(n) + '.txt'
>  output = open(new_output_file,'w')
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] HTML Parsing

[linuxian iandsd]

Another horrid solution


> #!/usr/bin/python
> # line number does not change so we use that

# the data we're looking for does not have a (unique) close tag (htmllib
> )
>
> import re, urllib2
> file=urllib2.urlopen('http://10.1.2.201/server-status')
> n=0
> for line in file:
>  n=n+1
>  if n==16:
>   print re.sub('requests.*','',line)[4:].strip()
>  elif n==17:
>   print re.sub('requests.*','',line)[4:].strip()
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Arguments ina separate file

[linuxian iandsd]

>
> import myargs


this one is clever
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Sending Mail

[linuxian iandsd]

i can send email w/ no problem using this :

import smtplib
smtp = smtplib.SMTP()
smtp.connect('smtp_server_here')
smtp.login('user_name', 'password')
smtp.sendmail(strFrom, strTo, msgRoot.as_string())
smtp.quit()
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] XMLRPC

[linuxian iandsd]

>
> looking for a way to run commands remotely on another computer.
>

I can run pretty much complicated commands on remote servers using only ssh.
I also run programs on remote server (when inside lan) using cgi module,
works perfectly.

maybe if you describe your problem i can provide you with some code.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[linuxian iandsd]

if i only wanted the last element ofthe list, i would  change the
"geoCode.getResults()' to return to me both the complete list & the new
elements in the list. it should be pretty easy from there. if what you need
is to sort the list & see whether a value have changed or not then you will
be better off working with dictionaries.



On Wed, Apr 23, 2008 at 6:26 AM, Vaibhav.bhawsar <[EMAIL PROTECTED]> wrote:

> i have this code to print every new element in a list only when the list
> length changes (while the list is updated by a thread running elsewhere)...I
> was wondering if there is a pythonic way to do this? how does one know when
> there is a new element in the list?
>
> prevlength = 0
> while geoCode.running:
> places = geoCode.getResults() #returns a list with most up to date
> elements..the list grows as the thread that updates it
> if len(places) > prevlength:
> print places[prevlength]
> prevlength = len(places)
>
>
> thank you!
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor