[Tutor] Removing/Handing large blocks of text

[Jesse Noller]

Hello,

I'm trying to do some text processing with python on a farily large
text file (actually, XML, but I am handling it as plaintext as all I
need to do is find/replace/move) and I am having problems with trying
to identify two lines in the text file, and remove everything in
between those two lines (but not the two lines) and then write the
file back (I know the file IO part).

I'm trying to do this with the re module - the two tags looks like:


...
a bunch of text (~1500 lines)
...


I need to identify the first tag, and the second, and unconditionally
strip out everything in between those two tags, making it look like:




I'm familiar with using read/readlines to pull the file into memory
and alter the contents via string.replace(str, newstr) but I am not
sure where to begin with this other than the typical open/readlines.

I'd start with something like:

re1 = re.compile('^\')
re2 = re.compile('^\<\/foo\>')

f = open('foobar.txt', 'r')
for lines in f.readlines()
match = re.match(re1, line)

But I'm lost after this point really, as I can identify the two lines,
but I am not sure how to do the processing.

thank you
-jesse
___
Tutor maillist  -  [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Removing/Handing large blocks of text

[Jesse Noller]

On Wed, 8 Dec 2004 15:11:55 +, Max Noel <[EMAIL PROTECTED]> wrote:
> 
> 
> 
> On Dec 8, 2004, at 14:42, Jesse Noller wrote:
> 
> > Hello,
> >
> > I'm trying to do some text processing with python on a farily large
> > text file (actually, XML, but I am handling it as plaintext as all I
> > need to do is find/replace/move) and I am having problems with trying
> > to identify two lines in the text file, and remove everything in
> > between those two lines (but not the two lines) and then write the
> > file back (I know the file IO part).
> 
> Okay, here are some hints: you need to identify when you enter a 
> block and when you exit a  block, keeping in mind that this may
> happen on the same line (e.g. blah). The rest is trivial.
> The rest of your message is included as a spoiler space if you want to
> find the solution by yourself -- however, a 17-line program that does
> that is included at the end of this message. It prints the resulting
> file to the standard out, for added flexibility: if you want the result
> to be in a file, just redirect stdout (python blah.py > out.txt).
> 
> Oh, one last thing: don't use readlines(), it uses up a lot of memory
> (especially with big files), and you don't need it since you're reading
> the file sequentially. Use the file iterator instead.
> 
> 
> 
> > I'm trying to do this with the re module - the two tags looks like:
> >
> > 
> > ...
> > a bunch of text (~1500 lines)
> > ...
> > 
> >
> > I need to identify the first tag, and the second, and unconditionally
> > strip out everything in between those two tags, making it look like:
> >
> > 
> > 
> >
> > I'm familiar with using read/readlines to pull the file into memory
> > and alter the contents via string.replace(str, newstr) but I am not
> > sure where to begin with this other than the typical open/readlines.
> >
> > I'd start with something like:
> >
> > re1 = re.compile('^\')
> > re2 = re.compile('^\<\/foo\>')
> >
> > f = open('foobar.txt', 'r')
> > for lines in f.readlines()
> > match = re.match(re1, line)
> >
> > But I'm lost after this point really, as I can identify the two lines,
> > but I am not sure how to do the processing.
> >
> > thank you
> > -jesse
> > ___
> > Tutor maillist  -  [EMAIL PROTECTED]
> > http://mail.python.org/mailman/listinfo/tutor
> 
> #!/usr/bin/env python
> 
> import sre
> 
> reStart = sre.compile('^\s*\')
> reEnd = sre.compile('\\s*$')
> 
> inBlock = False
> 
> fileSource = open('foobar.txt')
> 
> for line in fileSource:
>  if reStart.match(line): inBlock = True
>  if not inBlock: print line
>  if reEnd.match(line): inBlock = False
> 
> fileSource.close()
> 
> -- Max
> maxnoel_fr at yahoo dot fr -- ICQ #85274019
> "Look at you hacker... A pathetic creature of meat and bone, panting
> and sweating as you run through my corridors... How can you challenge a
> perfect, immortal machine?"
> 
> 


Thanks a bunch for all of your fast responses, they helped a lot -
I'll post what I cook up back to the list as soon as I complete it.
Thanks!

-jesse
___
Tutor maillist  -  [EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SSH using python

[Jesse Noller]

On 6/4/07, Chandrashekar <[EMAIL PROTECTED]> wrote:


Hi ,

Can anyone tell me how to do ssh to a machine using python and execute
programs on the remote machine? Thanks in advance.

Regards,
Chandru

--
Boardwalk for $500? In 2007? Ha!
Play Monopoly Here and 
Now(it's
 updated for today's economy) at Yahoo! Games.


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor



pSSH:
http://www.theether.org/pssh/

Paramiko:
http://www.lag.net/paramiko/

pExpect example:
http://www.palovick.com/code/python/python-ssh-client.php

Twisted Conch:
http://twistedmatrix.com/users/z3p/files/conch-talk.html
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] SSH commands in Python on Linux

[Jesse Noller]

On 8/10/05, Bernard Lebel <[EMAIL PROTECTED]> wrote:
> Hello,
> 
> I'm trying to make a script to send a SSH command from a Linux
> computer to another Linux compter.
> 
> The Python syntax I'm using...
> 
> 
> import os
> os.system( 'ssh [EMAIL PROTECTED] "ls"' )
> 
> 
> Now the problem is that I'm always asked for the password. So my
> question is two-fold:
> 
> 1- Is there a way to not be asked for the root password?
> 2- If not, then is it possible to feed the SSH password input with my
> Python script? I have about stdin redirection, but I have to admit
> that I'm a bit lost and I don't know if it applies to SSH input as
> well.
> 
> Right now I'm using the ls command to list processes, however
> ultimately I want to be able to kill some processes within a loop
> (this is for render farm management).
> 
> 
> 
> Thanks
> Bernard

Some of the things you are going to be bit by you've already ran into
- the first of which is authentication.

I would recommend looking at the following utilities:

http://www.theether.org/pssh/
http://sourceforge.net/projects/pyssh

Setting up private/public key authentication is going to allow for a
greate amount of secure automation. Barring that, use the pexpect
module to do the prompt handling, pexpect is bad simply due to the
fact you'll need to store your passwords plaintext within your program
which is a seriously risk.

Also note you'll need to take into account basic flow control for
Stdin, Stdout and Stderr - you need to watch for hosts that are not
found/don't exist and ssh commands that may or may not hang.

The PSSH program has excellent work arounds for all of these within it. 

-jesse
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor