[Tutor] Nested, line by line, file reading

[jon vs. python]

Hi everyone,
I have a file with this content:

"1
1
1
1
1
1
1
2
1
1"

I wanted a little script that would print the line containing "2" and every
line containing "1" after it. I've tried this:

>>> def p():
f = file("prueba.txt",'r')
for startline in f.read():
if startline.find("2") != -1:
print startline
for endline in f.read():
if endline.find("1") != -1:
print endline
break
f.close()


>>> p()
2

I found a way for doing it.

But still I don't really understand why I don't get two "1" lines printed.
It seems that every line is read in "for startline f.read()" so "for endline
in f.read()" will start reading but find no data, am I right?

Thanks, Jon.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[bob gailer]

jon vs. python wrote:
> Hi everyone,
> I have a file with this content:
>
> "1
> 1
> 1
> 1
> 1
> 1
> 1
> 2
> 1
> 1"
>
> I wanted a little script that would print the line containing "2" and 
> every line containing "1" after it. I've tried this:
>
> >>> def p():
> f = file("prueba.txt",'r')
> for startline in f.read():
> if startline.find("2") != -1:
> print startline
> for endline in f.read():
> if endline.find("1") != -1:
> print endline
> break
> f.close()
>
>
> >>> p()
> 2
>
> I found a way for doing it.
>
> But still I don't really understand why I don't get two "1" lines 
> printed. It seems that every line is read in "for startline f.read()" 
> so "for endline in f.read()" will start reading but find no data, am I 
> right?
 From the manual: "read([size]) Read at most size bytes from the file 
(less if the read hits EOF before obtaining size bytes). If the size 
argument is negative or omitted, read all data until EOF is reached."

Instead use readline().

Then realize that will also consume the entire file, so a nested 
readline() will return "".

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[Luis N]

On Dec 16, 2007 10:17 PM, jon vs. python <[EMAIL PROTECTED]> wrote:
> Hi everyone,
> I have a file with this content:
>
> "1
> 1
> 1
> 1
> 1
> 1
> 1
> 2
> 1
> 1"
>
> I wanted a little script that would print the line containing "2" and every
> line containing "1" after it. I've tried this:
>
> >>> def p():
> f = file("prueba.txt",'r')
> for startline in f.read():
> if startline.find("2") != -1:
> print startline
> for endline in f.read():
> if endline.find("1") != -1:
> print endline
> break
> f.close()
>
>
> >>> p()
> 2
>
> I found a way for doing it.
>
> But still I don't really understand why I don't get two "1" lines printed.
> It seems that every line is read in "for startline f.read()" so "for endline
> in f.read()" will start reading but find no data, am I right?
>
> Thanks, Jon.
>


Try something like:

show_ones = False

for line in f.read():
if line.find(2) != -1 or show_ones == True:
   print line
   show_ones = True
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[Bob Gailer]

[snip]

now ponder this:

f=file("prueba.txt.log")
for startline in f:
if startline.find("2") != -1:
for endline in f:
print endline
# etc.
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[bob gailer]

Luis N wrote:
> On Dec 16, 2007 10:17 PM, jon vs. python <[EMAIL PROTECTED]> wrote:
>   
>> Hi everyone,
>> I have a file with this content:
>>
>> "1
>> 1
>> 1
>> 1
>> 1
>> 1
>> 1
>> 2
>> 1
>> 1"
>>
>> I wanted a little script that would print the line containing "2" and every
>> line containing "1" after it. I've tried this:
>>
>> 
> def p():
>   
>> f = file("prueba.txt",'r')
>> for startline in f.read():
>> if startline.find("2") != -1:
>> print startline
>> for endline in f.read():
>> if endline.find("1") != -1:
>> print endline
>> break
>> f.close()
>>
>>
>> 
> p()
>   
>> 2
>>
>> I found a way for doing it.
>>
>> But still I don't really understand why I don't get two "1" lines printed.
>> It seems that every line is read in "for startline f.read()" so "for endline
>> in f.read()" will start reading but find no data, am I right?
>>
>> Thanks, Jon.
>>
>> 
>
>
> Try something like:
>
> show_ones = False
>
> for line in f.read():
> if line.find(2) != -1 or show_ones == True:
>print line
>show_ones = True
>   
See my post re use of readline instead of read

Also note that when testing booleans you don't need == True. Just write

if line.find(2) != -1 or show_ones:

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[Kent Johnson]

jon vs. python wrote:
> Hi everyone,
> I have a file with this content:
> 
> "1
> 1
> 1
> 1
> 1
> 1
> 1
> 2
> 1
> 1"
> 
> I wanted a little script that would print the line containing "2" and 
> every line containing "1" after it. I've tried this:
> 
>  >>> def p():
> f = file("prueba.txt",'r')
> for startline in f.read():

As Bob pointed out, f.read() reads the entire file into a single string. 
Iterating over a string yields individual characters, not lines. So 
startline is actually a single character.

> if startline.find("2") != -1:

More simply:
   if "2" in startline:

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[Alan Gauld]

"jon vs. python" <[EMAIL PROTECTED]> wrote in

> I wanted a little script that would print the line containing "2" 
> and every
> line containing "1" after it. I've tried this:
>
 def p():
>f = file("prueba.txt",'r')
>for startline in f.read():
>if startline.find("2") != -1:
>print startline
>for endline in f.read():
>if endline.find("1") != -1:
>print endline
>break
>f.close()

Its easier to iterate over the file itself

found2 = False
for line in file("prueba.txt"):
 if '2' in line: found2 = True
 if found2: print line
 else: continue

Should be close.

HTH,

-- 
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld 


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Nested, line by line, file reading

[Alan Gauld]

"Alan Gauld" <[EMAIL PROTECTED]> wrote 

> found2 = False
> for line in file("prueba.txt"):
> if '2' in line: found2 = True
> if found2: print line
> else: continue

Just after posting I realised it would be better to do:

found2 = False
for line in file("prueba.txt"):
found2 = found2 or line.startswith('2')
if found2: print line

The continue was superfluous and the if test 
less efficient that the boolean test.
line.startswith seems a more appropriate test too, 
and should be more efficient on long lines.

HTH,

-- 
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld




___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python logging module: two handlers writing to the same file - okay?

[Hans Fangohr]

Dear all,

thanks to everybody who replied to my question.

On 15 Dec 2007, at 16:34, Kent Johnson wrote:

> Ricardo Aráoz wrote:
>> Kent Johnson wrote:
>>> I don't know the answer, but it has nothing to do with the logging
>>> module. The question is, can the same file reliably be opened  
>>> twice for
>>> writing in the same module.
>>
>> Well, the question would actually be if the logging module is smart
>> enough to find out that both your filehandlers are referring to  
>> the same
>> file and open it only once.
>
> A quick glance at the logging module shows that it is not that  
> smart. It
> just opens the file twice; hence my original question.
>

To summarise the situation: the logging module is not clever enough  
to detect that I am opening the same file twice. If I use 'w' as the  
opening mode, then I know (and have an example program for this) that  
the logging 'fails': I suppose this can be tracked to the first part  
of the log file being overriden when the second filehandler attempts  
to log its first message, and thus opens the file (again) in 'w' mode.

Using 'a' (append) or 'a+' (?)  *seems* to solve this problem but we  
are still short of knowing as to whether this is guaranteed to work  
(on all platforms...) or whether it just happened to be okay on Mac  
OS X and (Debian) Linux (on which I have tested this).

I think my example raises another question (in particular in context  
with openening the files in 'w' mode): would it be worth contacting  
the developers of the logging module to suggest that it
- tracks which files it writes to and
- warns if the same file is opened twice (or more ofter) using 'w'?

Alternatively, one could enhance the internal logic of the logging  
module so that it is aware of the same file being used twice (or more  
times) so that it only opens the file once, but I suppose this raises  
some other questions (if the two filehandlers have been given  
different opening modes, say 'a' and 'w', which one should it take  
when opening the file only once?). Therefore, just issueing a warning  
may be the better solution: should be easy to implement, doesn't  
change the interface, and prevents (naive) users (like myself) from  
wasting time trying to track down the problem of the beginning of the  
log file missing.

What is the recommended method to make such a suggestion to the  
python team, or the people who look after the logging module?

Many thanks,

Hans





> Kent
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
>

--
Hans Fangohr
School of Engineering Sciences
University of Southampton
Phone: +44 (0) 238059 8345

Email: [EMAIL PROTECTED]
http://www.soton.ac.uk/~fangohr






___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python logging module: two handlers writing to the same file - okay?

[Kent Johnson]

Hans Fangohr wrote:
> Dear all,
> 
> thanks to everybody who replied to my question.
> 
> On 15 Dec 2007, at 16:34, Kent Johnson wrote:
> 
>> Ricardo Aráoz wrote:

> What is the recommended method to make such a suggestion to the python 
> team, or the people who look after the logging module?

You could post a feature request (possibly including a patch or 
suggested implementation) to the Python bug tracker:
http://www.python.org/dev/patches/

You could contact the developer directly via the feedback link here:
http://www.red-dove.com/python_logging.html

You could start a discussion on comp.lang.python.

Note: when I was researching my original answer I found a thread about a 
similar problem with shelve. The discussion was not sympathetic to the 
OP for two reasons:
- in general it's difficult to determine if the same file is being 
opened twice
- it is trying too hard to protect the programmer against himself.
http://groups.google.com/group/comp.lang.python/browse_frm/thread/4aafd9a8192cfbe7/5803ea195210a28f?hl=en&lnk=gst&q=open+file+twice#5803ea195210a28f
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python logging module: two handlers writing to the same file - okay?

[Ricardo Aráoz]

Hans Fangohr wrote:
> Dear all,
> 
> thanks to everybody who replied to my question.
> 
> On 15 Dec 2007, at 16:34, Kent Johnson wrote:
> 
>> Ricardo Aráoz wrote:
>>> Kent Johnson wrote:
 I don't know the answer, but it has nothing to do with the logging
 module. The question is, can the same file reliably be opened twice for
 writing in the same module.
>>>
>>> Well, the question would actually be if the logging module is smart
>>> enough to find out that both your filehandlers are referring to the same
>>> file and open it only once.
>>
>> A quick glance at the logging module shows that it is not that smart. It
>> just opens the file twice; hence my original question.
>>
> 
> To summarise the situation: the logging module is not clever enough to
> detect that I am opening the same file twice. If I use 'w' as the
> opening mode, then I know (and have an example program for this) that
> the logging 'fails': I suppose this can be tracked to the first part of
> the log file being overriden when the second filehandler attempts to log
> its first message, and thus opens the file (again) in 'w' mode.
> 
> Using 'a' (append) or 'a+' (?)  *seems* to solve this problem but we are
> still short of knowing as to whether this is guaranteed to work (on all
> platforms...) or whether it just happened to be okay on Mac OS X and
> (Debian) Linux (on which I have tested this).
> 
> I think my example raises another question (in particular in context
> with openening the files in 'w' mode): would it be worth contacting the
> developers of the logging module to suggest that it
> - tracks which files it writes to and
> - warns if the same file is opened twice (or more ofter) using 'w'?
> 
> Alternatively, one could enhance the internal logic of the logging
> module so that it is aware of the same file being used twice (or more
> times) so that it only opens the file once, but I suppose this raises
> some other questions (if the two filehandlers have been given different
> opening modes, say 'a' and 'w', which one should it take when opening
> the file only once?). Therefore, just issueing a warning may be the
> better solution: should be easy to implement, doesn't change the
> interface, and prevents (naive) users (like myself) from wasting time
> trying to track down the problem of the beginning of the log file missing.
> 


As far as I can see, the only reason in your example program to open the
same file twice is to use two different formatters (i.e. two different
type of lines) in the same log, if you'd drop the requirement for two
different format of line in the same log (which is itself somewhat
questionable) then you could use the same handler for both loggers, thus
opening only one file.


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python logging module: two handlers writing to the same file - okay?

[Ricardo Aráoz]

Kent Johnson wrote:
> Hans Fangohr wrote:
>> Dear all,
>>
>> thanks to everybody who replied to my question.
>>
>> On 15 Dec 2007, at 16:34, Kent Johnson wrote:
>>
>>> Ricardo Aráoz wrote:
> 
>> What is the recommended method to make such a suggestion to the python 
>> team, or the people who look after the logging module?
> 
> You could post a feature request (possibly including a patch or 
> suggested implementation) to the Python bug tracker:
> http://www.python.org/dev/patches/
> 
> You could contact the developer directly via the feedback link here:
> http://www.red-dove.com/python_logging.html
> 
> You could start a discussion on comp.lang.python.
> 
> Note: when I was researching my original answer I found a thread about a 
> similar problem with shelve. The discussion was not sympathetic to the 
> OP for two reasons:
> - in general it's difficult to determine if the same file is being 
> opened twice
> - it is trying too hard to protect the programmer against himself.
> http://groups.google.com/group/comp.lang.python/browse_frm/thread/4aafd9a8192cfbe7/5803ea195210a28f?hl=en&lnk=gst&q=open+file+twice#5803ea195210a28f

Why should it be all or nothing. Couldn't the programmer indicate that
both handlers use the same file? For example if instead of pointing to a
file the formatter points to another formatter then it is using the same
file of the pointed formatter.
Then both your points would be answered and the module would be a little
bit better (I don't think I'll ever need that feature as I believe that
a log should have all it's lines in the same format).



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python logging module: two handlers writing to the same file - okay?

[Kent Johnson]

Ricardo Aráoz wrote:
> Why should it be all or nothing. Couldn't the programmer indicate that
> both handlers use the same file?

It would be very easy to do this using StreamHandlers instead of 
FileHandlers. It would also be easy to make a FileHandler subclass that 
keeps a map from file name to file objects so if the same file is given 
to two instances it reuses the first one. This could be specified as the 
handler in the config file even.

Python has 'batteries included' but sometimes you have to add some 
wiring ;-)

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] python links

[Mark Alexiuk]

Hi All,

I have no experience using Python in an embedded application (industrial
microcontrollers) but I'm sure it can be done and feel that Python offers
many advantages for new OOP developers over C++.

I would appreciate any suggestions on embedded processors compatible with
Python or some variant and what libraries people find useful, such as
UnitTest and Logging.

What follows are some links and excerpts I found looking for information on
this topic.

Cheers,
Mark

===

*Open Embedded*

OpenEmbedded is a tool which allows developers to create a fully usable
Linux base for various embedded systems.

http://www.openembedded.org/

===

*Embedded Python*

Some modern *embedded* devices have enough memory and a fast enough CPU to
run a typical Linux-based environment, for example, and running CPython on
such devices is mostly a matter of compilation (or cross-compilation) and
tuning.

Devices which could be considered as "*embedded*" by modern standards and
which can run tuned versions of CPython include the following:

   -

   [image: [WWW]] Gumstix 
   -

   FIC [image: [WWW]] Neo1973  ( [image:
   [WWW]] Python on OpenMoko )

See also PythonForArmLinux and
Open*Embedded* .

http://wiki.python.org/moin/EmbeddedPython?highlight=%28embedded%29


===
Python compilers are available for some popular microcontrollers.
Pyastra[1]compiles for all Microchip
PIC12, PIC14 and PIC16 microcontrollers.
PyMite[2]compiles for "any
device in the AVR family that has at least 64 KiB program
memory and 4 KiB RAM". PyMite also targets (some) ARM microcontrollers.
Notice that these embedded Python compilers typically can only compile a
subset of the Python language for these devices.

http://en.wikibooks.org/wiki/Embedded_Systems/Embedded_Systems_Introduction

===

PyMite: A Flyweight Python Interpreter for 8-bit Architectures
http://python.fyxm.net/pycon/papers/pymite/

===

*George Belotsky is a software architect who has done extensive work on high
performance Internet servers as well as hard real time and embedded systems.
His technology interests include C++, Python and Linux.*

SOURCE: http://pythonology.org/success&story=carmanah

===
*Embedding Python with Boost.Python Part 1 *

Python has been called a language optimized for development speed. This puts
it in contrast with compiled languages like C and C++, which can be called
languages optimized for execution speed. This contrast between Python and
C/C++ often fuels a development methodology in Python circles: code the
application in Python, profile the application, and rewrite the performance
critical parts in C or C++. This makes the topic of developing hybrid
systems by extending Python very well covered in various Python
documentation.
SOURCE:
http://members.gamedev.net/sicrane/articles/EmbeddingPythonPart1.html

===

*Java Embedded Python*
Jepp embeds CPython in Java. It is safe to use in a heavily threaded
environment, it is quite fast and its stability is a main feature and goal.

Some benefits of CPython over Java-based languages:

   - Using the native Python interpreter may mean a massive speed
   improvement over Java-based languages.
   - Python is mature so authors needn't fear the interpreter will
   suddenly change widely-used features.
   - Access to the high quality Python modules, both native and
   Python-based.
   - Compilers and assorted Python tools are as mature as the language.
   - Python is an ideal language for your business logic. It is easy to
   learn, eminently readable and generally immune to programming gotchas.


SOURCE: http://jepp.sourceforge.net/

===
*DePython*

In the DePython  (Deeply Embeddable Python) project we have created a
stripped-down version of Python  that provides a platform for implementing
embedded systems in the Python language.  Specifically the DePython system
provides the following advantages of Javatm:

   - absolutely free, even for commercial use (including resale),
   - small footprint (<200K)
   - easy maintenance of software: objects and methods can be replaced
   ``on-the-fly''
   - reference counting garbage-collection giving more deterministic
   performance

Currently DePython runs on a Hitachi SH1 evaluation board with some extra
hardware.  The SH1 is a 32 bits RISC micro-controller running at 16Mhz.  T
he evaluation is equipped with 256Kb of RAM, 64Kb of ROM, two RS-232 ports,
and a number of I/O ports.

SOURCE: http://www.

Re: [Tutor] python links

[Michael Langford]

Cross compiling python can be a bit of a bear. There build process has
python scripts that assume that you're building python for the machine
you are building on. There is a patch to help for that which Chris
Lambacher has recently updated:
http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/

I've not tried it yet, as I've not had an applicable project. If you
use it, please let me know how it goes, as I love to see python on
these systems. Please keep me in the loop on this matter if you don't
mind...I like to see my two primary skill areas overlap as much as
possible.

Before Lambacher did his updates, I have had success on one Linux
platform and failure on another with the original patch.

Keep in mind the term "Embedding" in the python world means something
entirely unrelated to embedded systems. It means adding the
interpreter as a scripting language into an application, usually that
runs on a normal, desktop OS. For instance, the computer game
Civilization IV did this, so the graphics parts are written in C, and
the "game" part is written in python. It's a cool process, however
it's not what you're looking for. This will probably plague some of
your web searching and reading. I find looking for cross compiling
helps with this, as well as identifying embedded projects, such as
gumstix. If their stuff compiles on your target board, I've had some
success in the past with some of their stuff, (their buildroot I
think), although the python compile didn't work for my platform.

Disregard Jepp and Embedding Python with Boost.Python Part 1. They are
not the type of embedding you're looking for.

PyMite looks promising for the type of processor I think you're
interested in targeting (http://pymite.python-hosting.com/wiki/PyMite
is their new site). This could most likely be brought up for you by a
consultant if you don't meet sucess and then the actual application
code could then be written by you. With a little flexability on
processor, I am pretty sure you could get it running from their
description as long as you don't need floating point.

DePython doesn't appear to still be under active development (Broken
links, no visible source repository, etc). I usually run kicking and
screaming away from abandoned projects, and this looks like an
abandoned academic project to boot.

If you are going to go with a processor on which you can embed Linux,
there is a python in the buildroot tree, however buildroot can be
quite the crap shoot, depending on processor and mix of applications
(It didn't work on the last board I was using buildroot on).

 --Michael


-- 
Michael Langford
Phone: 404-386-0495
Consulting: http://www.RowdyLabs.com



On Dec 16, 2007 9:32 PM, Mark Alexiuk <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> I have no experience using Python in an embedded application (industrial
> microcontrollers) but I'm sure it can be done and feel that Python offers
> many advantages for new OOP developers over C++.
>
> I would appreciate any suggestions on embedded processors compatible with
> Python or some variant and what libraries people find useful, such as
> UnitTest and Logging.
>
> What follows are some links and excerpts I found looking for information on
> this topic.
>
> Cheers,
> Mark
>
> ===
>
> Open Embedded
>
>  OpenEmbedded is a tool which allows developers to create a fully usable
> Linux base for various embedded systems.
>
> http://www.openembedded.org/
>
> ===
>
> Embedded Python
>
>
> Some modern embedded devices have enough memory and a fast enough CPU to run
> a typical Linux-based environment, for example, and running CPython on such
> devices is mostly a matter of compilation (or cross-compilation) and tuning.
>
> Devices which could be considered as "embedded" by modern standards and
> which can run tuned versions of CPython include the following:
>
>
>  Gumstix
>
>
>  FIC Neo1973 ( Python on OpenMoko)
>
> See also PythonForArmLinux and OpenEmbedded.
>  http://wiki.python.org/moin/EmbeddedPython?highlight=%28embedded%29
>
>
> ===
> Python compilers are available for some popular microcontrollers. Pyastra
> [1] compiles for all Microchip PIC12, PIC14 and PIC16 microcontrollers.
> PyMite [2] compiles for "any device in the AVR family that has at least 64
> KiB program memory and 4 KiB RAM". PyMite also targets (some) ARM
> microcontrollers. Notice that these embedded Python compilers typically can
> only compile a subset of the Python language for these devices.
>
> http://en.wikibooks.org/wiki/Embedded_Systems/Embedded_Systems_Introduction
>
> ===
>
> PyMite: A Flyweight Python Interpreter for 8-bit
> Architectureshttp://python.fyxm.net/pycon/papers/pymite/
>
> ===
>
>  George Belotsky is a software architect who has done extensive work on high
> performance Internet ser