Re: Having both if() and for() statements in one liner

[Chris Angelico]

On Wed, Sep 18, 2013 at 3:55 PM, Joshua Landau  wrote:
> range(person == "simon" and 5)

+1 for the BOFH reference.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: *.csv to *.txt after adding columns

[Peter Otten]

Bryan Britten wrote:

> Hey, gang, I've got a problem here that I'm sure a handful of you will
> know how to solve. I've got about 6 *.csv files that I am trying to open;
> change the header names (to get rid of spaces); add two new columns, which
> are just the results of a string.split() command; drop the column I just
> split; and then finally export to *.txt files. Here's the code I'm using:
> 
> import os
> import csv
> 
> 
> fileHandle = 'Path/To/Data'
> varNames =
> 
'ID\tCaseNum\tDate\tTime\tBlock\tIUCR\tPrimaryType\tDescription\tLocDesc\tArrest\tDomestic\tBeat\tDistrict\tWard\tCommArea\tFBICode\tXCoord\tYCoord\tYear\tUpdatedOn\tLat\tLong\tLoc\n'
> 

You may be processing not just the .csv files, but also the .txt files from 
a previous run of your script. Remove them manually to verify my guess.
If I'm right you can (in order of my preference)

(1) write the output into another directory and also do (2)
(2) filter the input files with glob.glob("/path/to/data/*.csv") 
(3) or just add the two lines below to your current code.

> for csvFile in os.listdir(fileHandle):

  if not csvFile.endswith(".csv"):
  continue

> outFile = open(fileHandle + os.path.splitext(csvFile)[0] + '.txt',
> 'w') 
> inFile = open(fileHandle + csvFile, 'rb')
> reader = csv.reader(inFile, delimiter=',')
> rowNum = 0
> for row in reader:
> if rowNum == 0:
> outFile.write(varNames)
> rowNum += 1
> else:
> date, time = row[2].split()
> row.insert(3, date)
> row.insert(4, time)
> row.remove(row[2])
> outFile.write('\t'.join(row) + '\n')
> outFile.close()
> inFile.close()
> 
> 
> The issue I'm having is that the *.txt files I'm generating are empty. I
> assume some unraised error is being thrown, but I'm new to Python and am
> self taught, so I don't know where to look or how to troubleshoot.
> 
> When I run the code on just one file and print the output instead of
> writing it, it looks exactly like what I'd want. So I'm at a loss for
> where the problem is.
> 
> Any help is appreciated!


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: *.csv to *.txt after adding columns

[rusi]

On Wednesday, September 18, 2013 7:12:21 AM UTC+5:30, Bryan Britten wrote:
> Hey, gang, I've got a problem here that I'm sure a handful of you will know 
> how to solve. I've got about 6 *.csv files that I am trying to open; change 
> the header names (to get rid of spaces); add two new columns, which are just 
> the results of a string.split() command; drop the column I just split; and 
> then finally export to *.txt files. Here's the code I'm using:

Its generally a good idea in programming to separate separable problems. Since 
you say:

> The issue I'm having is that the *.txt files I'm generating are empty. I 
> assume some unraised error is being thrown, but I'm new to Python and am self 
> taught, so I don't know where to look or how to troubleshoot.

> When I run the code on just one file and print the output instead of writing 
> it, it looks exactly like what I'd want. So I'm at a loss for where the 
> problem is. 

you should probably write and have checked the code that handles one file. For 
example a function that takes as parameters the two: the input/output files 
(maybe names or handles), and then
- check that that function works for one file
- call it in your loop

[BTW your code seems to mixup the variable-names for file-names and handles]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: *.csv to *.txt after adding columns

[Dave Angel]

On 17/9/2013 22:28, Bryan Britten wrote:

> Dave -
>
> I can't print the output because there are close to 1,000,000 records. It 
> would be extremely inefficient and resource intensive to look at every row.

Not if you made a sample directory with about 3 files, each containing
half a dozen lines.

> Like I said, when I take just one file and run the code over the first
> few records I get what I'd expect to see. Here's an example(non-redacted 
> code):
>
> INPUT:
>
> import csv
>
> fileHandle = 'C:/Users/Bryan/Data Analysis/Crime Analysis/Data/'

Now, that directory specification ends with a slash.  So "+" will work
correctly.  But your original did not.

>
> varNames = 
> 'ID\tCaseNum\tDate\tTime\tBlock\tIUCR\tPrimaryType\tDescription\tLocDesc\tArrest\tDomestic\tBeat\tDistrict\tWard\tCommArea\tFBICode\tXCoord\tYCoord\tYear\tUpdatedOn\tLat\tLong\tLoc\n'
>
> outFile = open(fileHandle + 'ChiCrime01_02.txt', 'w')
> inFile = open(fileHandle + 'ChiCrime01_02.csv', 'rb')

Instead of changing this code, you could have switched to a directory
containing only one file.

>
> Like I said, the output is exactly what I want, but it doesn't seem to be 
> writing to the file and I don't know why. I said I didn't know if it was 
> raising an exception because I'm new to Python and I didn't know if there 
> were some methods that included "silent" errors where it would continue the 
> code but produce the wrong results, such as not writing my files. 
>

The only "silent" exception I know of is the one triggered by
sys.exit(), used to terminate the process.

> Lastly, why does everyone seem to push for os.path.join versus the method I 
> have used? Is it just a 'standard' that people like to see?
>

Because os.path.join will be smart enough to add the slashes only if
they are necessary.   That can be useful, especially if the directory
you're using as a prefix came from the user.


I think Peter's suggestion is probably right on;  you don't limit your
infiles to *.csv, so you will be processing *.txt files the second time.

Another useful debugging aid would have been to add print statements
right after opening the files, perhaps something like:

print "Reading :", infile.name
print "Writing:", outfile.name

If those names had been missing slashes, I would have been vindicated,
while if they were the same, you'd know Peter had nailed it.



-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Please use the Python Job Board for recruiting (was: Python Developers)

[Ben Finney]

Kelly Lurz  writes:

> List

Please do not use this discussion forum for job advertisements.

Instead, please use the Python Job Board designed for this purpose
http://www.python.org/community/jobs/>.

-- 
 \  “I don't know half of you half as well as I should like, and I |
  `\   like less than half of you half as well as you deserve.” —Bilbo |
_o__)  Baggins |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Remove \n, " " from tuple.

[Venkat Addala]

Hi all,

I have a tuple in this format, which is retrived using MySQLdb, now i want
to remove \n and extra spaces in this.

13L, 'ssDsC', 'CEs5s1, DsC', 'srylscetsmight\nghtscetylsse', '3', '3q25.1',
151531861L, 151546276L, '+', '1 kjafhkfhlad\fdfdsdf ghtscetylsse \ncontends
the sctivity of dsfdk srylsmine N-scetyltrsnsfersse thst scts ss s cstslyst
inbiotrsnsformstion fghjfg for srylsmine snd bvhghbh smine csrcinogens.
Bgskjdg in ssDsC dfkdkdf in bresst snd prostrste csncer.', '', 'Msdhurs
\nsgdfgvslingegowds', dstetime.dste(2013, 6, 14), None



--
Regards
Venkat Addala  
Computational Biologist
  O__  
 _/ /`_ ---
(*) \(*) --
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Remove \n, " " from tuple.

[Dave Angel]

On 18/9/2013 01:12, Venkat Addala wrote:

> Hi all,
>
> I have a tuple in this format, which is retrived using MySQLdb, now i want
> to remove \n and extra spaces in this.
>
> 13L, 'ssDsC', 'CEs5s1, DsC', 'srylscetsmight\nghtscetylsse', '3', '3q25.1',
> 151531861L, 151546276L, '+', '1 kjafhkfhlad\fdfdsdf ghtscetylsse \ncontends
> the sctivity of dsfdk srylsmine N-scetyltrsnsfersse thst scts ss s cstslyst
> inbiotrsnsformstion fghjfg for srylsmine snd bvhghbh smine csrcinogens.
> Bgskjdg in ssDsC dfkdkdf in bresst snd prostrste csncer.', '', 'Msdhurs
> \nsgdfgvslingegowds', dstetime.dste(2013, 6, 14), None
>

A tuple is immutable, so it cannot be done.

However, you can create a new tuple.  Easiest way is probably to convert
it to a list, process any strings in the list, and convert it back.


def convert(mylist):
res = []
for item in mylist:
if isinstance(item, str):
item = to be filled in
res.append(item)
return res


mytuple = .
temp = list(mytuple)
mytuple = tuple(convert(temp))


-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Remove \n, " " from tuple.

[Vlastimil Brom]

2013/9/18 Venkat Addala 

> Hi all,
>
> I have a tuple in this format, which is retrived using MySQLdb, now i want
> to remove \n and extra spaces in this.
>
> 13L, 'ssDsC', 'CEs5s1, DsC', 'srylscetsmight\nghtscetylsse', '3',
> '3q25.1', 151531861L, 151546276L, '+', '1 kjafhkfhlad\fdfdsdf ghtscetylsse
> \ncontends the sctivity of dsfdk srylsmine N-scetyltrsnsfersse thst scts ss
> s cstslyst inbiotrsnsformstion fghjfg for srylsmine snd bvhghbh smine
> csrcinogens. Bgskjdg in ssDsC dfkdkdf in bresst snd prostrste csncer.', '',
> 'Msdhurs \nsgdfgvslingegowds', dstetime.dste(2013, 6, 14), None
>
>
>
> --
> Regards
> Venkat Addala  
> Computational Biologist
>O__  
>  _/ /`_ ---
> (*) \(*) --
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
> Hi,
you can create a new tuple with the converted elements from the original one
using a list comprehension and the regular expression replacement, it could
be e.g.

>>> import re
>>> tuple([re.sub(r"\s+", " ", element) if isinstance(element, str) else
element for element in ("qw e", 1, 2, None, "U\nN IC", 'Q\tW E')])
('qw e', 1, 2, None, 'U N IC', 'Q W E')
>>>

re.sub(...) replaces one or more whitespace characters (space, newline, tab
...) with a single space - the pattern can be tweaked as needed (e.g. the
pattern r" *\n *| +" would only handle spaces and newlines explicitely).
This replacement is only applied on string elements of the tuple, others
are left unchanged
 if isinstance(element, str) is meant for  python 3, for python 2  if
isinstance(element, basestring) would be more appropriate
the elements are collected in a list using this list comprehension
and this intermediate list is finaly converted to a tuple if this immutable
container is needed.

hth,
  vbr
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: HID Feature Raport, Linux

[Anssi Saari]

Kasper Jepsen  writes:

> Hi,
>
> I have been using pywinusb.hid for a hid unit, using only feature reports.
> I like to get this code to run on raspbian PI, but i can not fint a good 
> library to support HID/feature reports?
> I am a USB newbie 
> Can anyone help me, or point me in a direction to send/recieve feature 
> reports without HID support?

I believe pyusb with libusb or libusbx would work. There's a summary of
the alternatives at
http://mcuee.blogspot.fi/2011/04/python-and-usb-hid-device.html if that
helps.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: HID Feature Raport, Linux

[Kasper Jepsen]

Den onsdag den 18. september 2013 12.10.29 UTC+2 skrev Anssi Saari:
> 
> 
> 
> > Hi,
> 
> >
> 
> > I have been using pywinusb.hid for a hid unit, using only feature reports.
> 
> > I like to get this code to run on raspbian PI, but i can not fint a good 
> > library to support HID/feature reports?
> 
> > I am a USB newbie 
> 
> > Can anyone help me, or point me in a direction to send/recieve feature 
> > reports without HID support?
> 
> 
> 
> I believe pyusb with libusb or libusbx would work. There's a summary of
> 
> the alternatives at
> 
> http://mcuee.blogspot.fi/2011/04/python-and-usb-hid-device.html if that
> 
> helps.

Hi,
Thx, i have been looking at this... but could not fint the way to send feature 
reports, does anyone have an example?
Kasper
-- 
https://mail.python.org/mailman/listinfo/python-list

iterating over a file with two pointers

[nikhil Pandey]

hi,
I want to iterate over the lines of a file and when i find certain lines, i 
need another loop starting from the next of that "CERTAIN" line till a few (say 
20) lines later.
so, basically i need two pointers to lines (one for outer loop(for each line in 
file)) and one for inner loop. How can i do that in python?
please help. I am stuck up on this.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Chris Angelico]

On Wed, Sep 18, 2013 at 9:12 PM, nikhil Pandey  wrote:
> hi,
> I want to iterate over the lines of a file and when i find certain lines, i 
> need another loop starting from the next of that "CERTAIN" line till a few 
> (say 20) lines later.
> so, basically i need two pointers to lines (one for outer loop(for each line 
> in file)) and one for inner loop. How can i do that in python?
> please help. I am stuck up on this.

After the inner loop finishes, do you want to go back to where the
outer loop left off, or should the outer loop continue from the point
where the inner loop stopped? In other words, do you want to locate
overlapping sections, or not? Both are possible, but the solutions
will look somewhat different.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: creating rectangle with qt-designer

[MRAB]

On 18/09/2013 04:41, Mohsen Pahlevanzadeh wrote:

Dear all,

I need to draw a rectangle , according to attached picture , you see a
rectangle in a form that enclosed with a text: "Rebuild Last Target".

Question: How can i draw a same rectangle with qt-designer?


That's a "groupbox". In Qt it's called "QGroupBox".

--
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Dave Angel]

On 18/9/2013 07:21, Chris Angelico wrote:

> On Wed, Sep 18, 2013 at 9:12 PM, nikhil Pandey  
> wrote:
>> hi,
>> I want to iterate over the lines of a file and when i find certain lines, i 
>> need another loop starting from the next of that "CERTAIN" line till a few 
>> (say 20) lines later.
>> so, basically i need two pointers to lines (one for outer loop(for each line 
>> in file)) and one for inner loop. How can i do that in python?
>> please help. I am stuck up on this.
>
> After the inner loop finishes, do you want to go back to where the
> outer loop left off, or should the outer loop continue from the point
> where the inner loop stopped? In other words, do you want to locate
> overlapping sections, or not? Both are possible, but the solutions
> will look somewhat different.
>

In addition, is this really a text file?  For binary files, you could
use seek(), and manage things yourself.  But that's not strictly legal
in a text file, and may work on one system, not on another.

I'd suggest you open the file twice, and get two file objects.  Then you
can iterate over them independently.

Or if the file is under a few hundred meg, just do a readlines, and do
the two iterators over that.  That way, the inner loop could just
iterate over a simple slice.



infile = open(  "rb")
lines = infile.readlines()
infile.close()

for index, line in enumerate(lines):
for inner in lines[index+1:20]:
 ...




-- 
DaveA

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Peter Otten]

nikhil Pandey wrote:

> hi,
> I want to iterate over the lines of a file and when i find certain lines,
> i need another loop starting from the next of that "CERTAIN" line till a
> few (say 20) lines later. so, basically i need two pointers to lines (one
> for outer loop(for each line in file)) and one for inner loop. How can i
> do that in python? please help. I am stuck up on this.

Here's an example that prints the three lines following a line containing a 
'*':

Example data:

$ cat tmp.txt
alpha
*beta
*gamma
delta
epsilon
zeta
*eta

The python script:

$ cat tmp.py
from itertools import islice, tee

with open("tmp.txt") as f:
while True:
for outer in f:
print outer,
if "*" in outer:
f, g = tee(f)
for inner in islice(g, 3):
print "   ", inner,
break
else:
break

The script's output:

$ python tmp.py
alpha
*beta
*gamma
delta
epsilon
*gamma
delta
epsilon
zeta
delta
epsilon
zeta
*eta
$ 

As you can see the general logic is relatively complex; it is likely that we 
can come up with a simpler solution if you describe your actual requirement 
in more detail.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: *.csv to *.txt after adding columns

[Bryan Britten]

Peter nailed it. Adding in the two lines of code to ensure I was just working 
with *.csv files fixed the problem. Thanks to everyone for the help and 
suggestions on best practices. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[nikhil Pandey]

On Wednesday, September 18, 2013 4:51:51 PM UTC+5:30, Chris Angelico wrote:
> On Wed, Sep 18, 2013 at 9:12 PM, nikhil Pandey  
> wrote:
> 
> > hi,
> 
> > I want to iterate over the lines of a file and when i find certain lines, i 
> > need another loop starting from the next of that "CERTAIN" line till a few 
> > (say 20) lines later.
> 
> > so, basically i need two pointers to lines (one for outer loop(for each 
> > line in file)) and one for inner loop. How can i do that in python?
> 
> > please help. I am stuck up on this.
> 
> 
> 
> After the inner loop finishes, do you want to go back to where the
> 
> outer loop left off, or should the outer loop continue from the point
> 
> where the inner loop stopped? In other words, do you want to locate
> 
> overlapping sections, or not? Both are possible, but the solutions
> 
> will look somewhat different.
> 
> 
> 
> ChrisA

Hi Chris,
After the inner loop finishes, I want to go back to the next line from where 
the outer loop was left i.e the lines of the inner loop will be traversed again 
in the outer loop.
1>>I iterate over lines of the file
2>> when i find a match in a certain line, i start another loop till some 
condition is met in the subsequent lines
3>> then i come back to where i left and repeat 1(ideally i want to delete that 
line in inner loop where that condition is met, but even if it is not deleted, 
its OK)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[nikhil Pandey]

On Wednesday, September 18, 2013 5:14:10 PM UTC+5:30, Peter Otten wrote:
> nikhil Pandey wrote:
> 
> 
> 
> > hi,
> 
> > I want to iterate over the lines of a file and when i find certain lines,
> 
> > i need another loop starting from the next of that "CERTAIN" line till a
> 
> > few (say 20) lines later. so, basically i need two pointers to lines (one
> 
> > for outer loop(for each line in file)) and one for inner loop. How can i
> 
> > do that in python? please help. I am stuck up on this.
> 
> 
> 
> Here's an example that prints the three lines following a line containing a 
> 
> '*':
> 
> 
> 
> Example data:
> 
> 
> 
> $ cat tmp.txt
> 
> alpha
> 
> *beta
> 
> *gamma
> 
> delta
> 
> epsilon
> 
> zeta
> 
> *eta
> 
> 
> 
> The python script:
> 
> 
> 
> $ cat tmp.py
> 
> from itertools import islice, tee
> 
> 
> 
> with open("tmp.txt") as f:
> 
> while True:
> 
> for outer in f:
> 
> print outer,
> 
> if "*" in outer:
> 
> f, g = tee(f)
> 
> for inner in islice(g, 3):
> 
> print "   ", inner,
> 
> break
> 
> else:
> 
> break
> 
> 
> 
> The script's output:
> 
> 
> 
> $ python tmp.py
> 
> alpha
> 
> *beta
> 
> *gamma
> 
> delta
> 
> epsilon
> 
> *gamma
> 
> delta
> 
> epsilon
> 
> zeta
> 
> delta
> 
> epsilon
> 
> zeta
> 
> *eta
> 
> $ 
> 
> 
> 
> As you can see the general logic is relatively complex; it is likely that we 
> 
> can come up with a simpler solution if you describe your actual requirement 
> 
> in more detail.

hi,
I want to iterate in the inner loop by reading each line till some condition is 
met.how can i do that. Thanks for this code.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: scipy 11 and scipy 12

[Oscar Benjamin]

On 18 September 2013 03:48, Steven D'Aprano  wrote:
> On Tue, 17 Sep 2013 20:06:44 -0400, Susan Lubbers wrote:
>
>> Our group is a python 2.7 which is installed in a shared area.  We have
>> scipy 11 installed in site-packages.  How would I install scipy 12 so
>> that I used the shared install of python but scipy 12 instead of 11?
>
> If you are using Python 2.6 or better, you should be able to include the
> option "--user" when installing Scipy using either pip or distutils. I
> haven't tried these, but:
>
> # using pip:
> pip install --install-option="--user" scipy

Is there a difference between --install-option="--user" and just
passing --user directly?

> If that fails, follow the advice given here to install from svn:
> http://stackoverflow.com/questions/2213551/installing-scipy-with-pip
>
>
> pip install --install-option="--user" git+http://github.com/scipy/scipy/
>
>
> Otherwise, if you are installing from source using distutils, add the
> --user option directly:
>
> python setup.py install --user scipy

To be clear any of the above options are for building scipy from
source which means you need a C compiler, a Fortran compiler and to
separately build/install BLAS/LAPACK:
http://stackoverflow.com/questions/7496547/python-scipy-needs-blas

The best instructions for meeting these requirements depend on your OS
(are you using Windows?).

For Python 2.7 I think that easy_install will be able to install from
the sourceforge binaries, e.g

easy_install --user scipy

but I may be wrong.

You'll need to ensure that you don't have a mismatch between
numpy/scipy versions and I don't know if easy_install will handle that
for you. This may mean separately installing numpy as well.


Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Peter Otten]

nikhil Pandey wrote:

> On Wednesday, September 18, 2013 5:14:10 PM UTC+5:30, Peter Otten wrote:

> I want to iterate in the inner loop by reading each line till some
> condition is met.how can i do that. Thanks for this code.

That's not what I had in mind when I asked you to

>> describe your actual requirement in more detail.

Anyway, change

[...]
>> f, g = tee(f)
>> for inner in islice(g, 3):
>> print "   ", inner,
>> break
[...]

to

f, g = tee(f)
for inner in g:
if some condition:
break
print "   ", inner,
break

in my example.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Roy Smith]

> > On Wed, Sep 18, 2013 at 9:12 PM, nikhil Pandey  
> > wrote:
> >> hi,
> >> I want to iterate over the lines of a file and when i find certain lines, 
> >> i need another loop starting from the next of that "CERTAIN" line till a 
> >> few (say 20) lines later.
> >> so, basically i need two pointers to lines (one for outer loop(for each 
> >> line in file)) and one for inner loop. How can i do that in python?
> >> please help. I am stuck up on this.
> [...]

In article ,
 Dave Angel  wrote:
[I hope I unwound the multi-layer quoting right]
> In addition, is this really a text file?  For binary files, you could
> use seek(), and manage things yourself.  But that's not strictly legal
> in a text file, and may work on one system, not on another.

Why is seek() not legal on a text file?  The only issue I'm aware of is 
the note at http://docs.python.org/2/library/stdtypes.html, which says:

"On Windows, tell() can return illegal values (after an fgets()) when 
reading files with Unix-style line-endings. Use binary mode ('rb') to 
circumvent this problem."

so, don't do that (i.e. read unix-line-terminated files on windows).  
But assuming you're not in that situation, it seems like something like 
this this should work:

> I'd suggest you open the file twice, and get two file objects.  Then you
> can iterate over them independently.

and use tell() to keep them in sync.  Something along the lines of (not 
tested):

f1 = open("my_file")
f2 = open("my_file")

while True:
   where = f1.tell()
   line = f1.readline()
   if not line:
  break
   if matches_pattern(line):
  f2.seek(where)
  for i in range(20):
 line = f2.readline()
 print line

Except for the specific case noted above (i.e. reading a unix file on a 
windows box, so don't do that), it doesn't matter that seek() does funny 
things with windows line endings, because tell() does the same funny 
things.  Doing f2.seek(f1.tell()) will get the two file pointers into 
the same place in both files.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Oscar Benjamin]

On 18 September 2013 13:56, Roy Smith  wrote:
>
>> > On Wed, Sep 18, 2013 at 9:12 PM, nikhil Pandey 
>> > wrote:
>> >> hi,
>> >> I want to iterate over the lines of a file and when i find certain lines,
>> >> i need another loop starting from the next of that "CERTAIN" line till a
>> >> few (say 20) lines later.
>> >> so, basically i need two pointers to lines (one for outer loop(for each
>> >> line in file)) and one for inner loop. How can i do that in python?
>> >> please help. I am stuck up on this.
>> [...]
>
> In article ,
>  Dave Angel  wrote:
> [I hope I unwound the multi-layer quoting right]
>> In addition, is this really a text file?  For binary files, you could
>> use seek(), and manage things yourself.  But that's not strictly legal
>> in a text file, and may work on one system, not on another.
>
> Why is seek() not legal on a text file?  The only issue I'm aware of is
> the note at http://docs.python.org/2/library/stdtypes.html, which says:
>
> "On Windows, tell() can return illegal values (after an fgets()) when
> reading files with Unix-style line-endings. Use binary mode ('rb') to
> circumvent this problem."
>
> so, don't do that (i.e. read unix-line-terminated files on windows).
> But assuming you're not in that situation, it seems like something like
> this this should work:
>
>> I'd suggest you open the file twice, and get two file objects.  Then you
>> can iterate over them independently.

There's no need to use OS resources by opening the file twice or to
screw up the IO caching with seek(). Peter's version holds just as
many lines as is necessary in an internal Python buffer and performs
the minimum possible amount of IO. I would expect this to be more
efficient as well as less error-prone on Windows.


Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: linregress and polyfit

[chitturk]

Thanks - that helps ... but it is puzzling because

np.random.normal(0.0,1.0,1) returns exactly one 
and when I checked the length of "z", I get 21 (as before) ... 


On Tuesday, September 17, 2013 9:34:39 PM UTC-5, Krishnan wrote:
> I created an xy pair
> 
> 
> 
> y = slope*x + intercept
> 
> 
> 
> then I added some noise to "y" using
> 
> 
> 
> numpy.random.normal - call it z 
> 
> 
> 
> I could recover the slope, intercept from (x,y) using linregress
> 
> BUT cannot calculate the slope, intercept from (x, z)
> 
> 
> 
> What is puzzling is that for both pairs (x,y) and (x,z) the
> 
> polyfit (order 1) works just fine (gives me the slope, intercept)
> 
> --
> 
> import numpy as np
> 
> import scipy
> 
> # create a straight line, y= slope*x + intercept 
> 
> x = np.linspace(0.0,10.0,21)  
> 
> slope = 1.0  
> 
> intercept = 3.0   
> 
> y = []  
> 
> for i in range(len(x)):  
> 
> y.append(slope*x[i] + intercept)  
> 
> # now create a data file with noise
> 
> z= []  
> 
> for i in range(len(x)):  
> 
> z.append(y[i] + 0.1*np.random.normal(0.0,1.0,1))  
> 
> # now calculate the slope, intercept using linregress
> 
> from scipy import stats  
> 
> # No error here this is OK, works for x, y
> 
> cslope, cintercept, r_value, p_value, std_err = stats.linregress(x,y)
> 
> print cslope, cintercept
> 
> # I get an error here
> 
> #ValueError: array dimensions must agree except for d_0
> 
> nslope, nintercept, nr_value, np_value, nstd_err = stats.linregress(x,z)  
> 
> print nslope, nintercept
> 
> # BUT polyfit works fine, polynomial or order 1 with both data sets
> 
> ncoeffs = scipy.polyfit(x,z,1)  
> 
> print ncoeffs
> 
> coeffs = scipy.polyfit(x,y,1)  
> 
> print coeffs

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Roy Smith]

> Dave Angel  wrote (and I agreed with):
>> I'd suggest you open the file twice, and get two file objects.  Then you
>> can iterate over them independently.


On Sep 18, 2013, at 9:09 AM, Oscar Benjamin wrote:
> There's no need to use OS resources by opening the file twice or to
> screw up the IO caching with seek().

There's no reason NOT to use OS resources.  That's what the OS is there for; to 
make life easier on application programmers.  Opening a file twice costs almost 
nothing.  File descriptors are almost as cheap as whitespace.

> Peter's version holds just as many lines as is necessary in an
> internal Python buffer and performs the minimum possible
> amount of IO.

I believe by "Peter's version", you're talking about:

> from itertools import islice, tee 
> 
> with open("tmp.txt") as f: 
> while True: 
> for outer in f: 
> print outer, 
> if "*" in outer: 
> f, g = tee(f) 
> for inner in islice(g, 3): 
> print "   ", inner, 
> break 
> else: 
> break 


There's this note from 
http://docs.python.org/2.7/library/itertools.html#itertools.tee:

> This itertool may require significant auxiliary storage (depending on how 
> much temporary data needs to be stored). In general, if one iterator uses 
> most or all of the data before another iterator starts, it is faster to use 
> list() instead of tee().


I have no idea how that interacts with the pattern above where you call tee() 
serially.  You're basically doing

with open("my_file") as f:
while True:
f, g = tee(f)

Are all of those g's just hanging around, eating up memory, while waiting to be 
garbage collected?  I have no idea.  But I do know that no such problems exist 
with the two file descriptor versions.






> I would expect this to be more
> efficient as well as less error-prone on Windows.
> 
> 
> Oscar
> 


---
Roy Smith
r...@panix.com



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Please use the Python Job Board for recruiting (was: Python Developers)

[Skip Montanaro]

On Wed, Sep 18, 2013 at 2:57 AM, Ben Finney  wrote:
> Kelly Lurz  writes:
>
>> List
>
> Please do not use this discussion forum for job advertisements.
>
> Instead, please use the Python Job Board designed for this purpose
> http://www.python.org/community/jobs/>.

Agreed. This recruiter also spammed the Chicago Python list. Although
the post was clearly Python-related, and he/she also used the template
we ask for on the job board (thus not completely clueless), I consider
these posts spam as said recruiter isn't considering the focus of the
various mailing lists before posting to them.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Neil Cerutti]

On 2013-09-13, Chris Angelico  wrote:
> On Sat, Sep 14, 2013 at 5:32 AM, Terry Reedy  wrote:
>> Poetry, including that in English, often *is* concerned with formatting.
>> Code is more like poetry than prose.
>>
>>
>>> You can take this
>>> paragraph of text, unwrap it, and then reflow it to any width you
>>> like, without materially changing my points.
>>
>>
>> But you cannot do that with poetry!
>
> Evangelical vicar in want of a portable second-hand font. Would
> dispose, for the same, of a portrait, in frame, of the Bishop-elect of
> Vermont.
>
> I think you could quite easily reconstruct the formatting of
> that, based on its internal structure. Even in poetry, English
> doesn't depend on its formatting nearly as much as Python does;
> and even there, it's line breaks, not indentation - so we're
> talking more like REXX than Python. In fact, it's not uncommon
> for poetry to be laid out on a single line with slashes to
> divide lines:

There's lots of poetry with significant indentation, though.
Imbuing the shape of the text on the page with significance is a
thing.

-- 
Neil Cerutti
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Chris Angelico]

On Thu, Sep 19, 2013 at 12:57 AM, Neil Cerutti  wrote:
> There's lots of poetry with significant indentation, though.
> Imbuing the shape of the text on the page with significance is a
> thing.

And you can do that with C code, too. Doesn't mean that indentation is
important to C; it means that you're layering different types of
information into a single piece of work. It's like Perl code drawn in
the shape of a camel - a beautiful hack.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Neil Cerutti]

On 2013-09-18, Chris Angelico  wrote:
> On Thu, Sep 19, 2013 at 12:57 AM, Neil Cerutti  wrote:
>> There's lots of poetry with significant indentation, though.
>> Imbuing the shape of the text on the page with significance is a
>> thing.
>
> And you can do that with C code, too. Doesn't mean that
> indentation is important to C; it means that you're layering
> different types of information into a single piece of work.
> It's like Perl code drawn in the shape of a camel - a beautiful
> hack.

I just meant you can't condense whitespace in a poem and retain
all its meaning. It will break certain kinds of quotation styles
in publications, as well.

-- 
Neil Cerutti
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Chris Angelico]

On Thu, Sep 19, 2013 at 1:08 AM, Neil Cerutti  wrote:
> On 2013-09-18, Chris Angelico  wrote:
>> On Thu, Sep 19, 2013 at 12:57 AM, Neil Cerutti  wrote:
>>> There's lots of poetry with significant indentation, though.
>>> Imbuing the shape of the text on the page with significance is a
>>> thing.
>>
>> And you can do that with C code, too. Doesn't mean that
>> indentation is important to C; it means that you're layering
>> different types of information into a single piece of work.
>> It's like Perl code drawn in the shape of a camel - a beautiful
>> hack.
>
> I just meant you can't condense whitespace in a poem and retain
> all its meaning. It will break certain kinds of quotation styles
> in publications, as well.

Sure. I'm still trying to work out if it's possible to deliver a
verbal speech with fancy information in its written version... English
is a fun language to tinker with!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Travis Griggs]

On Sep 18, 2013, at 5:07 AM, nikhil Pandey  wrote:

> On Wednesday, September 18, 2013 4:51:51 PM UTC+5:30, Chris Angelico wrote:
>> On Wed, Sep 18, 2013 at 9:12 PM, nikhil Pandey  
>> wrote:
>> 
>>> hi,
>> 
>>> I want to iterate over the lines of a file and when i find certain lines, i 
>>> need another loop starting from the next of that "CERTAIN" line till a few 
>>> (say 20) lines later.
>> 
>>> so, basically i need two pointers to lines (one for outer loop(for each 
>>> line in file)) and one for inner loop. How can i do that in python?
>> 
>>> please help. I am stuck up on this.
>> 
>> 
>> 
>> After the inner loop finishes, do you want to go back to where the
>> 
>> outer loop left off, or should the outer loop continue from the point
>> 
>> where the inner loop stopped? In other words, do you want to locate
>> 
>> overlapping sections, or not? Both are possible, but the solutions
>> 
>> will look somewhat different.
>> 
>> 
>> 
>> ChrisA
> 
> Hi Chris,
> After the inner loop finishes, I want to go back to the next line from where 
> the outer loop was left i.e the lines of the inner loop will be traversed 
> again in the outer loop.
> 1>>I iterate over lines of the file
> 2>> when i find a match in a certain line, i start another loop till some 
> condition is met in the subsequent lines
> 3>> then i come back to where i left and repeat 1(ideally i want to delete 
> that line in inner loop where that condition is met, but even if it is not 
> deleted, its OK)


Just curious, do you really need two loops and file handles? Without better 
details about what you're really doing, but as you've provided more detail, it 
seems to me that just iterating the lines of the file, and using a latch 
boolean to indicate when you should do additional processing on lines might be 
easier. I modified Chris's example input to look like:

alpha
*beta
gamma+
delta
epsilon
zeta
*eta
kappa
tau
pi+
omicron

And then shot it with the following:

#!/usr/bin/env python3
with open("samplein.txt") as file:
reversing = False
for line in (raw.strip() for raw in file):
if reversing:
print('', line[::-1], '')
reversing = not line.endswith('+')
else:
print(line)
reversing = line.startswith('*')

Which begins reversing lines as its working through them, until a different 
condition is met.

Travis Griggs


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[William Ray Wing]

On Sep 18, 2013, at 11:12 AM, Chris Angelico  wrote:

> On Thu, Sep 19, 2013 at 1:08 AM, Neil Cerutti  wrote:
>> On 2013-09-18, Chris Angelico  wrote:
>>> On Thu, Sep 19, 2013 at 12:57 AM, Neil Cerutti  wrote:
 There's lots of poetry with significant indentation, though.
 Imbuing the shape of the text on the page with significance is a
 thing.
>>> 
>>> And you can do that with C code, too. Doesn't mean that
>>> indentation is important to C; it means that you're layering
>>> different types of information into a single piece of work.
>>> It's like Perl code drawn in the shape of a camel - a beautiful
>>> hack.
>> 
>> I just meant you can't condense whitespace in a poem and retain
>> all its meaning. It will break certain kinds of quotation styles
>> in publications, as well.
> 
> Sure. I'm still trying to work out if it's possible to deliver a
> verbal speech with fancy information in its written version... English
> is a fun language to tinker with!
> 
> ChrisA
> -- 
> 

Just to add a data point on the importance of formatting in language.  New 
Testament Greek is/was written with no punctuation, no spaces between words, 
and no distinction between upper and lower case.  This frequently results in 
the Greek equivalent of the follow English:  "iamnowhereiameverywhere"  which 
can be "translated" as either "I am now here, I am everywhere."  _or_ "I am 
nowhere, I am everywhere."  In other words, two very different meanings.  So, 
without context or further information, the intent of the original writer can't 
be inferred.

-Bill
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[wxjmfauth]

>>> 1and 0
0
>>> 'a'or 1
'a'
>>> 5if True else 999
5


jmf

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Language design

[Neil Cerutti]

On 2013-09-18, wxjmfa...@gmail.com  wrote:
 1and 0
> 0
 'a'or 1
> 'a'
 5if True else 999
> 5

Curse you, FSR!

Oh, wait...

-- 
Neil Cerutti
-- 
https://mail.python.org/mailman/listinfo/python-list

lxml question -- creating an etree.Element attribute with ':' in the name

[Roy Smith]

I can create an Element with a 'foo' attribute by doing:

etree.Element('my_node_name', foo="spam")

But, how do I handle something like:

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";, since "xmlns:xsi" isn't 
a valid python identifier?

---
Roy Smith
r...@panix.com



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping characters from windows clipboard with win32clipboard from excel

[random832]

On Fri, Sep 13, 2013, at 12:09, random...@fastmail.us wrote:
> Anyway, to match behavior found in other applications when pasting from
> the clipboard, I would suggest using:
> 
> if s.contains('\0'): s = s[:s.index('\0')]
> 
> Which will also remove non-null bytes after the first null (but if the
> clipboard contains these, it won't be pasted into e.g. notepad).

I did some testing and confirmed you MUST do this - if you copy multiple
things from Excel, it may reuse this buffer and not clear the end of it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: lxml question -- creating an etree.Element attribute with ':' in the name

[Zachary Ware]

On Wed, Sep 18, 2013 at 1:59 PM, Roy Smith  wrote:
> I can create an Element with a 'foo' attribute by doing:
>
> etree.Element('my_node_name', foo="spam")
>
> But, how do I handle something like:
>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";, since "xmlns:xsi" 
> isn't a valid python identifier?
>

Try this:

etree.Element('my_node_with_invalid_identifiers_name', **{'xmlns:xsi': 'spam'})

HTH,

-- 
Zach
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping characters from windows clipboard with win32clipboard from excel

[stephen . boulet]

Thanks to everyone for their help. Using everyone's suggestions, this seems to 
work:

import win32clipboard, win32con

def getclipboard():
win32clipboard.OpenClipboard()
s = win32clipboard.GetClipboardData(win32con.CF_UNICODETEXT)
win32clipboard.CloseClipboard()
if '\0' in s:
s = s[:s.index('\0')]
return s
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: lxml question -- creating an etree.Element attribute with ':' in the name

[Burak Arslan]

On 09/18/13 21:59, Roy Smith wrote:
> I can create an Element with a 'foo' attribute by doing:
>
> etree.Element('my_node_name', foo="spam")
>
> But, how do I handle something like:
>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";, since "xmlns:xsi" 
> isn't a valid python identifier?
>
>

xmlns: is a prefix with a special meaning: it defines an xml namespaces
prefix. you should read about how they work.

The following:

Element('{http://www.w3.org/2001/XMLSchema-instance}my_node_name')

will generate a proper xmlns declaration for you. It may not be the same
every time, but it will do the job just as well.

btw, if you need to generate xml schemas, have a look at spyne:
http://spyne.io

Specifically:
https://github.com/arskom/spyne/blob/master/examples/xml/schema.py

best,
burak
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping characters from windows clipboard with win32clipboard from excel

[MRAB]

On 18/09/2013 20:28, stephen.bou...@gmail.com wrote:

Thanks to everyone for their help. Using everyone's suggestions, this seems to 
work:

import win32clipboard, win32con

def getclipboard():
 win32clipboard.OpenClipboard()
 s = win32clipboard.GetClipboardData(win32con.CF_UNICODETEXT)
 win32clipboard.CloseClipboard()
 if '\0' in s:
 s = s[:s.index('\0')]
 return s


That'll look for a null character and, if it finds one, then look for
it again. Of course, that probably isn't an issue in practice.

However, an alternative way of doing it is to use the .partition method:

 return s.partition('\0')[0]

--
https://mail.python.org/mailman/listinfo/python-list

Re: linregress and polyfit

[Dave Angel]

On 18/9/2013 09:38, chitt...@uah.edu wrote:

> Thanks - that helps ... but it is puzzling because
>
> np.random.normal(0.0,1.0,1) returns exactly one 
> and when I checked the length of "z", I get 21 (as before) ... 
>
>

I don't use Numpy, so this is just a guess, plus reading one web page.

According to:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.normal.html

the 3rd argument to normal should be a tuple.  So if you want a single
element, you should have made it (1,)

As for checking the 'length of "z"' did you just use the len() function?
That just tells you the first dimension.  Have you tried simply printing
out "z" ?

-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python GUI?

[Michael Torrie]

On 09/17/2013 10:19 AM, Benjamin Kaplan wrote:
> Sure. Every platform provides its own GUI library (Cocoa on Mac OS X,
> Win32 on Windows). Other programs that want to hook into yours, such
> as screen readers, are familiar with the platform's native GUI
> elements- it knows what a Win32 combo box is, and it knows how to read
> the text inside it.
> 
> The other way to make a GUI is to take a blank canvas and draw on it
> yourself. This is more flexible and provides a more consistent
> experience across platforms, but unless you specifically go out of
> your way to provide hooks for other programs to jump in, all they see
> is a bunch of pixels on the screen. In addition, drawing your own
> stuff won't necessarily give you the "normal for the operating system"
> behavior on other things, like tab behavior. It's possible for
> non-native GUI environments to mimic this behavior (and QT does a
> pretty good job of this), but there's always going to be little things
> that seem a bit off.
> 
> The situation is a bit more complicated because QT is the native
> toolkit on KDE, so in that environment, QT will be more "correct" than
> wx, which would be using GTK if present and plain X11 if it isn't.

I don't think the distinction you're drawing is really all that
important.  Almost all platforms now have accessibility APIs which are a
way better way to go than relying than trying to scrape the information
by peaking into GUI structures that could change.

And on Windows, while the Win32 widget stuff you speak of is a lowest
common denominator, few modern apps use it directly anymore.  MS Office
set the the example early on by creating a new widget set for every
release (using the blank canvas method you describe!).  .Net apps using
winforms use yet another widget set.  They all now use the theming API
to draw their widgets (just like Qt does on Windows), but do their own
even processing, etc.

And it turns out that wxWidget's use of the old win32 widgets means that
some widgets just look plain old and out of place on a new Windows 7 or
8 machine, though MS has tried to make the older widgets work better
with the new theming api:

http://wiki.wxwidgets.org/WxFAQ#Why_does_my_app_take_a_Windows-95-like_look_on_Windows_.3F

My point is, the former method isn't actually the best way to go in the
long run.  The second turns out to work out better, once things like
accessibility frameworks and theming apis are in place.  Perhaps in the
future MS Windows will just provide a drawing method and a theming api.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping characters from windows clipboard with win32clipboard from excel

[Dave Angel]

On 18/9/2013 15:40, MRAB wrote:

> On 18/09/2013 20:28, stephen.bou...@gmail.com wrote:
>> Thanks to everyone for their help. Using everyone's suggestions, this seems 
>> to work:
>>
>> import win32clipboard, win32con
>>
>> def getclipboard():
>>  win32clipboard.OpenClipboard()
>>  s = win32clipboard.GetClipboardData(win32con.CF_UNICODETEXT)
>>  win32clipboard.CloseClipboard()
>>  if '\0' in s:
>>  s = s[:s.index('\0')]
>>  return s
>>
> That'll look for a null character and, if it finds one, then look for
> it again. Of course, that probably isn't an issue in practice.
>
> However, an alternative way of doing it is to use the .partition method:
>
>   return s.partition('\0')[0]

So is the bug in Excel, in Windows, or in the Python library?  Somebody
is falling down on the job;  if Windows defines the string as ending at
the first null, then the Python interface should use that when defining
the text defined with CF_UNICODETEXT.

Or maybe it's an example of ill-defined Windows specs.

-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Dave Angel]

On 18/9/2013 10:36, Roy Smith wrote:

>> Dave Angel  wrote (and I agreed with):
>>> I'd suggest you open the file twice, and get two file objects.  Then you
>>> can iterate over them independently.
>
>
> On Sep 18, 2013, at 9:09 AM, Oscar Benjamin wrote:
>> There's no need to use OS resources by opening the file twice or to
>> screw up the IO caching with seek().
>
> There's no reason NOT to use OS resources.  That's what the OS is there for; 
> to make life easier on application programmers.  Opening a file twice costs 
> almost nothing.  File descriptors are almost as cheap as whitespace.
>
>> Peter's version holds just as many lines as is necessary in an
>> internal Python buffer and performs the minimum possible
>> amount of IO.
>
> I believe by "Peter's version", you're talking about:
>
>> from itertools import islice, tee 
>> 
>> with open("tmp.txt") as f: 
>> while True: 
>> for outer in f: 
>> print outer, 
>> if "*" in outer: 
>> f, g = tee(f) 
>> for inner in islice(g, 3): 
>> print "   ", inner, 
>> break 
>> else: 
>> break 
>
>
> There's this note from 
> http://docs.python.org/2.7/library/itertools.html#itertools.tee:
>
>> This itertool may require significant auxiliary storage (depending on how 
>> much temporary data needs to be stored). In general, if one iterator uses 
>> most or all of the data before another iterator starts, it is faster to use 
>> list() instead of tee().
>
>
> I have no idea how that interacts with the pattern above where you call tee() 
> serially.  You're basically doing
>
> with open("my_file") as f:
> while True:
>   f, g = tee(f)
>
> Are all of those g's just hanging around, eating up memory, while waiting to 
> be garbage collected?  I have no idea.  But I do know that no such problems 
> exist with the two file descriptor versions.
>
>
>
>
>
>
>> I would expect this to be more
>> efficient as well as less error-prone on Windows.
>> 
>> 
>> Oscar
>> 
>
>
> ---
> Roy Smith
> r...@panix.com
>
>
>
>
>
>  type="cite">Dave Angel < href="mailto:da...@davea.name";>da...@davea.name> wrote (and I agreed 
> with):I'd 
> suggest you open the file twice, and get two file objects.  Then 
> you type="cite">can iterate over them 
> independently.On Sep 
> 18, 2013, at 9:09 AM, Oscar Benjamin wrote: type="cite">There's no need to use OS resources by opening the file 
> twice or toscrew up the IO caching with 
> seek().There's no reason NOT to use OS 
> resources.  That's what the OS is there for; to make life easier on 
> application programmers.  Opening a file twice costs almost nothing. 
>  File descrip
 tors are almost as cheap as whitespace.Peter's version holds just as many lines as is necessary 
in aninternal Python buffer and 
performs the minimum possibleamount of IO.I believe 
by "Peter's version", you're talking 
about:from itertools import 
islice, tee with open("tmp.txt") as f:     while 
True:         for outer in f:             print outer,             if "*" in 
outer:       
          f, g = tee(f)                 for inner in 
islice(g, 3):             
        print "   ", inner,       
          break         
else:             
break There's this 
note from http://docs.python.org/2.7/library/itertools.html#itertools.tee";>http://docs.python.org/2.7/library/itertools.html#itertools.tee:This itertool may require significant auxiliary storage (depending 
on how much temporary data needs to be stored). In general, if one 
iterator uses most or all of the data before another iterator starts, it is 
faster to use list() instead of tee().I have no idea how that 
interacts with the pattern above where you call tee() serially.  You're 
basically doingwith open("my_file") as 
f:while True:f, g = 
tee(f)Are all of those g's just hanging around, 
eating up memory, while waiting to be garbage collected?  I have no idea. 
 But I do know that no such problems exist with the two file descriptor 
versions.I would expect this to be moreefficient as well as less 
error-prone on 
Windows.Oscar
>  class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 
> 0); font-family: Helvetica; font-style: normal; font-variant: normal; 
> font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; 
> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; 
> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; 
> -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: 
> none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; 
> font-size: medium; "> class="Apple-interchange-newline">---Roy Smith href="mailto:r...@panix.com";>r...@panix.com
> 
> 
>

And if you're willing to ignore the possibility that the text file has
unix line endings, I'm willing to ignore the possibility that the text
file has a huge number of lines.  Everything is MUCH simpler if one
assumes re

Re: Stripping characters from windows clipboard with win32clipboard from excel

[Neil Hodgson]

Dave Angel:


So is the bug in Excel, in Windows, or in the Python library?  Somebody
is falling down on the job;  if Windows defines the string as ending at
the first null, then the Python interface should use that when defining
the text defined with CF_UNICODETEXT.


   Everything is performing correctly. win32clipboard is low-level 
direct access to the Win32 clipboard API. A higher level API which is 
more easily used from Python could be defined on top of this if anyone 
was motivated.


   Neil

--
https://mail.python.org/mailman/listinfo/python-list

Python with Delphi

[Errol Anderson]

Is there anyone out there who uses Python with Delphi, or knows someone who
uses Python with Delphi or who used to use Python with Delphi?  The latest
version of "Python for Delphi" (P4D) works fine with Python 2.5 and Delphi
2007, but not for Python 2.7.  Any assistance gratefully received.

 

Errol Anderson

 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Stripping characters from windows clipboard with win32clipboard from excel

[Dave Angel]

On 18/9/2013 17:40, Neil Hodgson wrote:

> Dave Angel:
>
>> So is the bug in Excel, in Windows, or in the Python library?  Somebody
>> is falling down on the job;  if Windows defines the string as ending at
>> the first null, then the Python interface should use that when defining
>> the text defined with CF_UNICODETEXT.
>
> Everything is performing correctly. win32clipboard is low-level 
> direct access to the Win32 clipboard API. A higher level API which is 
> more easily used from Python could be defined on top of this if anyone 
> was motivated.
>
> Neil

Clearly you miss the point.  If the clipboard API is defined to return a
null-terminated string, then the problem is in the Python library which
doesn't do a strlen() (or the wide-character equivalent;  I forget its
name) on the results.

But there's a big if there.  Somebody is either ill specified or poorly
implemented here.

-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mutlifile inheritance problem

[Peter Cacioppi]

This is a very old topic, but here is a trick for single inheritance. (The 
problem you allude to isn't restricted to multiple inheritance).

Any class with a single parent simply defines this function.


def mySuper(self) : 
return super(self.__class__, self)
  
And then any parent function be referenced like
self.mySuper().foo()

This includes __init__.

You can read more here.

http://atlee.ca/blog/posts/blog20081121python-reload-danger-here-be-dragons.html


On Thursday, March 21, 2002 2:03:23 PM UTC-7, Marc wrote:
> I have classes defined in different files and would like to inherit
> from a class in file A.py for a class in file B.py but am running into
> problems.  I'm using Python 1.5.2 on Windows NT
> 
> Here's a specific example:
> 
> 
> file cbase01.py:
> 
> class CBase:
> 
> def __init__(self):
> self.cclass = None
> print "cbase"
> 
> class CImStream(CBase):
> 
> def __init(self):
> CBase.__init__(self)
> print "CImStream"
> 
> *
> in file wrappers_A01.py: 
> 
> import cbase01
> reload(cbase01)
> 
> class ImStream_SavedBitmaps(cbase01.CImStream):
> 
> def __init__(self):
> cbase.CImStream.__init__(self)
> print "SavedBitmaps"
> 
> **
> in file sequencer01.py
> 
> import cbase01# the offending lines, program works 
> reload(cbase01)   # if I comment these out.
> 
> class Sequencer:
> 
> def Append(self, item):
> pass
> 
> *
> in test02.py
> 
> import wrappers_A01
> reload(wrappers_A01)
> 
> import sequencer01
> reload(sequencer01)
> 
> x0 = wrappers_A01.ImStream_SavedBitmaps()
> ***
> 
> If I run test02 I get the traceback
> 
> Traceback (innermost last):
>   File "", line 1, in ?
>   File "D:\PythonCode\pna\eyeTracking\tests\test02.py", line 15, in ?
> x0 = wrappers_A01.ImStream_SavedBitmaps()
>   File "D:\PythonCode\pna\eyeTracking\tests\wrappers_A01.py", line 21,
> in __init__
> cbase.CImStream.__init__(self)
> TypeError: unbound method must be called with class instance 1st
> argument
> 
> 
> Any ideas what I am doing wrong?
> 
> Thanks,
> Marc
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Tryign to send mail via a python script by using the local MTA

[Dave Angel]

On 18/9/2013 18:31, Dennis Lee Bieber wrote:

> On Tue, 17 Sep 2013 18:34:50 -0400, William Ray Wing 
> declaimed the following:
>
>
>>
>>I think you need to read up on some of the most basic fundamentals of tcp/ip 
>>networking, i.e., the basis of the global internet.  EVERY network packet 
>>(and I do mean every) packet in an IP network carries both a source and a 
>>destination address in its header.  These are fundamentally necessary in 
>>order to allow the gateway router at the originating site to direct an 
>>outgoing packet to its destination, and allow the receiving host at the 
>>destination site to craft reply packets.
>>
>
>   Even worse -- IP packets are wrapped by Ethernet packets which use MAC
> addresses for direct routing between nodes... Granted, those MAC addresses
> may not propagate beyond the next defined gateway IP host, but they do
> (theoretically) identify the exact NIC that sent the packet.

Noting reliable about the MAC adress. Many systems permit or even
encourage spoofing.  And it gets overwritten at each hop, so the other
end cannot easily determine the number a client may have made up.

The first time I added a router to my cable modem, I had to tell it to
use (spoof) the old MAC address so that the cable modem didn't have to
be reprogrammed.

-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mutlifile inheritance problem

[Peter Cacioppi]

One more comment - my trick has some utility with multiple inheritance, but you 
really need to understand super() to and method resolution ordering in that 
case (as, I suppose, you ought to whenever you cross the Rubicon beyond single 
inheritance). So it's a nice trick but YMMV

On Wednesday, September 18, 2013 4:54:00 PM UTC-7, Peter Cacioppi wrote:
> This is a very old topic, but here is a trick for single inheritance. (The 
> problem you allude to isn't restricted to multiple inheritance).
> 
> 
> 
> Any class with a single parent simply defines this function.
> 
> 
> 
> 
> 
> def mySuper(self) : 
> 
> return super(self.__class__, self)
> 
>   
> 
> And then any parent function be referenced like
> 
> self.mySuper().foo()
> 
> 
> 
> This includes __init__.
> 
> 
> 
> You can read more here.
> 
> 
> 
> http://atlee.ca/blog/posts/blog20081121python-reload-danger-here-be-dragons.html
> 
> 
> 
> 
> 
> On Thursday, March 21, 2002 2:03:23 PM UTC-7, Marc wrote:
> 
> > I have classes defined in different files and would like to inherit
> 
> > from a class in file A.py for a class in file B.py but am running into
> 
> > problems.  I'm using Python 1.5.2 on Windows NT
> 
> > 
> 
> > Here's a specific example:
> 
> > 
> 
> > 
> 
> > file cbase01.py:
> 
> > 
> 
> > class CBase:
> 
> > 
> 
> > def __init__(self):
> 
> > self.cclass = None
> 
> > print "cbase"
> 
> > 
> 
> > class CImStream(CBase):
> 
> > 
> 
> > def __init(self):
> 
> > CBase.__init__(self)
> 
> > print "CImStream"
> 
> > 
> 
> > *
> 
> > in file wrappers_A01.py: 
> 
> > 
> 
> > import cbase01
> 
> > reload(cbase01)
> 
> > 
> 
> > class ImStream_SavedBitmaps(cbase01.CImStream):
> 
> > 
> 
> > def __init__(self):
> 
> > cbase.CImStream.__init__(self)
> 
> > print "SavedBitmaps"
> 
> > 
> 
> > **
> 
> > in file sequencer01.py
> 
> > 
> 
> > import cbase01# the offending lines, program works 
> 
> > reload(cbase01)   # if I comment these out.
> 
> > 
> 
> > class Sequencer:
> 
> > 
> 
> > def Append(self, item):
> 
> > pass
> 
> > 
> 
> > *
> 
> > in test02.py
> 
> > 
> 
> > import wrappers_A01
> 
> > reload(wrappers_A01)
> 
> > 
> 
> > import sequencer01
> 
> > reload(sequencer01)
> 
> > 
> 
> > x0 = wrappers_A01.ImStream_SavedBitmaps()
> 
> > ***
> 
> > 
> 
> > If I run test02 I get the traceback
> 
> > 
> 
> > Traceback (innermost last):
> 
> >   File "", line 1, in ?
> 
> >   File "D:\PythonCode\pna\eyeTracking\tests\test02.py", line 15, in ?
> 
> > x0 = wrappers_A01.ImStream_SavedBitmaps()
> 
> >   File "D:\PythonCode\pna\eyeTracking\tests\wrappers_A01.py", line 21,
> 
> > in __init__
> 
> > cbase.CImStream.__init__(self)
> 
> > TypeError: unbound method must be called with class instance 1st
> 
> > argument
> 
> > 
> 
> > 
> 
> > Any ideas what I am doing wrong?
> 
> > 
> 
> > Thanks,
> 
> > Marc

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mutlifile inheritance problem

[Ned Batchelder]

On 9/18/13 7:54 PM, Peter Cacioppi wrote:

This is a very old topic, but here is a trick for single inheritance. (The 
problem you allude to isn't restricted to multiple inheritance).

Any class with a single parent simply defines this function.


 def mySuper(self) :
 return super(self.__class__, self)
   
And then any parent function be referenced like

 self.mySuper().foo()


You can't use self.__class__ for super, it won't give you the right 
class if you are using subclasses:


class MyBase(object):
def mySuper(self):
return super(self.__class__, self)

def say(self):
print "This is MyBase!"


class Thing(MyBase):
def say_it(self):
print "The parent:"
self.mySuper().say()

def say(self):
print "this is Thing!"

class SubThing(Thing):
pass

thing = Thing()
thing.say_it()

sub = SubThing()
sub.say_it()

This produces:

The parent:
This is MyBase!
The parent:
this is Thing!

super() takes a class and an instance for a reason. If you could use 
self.__class__ for the class, then it would only take the instance. 
Super() needs to know the instance, but also needs to know the class 
it's being called from.


--Ned.


This includes __init__.

You can read more here.

http://atlee.ca/blog/posts/blog20081121python-reload-danger-here-be-dragons.html


On Thursday, March 21, 2002 2:03:23 PM UTC-7, Marc wrote:

I have classes defined in different files and would like to inherit
from a class in file A.py for a class in file B.py but am running into
problems.  I'm using Python 1.5.2 on Windows NT

Here's a specific example:


file cbase01.py:

class CBase:

 def __init__(self):
 self.cclass = None
 print "cbase"

class CImStream(CBase):

 def __init(self):
 CBase.__init__(self)
 print "CImStream"

*
in file wrappers_A01.py:

import cbase01
reload(cbase01)

class ImStream_SavedBitmaps(cbase01.CImStream):

 def __init__(self):
 cbase.CImStream.__init__(self)
 print "SavedBitmaps"

**
in file sequencer01.py

import cbase01# the offending lines, program works
reload(cbase01)   # if I comment these out.

class Sequencer:

 def Append(self, item):
 pass

*
in test02.py

import wrappers_A01
reload(wrappers_A01)

import sequencer01
reload(sequencer01)

x0 = wrappers_A01.ImStream_SavedBitmaps()
***

If I run test02 I get the traceback

Traceback (innermost last):
   File "", line 1, in ?
   File "D:\PythonCode\pna\eyeTracking\tests\test02.py", line 15, in ?
 x0 = wrappers_A01.ImStream_SavedBitmaps()
   File "D:\PythonCode\pna\eyeTracking\tests\wrappers_A01.py", line 21,
in __init__
 cbase.CImStream.__init__(self)
TypeError: unbound method must be called with class instance 1st
argument


Any ideas what I am doing wrong?

Thanks,
Marc


--
https://mail.python.org/mailman/listinfo/python-list

Re: mutlifile inheritance problem

[Steven D'Aprano]

On Wed, 18 Sep 2013 20:38:10 -0400, Ned Batchelder wrote:

> super() takes a class and an instance for a reason. If you could use
> self.__class__ for the class, then it would only take the instance.
> Super() needs to know the instance, but also needs to know the class
> it's being called from.

You're correct, of course, but I would word the reason why slightly 
differently.

super needs to know which instance it is being called from, but it also 
needs to know which class the method is defined in. Hence, given:

class Spam(Food):
def method(self):
print("Spam spam spam!")
super(Spam, self).method()

class PickledSpam(Spam):
pass


obj = PickledSpam()
obj.method()

super is being called from a PickledSpam instance, self, but the call is 
defined in the Spam class. super needs to know about both of them.

Of course, this hinges on how one defines "called from". I think it is 
helpful to understand methods as being called from the instance doing the 
calling rather than the class where they are defined.


-- 
Steven
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Steven D'Aprano]

On Wed, 18 Sep 2013 05:14:23 -0700, nikhil Pandey wrote:

> I want to iterate in the inner loop by reading each line till some
> condition is met.how can i do that. Thanks for this code.

while not condition:
read line


Re-write using Python syntax, and you are done.



-- 
Steven
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: scipy 11 and scipy 12

[Steven D'Aprano]

On Wed, 18 Sep 2013 13:28:44 +0100, Oscar Benjamin wrote:

> On 18 September 2013 03:48, Steven D'Aprano  wrote:
>> On Tue, 17 Sep 2013 20:06:44 -0400, Susan Lubbers wrote:
>>
>>> Our group is a python 2.7 which is installed in a shared area.  We
>>> have scipy 11 installed in site-packages.  How would I install scipy
>>> 12 so that I used the shared install of python but scipy 12 instead of
>>> 11?
>>
>> If you are using Python 2.6 or better, you should be able to include
>> the option "--user" when installing Scipy using either pip or
>> distutils. I haven't tried these, but:
>>
>> # using pip:
>> pip install --install-option="--user" scipy
> 
> Is there a difference between --install-option="--user" and just passing
> --user directly?

*shrug*

I don't have any experience with pip, so I don't know.


> For Python 2.7 I think that easy_install will be able to install from
> the sourceforge binaries, e.g
> 
> easy_install --user scipy
> 
> but I may be wrong.

If I recall correctly, and I may not, easy_install doesn't support per-
user installs with the --user option.




-- 
Steven
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python GUI?

[llanitedave]

On Friday, September 13, 2013 10:31:17 AM UTC-7, Eamonn Rea wrote:
> I don't like the idea of being able to drag and drop anything in the 
> programming world. Outside of that, I use D&D programs a lot. I got into GUI 
> programming because I thought that I could get away from them, but I guess 
> not.
> 
> 
> 
> Maybe I'm against them because if I can't code, I don't have anything else to 
> do with my time. If I don't program, the only other thing I have to do is... 
> well... nothing. So, because of this, they're making programming easier... by 
> not coding as much. Oh well, guess coding is dead :(

Why do you feel the need to project your personal issues onto the whole world?  
Nobody is forcing you to use Drag and Drop -- you simply have the option to do 
so, or not.  I don't use it either, but I sure don't fill up with angst just 
because someone else might actually enjoy using it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: iterating over a file with two pointers

[Steven D'Aprano]

On Wed, 18 Sep 2013 04:12:05 -0700, nikhil Pandey wrote:

> hi,
> I want to iterate over the lines of a file and when i find certain
> lines, i need another loop starting from the next of that "CERTAIN" line
> till a few (say 20) lines later. so, basically i need two pointers to
> lines (one for outer loop(for each line in file)) and one for inner
> loop. How can i do that in python? please help. I am stuck up on this.

No, you don't "need" two pointers to lines. That is just one way to solve 
this problem. You can solve it many ways.

One way, for small files (say, under one million lines), is to read the 
whole file into a list, then have two pointers to a line:

lines = file.readlines()
p = q = 0

while p < len(lines):
print(lines[p])
p += 1


then advance the pointers p and q as needed. This is the most flexible 
way to do it: you can have as many pointers as needed, you can back-
track, jump forward, jump back, and it is all high-speed random-access 
memory accesses. Except for the initial readlines, none of it is slow I/O 
processing.


Another solution is to use a state-machine:


for line in somefile:
if state == SCANNING:
do_something()
elif state == PROCESSING:
do_something_else()
elif state == WOBBLING:
wobble()
state = adjust_state(line)


You can combine the two, of course, and have a state machine with 
multiple pointers to a list of lines.

Using itertools.tee, you can potentially combine these solutions with the 
straightforward for-loop over a list. The danger of itertools.tee is that 
it may use as much memory as reading the entire file into memory at once, 
but the benefit is that it may use much less. But personally, I find list-
based processing with random-access by index much easier to understand 
that itertools.tee solutions.



-- 
Steven
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: lxml question -- creating an etree.Element attribute with ':' in the name

[dieter]

Roy Smith  writes:

> But, how do I handle something like:
>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";, since "xmlns:xsi" 
> isn't a valid python identifier?

Read about "lxml"'s "namespace" support.

-- 
https://mail.python.org/mailman/listinfo/python-list