[Tutor] sorting distances for nearest neighbor

2014-08-29 Thread LN A-go-go
Python Gurus,

How do you sort distances to prepare for nearest neighbor when the results 
returned look like this from the below statements?
DR=distance(n,XCOORDINATE, YCOORDINATE, IDCODE)
DZ=switch(n,XCOORDINATE, YCOORDINATE, IDCODE)



Distance from 1 to
16 is 43763.0757603
19 is 126516.988978
18 is 131011.694135
15 is 156251.651191
17 is 157090.833596
14 is 170437.744646
13 is 174344.799177
20 is 175368.495745
11 is 235415.474428
10 is 236267.509404
12 is 240007.657586
9 is 253298.673506
4 is 318134.525162
2 is 342924.627433
5 is 344436.98248
6 is 371599.399488
3 is 374494.086068
8 is 389923.9631
7 is 393889.530199
Distance from 2 to
3 is 32471.4335994
9 is 126093.246845
6 is 135032.661234
10 is 145083.703082
11 is 153462.088152
12 is 155269.121206
4 is 156434.810704
7 is 161054.471841
5 is 161119.250867
20 is 169483.547284
8 is 173243.878103
17 is 210880.788362
18 is 216248.340803
19 is 218147.66971
13 is 220831.360318
14 is 225608.192449
15 is 226192.408803
16 is 305477.356444
1 is 342924.627433
Distance from 3 to.

LN___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


[Tutor] extracting a cPickle/pickle file from inside a zip file

2014-08-29 Thread diliup gabadamudalige
HI! To all,

with open(from_dir + "\\" + name_py,"wb") as outfile:
cPickle.dump(str1,outfile)
after doing above I zip the file with some other files into a password
secured zip file using win rar.

I can extract all other files except the py file.

pload = cPickle.load(zf.open("my.py",pwd=password))
this gives an EOF

. Does any body know why?
Can anyone show how to extract a pickle file from a zipfile?

Thanks in advance


Diliup Gabadamudalige

http://www.diliupg.com
http://soft.diliupg.com/

**
This e-mail is confidential. It may also be legally privileged. If you are
not the intended recipient or have received it in error, please delete it
and all copies from your system and notify the sender immediately by return
e-mail. Any unauthorized reading, reproducing, printing or further
dissemination of this e-mail or its contents is strictly prohibited and may
be unlawful. Internet communications cannot be guaranteed to be timely,
secure, error or virus-free. The sender does not accept liability for any
errors or omissions.
**
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] sorting distances for nearest neighbor

2014-08-29 Thread R. Alan Monroe
> How do you sort distances to prepare for nearest neighbor when the
> results returned look like this

Have you read up on the "key" parameter for Python's sort() function?
You can use that, along with split() to do this.

Alan

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] extracting a cPickle/pickle file from inside a zip file

2014-08-29 Thread Alan Gauld

On 29/08/14 10:00, diliup gabadamudalige wrote:


pload = cPickle.load(zf.open("my.py",pwd=password))
this gives an EOF

. Does any body know why?


I've never used zipfile so I don't know, but could it be
related to the fact that your pickle file is binary?
Do you need any flags to tell zipfile that you're
extracting a binary file?

Just a thought,

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] sorting distances for nearest neighbor

2014-08-29 Thread Dave Angel
LN A-go-go  Wrote in message:
>
Please tell your email program to use text when posting here, The
 html you used can cause numerous problems. One of them prevents
 me from quoting context.

If you can get your data to a list of tuples, then you can sort
 that list by something like
   sorteddata = sorted (data, key = dist)

Of course you have to write dist. Something like

def dist (tup):
  return distance (tup [3], tup [0], tup [1], tup [2]

But of course this depends greatly on the contents of the tuple. 

You can also skip the sorting if you just want the lowest value by
 doing

 mintuple = min (data, key= dist)

Or if you need the 3 closest,  you can use heapq.nsmallest




-- 
DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Import from project's lib directory?

2014-08-29 Thread leam hall
On Thu, Aug 28, 2014 at 6:39 PM, Cameron Simpson  wrote:
> On 28Aug2014 22:36, Alan Gauld  wrote:
>>
>> On 28/08/14 19:03, leam hall wrote:
>>>
>>> python 2.4.3 on Red Hat Linux.
>>>
>>> I'm starting a project and want to break the scripts into "input",
>>> "output", and "ddl". I'd like to have a "lib" library directory for
>>> local modules. The library directory would be at the same level as the
>>> others.
>>>
>>> How do I get the library modules?
>>
>>
>> Add lib to the sys.path list?
>>
>> That may not be viable if you need it to be portable across systems,
>> although using os.getcwd to locate the current folder or an
>> environment variable to store the app root folder might be
>> options there.

The code will be imported to other machines so whatever I'm doing must
be portable. Right now my code looks like this:

###
## lib/mymodule.py

#!/usr/bin/env python

def howdy():
print("in mymodule")

###
## input/test_lib.py

#!/usr/bin/env python

from ..lib import mymodule
mymodule.howdy()

###
## Directory tree:
.
./input
./input/__init.py__
./input/test_lib.py
./lib
./lib/mymodule.py
./lib/__init.py__
./output


## In input, running test_lib.py
 ./test_lib.py
Traceback (most recent call last):
  File "./test_lib.py", line 3, in 
from ..lib import mymodule
ValueError: Attempted relative import in non-package


## the __init.py__ files are empty.


-- 
Mind on a Mission
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Import from project's lib directory?

2014-08-29 Thread leam hall
On Fri, Aug 29, 2014 at 9:39 AM, diliup gabadamudalige
 wrote:
> this is the easiest way to do this
>
> # one.py is in the \data\ directory
> # two.py is in the \data\img\ directory
> # three.py is in the \data\img\sc\ directory
>
> ## this program is in the learn_music dir
>
> ## expanding the file path add a . before the new dir
>
> ## each directory you need to load a module from needs to have __init__.py
> file inside the directory. It can be an empty file but it needs to be there.
> That's how python knows that you can load a module from this directory. my
> one.py, two.py and three.py modules each have a string called a in it. You
> can have classes or functions which can be called in the same way.
>
> from data import one  # / data/
> from data .img import two  # /data/img/
> from data .img.sc import three  # /data/img/sc/
>
> print one.a, two.a, three.a
>
> hope this helps


So far this doesn't work. I'm pretty convinced it's either a Python
version issue (2.4.3 and 2.6.6.) or pure Operator Error. Leaning
towards the latter.

Thanks!

Leam
-- 
Mind on a Mission
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] extracting a cPickle/pickle file from inside a zip file

2014-08-29 Thread diliup gabadamudalige
http://stackoverflow.com/questions/3006727/load-a-pickle-file-from-a-zipfile

zf = czipfile.ZipFile(r"cpickletest.zip")
pyfromzip = zf.read ("cPickletest.py", pwd="python")
print pyfromzip, type(pyfromzip)

this prints the following to the console

(dp1
S'kola.jpg'
p2
S'CSVTQq#w.@lm'
p3
sS'p2.jpg'
p4
S'CSVT$S.@lm'
p5
sS'p1.jpg'
p6
S'CSVT3S.@lm'
p7
s. 

most probably this is the dictionary file written to disk as the keys and
values are all there.

now if i do this

outfile = cStringIO.StringIO()
outfile.write(pyfromzip)
z=cPickle.load(outfile)
it returns -> EOFError:



I tried various experiments and the code below worked.

filenames # this is a dictionary
with open(p + "\\coded\\" + name_py,"wb") as outfile:
cPickle.dump(str(filenames), outfile) # pickled dumped the
dictionary as a string

now the file is locked up in a pwd prtected zipfile using winrar

zf = czipfile.ZipFile(r"cpickletest.zip")
pyfromzip = zf.read ("cPickletest.py", pwd="password")
c = pyfromzip.partition('"')[-1].rpartition('"')[0]
c = eval(c)
print c, type(c) # this gave back the original dictionary

can someone think of a better way?




On Fri, Aug 29, 2014 at 6:25 PM, diliup gabadamudalige 
wrote:

> Alan, thanks for the reply.
> nope. I tried both ways. Btw I am using Python 2.7.8.
> This seem to be a problem with python itself.
>
>
> http://stackoverflow.com/questions/3006727/load-a-pickle-file-from-a-zipfile
>
> zf = czipfile.ZipFile(r"cpickletest.zip")
> pyfromzip = zf.read ("cPickletest.py", pwd="python")
> print pyfromzip, type(pyfromzip)
>
> this prints the following to the console
>
> (dp1
> S'kola.jpg'
> p2
> S'CSVTQq#w.@lm'
> p3
> sS'p2.jpg'
> p4
> S'CSVT$S.@lm'
> p5
> sS'p1.jpg'
> p6
> S'CSVT3S.@lm'
> p7
> s. 
>
> most probably this is the dictionary file written to disk as the keys and
> values are all there.
>
> now if i do this
>
> outfile = cStringIO.StringIO()
> outfile.write(pyfromzip)
> z=cPickle.load(outfile)
> it returns -> EOFError:
>
>
>
> On Fri, Aug 29, 2014 at 5:01 PM, Alan Gauld 
> wrote:
>
>> On 29/08/14 10:00, diliup gabadamudalige wrote:
>>
>>  pload = cPickle.load(zf.open("my.py",pwd=password))
>>> this gives an EOF
>>>
>>> . Does any body know why?
>>>
>>
>> I've never used zipfile so I don't know, but could it be
>> related to the fact that your pickle file is binary?
>> Do you need any flags to tell zipfile that you're
>> extracting a binary file?
>>
>> Just a thought,
>>
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.flickr.com/photos/alangauldphotos
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
>
> --
> Diliup Gabadamudalige
>
> http://www.diliupg.com
> http://soft.diliupg.com/
>
>
> **
> This e-mail is confidential. It may also be legally privileged. If you are
> not the intended recipient or have received it in error, please delete it
> and all copies from your system and notify the sender immediately by return
> e-mail. Any unauthorized reading, reproducing, printing or further
> dissemination of this e-mail or its contents is strictly prohibited and may
> be unlawful. Internet communications cannot be guaranteed to be timely,
> secure, error or virus-free. The sender does not accept liability for any
> errors or omissions.
>
> **
>
>


-- 
Diliup Gabadamudalige

http://www.diliupg.com
http://soft.diliupg.com/

**
This e-mail is confidential. It may also be legally privileged. If you are
not the intended recipient or have received it in error, please delete it
and all copies from your system and notify the sender immediately by return
e-mail. Any unauthorized reading, reproducing, printing or further
dissemination of this e-mail or its contents is strictly prohibited and may
be unlawful. Internet communications cannot be guaranteed to be timely,
secure, error or virus-free. The sender does not accept liability for any
errors or omissions.
**
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


[Tutor] Python 3.4.1 question for Mac users

2014-08-29 Thread Richard Dillon
I’m teaching myself Python 3.4.1 on a Mac and the book I’m using is written for 
Windows users.

I’m trying to open a file on the desktop and I created a path using the  
example in the book.

 

Any Mac users out there with a solution? My main drive is named “OS”.

Here’s my code:


  

def main():

my_file = input('Enter file to open: ')

infile = open(r'\OS\Users\richarddillon\Desktop\my_file','r')

file_contents = infile.read()

infile.close()

print(file.contents)

 

main()___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Python 3.4.1 question for Mac users

2014-08-29 Thread Chris “Kwpolska” Warrick
On Fri, Aug 29, 2014 at 5:01 PM, Richard Dillon  wrote:
> I’m teaching myself Python 3.4.1 on a Mac and the book I’m using is written
> for Windows users.
>
> I’m trying to open a file on the desktop and I created a path using the
> example in the book.
>
>
>
> Any Mac users out there with a solution? My main drive is named “OS”.
>
> Here’s my code:
>
>
>
> def main():
>
> my_file = input('Enter file to open: ')
>
> infile = open(r'\OS\Users\richarddillon\Desktop\my_file','r')
>
> file_contents = infile.read()
>
> infile.close()
>
> print(file.contents)
>
>
>
> main()
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>



-- 
Chris “Kwpolska” Warrick 
PGP: 5EAAEA16
stop html mail | always bottom-post | only UTF-8 makes sense
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Python 3.4.1 question for Mac users

2014-08-29 Thread Chris “Kwpolska” Warrick
Sorry for the errorneous quote-only response, gmail managed to send it
without me typing a thing.

On Fri, Aug 29, 2014 at 5:01 PM, Richard Dillon  wrote:
> infile = open(r'\OS\Users\richarddillon\Desktop\my_file','r')

On the Mac, you don’t need to specify a drive there; the main drive is just /.

Also, the path separator is a forward slash (/).  This also lets you
remove the r in front of the string, which is needed for backslashes
unless you escape them.  The corrected code is:

infile = open('/Users/richarddillon/Desktop/my_file', 'r')

-- 
Chris “Kwpolska” Warrick 
PGP: 5EAAEA16
stop html mail | always bottom-post | only UTF-8 makes sense
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Python 3.4.1 question for Mac users

2014-08-29 Thread Al Bull

-Original Message-
From: Tutor [mailto:tutor-bounces+a.bull=pubdmgroup@python.org] On Behalf 
Of Chris “Kwpolska” Warrick
Sent: Friday, August 29, 2014 11:40 AM
To: Richard Dillon
Cc: tutor@python.org
Subject: Re: [Tutor] Python 3.4.1 question for Mac users

Sorry for the errorneous quote-only response, gmail managed to send it without 
me typing a thing.

On Fri, Aug 29, 2014 at 5:01 PM, Richard Dillon  wrote:
> infile = open(r'\OS\Users\richarddillon\Desktop\my_file','r')

On the Mac, you don’t need to specify a drive there; the main drive is just /.

Also, the path separator is a forward slash (/).  This also lets you remove the 
r in front of the string, which is needed for backslashes unless you escape 
them.  The corrected code is:

infile = open('/Users/richarddillon/Desktop/my_file', 'r')

--
Chris “Kwpolska” Warrick 
PGP: 5EAAEA16
stop html mail | always bottom-post | only UTF-8 makes sense 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Al Bull] 

Isn't he still mixing a string and a string variable is his open statement?   
My_file is a string variable.   I'm still learning Python myself so I could 
very well be incorrect.. :)



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


[Tutor] Printing a list count - Help

2014-08-29 Thread Derek Jenkins
Hi everybody,

I have a list that I want to go through and finally print a total
count of particular items. In this case, I want to print the result of
how many A's and B's are in the list.

honor_roll_count = 0
student_grades = ["A", "C", "B", "B", "C", "A", "F", "B", "B", "B", "C", "A"]

for grades in student_grades:
honor_roll_count = honor_roll_count + 1
if grades == "A" or grades == "B":
print honor_roll_count

The above code prints 8 lines, each being an entry for which item in
the list was either A or B. Again, I'm looking for the result to be
the number 8 itself - the total number of instances that A or B occurs
in the list.

I'm sure there are a handful of easy solutions for this, I just
haven't hit the right combo yet and I thought I'd reach out to the
tutor list for a bit of advice.

Please and thank you!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] sorting distances for nearest neighbor

2014-08-29 Thread Danny Yoo
> How do you sort distances to prepare for nearest neighbor when the results
> returned look like this from the below statements?


It's unclear what you're asking.


1.  Are you showing us the file content in the raw form:

> Distance from 1 to
> 16 is 43763.0757603
> 19 is 126516.988978
> 18 is 131011.694135
[text cut]

and you're asking how to parse the content that you presented?

2.  Or are you asking how to use the sort routines in Python?

3.  Or are you asking how to write nearest neighbor?

4.  Or are you asking for something else?  I have no idea how to
interpret what you mean by:

DR=distance(n,XCOORDINATE, YCOORDINATE, IDCODE)
DZ=switch(n,XCOORDINATE, YCOORDINATE, IDCODE)

in your problem statement, given you don't say what DR, distance,
switch, n, or IDCODE are.  So I don't understand the terms being used
in large portions of your problem statement.


The question you're presenting has much more ambiguity than I'm
comfortable with.  Rather than try to cover all the bases, if you can
scope the request to something more manageable, it'll probably make it
easier to answer.   Please clarify.  Thanks!
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] extracting a cPickle/pickle file from inside a zip file

2014-08-29 Thread Danny Yoo
> now if i do this
>
> outfile = cStringIO.StringIO()
> outfile.write(pyfromzip)
> z=cPickle.load(outfile)


Do you have to rewind the outfile so that the read is properly
positioned?  The following interaction:

#
>>> import StringIO
>>> out = StringIO.StringIO()
>>> out.write("hello world")
>>> out.read()
''
#

shows that if we immediately read what you write, we get nothing: the
file position is not at the beginning.  But as soon as we do a seek():


##
>>> out.seek(0)
>>> out.read()
'hello world'
##

then we're good.



Alternatively, can you do this instead?

#
>>> out = StringIO.StringIO("hello, this is a test")
>>> out.read()
'hello, this is a test'
#

In your case, construct the StringIO object with the content of
pyfromzip, up front, rather than with a separate call to write().
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Printing a list count - Help

2014-08-29 Thread Danny Yoo
>
> The above code prints 8 lines, each being an entry for which item in
> the list was either A or B. Again, I'm looking for the result to be
> the number 8 itself - the total number of instances that A or B occurs
> in the list.


Hi Derek,

Hint: the code currently prints a variable within the loop.  But you
can print the value of your variable _outside_ the loop, after all the
student_grades have been processed.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Printing a list count - Help

2014-08-29 Thread Derek Jenkins
honor_roll_count = 0
student_grades = ["A", "C", "B", "B", "C", "A", "F", "B", "B", "B", "C", "A"]

for grades in student_grades:
if grades == "A" or grades == "B":
honor_roll_count = honor_roll_count + 1
print honor_roll_count



That works more to my liking. Thanks a million, Danny and Bob!

On Fri, Aug 29, 2014 at 3:42 PM, boB Stepp  wrote:
> On Fri, Aug 29, 2014 at 2:17 PM, Derek Jenkins  
> wrote:
>> Hi everybody,
>>
>> I have a list that I want to go through and finally print a total
>> count of particular items. In this case, I want to print the result of
>> how many A's and B's are in the list.
>>
>> honor_roll_count = 0
>> student_grades = ["A", "C", "B", "B", "C", "A", "F", "B", "B", "B", "C", "A"]
>>
>> for grades in student_grades:
>> honor_roll_count = honor_roll_count + 1
>> if grades == "A" or grades == "B":
>> print honor_roll_count
>>
>
> Are you sure you have your increment of honor_roll_count where you
> want it? As it is placed you are counting how many grades of any kind
> there are.
>
> --
> boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Import from project's lib directory?

2014-08-29 Thread leam hall
Am I asking the wrong question? How do older apps with older versions
of python (2.4.x) separate code into sub-directories? Do they?

-- 
Mind on a Mission
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Printing a list count - Help

2014-08-29 Thread Derek Jenkins
Alex,

Thanks for taking this one step further! I do appreciate it... +1

On Fri, Aug 29, 2014 at 3:48 PM, Alex Kleider  wrote:
> On 2014-08-29 12:17, Derek Jenkins wrote:
>>
>> Hi everybody,
>>
>> I have a list that I want to go through and finally print a total
>> count of particular items. In this case, I want to print the result of
>> how many A's and B's are in the list.
>>
>> honor_roll_count = 0
>> student_grades = ["A", "C", "B", "B", "C", "A", "F", "B", "B", "B", "C",
>> "A"]
>>
>> for grades in student_grades:
>> honor_roll_count = honor_roll_count + 1
>> if grades == "A" or grades == "B":
>> print honor_roll_count
>>
>> The above code prints 8 lines, each being an entry for which item in
>> the list was either A or B. Again, I'm looking for the result to be
>> the number 8 itself - the total number of instances that A or B occurs
>> in the list.
>
>
> Try the following:
> print("Running Python3 script: 'tutor.py'...")
>
>
> student_grades = ["A", "C", "B", "B", "C", "A", "F",
> "B", "B", "B", "C", "A"]
>
> grades = {}
>
> for grade in student_grades:
> grades[grade] = grades.get(grade, 0) + 1
>
> for grade in sorted(grades.keys()):
> print("'{}': {}".format(grade, grades[grade]))
>
> If you are using Python 2, I believe the get method is called something
> else; you can look it up if need be.
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Printing a list count - Help

2014-08-29 Thread Alan Gauld

On 29/08/14 20:17, Derek Jenkins wrote:

Hi everybody,

I have a list that I want to go through and finally print a total
count of particular items. In this case, I want to print the result of
how many A's and B's are in the list.

honor_roll_count = 0
student_grades = ["A", "C", "B", "B", "C", "A", "F", "B", "B", "B", "C", "A"]

for grades in student_grades:
 honor_roll_count = honor_roll_count + 1
 if grades == "A" or grades == "B":
 print honor_roll_count



lists have a count() method.
try this:

honor_roll_count = student_grades.count('A') + student_grades.count('B')

For more complex data sets you could use the itertools groupby function too.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Crush
I am now able to print stderr to the screen, however I still can not run 
conditional statements against stderr. 

Please see the screenshots(SS) here...

https://www.dropbox.com/sh/31wyjtvqymo94uk/AAAZaxwB27nw1nmz7tz69I5La?dl=0

Bo 

> On Aug 28, 2014, at 6:12 PM, Cameron Simpson  wrote:
> 
>> On 28Aug2014 09:42, Crush  wrote:
>> As far as the pipe in...
>> 
>> "avconv -v verbose -re -analyzeduration 1000 
>> -ihttp://localhost:6498/ms2/1382097438004/0MediaPlayer+0+/octoshape+hVV+octolive.americaone.com+V+aone+V+live+V+ONECONNXTDEMO1_HD_flv/aoneliveONECONNXTDEMO1HDflv
>>  -c:v rawvideo -c:a pcm_s16le -ar 48000 -f nut - | ./bmdplay -m 12 -f pipe:0"
>> 
>> ...it has to be there.
> 
> Indeed. We're not arguing about that. Leave it there for.
> 
>> Currently, i am running a shell script that is handling this decoding 
>> process i.e. the real long avconv/bmdplay command. So you are saying leave 
>> the shell script and invoke it from a python script...something like "p = 
>> subprocess.Popen("./my_script.sh")? How will the above allow me to interact 
>> with stderr in oppose to running the avconv/bmdplay command directly in 
>> python?
> 
> No, we're no saying that.
> 
> You could (and presumably do) have this long incantation in your shell 
> script.  As far as the Python stuff goes, that does not change anything: you 
> can invoke your shell script or invoke your long command directly. Either way 
> you need to grab the stderr on the fly and work with it.
> 
> Personally, I would keep the pipeline in the shell script where it is easier 
> to edit and invoke the script from Python (easier to read). Your call.
> 
>> As far as not using "shell=True," if I dont have it set to True, the 
>> terminal hangs and will not display stderr to the screen as it is currently 
>> does.
> 
> Let's tackle shell=True later.
> 
> What you need to do is examine stderr before the shell script/pipeline is 
> complete, and for that you need to separate starting the process, reading 
> stderr, and waiting for process completion into distinct parts, which was 
> what my example code was doing. I'll recite it again here:
> 
>  from subprocess import Popen, PIPE
> 
>  P = Popen("avconv ... lots of arguments...", shell=True, stderr=PIPE)
> 
>  for line in P.stderr:
>  ... examine the line from stderr ...
> 
>  # ok, we have read all of stderr now
>  xit = P.wait()
>  if xit != 0:
>  ... command was unsuccessful, complain, maybe abort ...
> 
> Please try adapting your Python code to that and see where you end up.
> Put a print statement as the first line in the for-loop so you can see if you 
> are receiving stderr lines as intended.
> 
> Cheers,
> Cameron Simpson 
> 
> As soon as we started programming, we found to our surprise that it wasn't as
> easy to get programs right as we had thought. Debugging had to be discovered.
> I can remember the exact instant when I realized that a large part of my life
> from then on was going to be spent in finding mistakes in my own programs.
>- Maurice Wilkes discovers debugging, 1949
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Import from project's lib directory?

2014-08-29 Thread Peter Otten
leam hall wrote:

> Am I asking the wrong question? How do older apps with older versions
> of python (2.4.x) separate code into sub-directories? Do they?

Even new versions allow relative imports only inside packages. Given a tree

$ tree
.
├── alpha
│   ├── beta
│   │   ├── __init__.py
│   │   └── one.py
│   ├── gamma
│   │   ├── __init__.py
│   │   └── two.py
│   └── __init__.py
└── elsewhere
├── __init__.py
└── three.py

You have to ensure that the *parent* of alpha is in sys.path. Then you can 
refer from alpha/beta/one.py to alpha/gamma/two.py with the statement

# in file alpha/beta.one.py
from ..gamma import two

but as far as I know no combination of dots lets you refer to modules 
outside the package (like elswhere/three.py in the example). You need an 
absolute import for that.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] extracting a cPickle/pickle file from inside a zip file

2014-08-29 Thread Danny Yoo
Hi Dillup,

One other comment other than the zipfile stuff:  you might want to
consider something other than pickle format if you want
interchangeability with other tools.  JSON, for example, is pretty
well-supported in the json library:

 https://docs.python.org/2/library/json.html

and it has an interface similar to that of pickle, so it's easy to switch to it.

I recommend this instead of pickle, unless your situation favors
pickle.  JSON has a looser coupling to Python, and more importantly,
fewer security concerns.  Pickle can do some very dynamic stuff,
including eval-like behavior, which can be a worry.  See:
http://stackoverflow.com/questions/10282175/attacking-pythons-pickle
for example.

If you're trying to represent dictionaries of string keys and values,
that's something that JSON can handle very well.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Danny Yoo
On Fri, Aug 29, 2014 at 2:13 PM, Crush  wrote:
> I am now able to print stderr to the screen, however I still can not run
> conditional statements against stderr.
>
> Please see the screenshots(SS) here...
>
> https://www.dropbox.com/sh/31wyjtvqymo94uk/AAAZaxwB27nw1nmz7tz69I5La?dl=0


Hi Bo,


Consider case sensitivity.


>>> "a" == "A"
False



Also, in the screenshots you're presenting, you very much should want
to do a copy-and-paste of the text content and include it in your
question text.  The reason is because, for all the Tutor folks who
aren't in a GUI right now, they can't see your program.


For the sake of the others on the list: your first screenshot has the following:


##
for line in p.stderr:
print line
if "segmentation" in line:
print "Yes it is"
...
###


and your second screenshot has the content:


##
...
  11265 Segmentation fault   | ./bmdplay -m 12 -f pipe:0

Gave up
##
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Danny Yoo
Hi Bo,

One other thing: if you can avoid running commands as root, I'd
strongly suggest doing so.  Your second screenshot shows that you're
running as root superuser, and the imaginary security demon that sits
on my left shoulder is laughing uproariously as we speak.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Import from project's lib directory?

2014-08-29 Thread leam hall
On Fri, Aug 29, 2014 at 5:16 PM, Peter Otten <__pete...@web.de> wrote:
> leam hall wrote:
>
>> Am I asking the wrong question? How do older apps with older versions
>> of python (2.4.x) separate code into sub-directories? Do they?

> You have to ensure that the *parent* of alpha is in sys.path. Then you can
> refer from alpha/beta/one.py to alpha/gamma/two.py with the statement


Ah, so I can use sys.path.insert()! That's what I needed, thanks!

Leam
-- 
Mind on a Mission
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Crush
Ok so no links to dropbox? Man im confused...i thought links to dropbox were ok?

Wow i feel like such an idiot haha. I should have caught that capitolization 
error! 

Im sorry to have waisted your time.

Bo 

> On Aug 29, 2014, at 6:06 PM, Danny Yoo  wrote:
> 
>> On Fri, Aug 29, 2014 at 2:13 PM, Crush  wrote:
>> I am now able to print stderr to the screen, however I still can not run
>> conditional statements against stderr.
>> 
>> Please see the screenshots(SS) here...
>> 
>> https://www.dropbox.com/sh/31wyjtvqymo94uk/AAAZaxwB27nw1nmz7tz69I5La?dl=0
> 
> 
> Hi Bo,
> 
> 
> Consider case sensitivity.
> 
> 
 "a" == "A"
> False
> 
> 
> 
> Also, in the screenshots you're presenting, you very much should want
> to do a copy-and-paste of the text content and include it in your
> question text.  The reason is because, for all the Tutor folks who
> aren't in a GUI right now, they can't see your program.
> 
> 
> For the sake of the others on the list: your first screenshot has the 
> following:
> 
> 
> ##
> for line in p.stderr:
>print line
>if "segmentation" in line:
>print "Yes it is"
>...
> ###
> 
> 
> and your second screenshot has the content:
> 
> 
> ##
> ...
>  11265 Segmentation fault   | ./bmdplay -m 12 -f pipe:0
> 
> Gave up
> ##
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Crush
Haha Yes I am aware of people like you who are just itching to exploit 
vulnerabilities like that; however, the programs my company uses to broadcast 
will only run as root. 

Bo 

> On Aug 29, 2014, at 6:12 PM, Danny Yoo  wrote:
> 
> Hi Bo,
> 
> One other thing: if you can avoid running commands as root, I'd
> strongly suggest doing so.  Your second screenshot shows that you're
> running as root superuser, and the imaginary security demon that sits
> on my left shoulder is laughing uproariously as we speak.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Cameron Simpson

On 29Aug2014 18:31, Crush  wrote:

Ok so no links to dropbox? Man im confused...i thought links to dropbox were ok?


That was my fault. I said:

  If your screenshot is inherently graphical, publish it elsewhere
  (eg save it to Dropbox and get a "public link") and mention the
  link in your post, with some textual description.

But unless you're showing us some drawing, this is usually not desirable. In 
your case, the screenshot is (apparently) of some code, and it is far better to 
just include the code in your post, as text.


There are many advantages to text, including:

  It is MUCH MUCH smaller. (Applies to screenshot attachments, too.)

  It keeps the text with the message.

  Further, because it is in the message, it keeps the text in the mail 
  archives, where it may be searche fdor. Again, because it is in the message, 
  it _stays_ in the archives. Your dropbox link might go away at any time in 
  the future.


  Being text, part of the message, it is presented in _my_ preferred font. This 
  is just nicer, and for some people with sight issues, very important. I once 
  had a student who needed to get up very close to the screen, _and_ use a 
  magnifying glass. Forcing him to fetch a screenshot would not be nice.


  It arrives in the message, not requiring a separate act to fetch it (versus a 
  Dropbox link). For example, your message arrives and I have to open a web 
  browser to go and get your screenshot. Suppose I'm offline. This is 
  realistic: my laptop fetches my email in the background all the time, but I 
  visit the "python" folder at ad hoc times. If I'm on a long train trip, that 
  is an ideal time to catch up on mailing lists. I can read because the 
  messages were fetched earlier. I can reply, because my laptop has a mail 
  system which will queue messages and send when I'm online again. But I 
  _can't_ go and get your URL, because I'm offline.


Please use inline plain text when you can. It helps everyone.


Wow i feel like such an idiot haha. I should have caught that capitolization 
error!


No worries.

Cheers,
Cameron Simpson 

The impossible we understand right away - the obvious takes a little longer.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Interacting with stderr

2014-08-29 Thread Cameron Simpson

On 29Aug2014 18:35, Crush  wrote:

On Aug 29, 2014, at 6:12 PM, Danny Yoo  wrote:
One other thing: if you can avoid running commands as root, I'd
strongly suggest doing so.  Your second screenshot shows that you're
running as root superuser, and the imaginary security demon that sits
on my left shoulder is laughing uproariously as we speak.


Haha Yes I am aware of people like you who are just itching to exploit 
vulnerabilities like that; however, the programs my company uses to broadcast 
will only run as root.


That's a really bad design decision, and rarely necessary. It is often _easy_, 
but it leaves your systems at much greater risk.


Cheers,
Cameron Simpson 

Please do not send me Microsoft Word files.
http://en.nothingisreal.com/wiki/Please_don't_send_me_Microsoft_Word_documents
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


[Tutor] Fwd: Re: extracting a cPickle/pickle file from inside a zip file

2014-08-29 Thread Danny Yoo
-- Forwarded message --
From: "diliup gabadamudalige" 
Date: Aug 29, 2014 8:34 PM
Subject: Re: [Tutor] extracting a cPickle/pickle file from inside a zip file
To: "Danny Yoo" 
Cc:

Dear Danny,
Thank for your invaluable advice which is much appreciated. My code finally
ends up being exe-fied with py2exe and I have a bit of a limitation there.
I have to select packages which are compatible with it. Haven't tried json
or checked whether it is compatible but I will do so.
On another note which is a better exe-fier? py2exe or  cxfreeze? I would
like your personal opinion if you have any knowledge on this matter.

Thank you once again for your invaluable help.

Diliupg


On Sat, Aug 30, 2014 at 3:24 AM, Danny Yoo  wrote:

> Hi Dillup,
>
> One other comment other than the zipfile stuff:  you might want to
> consider something other than pickle format if you want
> interchangeability with other tools.  JSON, for example, is pretty
> well-supported in the json library:
>
>  https://docs.python.org/2/library/json.html
>
> and it has an interface similar to that of pickle, so it's easy to switch
> to it.
>
> I recommend this instead of pickle, unless your situation favors
> pickle.  JSON has a looser coupling to Python, and more importantly,
> fewer security concerns.  Pickle can do some very dynamic stuff,
> including eval-like behavior, which can be a worry.  See:
> http://stackoverflow.com/questions/10282175/attacking-pythons-pickle
> for example.
>
> If you're trying to represent dictionaries of string keys and values,
> that's something that JSON can handle very well.
>



-- 
Diliup Gabadamudalige

http://www.diliupg.com
http://soft.diliupg.com/

**
This e-mail is confidential. It may also be legally privileged. If you are
not the intended recipient or have received it in error, please delete it
and all copies from your system and notify the sender immediately by return
e-mail. Any unauthorized reading, reproducing, printing or further
dissemination of this e-mail or its contents is strictly prohibited and may
be unlawful. Internet communications cannot be guaranteed to be timely,
secure, error or virus-free. The sender does not accept liability for any
errors or omissions.
**
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor