Re: [Tutor] Python Job Scheduling package

[bodsda]

Have you thought about writing your own? Others have posted some useful links, 
but in all honesty you could hack something together to achieve that in next to 
no time

Bodsda 
Sent from my BlackBerry® wireless device

-Original Message-
From: harish bansal 
Sender: tutor-bounces+bodsda=googlemail@python.org
Date: Thu, 13 Oct 2011 15:23:04 
To: 
Subject: [Tutor] Python Job Scheduling package

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] 'object' class

[Max S.]

I have seen classes created with 'class Class_Name:' and 'class
Class_Name(object):'.  I'm using the latter, just in case it has some sort
of method that could be useful that I don't know about, but *are *there any
methods in the 'object' class?  And if so, what are they?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] 'object' class

[Rafael Durán Castañeda]

2011/10/14 Max S. 

> I have seen classes created with 'class Class_Name:' and 'class
> Class_Name(object):'.  I'm using the latter, just in case it has some sort
> of method that could be useful that I don't know about, but *are *there
> any methods in the 'object' class?  And if so, what are they?
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
> This new style vs old 
> styleclasses question, 
> you can find a lot of information if you search about it.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Windows vs Linux processing speed.

[Tony Pelletier]

Hi,

I have a question regarding the speed of my program on linux in comparison
to windows.

I'm using geopy and contacting Google for geocodes for records in a csv I
created.  Like such:

 try:
reader = csv.reader(open(filename, "r"))
for row in reader:
if row: # Checking to see if it's a valid row so it doesn't blow
up on an empty row.
username, address, address2, city, state, zip, country =
row[0:7]
location = address + ', ' + city + ' ' + state + ' ' + zip +
' ' + country
try:
place, (lat, lng) =  g.geocode(location) # Contact
Google for Geocodes

I mainly use windows, but I tend to use Arch Linux at home.  When I run it
via windows, it's relatively slow and didn't really bother me, but when I
ran it at home I got through 15 or so and got a message back from google
saying  I was exceeding the allowed amount.  Or something to that effect.
 Basically, I was exceeding the 10 per second that's allowed.

So, my question is.  Why is it running so much faster on linux?  Is it the
way that linux is handling the socket?  Does windows open and close it
whereas linux might leave it open and just pump data through?

It's not really a problem since I just added sleep time, but I'm curious.

Thanks
Tony
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[Prasad, Ramit]

From: tutor-bounces+ramit.prasad=jpmorgan@python.org 
[mailto:tutor-bounces+ramit.prasad=jpmorgan@python.org] On Behalf Of Tony 
Pelletier
Sent: Friday, October 14, 2011 11:46 AM
To: tutor@python.org
Subject: [Tutor] Windows vs Linux processing speed.

Hi, 

I have a question regarding the speed of my program on linux in comparison to 
windows.

I'm using geopy and contacting Google for geocodes for records in a csv I 
created.  Like such:

 try:
        reader = csv.reader(open(filename, "r"))
        for row in reader:
            if row: # Checking to see if it's a valid row so it doesn't blow up 
on an empty row.
                username, address, address2, city, state, zip, country = 
row[0:7]
                location = address + ', ' + city + ' ' + state + ' ' + zip + ' 
' + country
                try:
                    place, (lat, lng) =  g.geocode(location) # Contact Google 
for Geocodes

I mainly use windows, but I tend to use Arch Linux at home.  When I run it via 
windows, it's relatively slow and didn't really bother me, but when I ran it at 
home I got through 15 or so and got a message back from google saying  I was 
exceeding the allowed amount.  Or something to that effect.  Basically, I was 
exceeding the 10 per second that's allowed.  

So, my question is.  Why is it running so much faster on linux?  Is it the way 
that linux is handling the socket?  Does windows open and close it whereas 
linux might leave it open and just pump data through?  

It's not really a problem since I just added sleep time, but I'm curious.

Thanks
Tony
==

I think you might get a better response for this question on the main Python 
list. You say that at home you use Arch Linux, which implies you are running it 
from Windows in a different physical location. My initial thought is that your 
original run might have been from a location "far" from Google or with network 
congestion. Have you tried Windows and Linux from the same location? What 
version of Windows and Python for each?

Not sure location should make *that* much of a difference though.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[John Fabiani]

On Friday, October 14, 2011 09:45:57 am Tony Pelletier wrote:
> Hi,
> 
> I have a question regarding the speed of my program on linux in comparison
> to windows.
> 
> I'm using geopy and contacting Google for geocodes for records in a csv I
> created.  Like such:
> 
>  try:
> reader = csv.reader(open(filename, "r"))
> for row in reader:
> if row: # Checking to see if it's a valid row so it doesn't
> blow up on an empty row.
> username, address, address2, city, state, zip, country =
> row[0:7]
> location = address + ', ' + city + ' ' + state + ' ' + zip
> + ' ' + country
> try:
> place, (lat, lng) =  g.geocode(location) # Contact
> Google for Geocodes
> 
> I mainly use windows, but I tend to use Arch Linux at home.  When I run it
> via windows, it's relatively slow and didn't really bother me, but when I
> ran it at home I got through 15 or so and got a message back from google
> saying  I was exceeding the allowed amount.  Or something to that effect.
>  Basically, I was exceeding the 10 per second that's allowed.
> 
> So, my question is.  Why is it running so much faster on linux?  Is it the
> way that linux is handling the socket?  Does windows open and close it
> whereas linux might leave it open and just pump data through?
> 
> It's not really a problem since I just added sleep time, but I'm curious.
> 
> Thanks
> Tony

It's a very good question that I'd like to hear the answer too.  When ever I 
run pure python on linux is always runs faster (this has been my experience).  
I of course have not a clue as to why (or I might not be monitoring this 
list).  But I bet other do know!

Johnf
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[bob gailer]

On 10/14/2011 12:45 PM, Tony Pelletier wrote:

Hi,

I have a question regarding the speed of my program on linux in 
comparison to windows.

[snip]

Speed of a pure Python program depends on

 * processor speed
 * competition for resources from other processes
 * perhaps RAM

A good starting point -create a benchmark program that uses only CPU - 
no disk, no internet.

Example:

import time
n = 10 # a guess
start = time.time()
for i in range(n):pass
print time.time() - start

Be sure n is large enough to create a run time of at least several seconds.

Run it on Linux and on Windows.

Do a speed check on your internet connections.

Then modify the program to open & read the csv file inside the loop. You 
will have to reduce n when you add disk i/o


import time, csv
n = 1 # a guess
start = time.time()
for i in range(n):
  for line in csv.reader(open(filename, "r")):pass
print time.time() - start
HTH

--
Bob Gailer
919-636-4239
Chapel Hill NC

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] 'object' class

[Dave Angel]

On 10/14/2011 10:17 AM, Rafael Durán Castañeda wrote:

2011/10/14 Max S.


I have seen classes created with 'class Class_Name:' and 'class
Class_Name(object):'.  I'm using the latter, just in case it has some sort
of method that could be useful that I don't know about, but *are *there
any methods in the 'object' class?  And if so, what are they?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

This new style vs old 
styleclasses question, you 
can find a lot of information if you search about it.

Things not covered by that wiki page,

1) in Python 3.x, all classes are new-style, regardless of which syntax 
you use.


2) I believe super() is new to new-style classes.  In any case the 
documentation for it seem to assume new-style.


3) you could have answered your own question about methods in a 
particular class.  Just use 'dir'


>>> dir(object)
['__class__', '__delattr__', '__doc__', '__format__', 
'__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', 
'__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', 
'__subclasshook__']

>>>


--

DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[Steven D'Aprano]

bob gailer wrote:

On 10/14/2011 12:45 PM, Tony Pelletier wrote:

Hi,

I have a question regarding the speed of my program on linux in 
comparison to windows.

[snip]

Speed of a pure Python program depends on

 * processor speed
 * competition for resources from other processes
 * perhaps RAM


"Perhaps" RAM? The amount of memory is close to the easiest, cheapest 
way to speed up a program: more RAM means it is easier and faster to 
allocate memory in the process, which means more time performing *your* 
code and less time hunting for blocks of memory. More RAM means less 
chances you will start using virtual memory, that is, paging blocks of 
memory out of RAM onto the hard drive, which is around a *thousand* 
times slower than RAM. You can never have too much RAM :)


Very few program's speed are greatly dependent on raw processor speed. 
Processor speed is one of the great marketing gimmicks of all time. Of 
course it has *some* effect, but the bottleneck is almost never the CPU, 
and usually the speed of getting data and/or code out of RAM and onto 
the CPU and from their into the core for the instructions to be 
executed. CPU cache faults are really, really expensive, so the bigger 
the pipeline into the core, the fewer the cache faults.


But the most important factor determining the speed of a program is what 
it actually does, and the algorithm it uses to do it.



--
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] 'object' class

[Steven D'Aprano]

Dave Angel wrote:

2) I believe super() is new to new-style classes.  In any case the 
documentation for it seem to assume new-style.



Yes, super() only works for new style classes.

>>> class Test:
... def method(self):
... super(Test, self).method()
...
>>> t = Test()
>>> t.method()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in method
TypeError: super() argument 1 must be type, not classobj


Likewise for property(). property is particularly tricky, because it 
*appears* to work for old style classes:


>>> class Test:
... def __init__(self, value):
... self._x = value
... def getx(self):
... print "Getting x"
... return self._x
... def setx(self, value):
... print "Setting x"
... self._x = value
... x = property(getx, setx)
...
>>> t = Test(42)
>>> t.x
Getting x
42

but actually doesn't work correctly:

>>> t.x = 12
>>> t.x
12



--
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[Steven D'Aprano]

Tony Pelletier wrote:


So, my question is.  Why is it running so much faster on linux?  Is it the
way that linux is handling the socket?  Does windows open and close it
whereas linux might leave it open and just pump data through?


Perhaps; you'll have to read the source code to see if there are 
differences in geopy, or in the code that it relies on. But I suspect 
you *might* be seeing this bug:


http://www.mail-archive.com/python-dev@python.org/msg40692.html

It's a long thread, but the summary is: reading data over the Internet 
on Windows using some versions of Python is *sometimes* EXTREMELY slow 
compared to Linux or third party tools. This is due to a bug in the 
httplib module, which used a naive way of concatenating many little 
strings. Most of the time, Python has an optimization that can disguise 
how slow this is, but occasionally the optimization can fail.




--
Steven

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[Tony Pelletier]

Thanks everyone.

Well, I've added a timer to the code and I'm running it from my house.  I'm
getting the same times here from both laptops which I didn't expect at all.
 I guess I'll have to take my linux laptop to work and see what my tests
produce there.

And as an FYI, my windows laptop is actually much more capable than my linux
laptop.
Linux is an IBM Thinkpad T61
Windows is an Alienware M11x R2( i7 with 4gb of ram)

Thanks again.
Tony

On Fri, Oct 14, 2011 at 7:01 PM, Steven D'Aprano wrote:

> Tony Pelletier wrote:
>
>  So, my question is.  Why is it running so much faster on linux?  Is it the
>> way that linux is handling the socket?  Does windows open and close it
>> whereas linux might leave it open and just pump data through?
>>
>
> Perhaps; you'll have to read the source code to see if there are
> differences in geopy, or in the code that it relies on. But I suspect you
> *might* be seeing this bug:
>
> http://www.mail-archive.com/**python-...@python.org/**msg40692.html
>
> It's a long thread, but the summary is: reading data over the Internet on
> Windows using some versions of Python is *sometimes* EXTREMELY slow compared
> to Linux or third party tools. This is due to a bug in the httplib module,
> which used a naive way of concatenating many little strings. Most of the
> time, Python has an optimization that can disguise how slow this is, but
> occasionally the optimization can fail.
>
>
>
>
> --
> Steven
>
> __**_
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Job Scheduling package

[Japhy Bartlett]

Is this a thing people would use?  I've built this as part of a larger
project.. do you think it'd be worth splitting out and polishing up?

On Oct 14, 2011 3:55 AM,  wrote:

Have you thought about writing your own? Others have posted some useful
links, but in all honesty you could hack something together to achieve that
in next to no time

Bodsda
Sent from my BlackBerry® wireless device


-Original Message-
From: harish bansal 
Sender: tutor-bounces+bo...
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[Alan Gauld]

On 14/10/11 17:45, Tony Pelletier wrote:

Hi,

I have a question regarding the speed of my program on linux in
comparison to windows.

I'm using geopy and contacting Google for geocodes for records in a csv
I created.
snip...

I mainly use windows, but I tend to use Arch Linux at home...

So, my question is.  Why is it running so much faster on linux?


Its impossible to tell from what you've told us.

Is the hardware identical? What about the network connections?

In your application I'd expect network delays to be by far the biggest 
factor in program speed and dwarf the PC/OS issues, (unless there was a 
huge issue with the networking code - see Steven's post re httplib)


Have you tried profiling the code to see how the time is spent on each 
system?


Is there contention on your network at one of the locations? That could 
account for large differences too.


Whatever the reason its unlike;y to be a Linux v Windows issue per se, 
its much ore likely to be either a difference in Python implementation 
or a difference in network configuration.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Windows vs Linux processing speed.

[Japhy Bartlett]

In this situation, the network connection is almost certainly the
bottleneck;  maybe CPU speed, if they are drastically different, but
it seems like you're running relatively comparable hardware.  RAM is
almost never a *speed* bottleneck, until you start working with
datasets larger than you can hold in memory and start dealing with
disk i/o.

If it's not that, I think that Steven is probably on the right track.


- Japhy

On Fri, Oct 14, 2011 at 11:24 PM, Tony Pelletier
 wrote:
> Thanks everyone.
> Well, I've added a timer to the code and I'm running it from my house.  I'm
> getting the same times here from both laptops which I didn't expect at all.
>  I guess I'll have to take my linux laptop to work and see what my tests
> produce there.
> And as an FYI, my windows laptop is actually much more capable than my linux
> laptop.
> Linux is an IBM Thinkpad T61
> Windows is an Alienware M11x R2( i7 with 4gb of ram)
> Thanks again.
> Tony
> On Fri, Oct 14, 2011 at 7:01 PM, Steven D'Aprano 
> wrote:
>>
>> Tony Pelletier wrote:
>>
>>> So, my question is.  Why is it running so much faster on linux?  Is it
>>> the
>>> way that linux is handling the socket?  Does windows open and close it
>>> whereas linux might leave it open and just pump data through?
>>
>> Perhaps; you'll have to read the source code to see if there are
>> differences in geopy, or in the code that it relies on. But I suspect you
>> *might* be seeing this bug:
>>
>> http://www.mail-archive.com/python-dev@python.org/msg40692.html
>>
>> It's a long thread, but the summary is: reading data over the Internet on
>> Windows using some versions of Python is *sometimes* EXTREMELY slow compared
>> to Linux or third party tools. This is due to a bug in the httplib module,
>> which used a naive way of concatenating many little strings. Most of the
>> time, Python has an optimization that can disguise how slow this is, but
>> occasionally the optimization can fail.
>>
>>
>>
>> --
>> Steven
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor