[Tutor] converting encoded symbols from rss feed?

[Serdar Tumgoren]

Hi everyone,
I just tried my hand at parsing an RSS 2.0 feed using Universal Feed
Parser and it worked beautifully.  My one point of confusion -- I'm
not sure how to convert encoded characters back to their
human-readable ascii/unicode.

Not sure if it matters, but the feed I'm dealing with is using xml
version 1.0 and "windows-1252" encoding.

Here are some examples of the encoded characters I'm trying to convert:

  – (symbol as it appears in the original xml file)
  –  (symbol as it appears in ipython shell after
using Universal Feed Parser)

What I'd like to do is process all of these xml items, convert the
encoded characters to readable text, and then pop the items in a
database.

So to my question -- can anyone point me to documentation on how to
perform this conversion? I didn't find anything explicit in the
Universal Feed Parser docs, but I figured Python might have a library
that handles this kind of thing.

Any help is greatly appreciated.

Regards,

Serdar
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] converting encoded symbols from rss feed?

[Wayne]

On Wed, Jun 17, 2009 at 7:30 AM, Serdar Tumgoren wrote:

> Here are some examples of the encoded characters I'm trying to
> convert:
>
>  – (symbol as it appears in the original xml file)
>  –  (symbol as it appears in ipython shell after
> using Universal Feed Parser)
> 


I've never played around much, but & is the HTML code for an &.

So if you have – it will show up as –.

I have no clue if the latter is any type of special character or something,
though.

Upon searching for – in google, I came up with this:
http://www.siber-sonic.com/mac/charsetstuff/Soniccharset.html

HTH,
Wayne
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] converting encoded symbols from rss feed?

[Serdar Tumgoren]

> Upon searching for – in google, I came up with this:
> http://www.siber-sonic.com/mac/charsetstuff/Soniccharset.html

The character table definitely helps. Thanks.

Some additional googling suggests that I need to unescape HTML
entities. I'm planning to try the below approach from Frederik Lundh.
It relies on the "re" and "htmlentitydefs" modules.

http://effbot.org/zone/re-sub.htm#unescape-html

I'll report back with my results. Meantime, I welcome any other suggestions.

Thanks!
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] distutils MANIFEST.in

[spir]

No one has a clue about this?

Le Tue, 16 Jun 2009 22:59:24 +0200,
spir  s'exprima ainsi:

> Hello,
> 
> a question for people who know how to write MANIFEST.in:
> How to tell to simply include all files in the package (and subdirs)? I
> tried:
> 
> recursive-include *.*
> ==> warning: sdist: MANIFEST.in, line 1: 'recursive-include' expects 
>   ...
> 
> recursive-include . *.*
> ==> warning: no files found matching '*.*' under directory '.'
> 
> recursive-include pijnu *.*
> (pijnu is the name of the package)
> ==> warning: no files found matching '*.*' under directory 'pijnu'
> 
> As a consequence, MANIFEST only includes:
> README
> setup.py
> and install installs nothing else the egg_info file.
> 
> Denis
> --
> la vita e estrany
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
> 


--
la vita e estrany
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] distutils MANIFEST.in

[Chris Fuller]

Use os.path.walk or similar to build the file before you call setup().

Cheers
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] distutils MANIFEST.in

[spir]

Le Wed, 17 Jun 2009 12:51:09 -0500,
Chris Fuller  s'exprima ainsi:

> 
> Use os.path.walk or similar to build the file before you call setup().
> 
> Cheers

Thanks, that's what I thought I would end up doing...

Denis
--
la vita e estrany
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] New to programming and python first minimilistic program (Bottles of beer), , please comment!

[matthew andriani]

Hi Guys,

I wrote this program for some practice to get into python..I'm trying to find 
fun ways to learn the language so if anyone has a challenge on this basic level 
using loops that sounds exciting please let me know.. If you think this program 
can be improved please let me know too :)

b = "bottles of beer"
w = "on the wall"
a = 100
while a != 0:
a = a-1
print a,b,w,a,b,"if one of those bottles should happen to fall there'll 
be",a-1,b,w


Thanks for the feedback..

Cheers,
Skellem.



_
Drag n’ drop—Get easy photo sharing with Windows Live™ Photos.

http://www.microsoft.com/windows/windowslive/products/photos.aspx___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] New to programming and python first minimilistic program (Bottles of beer), , please comment!

[vince spicer]

Like in any language there any number for ways, heres another

b = "bottles of beer"
w = "on the wall"
bottles = range(1, 101)
bottles.reverse()
for bottle in bottles:
print " %s %s %s if one of those bottles should happen to fall there'll
be %s %s %s" % (bottle, b,w, bottle-1, b,w)

Vince

2009/6/17 matthew andriani 

>  Hi Guys,
>
> I wrote this program for some practice to get into python..I'm trying to
> find fun ways to learn the language so if anyone has a challenge on this
> basic level using loops that sounds exciting please let me know.. If you
> think this program can be improved please let me know too :)
>
> b = "bottles of beer"
> w = "on the wall"
> a = 100
> while a != 0:
> a = a-1
> print a,b,w,a,b,"if one of those bottles should happen to fall there'll
> be",a-1,b,w
>
>
> Thanks for the feedback..
>
> Cheers,
> Skellem.
>
>
>
> --
> What can you do with the new Windows Live? Find 
> out
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] New to programming and python first minimilistic program (Bottles of beer), , please comment!

[Gregor Lingl]

matthew andriani schrieb:


Hi Guys,

I wrote this program for some practice to get into python..I'm trying 
to find fun ways to learn the language so if anyone has a challenge on 
this basic level using loops that sounds exciting please let me know.. 
If you think this program can be improved please let me know too :)


b = "bottles of beer"
w = "on the wall"
a = 100
while a != 0:
a = a-1
print a,b,w,a,b,"if one of those bottles should happen to fall 
there'll be",a-1,b,w



If you don't want the output of your program to be considered as a 
somewhat strange joke, you should better write:


b = "bottles of beer"
w = "on the wall"
a = 100
while a != 0:
  print a,b,w,a,b,"if one of those bottles should happen to fall 
there'll be",a-1,b,w

  a = a-1

That means: determine carefully the order of statements in the body of a 
loop.


Regards,
Gregor


Thanks for the feedback..

Cheers,
Skellem.




What can you do with the new Windows Live? Find out 




___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
  



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] New to programming and python first minimilistic program (Bottles of beer), , please comment!

[Emile van Sebille]

On 6/17/2009 1:46 PM matthew andriani said...

Hi Guys,

I wrote this program for some practice to get into python..I'm trying to 
find fun ways to learn the language so if anyone has a challenge on this 
basic level using loops that sounds exciting please let me know.. 


Exciting?

How about writing a sort utility?  I know there are easy ways in python, 
but as an exercise, put the words from the second quoted line above in 
alpha order by examining each word.  Or how about enumerating the 
possible outcomes of throwing three dice?  Once that's done, leverage 
python's capabilities and see how short a program will give the same 
results.


Emile

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] converting encoded symbols from rss feed?

[Serdar Tumgoren]

Hey everyone,
For the moment, I opted to use string replacement as my "solution."

So for the below string containing the HTML decimal represenation for en dash:

>>>x = "The event takes place June 17 – 19"
>>>x.replace('–', '-')
'The event takes place June 17 - 19'

It works in my case since this seems to be the only code that
Universal Feed Parser didn't properly translate, but of course not an
ideal solution. I assume this path will require me to build a
character reference dictionary as I encounter more character codes.

I also tried wrestling with character conversion:

>>>unichr(150)
u'\x96'

Not sure where to go from there...
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] List Splicing

[Robert Berman]

Greetings,

I am working on a 'simple' algorithm to solve the problem called PRIME1
explained at http://www.spoj.pl/problems/PRIME1/. 

I do have an algorithm based on the Sieve of Eratosthenes and it does
work as I am failing the project not because of a computational error
but because of the dreaded TLE (time limit exceeded) designator. I have
determined there are at least 3 areas for improvement. The first is
within my code where I am creating a list of the first million primes. 
The code is as follows:

def BuildSieve(itemsin):
TheSieve=list()
TheSieve = range(0,itemsin+1)
TheSieve[1]=0
for i in range(2,itemsin+1):
if (TheSieve[i] > 0):
j = i + i
while (j <= itemsin):
TheSieve[j] = 0
j+=i
return TheSieve

It is called with PrimaryList = BuildSieve(100)

I have read that list splicing rather than looping to set a range of
items to zero is the best approach. Given an example  list  of

In [53]: l1
Out[53]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Since 3 is a prime, we can eliminate all multiples of 3. Within l1,
these are expressed as

In [52]: l1[n+n:len(l1):n]
Out[52]: [6, 9, 12]

when n = 3. ( do know 12 would have been eliminated by the prime number
2)

It would be great if I could say l1[n+n:len(l1):n] = 0 but obviously
that will fail for obvious reasons. I am looking for the right hand side
of the statement to set a list within the list to all zeros. Also, if
anyone had a relatively simple explanation why in Python this is faster
than a loop, I would really like to understand it.

Thanks for the input.

Robert



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Emile van Sebille]

On 6/17/2009 3:03 PM Robert Berman said...

Greetings,

I am working on a 'simple' algorithm to solve the problem called PRIME1
explained at http://www.spoj.pl/problems/PRIME1/. 


I do have an algorithm based on the Sieve of Eratosthenes and it does
work as I am failing the project not because of a computational error
but because of the dreaded TLE (time limit exceeded) designator. I have
determined there are at least 3 areas for improvement. The first is
within my code where I am creating a list of the first million primes. 


It looks more like it creates a list 101 in length where the 
non-zero numbers in that list are primes.  I'd try BuildSieve(100) until 
BuildSieve was returning just primes.


Or, are you trying to create a list of the primes under 100?

Optimization should proceed once you've got it working.

Emie

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Robert Berman]

Emile,

Thank your for your comments. I do have a list running from 0-101.
Yes, it is true, I only needed 0 - 10 and yes I will change it.
However, if you use primearray as a sieve of all primes 2-100 you
will see it works quite well. Printing a range, say primearray[21]
through primearray[50] will give you all prime numbers between 21 and
50.

23
29
31
37
41
43
47

73000 to 99,123 works nicely, too. I did, in the beginning use a list of
100 (actually (101) and that worked well so thank you for corroborating
the approach.

Another question I will ask after this one is resolved involves the
methodology I use to find primes well over one million.

However, for the time being, can you perhaps share some suggestions on
list splicing?

Thank you for your assistance.

Robert




On Wed, 2009-06-17 at 15:25 -0700, Emile van Sebille wrote:
> On 6/17/2009 3:03 PM Robert Berman said...
> > Greetings,
> > 
> > I am working on a 'simple' algorithm to solve the problem called PRIME1
> > explained at http://www.spoj.pl/problems/PRIME1/. 
> > 
> > I do have an algorithm based on the Sieve of Eratosthenes and it does
> > work as I am failing the project not because of a computational error
> > but because of the dreaded TLE (time limit exceeded) designator. I have
> > determined there are at least 3 areas for improvement. The first is
> > within my code where I am creating a list of the first million primes. 
> 
> It looks more like it creates a list 101 in length where the 
> non-zero numbers in that list are primes.  I'd try BuildSieve(100) until 
> BuildSieve was returning just primes.
> 
> Or, are you trying to create a list of the primes under 100?
> 
> Optimization should proceed once you've got it working.
> 
> Emie
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Robert Berman]

Emille,

Thank you for the example of list splicing.  Do you know if this is
faster than a more conventional loop statement as in my code for
primearray which is in my original post (reprinted here)

The code is as follows:

def BuildSieve(itemsin):
TheSieve=list()
TheSieve = range(0,itemsin+1)
TheSieve[1]=0
for i in range(2,itemsin+1):
if (TheSieve[i] > 0):
j = i + i
while (j <= itemsin):
TheSieve[j] = 0
j+=i
return TheSieve

It is called with PrimaryList = BuildSieve(100)

Again, thank you for your help.

Robert


On Wed, 2009-06-17 at 17:01 -0700, Emile van Sebille wrote:

> On 6/17/2009 4:48 PM Robert Berman said...
> > Emile,
> > 
> > Thank your for your comments. I do have a list running from 0-101.
> > Yes, it is true, I only needed 0 - 10 and yes I will change it.
> > However, if you use primearray 
> 
> you haven't posted the primearray code...
> 
> 
> > 
> > However, for the time being, can you perhaps share some suggestions on
> > list splicing?
> 
> So, this part of your original post--
>  > Out[53]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
>  >
>  > Since 3 is a prime, we can eliminate all multiples of 3. Within l1,
>  > these are expressed as
>  >
>  > In [52]: l1[n+n:len(l1):n]
>  > Out[52]: [6, 9, 12]
>  >
>  > when n = 3. ( do know 12 would have been eliminated by the prime
>  > number 2)
>  >
>  > It would be great if I could say l1[n+n:len(l1):n] = 0
> 
> but you can say:
> 
> for ii in l1[n+n:len(l1):n]: l1[ii] = 0
> 
> Is something like that what you're after?
> 
> Emile
> 
> 
> 
>  > but obviously
>  >
>  > that will fail for obvious reasons. I am looking for the right hand
>  > side of the statement to set a list within the list to all zeros.
> ---
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Emile van Sebille]

On 6/17/2009 4:48 PM Robert Berman said...

Emile,

Thank your for your comments. I do have a list running from 0-101.
Yes, it is true, I only needed 0 - 10 and yes I will change it.
However, if you use primearray 


you haven't posted the primearray code...




However, for the time being, can you perhaps share some suggestions on
list splicing?


So, this part of your original post--
> Out[53]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
>
> Since 3 is a prime, we can eliminate all multiples of 3. Within l1,
> these are expressed as
>
> In [52]: l1[n+n:len(l1):n]
> Out[52]: [6, 9, 12]
>
> when n = 3. ( do know 12 would have been eliminated by the prime
> number 2)
>
> It would be great if I could say l1[n+n:len(l1):n] = 0

but you can say:

for ii in l1[n+n:len(l1):n]: l1[ii] = 0

Is something like that what you're after?

Emile



> but obviously
>
> that will fail for obvious reasons. I am looking for the right hand
> side of the statement to set a list within the list to all zeros.
---

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Wayne]

On Wed, Jun 17, 2009 at 7:11 PM, Robert Berman  wrote:

>  Emille,
>
> Thank you for the example of list splicing.  Do you know if this is  faster
> than a more conventional loop statement as in my code for primearray which
> is in my original post (reprinted here)
>
> The code is as follows:
>
> def BuildSieve(itemsin):
> TheSieve=list()
> TheSieve = range(0,itemsin+1)
> TheSieve[1]=0
> for i in range(2,itemsin+1):
> if (TheSieve[i] > 0):
> j = i + i
> while (j <= itemsin):
> TheSieve[j] = 0
> j+=i
> return TheSieve
>
> It is called with PrimaryList = BuildSieve(100)
>

I'm curious if it wouldn't be faster to use xrange. I know for certain
instead of range(0, itemsin+1) you could use TheSieve = [1, 2]
TheSieve.extend(range(3, itemsin+1,2))

Which would save you at least half the time in your generation. Since your
first step should be removing multiples of two, that should be a big help.

HTH,
Wayne
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Emile van Sebille]

On 6/17/2009 5:11 PM Robert Berman said...

Emille,

Thank you for the example of list splicing.  Do you know if this is  
faster than a more conventional loop statement


Faster can be exactly determined using timeit.  (for some definition of 
exact -- the one we use mostly around here anyway)


So, for the bits you want to know faster about, you'd do something like:

>>> from timeit import Timer
>>> t = Timer("aa = range(0,100,2)")
>>> t.timeit()
1.6688069739620928
>>> t = Timer("aa = range(100)[::2]")
>>> t.timeit()
3.3360306112492282
>>>

I'm not sure this applies specifically to what you're doing, and 
crafting what you are timing is important, but read up on timeit and try 
it out.



as in my code for 
primearray which is in my original post (reprinted here)


Well, there's the confusion.  This is the code for something called 
BuildSieve.  In your prior post you show:


-
Printing a range, say primearray[21]
through primearray[50] will give you all prime numbers between 21 and
50.

23
29
31
37
41
43
47
-

... but this isn't what BuildSieve yields:

>>> BuildSieve(20)
[0, 0, 2, 3, 0, 5, 0, 7, 0, 0, 0, 11, 0, 13, 0, 0, 0, 17, 0, 19, 0]

So I still don't know what primearray is/does.

Emile



The code is as follows:

def BuildSieve(itemsin):
TheSieve=list()
TheSieve = range(0,itemsin+1)
TheSieve[1]=0
for i in range(2,itemsin+1):
if (TheSieve[i] > 0):
j = i + i
while (j <= itemsin):
TheSieve[j] = 0
j+=i
return TheSieve

It is called with PrimaryList = BuildSieve(100)

Again, thank you for your help.

Robert


On Wed, 2009-06-17 at 17:01 -0700, Emile van Sebille wrote:

On 6/17/2009 4:48 PM Robert Berman said...
> Emile,
> 
> Thank your for your comments. I do have a list running from 0-101.

> Yes, it is true, I only needed 0 - 10 and yes I will change it.
> However, if you use primearray 


you haven't posted the primearray code...


> 
> However, for the time being, can you perhaps share some suggestions on

> list splicing?

So, this part of your original post--
 > Out[53]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
 >
 > Since 3 is a prime, we can eliminate all multiples of 3. Within l1,
 > these are expressed as
 >
 > In [52]: l1[n+n:len(l1):n]
 > Out[52]: [6, 9, 12]
 >
 > when n = 3. ( do know 12 would have been eliminated by the prime
 > number 2)
 >
 > It would be great if I could say l1[n+n:len(l1):n] = 0

but you can say:

for ii in l1[n+n:len(l1):n]: l1[ii] = 0

Is something like that what you're after?

Emile



 > but obviously
 >
 > that will fail for obvious reasons. I am looking for the right hand
 > side of the statement to set a list within the list to all zeros.
---

___
Tutor maillist  -  Tutor@python.org 
http://mail.python.org/mailman/listinfo/tutor




___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Robert Berman]

Wayne,

Thank you for the suggestion. I will let you know how well this plays
out.

Robert




On Wed, 2009-06-17 at 19:26 -0500, Wayne wrote:
> On Wed, Jun 17, 2009 at 7:11 PM, Robert Berman 
> wrote:
> Emille,
> 
> Thank you for the example of list splicing.  Do you know if
> this is  faster than a more conventional loop statement as in
> my code for primearray which is in my original post (reprinted
> here)
> 
> 
> The code is as follows:
> 
> def BuildSieve(itemsin):
> TheSieve=list()
> TheSieve = range(0,itemsin+1)
> TheSieve[1]=0
> for i in range(2,itemsin+1):
> if (TheSieve[i] > 0):
> j = i + i
> while (j <= itemsin):
> TheSieve[j] = 0
> j+=i
> return TheSieve
> 
> It is called with PrimaryList = BuildSieve(100)
> 
> 
> I'm curious if it wouldn't be faster to use xrange. I know for certain
> instead of range(0, itemsin+1) you could use TheSieve = [1, 2]
> TheSieve.extend(range(3, itemsin+1,2))
> 
> Which would save you at least half the time in your generation. Since
> your first step should be removing multiples of two, that should be a
> big help.
> 
> HTH,
> Wayne
> 
> 

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Robert Berman]

Emille,

Thank you very much for the information on timeit. I will investigate
and use it.

>>... but this isn't what BuildSieve yields:

 >>> BuildSieve(20)
>>[0, 0, 2, 3, 0, 5, 0, 7, 0, 0, 0, 11, 0, 13, 0, 0, 0, 17, 0, 19, 0]

>>So I still don't know what primearray is/does.

If you look at or print all non zero numbers you have the primes between
2 and 20.

Robert


On Wed, 2009-06-17 at 17:30 -0700, Emile van Sebille wrote:

> On 6/17/2009 5:11 PM Robert Berman said...
> > Emille,
> > 
> > Thank you for the example of list splicing.  Do you know if this is  
> > faster than a more conventional loop statement
> 
> Faster can be exactly determined using timeit.  (for some definition of 
> exact -- the one we use mostly around here anyway)
> 
> So, for the bits you want to know faster about, you'd do something like:
> 
>  >>> from timeit import Timer
>  >>> t = Timer("aa = range(0,100,2)")
>  >>> t.timeit()
> 1.6688069739620928
>  >>> t = Timer("aa = range(100)[::2]")
>  >>> t.timeit()
> 3.3360306112492282
>  >>>
> 
> I'm not sure this applies specifically to what you're doing, and 
> crafting what you are timing is important, but read up on timeit and try 
> it out.
> 
> 
> > as in my code for 
> > primearray which is in my original post (reprinted here)
> 
> Well, there's the confusion.  This is the code for something called 
> BuildSieve.  In your prior post you show:
> 
> -
> Printing a range, say primearray[21]
> through primearray[50] will give you all prime numbers between 21 and
> 50.
> 
> 23
> 29
> 31
> 37
> 41
> 43
> 47
> -
> 
> ... but this isn't what BuildSieve yields:
> 
>  >>> BuildSieve(20)
> [0, 0, 2, 3, 0, 5, 0, 7, 0, 0, 0, 11, 0, 13, 0, 0, 0, 17, 0, 19, 0]
> 
> So I still don't know what primearray is/does.
> 
> Emile
> 
> > 
> > The code is as follows:
> > 
> > def BuildSieve(itemsin):
> > TheSieve=list()
> > TheSieve = range(0,itemsin+1)
> > TheSieve[1]=0
> > for i in range(2,itemsin+1):
> > if (TheSieve[i] > 0):
> > j = i + i
> > while (j <= itemsin):
> > TheSieve[j] = 0
> > j+=i
> > return TheSieve
> > 
> > It is called with PrimaryList = BuildSieve(100)
> > 
> > Again, thank you for your help.
> > 
> > Robert
> > 
> > 
> > On Wed, 2009-06-17 at 17:01 -0700, Emile van Sebille wrote:
> >> On 6/17/2009 4:48 PM Robert Berman said...
> >> > Emile,
> >> > 
> >> > Thank your for your comments. I do have a list running from 0-101.
> >> > Yes, it is true, I only needed 0 - 10 and yes I will change it.
> >> > However, if you use primearray 
> >>
> >> you haven't posted the primearray code...
> >>
> >> 
> >> > 
> >> > However, for the time being, can you perhaps share some suggestions on
> >> > list splicing?
> >>
> >> So, this part of your original post--
> >>  > Out[53]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
> >>  >
> >>  > Since 3 is a prime, we can eliminate all multiples of 3. Within l1,
> >>  > these are expressed as
> >>  >
> >>  > In [52]: l1[n+n:len(l1):n]
> >>  > Out[52]: [6, 9, 12]
> >>  >
> >>  > when n = 3. ( do know 12 would have been eliminated by the prime
> >>  > number 2)
> >>  >
> >>  > It would be great if I could say l1[n+n:len(l1):n] = 0
> >>
> >> but you can say:
> >>
> >> for ii in l1[n+n:len(l1):n]: l1[ii] = 0
> >>
> >> Is something like that what you're after?
> >>
> >> Emile
> >>
> >>
> >>
> >>  > but obviously
> >>  >
> >>  > that will fail for obvious reasons. I am looking for the right hand
> >>  > side of the statement to set a list within the list to all zeros.
> >> ---
> >>
> >> ___
> >> Tutor maillist  -  Tutor@python.org 
> >> http://mail.python.org/mailman/listinfo/tutor
> > 
> > 
> > 
> > ___
> > Tutor maillist  -  Tutor@python.org
> > http://mail.python.org/mailman/listinfo/tutor
> 
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Fast way to access items in a dictionary

[Elisha Rosensweig]

Hi,

Till now, when I receive a dictionary that I'm not sure contains a certain
key, I would use the following template to access a given key:

if 'someKey' in dict.keys():
   someData = dict['someKey']

is there a faster way to do this? accessing a key that does not exist will
through an exception, which is just as tiresome...

Thanks

Elisha
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[bob gailer]

Robert Berman wrote:

Greetings,

I am working on a 'simple' algorithm to solve the problem called PRIME1
explained at http://www.spoj.pl/problems/PRIME1/. 


I do have an algorithm based on the Sieve of Eratosthenes and it does
work as I am failing the project not because of a computational error
but because of the dreaded TLE (time limit exceeded) designator. I have
determined there are at least 3 areas for improvement. The first is
within my code where I am creating a list of the first million primes. 
The code is as follows:


def BuildSieve(itemsin):
TheSieve=list()
TheSieve = range(0,itemsin+1)
TheSieve[1]=0
for i in range(2,itemsin+1):
if (TheSieve[i] > 0):
j = i + i
while (j <= itemsin):
TheSieve[j] = 0
j+=i
return TheSieve
  

Observations:

By convention we like to use names beginning with CAPS for class names 
and names beginning with lower for variable names. Not a requirement, 
but it makes it easier for us to read the code.


There is no need for TheSieve=list() as the next statement overrides that.

There is no need to store the actual numbers in the list, as the list 
index can serve that purpose.
You could instead build a list of 2 0s followed by all 1s: The Sieve = 
[0,0] + [1]*(itemsin-2).



It is called with PrimaryList = BuildSieve(100)
  




I have read that list splicing rather than looping to set a range of
items to zero is the best approach. Given an example  list  of

In [53]: l1
Out[53]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Since 3 is a prime, we can eliminate all multiples of 3. Within l1,
these are expressed as

In [52]: l1[n+n:len(l1):n]
Out[52]: [6, 9, 12]

when n = 3. ( do know 12 would have been eliminated by the prime number
2)

It would be great if I could say l1[n+n:len(l1):n] = 0 but obviously
that will fail for obvious reasons. I am looking for the right hand side
of the statement to set a list within the list to all zeros. Also, if
anyone had a relatively simple explanation why in Python this is faster
than a loop, I would really like to understand it.

Thanks for the input.

Robert



___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

  



--
Bob Gailer
Chapel Hill NC
919-636-4239
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fast way to access items in a dictionary

[Wayne]

On Wed, Jun 17, 2009 at 7:39 PM, Elisha Rosensweig wrote:

> Hi,
>
> Till now, when I receive a dictionary that I'm not sure contains a certain
> key, I would use the following template to access a given key:
>
> if 'someKey' in dict.keys():
>someData = dict['someKey']
>
> is there a faster way to do this? accessing a key that does not exist will
> through an exception, which is just as tiresome...


sure is:

In [97]: mydict = {'knights':'say ni', 'death':'pointy teeth'}

In [98]: mydict.get('spam')

In [99]: mydict.get('knights')
Out[99]: 'say ni'

In [100]: if mydict.get(3):
   .: print 'Hi'
   .:
   .:

In [101]: if mydict.get('death'):
   .: print mydict['death']
   .:
   .:
pointy teeth

HTH,
Wayne
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fast way to access items in a dictionary

[Emile van Sebille]

On 6/17/2009 5:39 PM Elisha Rosensweig said...

Hi,

Till now, when I receive a dictionary that I'm not sure contains a 
certain key, I would use the following template to access a given key:


if 'someKey' in dict.keys():
   someData = dict['someKey']

is there a faster way to do this? 


Looks like a simple 'in' is faster both when it's there...

>>> Timer("'D' in  {'D':123}.keys()").timeit()
0.93669924584355613
>>> Timer("'D' in  {'D':123}").timeit()
0.34678047105990117

... and when it isn't...

>>> Timer("'E' in  {'D':123}.keys()").timeit()
0.99194670371434768
>>> Timer("'E' in  {'D':123}").timeit()
0.34795386410769424
>>>

Emile

accessing a key that does not exist 
will through an exception, which is just as tiresome...


Thanks

Elisha




___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] List Splicing

[Luke Paireepinart]

Robert Berman wrote:

Emille,

Thank you for the example of list splicing.  Do you know if this is  
faster than a more conventional loop statement as in my code for 
primearray which is in my original post (reprinted here)
As has been mentioned, you will want to profile your code to know what 
is truly faster.
however, the main idea in speeding up your code is to move as much of 
the actual operations to C functions as possible.

For example,
x = []
for i in range(1):
   x.append(0)

is going to be quite a bit slower than
x = [0] * 1


>>> t = Timer("""x = [0] * 1000""")
>>> t.timeit()
6.623384202829115
>>> t = Timer("""x = []
for i in range(1000):
   x.append(0)""")
>>> t.timeit()
141.14222554321785

As you can see, the for loop version is about 20 times slower than the 
other version.

But they both accomplish the same thing.
The key here is that the first one, using the "[] * var" syntax, does 
not iterate over items individually.
This is a gross oversimplification, but think of the python interpreter 
like this:
you take your code to a copy shop (a la Kinkos).  however, they do not 
have any copy machines... instead they will rewrite it by hand, line by 
line.

and say they're going to charge you 20 cents per line.
Therefore you'd want to minimize the lines of code to save money, right?

However, in Python, the interpreter isn't executing just line-by-line 
(otherwise we'd have no loops)

so what it's essentially seeing  in the two examples are, first:
x = [0] * 1000

and for the second example:
x = []
x.append(0)
x.append(0)
x.append(0)
x.append(0)
x.append(0)
. about 992 more times ...
x.append(0)
x.append(0)
x.append(0)

So you are getting charged to "copy" each of these lines.  So what 
originally looked like only 2 or 3 more lines of code is in fact many 
hundreds more instructions.
This analogy refers to the overhead of the interpreter - just to call 
functions and do other basic things, the overhead is much higher than it 
would be in a compiled language such as C.


Now you may be thinking "but behind the scenes, x = [0] * 1000 is a lot 
of instructions as well!"  and you're correct.
However, here's where the key of Python comes in: that "behind the 
scenes" stuff is written over a period of years by very experienced 
coders, and has been optimized to heck.  Also, it's typically going to 
be written in C and compiled to native code, which is *a lot* faster 
than interpreted Python is.


So basically when you're trying to optimize Python, remember that the 
cost of executing each individual line is not that high, for the most 
part (of course there are exceptions).
What you want to do is minimize the amount of lines that are interpreted 
and maximize the amount of code that is executed by the interpreter's 
libraries, which are highly efficient C code.  While in another language 
you may think "I can do this faster if I do it myself, because the 
language's libraries are probably too generic to be faster than a custom 
implementation" but if you want to write fast Python, you have to think 
"I want to try to do as little as possible most of the time, so I can 
transfer most of the computational load over to the libraries and away 
from my code."


Just some generic speculation on making stuff more efficient...

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fast way to access items in a dictionary

[Luke Paireepinart]

Looks like a simple 'in' is faster both when it's there...

>>> Timer("'D' in  {'D':123}.keys()").timeit()
0.93669924584355613
>>> Timer("'D' in  {'D':123}").timeit()
0.34678047105990117

... and when it isn't...

>>> Timer("'E' in  {'D':123}.keys()").timeit()
0.99194670371434768
>>> Timer("'E' in  {'D':123}").timeit()
0.34795386410769424
>>>


That's because dictionaries are iterable by default, so if you're 
calling keys() you're just adding the overhead of another function call, 
and (I think) also generating a whole list as well (but keys() may 
return an iterable as well.)

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Apparent incosistency with Python interperter in IDLE

[Karen Palen]

I am an experienced C/C++/C# programmer who is just starting out with Python. 
My first attempt at IDLE worked great with the tutorial, but seems to have a 
problem with my first real app!

I am trying to get the Python subprocess/Popen module working with the example 
from the Python 3 documentation page:
>>> import subprocess
>>> subprocess.call('ls')
ArchiveFirefox_wallpaper.png  result.txt ...

which is what I expect.

However this same input to the IDLE "Python Shell" yields:

Python 3.0.1+ (r301:69556, Apr 15 2009, 15:59:22) 
[GCC 4.3.3] on linux2
Type "copyright", "credits" or "license()" for more information.
 No Subprocess 
>>> import subprocess
>>> subprocess.call('ls')
0
>>> 


Which appears to be merely the return code, not the stdout!

It looks like I need to set some variable in IDLE, but I can't figure out 
exactly what is needed here.

Can anyone point me to an answer?

Karen


  
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Apparent incosistency with Python interperter in IDLE

[Luke Paireepinart]

Karen Palen wrote:


Which appears to be merely the return code, not the stdout!

It looks like I need to set some variable in IDLE, but I can't figure out 
exactly what is needed here.

Can anyone point me to an answer?
  


Well, Karen, according to my IDLE (admittedly it's 2.6 not 3.0 so it may 
not agree), call only returns the return code.

You can check on your system by doing the following:

IDLE 2.6.2 
>>> import subprocess

>>> help(subprocess.call)

which will print out the docstring for the method:

Help on function call in module subprocess:

call(*popenargs, **kwargs)
   Run command with arguments.  Wait for command to complete, then
   return the returncode attribute.
  
   The arguments are the same as for the Popen constructor.  Example:
  
   retcode = call(["ls", "-l"])


I believe what's happening is that you are confused about the presence 
of STDOUT in IDLE.


In a normal console window, when you call a program (such as "ls"), 
stdout is directed at the screen (the console window.)

So what's happening in the first example (the non-idle one) is:

open subprocess and execute an "ls" call
   - the subprocess prints out the result of the "ls" call
   -subprocess finishes
process finishes

Now the key here is that in this case, both of these have the same 
stdout (the screen), which is the default stdout.


However, when you run from an IDLE prompt, your stdout is not going to 
be the default.

check it out if you want:

>>> import sys
>>> print sys.stdout


Notice that this is not a standard stdout.
In a normal Python window,

>>> import sys
>>> print sys.stdout
', mode 'w' at 0x00A43070>

it's not the same stdout as IDLE has.

So what's happening when you run the command from IDLE:
open subprocess and execute an "ls" call
   -subprocess starts, executes "ls", prints result to _its_ stdout 
(which is not IDLE's stdout, remember!)

   -subprocess executes, output goes bye bye
your process ends as well

So the problem is that the stdout of the "ls" command is appearing in 
some location that you cannot see.
As for ways to remedy this - I don't know.  The idea here, though, is 
that even though the regular Python version has the side-effect that it 
outputs it in the console, that's not necessarily what you want it to 
do.  The reason is that you have no way to access that data.


Also you mentioned that IDLE prints out a 0.  This is only something 
that IDLE's interpreter does.

For example
>>> x = "hello"
>>> x
'hello'

IDLE echoes the value of x.  However, if you had the code
x = "hello"
x

and you ran it (not in IDLE's interpreter) you would not get any output.

What you should learn from this is
1- IDLE is weird in many ways
2 - that subprocess.call() is not meant to get data back from a call 
(look at subprocess.Popen and its communicate() method for that)
3 - sometimes things that appear to be outputs are 
side-effects/artifacts of the environment you're running them in and 
can't be depended upon.


HTH,
-Luke
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor