Re: Multi thread reading a file

[Stefan Behnel]

Gabriel Genellina wrote:
> En Tue, 30 Jun 2009 22:52:18 -0300, Mag Gam  escribió:
> 
>> I am very new to python and I am in the process of loading a very
>> large compressed csv file into another format.  I was wondering if I
>> can do this in a multi thread approach.
> 
> Does the format conversion involve a significant processing time? If
> not, the total time is dominated by the I/O time (reading and writing
> the file) so it's doubtful you gain anything from multiple threads.

Well, the OP didn't say anything about multiple processors, so multiple
threads may not help wrt. processing time. However, if the file is large
and the OS can schedule the I/O in a way that a seek disaster is avoided
(although that's hard to assure with today's hard disk storage density, but
SSDs may benefit), multiple threads reading multiple partial streams may
still reduce the overall runtime due to increased I/O throughput.

That said, the OP was mentioning that the data was compressed, so I doubt
that the I/O bandwidth is a problem here. As another poster put it: why
bother? Run a few benchmarks first to see where (and if!) things really get
slow, and then check what to do about the real problem.

Stefan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: invoking a method from two superclasses

[Carl Banks]

On Jun 30, 9:15 pm, "Gabriel Genellina" 
wrote:
> En Tue, 30 Jun 2009 21:34:02 -0300, Mitchell L Model  
>  escribi :
>
> > Allow me to add to my previous question that certainly the superclass
> > methods can be called explicitly without resorting to super(), e.g.:
>
> >     class C(A, B):
> >         def __init__(self):
> >             A.__init__(self)
> >             B.__init__(self)
>
> > My question is really whether there is any way of getting around the
> > explicit class names by using super() and if not, shouldn't the  
> > documentation
> > of super point out that if more than one class on the mro defines a  
> > method
> > only the first will get called?
>
> super returns [a proxy to] the *next* class in the MRO chain; it may or  
> may not be a superclass of C (it may be a sibling class instead).

It could be even a niece class.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Basic question from pure beginner

[sato.ph...@gmail.com]

I have been going through some Python Programming exercises while
following the MIT OpenCourseWare Intro to CS syllabus and am having
some trouble with the first "If" exercise listed on this page:

http://en.wikibooks.org/wiki/Python_Programming/Conditional_Statements#If_Exercises

I have been able to make the module quit after entering a password
three times, but can't get it to quit right away after the correct one
is entered.  I know this is really basic, please forgive me.  I have
no programming experience and have just started going through these
tutorials.

My code is here:

http://python.pastebin.com/m6036b52e

-daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Making code run in both source tree and installation path

[Carl Banks]

On Jun 29, 9:20 am, Javier Collado  wrote:
> I've seen different approaches:
> - distutils trick in setup.py to modify the installed script (i.e.
> changing a global variable value) so that it has a reference to the
> data files location.


One of my biggest complaints about distutils is that it doesn't do
this, a limitation made worse by the fact that distutils allows you to
specify an alternate data file directory, but your scripts have no way
to know what that alternate directory is installed.  Which really
limits the usefulness of that functionality.

The most common way I've seen people work around this issue is to
throw their data files into the package directories.  Yuck.

At one point I hacked up a setup.py file to look in the distutils data
structure and pull out the data install location, and wrote out a
trivial python file listing that location.  (The distutils build data
structure is helpfully returned by setup function.)  I never felt good
about it, and it wouldn't work if the install was done in steps (such
as build as user, install as superuser).

If you care to navigate the murky internals of distutils to do this in
a more elegant way, more power to you, and if so I'd recommend doing
it that way.


> - Heuristic in the package code to detect when it's being executed
> from the source tree and when it has been the installed
> - Just using an environment variable that the user must set according
> to his needs

I guess I'd combine these two.  Make a sensible guess about where the
data files are by checking out the environment, but if the data files
aren't there (such as if the user installs to a different data
location) then they are responsible for setting an env variable or
config option.



> I guess that there are other options, for example, maybe using
> buildout. What would you say it's the best/more elegant option to
> solve this problem?

Another option is not to use distutils at all.  If you are writing an
application, I think that would be a good idea.  I don't think
applications really need to be located in site-packages.

If you have any C-extensions it might complicate this.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[Peter Otten]

sato.ph...@gmail.com wrote:

> I have been going through some Python Programming exercises while
> following the MIT OpenCourseWare Intro to CS syllabus and am having
> some trouble with the first "If" exercise listed on this page:
> 
> 
http://en.wikibooks.org/wiki/Python_Programming/Conditional_Statements#If_Exercises
> 
> I have been able to make the module quit after entering a password
> three times, but can't get it to quit right away after the correct one
> is entered.  I know this is really basic, please forgive me.  I have
> no programming experience and have just started going through these
> tutorials.
> 
> My code is here:
> 
> http://python.pastebin.com/m6036b52e
> 
> -daniel

Some hints: 

(1) Have the while loop check for both the correct password and the number 
of tries. Pseudocode:

while  and :
   ...

(2) After the loop has terminated you can check whether the password was 
entered correctly and print the appropriate message.

Aside: you can't mix if...else with other statements

correct:

if cond:
   print "sunny"
else:
   print "cloudy"
print "weather"

wrong: 

if cond:
   print "norwegian"
print "blue" 
else: # syntax error, else must immediately follow the indented if-block:
   print "moon"

you could fix it like so:

if cond:
   print "norwegian"
print "blue"
if not cond:
print "moon"

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[alex23]

On Jul 1, 3:38 pm, "sato.ph...@gmail.com" 
wrote:
> I have been able to make the module quit after entering a password
> three times, but can't get it to quit right away after the correct one
> is entered.  

Not with the code you pasted, you haven't. There's a missing colon on
line 7 & line 9 isn't indented properly. It's always best if we're
referring to the same source when trying to help with a problem.

The problem you're having, though, has to do with the while loop and
the fact that it's only checking one condition: has the 'count'
counter been incremented to 3. What you need to do is _also_ check for
the success condition:

is_confirmed = False
while count != 3 or is_confirmed:
  guess = raw_input("Enter your password: ")
  guess = str(guess)
  if guess != password:
print "Access Denied"
count = count + 1
  else:
print "Password Confirmed"
is_confirmed = True

This also provides you with a nice boolean that shows whether or not
the password was confirmed, which may be useful for latter code.

However, whenever you want to loop a set number of times, it's usually
better to use a 'for' loop instead:

PASSWORD = 'qwerty'
MAXRETRY = 3
for attempt in xrange(MAXRETRY):
  guess = str(raw_input('Enter your password: '))
  is_complete = guess == PASSWORD
  if is_complete:
print 'Password confirmed'
break # this exits the for loop
  else:
print 'Access denied: attempt %s of %s' % (attempt+1, MAXRETRY)

This saves you from the burden of creating, incrementing & testing
your own counter.

If there's anything here that isn't clear, please don't hesitate to
ask.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[sato.ph...@gmail.com]

Thank you for all of the help.  With your assistance and help from the
Python Tutor mailing list I was able to come up with the following
code:

password = "qwerty"
correct_password_given = False
guess = "0"
count = 0
while count != 3 and not correct_password_given :
   guess = raw_input("Enter your password: ")
   guess = str(guess)
   if guess != password:
   print "Access Denied"
   count = count + 1
   else:
   print "Password Confirmed"
   correct_password_given = True


If I understand it correctly, the function will continue to loop until
either the count == 3 or until correct_password_give = True,
satisfying the two conditions of the assignment, which were to lock a
user out after three failed password attempts and to print "Access
Granted" and end the module if the correct password is given.  Am I
understanding this correctly?

Thanks!

On Jul 1, 12:06 am, alex23  wrote:
> On Jul 1, 3:38 pm, "sato.ph...@gmail.com" 
> wrote:
>
> > I have been able to make the module quit after entering a password
> > three times, but can't get it to quit right away after the correct one
> > is entered.  
>
> Not with the code you pasted, you haven't. There's a missing colon on
> line 7 & line 9 isn't indented properly. It's always best if we're
> referring to the same source when trying to help with a problem.
>
> The problem you're having, though, has to do with the while loop and
> the fact that it's only checking one condition: has the 'count'
> counter been incremented to 3. What you need to do is _also_ check for
> the success condition:
>
> is_confirmed = False
> while count != 3 or is_confirmed:
>   guess = raw_input("Enter your password: ")
>   guess = str(guess)
>   if guess != password:
>     print "Access Denied"
>     count = count + 1
>   else:
>     print "Password Confirmed"
>     is_confirmed = True
>
> This also provides you with a nice boolean that shows whether or not
> the password was confirmed, which may be useful for latter code.
>
> However, whenever you want to loop a set number of times, it's usually
> better to use a 'for' loop instead:
>
> PASSWORD = 'qwerty'
> MAXRETRY = 3
> for attempt in xrange(MAXRETRY):
>   guess = str(raw_input('Enter your password: '))
>   is_complete = guess == PASSWORD
>   if is_complete:
>     print 'Password confirmed'
>     break # this exits the for loop
>   else:
>     print 'Access denied: attempt %s of %s' % (attempt+1, MAXRETRY)
>
> This saves you from the burden of creating, incrementing & testing
> your own counter.
>
> If there's anything here that isn't clear, please don't hesitate to
> ask.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Learning to use decorators with classes

[Bruno Desthuilliers]

sk8in_zo...@yahoo.com.au a écrit :

--- On Tue, 30/6/09, Bruno Desthuilliers 
 wrote:


(snip)


This can't work, and it's a FAQ FWIW - but since there's no
official c.l.py FAQ, we won't hold it against you !-)



Can you please point me to the FAQ related to this snippet. I would be grateful.


Well, as I said, there's *no* c.l.py FAQ (I mean, any half-official text 
document listing frequently asked questions and their canonical 
answers), so I just can't provide any pointer here :-/


I guess we all here love talking about Python (and showing out our 
science, knowledge and wisdom) too much to maintain a FAQ !-)


(snip)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[Bruno Desthuilliers]

sato.ph...@gmail.com a écrit :

Thank you for all of the help.  With your assistance and help from the
Python Tutor mailing list I was able to come up with the following
code:

password = "qwerty"
correct_password_given = False
guess = "0"


You could just use None here:

  guess=None



count = 0
while count != 3 and not correct_password_given :
   guess = raw_input("Enter your password: ")
   guess = str(guess)


IIRC, raw_input() already returns a string !-)


   if guess != password:
   print "Access Denied"
   count = count + 1


Or:
 count += 1


   else:
   print "Password Confirmed"
   correct_password_given = True


If I understand it correctly, the function will continue to loop until
either the count == 3 or until correct_password_give =  True,


There are many ways to write a same boolean test, and some are easier to 
understand. Your test could be written:


  while not (count == 3 or correct_password_given) :
 # proceed

Does it answer your question ?-)

As a general rule:

* not A and not B <=> not (A or B)
* not A or not B  <=> not (A and B)



satisfying the two conditions of the assignment, which were to lock a
user out after three failed password attempts and to print "Access
Granted" and end the module if the correct password is given.


Well, given your snippet, execution indeed terminates after either 
successful 'login' or 3 failed attempts, but that's mostly because 
there's no code to execute after the while loop...


--
http://mail.python.org/mailman/listinfo/python-list

Re: Direct interaction with subprocess - the curse of blocking I/O

[Lawrence D'Oliveiro]

In message , ryles wrote:

> On Jun 29, 5:43 pm, Scott David Daniels  wrote:
>
>>and I personally wouldn't have it any other way.  Simulating a shell
>> with hooks on its I/O should be so complicated that a "script kiddie"
>> has trouble writing a Trojan.
> 
> +1 QOTW

Trouble is, script kiddies, by definition, don't need to write anything. 
They just need to download something somebody else has already written.

-- 
Lawrence "I counter your +1 with my -1" D'Oliveiro

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[Lawrence D'Oliveiro]

In message , João 
Valverde wrote:

> Simple example usage case: Insert string into data structure in sorted
> order if it doesn't exist, else retrieve it.

the_set = set( ... )

if str in the_set :
... "retrieval" case ...
else :
the_set.add(str)
#end if

Want sorted order?

sorted(tuple(the_set))

What could be simpler?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[Lawrence D'Oliveiro]

In message , Miles 
Kaufmann wrote:

> On Jun 26, 2009, at 2:23 AM, Chris Rebert wrote:
> 
>> That's pretty much the bisect module in a nutshell. It manipulates a
>> sorted list using binary search.
> 
> With O(n) insertions and removals, though.  A decent self-balancing
> binary tree will generally do those in O(log n).

For values of n below, say, a million, would you even notice the difference 
on modern hardware?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 376

[Paul Rubin]

Joachim Strömbergson  writes:
> http://docs.python.org/library/hashlib.html
> I would suggest to use the SHA-256 in the library.

I agree with this.  We should even move towards supporting x509 and/or
gpg signatures in eggs, similar to signed .jar files.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[Lawrence D'Oliveiro]

In message , João Valverde 
wrote:

> But a dict can't be used to implement a (sorted) table ADT.

for key in sorted(the_dict.keys(), cmp = ... whatever ordering criteria you 
like ...) :
... do something with the_dict[key] ...
#end for


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 376

[Richard Brodie]

"Joachim Strömbergson"  wrote in message 
news:mailman.2422.1246418400.8015.python-l...@python.org...

> Even so, choosing md5 in 2009 for something that (hopefully) will be
> used in years is a bad design decision. It creates a dependency for to
> an algorithm that all sensible recommendations point you to move away
> from.

Why not write the field as algorithm:value?

e.g. sha1:8590b685654367e3eba70dc00df7e45e88c21da4

Installers can fallback to using hashlib.new(), so you can plug in a new
algorithm without changing the PEP or the installer code. 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[Paul Rubin]

Lawrence D'Oliveiro  writes:
> > With O(n) insertions and removals, though.  A decent self-balancing
> > binary tree will generally do those in O(log n).
> 
> For values of n below, say, a million, would you even notice the difference 
> on modern hardware?

Huh?  Yes, of course you'd notice, unless you were doing the operation
just once or something like that.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP 376

[Paul Rubin]

"Richard Brodie"  writes:
> Why not write the field as algorithm:value?
> e.g. sha1:8590b685654367e3eba70dc00df7e45e88c21da4

This is reasonable, though I would deprecate md5 and sha1 already, and
start with sha256.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: string character count

[Iain King]

On Jun 30, 6:27 pm, noydb  wrote:
> If I have a string for a file name such that I want to find the number
> of characters to the left of the dot, how can that be done?
>
> I did it this way:
> x = "text12345.txt"
> dot = x.find('.')
> print dot
>
> Was curious to see what method others would use - helps me learn.  I
> guess I was most curious to see if it could be done in one line.
>
> And, how would a char count be done with no dot -- like if the string
> were "textstringwithoutdot" or "no dot in text string"?

import os
root, ext = os.path.splitext(x)
print len(root)

or in one line (assuming you've imported os):
print len(os.path.splitext(x)[0])


Iain
-- 
http://mail.python.org/mailman/listinfo/python-list

A question about fill_free_list(void) function

[Pedram]

Hello community,
I'm reading the CPython interpreter source code,
first, if you have something that I should know for better reading
this source code, I would much appreciate that :)
second, in intobject.c file, I read the following code in
fill_free_list function that I couldn't understand:
while (--q > p)
Py_TYPE(q) = (struct _typeobject *)(q-1);
Py_TYPE(q) = NULL;

What does it mean?

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with uuid package when embedding a python interpreter

[Jérôme Fuselier]

On Jun 30, 7:02 pm, "Gabriel Genellina" 
wrote:
> En Tue, 30 Jun 2009 13:05:42 -0300, Jérôme Fuselier
>  escribió:
>
>
>
> >   I've tried to import a script in an embedded python intrepreter but
> > this script fails when it imports the uuid module. I have a
> > segmentation fault in Py_Finalize().
>
> > #include "Python.h"
>
> > void test() {
> >     Py_Initialize();
> >     PyImport_Import(PyString_FromString("uuid"));
> >     Py_Finalize();
> > }
>
> > main(int argc, char **argv)
> > {
> >     for (i=0 ; i < 10; i++)
> >         test();
> > }
>
> > For my application, I have to call Py_initialize and Py_Finalize
> > several times so factorizing them in the main function is not an easy
> > solution for me.
>
> Are you sure you can't do that? Not even using Py_IsInitialized? Try to
> avoid repeatedly calling Py_Initialize - won't work.
>
> Python 2.x does not have a way to "un-initialize" an extension module
> (that's a big flaw in Python design). Modules that contain global state
> are likely to crash the interpreter when used by the second time. (Python
> 3 attempts to fix that)
>
> --
> Gabriel Genellina

Hi Gabriel,
  Thanks for your response. I can modify my code to call
Py_IsInitialized which I didn't know before and this works well. The
problem is that I did not own the process which calls Py_Initialize
and Py_Finalize. I'm able to call Py_Initialize correctly but I can't
call Py_Finalize so this is not perfect.

At least I have a temporary solution which is not too bad.
Thanks,
Jerome
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Passing parameters for a C program in Linux.

[bobicanprogram]

On Jun 30, 6:46 am, "venutaurus...@gmail.com"
 wrote:
> Hi all,
>I have to write an automted script which will test my c
> program. That program when run will ask for the commands. For example:
>
> local-host# ./cli
> Enter 1 for add
> Enter 2 for sub
> Enter 3 for mul
> 1---Our option
> Enter two numbers
> 44 33   Our option
> Result is 77
>
> As shown in the above example it lists all the options and waits for
> user input and once given, it does some operations and waits for some
> more. This has to be automated.
>
> Can someone give suggestions how to do this in Python (Linux Platform
> in particular).
>
> Thank you,
> Venu.


Take a look at the examples here:

http://www.icanprogram.com/06py/main.html

You can probably do what you want using the SIMPL toolkit quite
easily.

bob
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[Steven D'Aprano]

On Wed, 01 Jul 2009 20:39:06 +1200, Lawrence D'Oliveiro wrote:

> In message , João
> Valverde wrote:
> 
>> Simple example usage case: Insert string into data structure in sorted
>> order if it doesn't exist, else retrieve it.
> 
> the_set = set( ... )
> 
> if str in the_set :
> ... "retrieval" case ...

Not terribly useful, because there's no way of attaching any data to 
retrieve to the key. Apart from the case of interning strings (or other 
keys), you'd probably want a dict rather than a set:

if str in the_dict:
return the_dict[str]
else:
the_dict[str] = something_or_other


> else :
> the_set.add(str)
> #end if


> Want sorted order?
> 
> sorted(tuple(the_set))
> 
> What could be simpler?

Dropping the entirely superfluous creation of a tuple:

sorted(the_set)

That's a reasonable approach for small sets of data, but it isn't very 
useful if the_set contains 4GB of keys, because you have to duplicate the 
keys, then sort them. More effective would be a data structure that was 
always sorted. This has the further advantage that it's even simpler than 
sorted(the_set):

the_set



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

what will be best framework to select to develop Multi-Level Marketing (network marketing) application in python?

[gganesh]

hi group,
I have to build a software for Multi-Level Marketing (network
marketing),There are several Frameworks available in python ,i could
like to know the best Framework suitable to develop an application on
Multi-Level Marketing .
Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

infinite recursion in pickle.load()

[Thomas]

we regularly pickle and unpickle data from the same script (mostly 
dictionaries).


sometimes, a file written this way cannot be unpickled with 
pickle.load(), due to an infinite recursion with __getattr__ in 
codecs.py. here is a python2.5 stack trace excerpt:


/usr/local/lib/python2.5/pickle.py in load(file)
   1403
   1404 def load(file):
-> 1405 return Unpickler(file).load()
   1406
   1407 def loads(str):

/usr/local/lib/python2.5/pickle.py in load(self)
891 while 1:
892 key = read(1)
--> 893 dispatch[key](self)
894 except _Stop, stopinst:
895 return stopinst.value

/usr/local/lib/python2.5/pickle.py in load_build(self)
   1248 state = stack.pop()
   1249 inst = stack[-1]
-> 1250 setstate = getattr(inst, "__setstate__", None)
   1251 if setstate:
   1252 setstate(state)

/usr/local/lib/python2.5/codecs.py in __getattr__(self, name, getattr)
328 """ Inherit all other methods from the underlying stream.
329 """
--> 330 return getattr(self.stream, name)
331
332 def __enter__(self):

/usr/local/lib/python2.5/codecs.py in __getattr__(self, name, getattr)
328 """ Inherit all other methods from the underlying stream.
329 """
--> 330 return getattr(self.stream, name)
331
332 def __enter__(self):

...

The last frame repeats ad infinitum.

'inst' in the third frame is a 

The problem is the same with cPickle.

This looks somewhat related to this Python issue, which is only about 
the exception reporting:

http://bugs.python.org/issue5508
(The title of the issue is the error you get when running our code in 
python2.6).


Any idea how to go about that?

T.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Timeout when connecting to sybase DBS

[eranlevi]

On Jun 30, 7:48 pm, s...@pobox.com wrote:
>     Gil> I have looked for a timeout parameter to limit the 4 minutes to
>     Gil> something more reasonable but couldn't find one.  Can anyone please
>     Gil> help?
>
>     Gil> BTW, I'm using Sybase.connect(, , ,
>     Gil> datetime='auto')
>
> We use the Sybase module where I work, but I've never encountered this
> problem (or a timeout parameter of any kind).  At any rate, you'll probably
> have more luck asking on the python-sybase mailing list:
>
>     python-sybase-m...@lists.sourceforge.net
>
> --
> Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/
>     when i wake up with a heart rate below 40, i head right for the espresso
>     machine. -- chaos @ forums.usms.org

Are you saying, that when you trying to connect to a sybase DBS server
and the DBS or the server is down, you get an error after a few
seconds and not after a few minutes? Which python version are you
using?

I'll try to ask in the python-sybase mailing list

Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Timeout when connecting to sybase DBS

[eranlevi]

On Jul 1, 3:56 pm, eranlevi  wrote:
> On Jun 30, 7:48 pm, s...@pobox.com wrote:
>
>
>
> >     Gil> I have looked for a timeout parameter to limit the 4 minutes to
> >     Gil> something more reasonable but couldn't find one.  Can anyone please
> >     Gil> help?
>
> >     Gil> BTW, I'm using Sybase.connect(, , ,
> >     Gil> datetime='auto')
>
> > We use the Sybase module where I work, but I've never encountered this
> > problem (or a timeout parameter of any kind).  At any rate, you'll probably
> > have more luck asking on the python-sybase mailing list:
>
> >     python-sybase-m...@lists.sourceforge.net
>
> > --
> > Skip Montanaro - s...@pobox.com -http://www.smontanaro.net/
> >     when i wake up with a heart rate below 40, i head right for the espresso
> >     machine. -- chaos @ forums.usms.org
>
> Are you saying, that when you trying to connect to a sybase DBS server
> and the DBS or the server is down, you get an error after a few
> seconds and not after a few minutes? Which python version are you
> using?
>
> I'll try to ask in the python-sybase mailing list
>
> Thanks

There's no such group as python-sybase :-(

Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A question about fill_free_list(void) function

[Pedram]

Oh... I got it! :)
I found this line in ctypes.h:
#define Py_TYPE(q) = ((PyObject *)(q))->ob_next;
So those lines are trying to set the blocks type from rear to front.

But I still don't know why after the while (when q is equal to p), the
Py_TYPE(q) set to NULL!
-- 
http://mail.python.org/mailman/listinfo/python-list

deleting certain entries in numpy array

[Sebastian Schabe]

Hello everybody,

I'm new to python and numpy and have a little/special problem:

I have an numpy array which is in fact a gray scale image mask, e.g.:

mask =
array([[  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0, 255, 255, 255,   0,   0, 255,   0],
   [  0,   0, 255, 255, 255,   0,   0, 255,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0],
   [  0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=uint8)

and I have another array of (y, x) positions in that mask (first two 
values of each row):


pos =
array([[  3.,   2.,   0.,   0.],
   [  3.,   4.,   0.,   0.],
   [  5.,   2.,   0.,   0.],
   [  5.,   4.,   0.,   0.],
   [  6.,   2.,   0.,   0.],
   [  6.,   7.,   0.,   0.],
   [  0.,   0.,   0.,   0.],
   [  8.,   8.,   0.,   0.]])

and now I only want to keep all lines from 2nd array pos with those 
indices that are nonzero in the mask, i.e. line 3-6 (pos[2]-pos[5]).


F.e. line 4 in pos has the values (5, 4) and mask[5][4] is nonzero, so I 
want to keep it. While line 2 (pos[1]) has the values (4, 6) and 
mask[4][6] is zero, so shall be discarded.


I want to avoid a for loop (if possible!!!) cause I think (but don't 
know) numpy array are handled in another way. I think numpy.delete is 
the right function for discarding the values, but I don't know how to 
build the indices.




Maybe someone can help me with that or suggest which way to do this

Sebastian
--
http://mail.python.org/mailman/listinfo/python-list

Re: invoking a method from two superclasses

[Scott David Daniels]

> Mitchell L Model wrote:

Sorry, after looking over some other responses, I went back and re-read
your reply.  I'm just making sure here, but:

> Scott David Daniels wrote:
Below compressed for readability in comparison:

   class A:
   def __init__(self): super().__init__(); print('A')
   class B:
   def __init__(self): super().__init__(); print('B')
   class C(A, B):
   def __init__(self): super().__init__(); print('C')
   C()
And, if you are doing it with a message not available in object:


Renamed to disambiguate later discussion

   class root:
   def prints(self): print('root') # or pass if you prefer
   class D(root):
   def prints(self): super().prints(); print('D')
   class E(root):
   def prints(self): super().prints(); print('E')
   class F(D, E):
   def prints(self): super().prints(); print('F')
   F().prints()



What I was missing is that each path up to and including the top of the diamond
must include a definition of the method, along with super() calls to move the 
method
calling on its way up.


Actually, not really true.  In the F through root example, any of the
"prints" methods except that on root may be deleted and the whole thing
works fine.  The rootward (closer to object) end must contain the method
in question, possibly only doing a pass as the action, and _not_ calling
super.  The other methods (those in D, E, and F above are all optional
(you can freely comment out those methods where you like), but each
should call super() in their bodies.

Note that you can also add a class:
class G(E, D):
def prints(self): super().prints(); print('G')
G().prints()
Also note that the inheritances above can be "eventually inherits from"
as well as direct inheritance.


Is this what the documentation means by "cooperative multiple inheritance"?


Yes, the phrase is meant to convey "No magic (other than super itelf) is
involved in causing the other methods to be invoked.  If you want "all"
prints methods called, make sure all but the last method do super calls.
Of course, any method that doesn't do the super call will be the last by
definition (no flow control then), but by putting the non-forwarding
version at or below the lower point of the diamond, the mro order
guarantees that you will have a "good" place to stop.

Think of the mro order as a solution to the partial order constraints
that a class must appear before any of its direct superclasses, and (by
implication) after any of its subclasses.


If your correction of my example, if you remove super().__init__ from B.__init__
the results aren't affected, because object.__init__ doesn't do anything and
B comes after A in C's mro. However, if you remove super().__init__ from
A.__init__, it stops the "supering" process dead in its tracks.


Removing the super from B.__init__ means that you don't execute
object.__init__.  It turns out that object does nothing in its __init__,
but without knowing that, removing the super from B.__init__ is also a
mistake.

So, you may well already have it all right, but as long as I'm putting
in the effort to get the "operational rules about using super" out, I
thought I'd fill in this last little bit.

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Multi thread reading a file

[Scott David Daniels]

Gabriel Genellina wrote:

...
def convert(in_queue, out_queue):
  while True:
row = in_queue.get()
if row is None: break
# ... convert row
out_queue.put(converted_line)


These loops work well with the two-argument version of iter,
which is easy to forget, but quite useful to have in your bag
of tricks:

def convert(in_queue, out_queue):
for row in iter(in_queue.get, None):
# ... convert row
out_queue.put(converted_line)

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: It's ...

[J. Cliff Dyer]

On Tue, 2009-06-30 at 13:24 -0700, Beni Cherniavsky wrote:
> On Jun 24, 11:40 pm, "J. Cliff Dyer"  wrote:
> > Also note that you can iterate over a file several times:
> >
> > f = open('foo.txt')
> > for line in f:
> > print line[0]  # prints the first character of every line
> > for line in f:
> > print line[1]  #prints the second character of every line
> >
> No, you can't.  The second loop prints nothing!
> A file by default advances forward.  Once you reach the end, you stay
> there.

You are, of course, absolutely right.  Sorry for the misinformation.

:(

-- 
http://mail.python.org/mailman/listinfo/python-list

a little wsgi framework

[timmyt]

i'm interested in getting opinions on a small wsgi framework i
assembled from webob, sqlalchemy, genshi, and various code fragments i
found on the inter-tubes

here is the interesting glue - any comments / suggestions would be
much appreciated

--
the wsgi app
--
def application(environ, start_response):
path = environ.get('PATH_INFO', '').lstrip('/')

for regex, callback in urls:
match = re.search(regex, path)

if match:
environ['myapp.url_args'] = match.groups()
request = webob.Request(environ)

try:
return callback(request, start_response)
except Exception, ex:
start_response('500 Internal Server Error', [('Content-
Type', 'text/plain')])
return [traceback.format_exc()]

start_response('404 Not Found', [('Content-Type', 'text/plain')])
return ["Couldn't find the URL specified."]


--
the controller decorator
--
def web_decorator(filename, method='html'):

def decorator(target):

def wrapper(request, start_response):

#genshi TemplateLoader
template = loader.load(filename)

global config

try:
return_dict = target(request, start_response)
return_string = template.generate(**return_dict).render
(method)
config['database.Session'].commit()
except:
config['database.Session'].rollback()
raise
finally:
config['database.Session'].remove()

#TODO: alter 'Content-Type' per method being passed
start_response('200 OK', [('Content-Type', 'text/html')])
return [return_string]

return wrapper

return decorator
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Getting input the scanf way

[Bruno Desthuilliers]

Mr.SpOOn a écrit :

Hi,
I need to do some kind of interactive command line program.

I mean: I run the program, it asks me for input, I type something and
then I get the output or other questions.
I'm not sure what is the right way to achieve this.


Simplest : use raw_input()
Friendlier (at least on *n*x platforms) : use readline.

HTH
--
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[Bruno Desthuilliers]

Charles Yeomans a écrit :

Please don't top-post (not corrected)


Let me offer a bit of editing.

First, using the condition count != 3 is perhaps risky.  A mistake or a 
change in logic in the loop body might result in an infinite loop.  So 
instead I suggest


while count < 3...

Second, I'd suggest storing the value 3 in a variable with a name that 
describes it.


Pretty sensible suggestion, but then:


MaxAttempts = 3


follow conventions instead :

MAX_ATTEMPS = 3


(snip)

Finally, I'd remove correct_password_given from the loop test, and 
replace it with a break statement when the correct password is entered.


Then using a while loop and manually incrementing a counter is a waste 
of time - using a for loop would be simplier and less error-prone.



--
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[Bruno Desthuilliers]

Scott David Daniels a écrit :
(snip)


And even simpler:
PASSWORD = "qwerty"
MAXRETRY = 3
for attempt in range(MAXRETRY):
if raw_input('Enter your password: ') == PASSWORD:
print 'Password confirmed'
break # this exits the for loop
print 'Access denied: attempt %s of %s' % (attempt+1, MAXRETRY)
else:
# The else for a for statement is not executed for breaks,
# So indicates the end of testing without a match
raise SystemExit # Or whatever you'd rather do.


Which is probably the most pythonic implementation.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why re.match()?

[Duncan Booth]

kj  wrote:

> 
> 
> For a recovering Perl-head like me it is difficult to understand
> why Python's re module offers both match and search.  Why not just
> use search with a beginning-of-string anchor?  I find it particularly
> puzzling because I have this (possibly mistaken) idea that the
> Python design philosophy tends towards minimalism, a sort of Occam's
> razor, when it comes to language entities; i.e. having re.match
> along with re.search seems to me like an "unnecessary multiplication
> of entities".  What am I missing?
> 
RTFM?

http://docs.python.org/library/re.html#matching-vs-searching says:

> Note that match may differ from search even when using a regular
> expression beginning with '^': '^' matches only at the start of the
> string, or in MULTILINE mode also immediately following a newline. The
> “match” operation succeeds only if the pattern matches at the start of
> the string regardless of mode, or at the starting position given by
> the optional pos argument regardless of whether a newline precedes
> it. 

So, for example:

>>> re.compile("c").match("abcdef", 2)
<_sre.SRE_Match object at 0x02C09B90>
>>> re.compile("^c").search("abcdef", 2)
>>>

The ^ anchors you to the start of the string (or start of line in multiline 
mode) even if you specify a non-zero position. The match method just tells 
you if the pattern matches at the specified starting position regardless of 
whether it is the start of the string.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Direct interaction with subprocess - the curse of blocking I/O

[spillz]

On Jun 29, 3:15 pm, Pascal Chambon  wrote:
> Hello everyone
>
> I've had real issues with subprocesses recently : from a python script,
> on windows, I wanted to "give control" to a command line utility, i.e
> forward user in put to it and display its output on console. It seems
> simple, but I ran into walls :


If you are willing to have a wxPython dependency, wx.Execute handles
non-blocking i/o with processes on all supported platforms

more discussion of python's blocking i/o issues here:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/a037349e18f99630/60886b8beb55cfbc?q=#60886b8beb55cfbc
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: deleting certain entries in numpy array

[Ben Finney]

Sebastian Schabe  writes:

> I want to avoid a for loop (if possible!!!) cause I think (but don't
> know) numpy array are handled in another way.

Yes, Numpy arrays can be indexed logically by a boolean array.

> I think numpy.delete is the right function for discarding the values,
> but I don't know how to build the indices.

You don't need to discard the values. You can get a new array which is a
filtered version of an existing array, by using an array of bool values
as the index to the old array::

>>> import numpy
>>> mask = numpy.array([
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0, 255, 255, 255,   0,   0, 255,   0],
... [  0,   0, 255, 255, 255,   0,   0, 255,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... [  0,   0,   0,   0,   0,   0,   0,   0,   0],
... ], dtype=numpy.uint8)
>>> mask > 0
array([[False, False, False, False, False, False, False, False, False],
   [False, False, False, False, False, False, False, False, False],
   [False, False, False, False, False, False, False, False, False],
   [False, False, False, False, False, False, False, False, False],
   [False, False, False, False, False, False, False, False, False],
   [False, False,  True,  True,  True, False, False,  True, False],
   [False, False,  True,  True,  True, False, False,  True, False],
   [False, False, False, False, False, False, False, False, False],
   [False, False, False, False, False, False, False, False, False],
   [False, False, False, False, False, False, False, False, False]],
   dtype=bool)
>>> mask[mask > 0]
array([255, 255, 255, 255, 255, 255, 255, 255], dtype=uint8)

However, your case is somewhat more tricky: you need to construct the
boolean array based on coordinates from a separate array. That doesn't
require a for loop statement, but AFAICT it does require manually
generating the array of bools. I've done it with a list comprehension::

>>> pos = numpy.array([
... [  3.,   2.,   0.,   0.],
... [  3.,   4.,   0.,   0.],
... [  5.,   2.,   0.,   0.],
... [  5.,   4.,   0.,   0.],
... [  6.,   2.,   0.,   0.],
... [  6.,   7.,   0.,   0.],
... [  0.,   0.,   0.,   0.],
... [  8.,   8.,   0.,   0.],
... ])
>>> pos[numpy.array(
... [mask[int(x), int(y)] > 0 for (x, y) in pos[:, 0:2]])]
array([[ 5.,  2.,  0.,  0.],
   [ 5.,  4.,  0.,  0.],
   [ 6.,  2.,  0.,  0.],
   [ 6.,  7.,  0.,  0.]])

Here it is again, showing my working steps::

>>> pos[:, 0:2]
array([[ 3.,  2.],
   [ 3.,  4.],
   [ 5.,  2.],
   [ 5.,  4.],
   [ 6.,  2.],
   [ 6.,  7.],
   [ 0.,  0.],
   [ 8.,  8.]])
>>> [(int(x), int(y)) for (x, y) in pos[:, 0:2]]
[(3, 2), (3, 4), (5, 2), (5, 4), (6, 2), (6, 7), (0, 0), (8, 8)]
>>> [mask[int(x), int(y)] for (x, y) in pos[:, 0:2]]
[0, 0, 255, 255, 255, 255, 0, 0]
>>> [mask[int(x), int(y)] > 0 for (x, y) in pos[:, 0:2]]
[False, False, True, True, True, True, False, False]
>>> numpy.array(
... [mask[int(x), int(y)] > 0 for (x, y) in pos[:, 0:2]])
array([False, False,  True,  True,  True,  True, False, False],
dtype=bool)
>>> pos[numpy.array(
... [mask[int(x), int(y)] > 0 for (x, y) in pos[:, 0:2]])]
array([[ 5.,  2.,  0.,  0.],
   [ 5.,  4.,  0.,  0.],
   [ 6.,  2.,  0.,  0.],
   [ 6.,  7.,  0.,  0.]])

-- 
 \“Holy knit one purl two, Batman!” —Robin |
  `\   |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

problems displaying results in ibm_db

[digz]

This simple db connectivity and select test program works in the
interactive mode

>>> import ibm_db
>>> conn = ibm_db.connect( 'DATABASE', 'user', 'passwd')
>>> result = ibm_db.exec_immediate( conn, 'SELECT * FROM TEST FETCH FIRST 1 
>>> ROWS ONLY')
>>> row = ibm_db.fetch_tuple( result )
>>> for r in row:
...print r
...
13
ABC
4
2009-05-11

The same executed from a script using python testdb.py does not yield
anything , What am I doing wrong ?

import ibm_db
conn = ibm_db.connect( 'DATABASE', 'user', 'pwd')
result = ibm_db.exec_immediate( conn, 'SELECT * FROM TEST FETCH FIRST
1 ROWS ONLY')
row = ibm_db.fetch_tuple( result )
for r in row:
  print r

==
>python testdb.py
database value after unicode conversion is : N
database value sent to CLI is : N
>
==
-- 
http://mail.python.org/mailman/listinfo/python-list

problem installing python 2.6.2 from tarball

[Paul Simon]

I have just finished going through the usual ./configure, make, etc. and now 
have a load of stuff in my home directory that I think doesn't belong there. 
Apparently I did the installation from my home directory (linux) and have a 
directory  in my home directory "Python2.6.2" with subdirectories like 
"build", "Demo", "Doc", "Include", "Lib", etc.  There are many files under 
/usr/bin/local/  which appear to be duplicates.

This looks like a mess to me and would like some help to sort this out.

Paul Simon 


-- 
http://mail.python.org/mailman/listinfo/python-list

Trying to use sets for random selection, but the pop() method returns items in order

[Mario Garcia]

Im trying to use sets for doing statistics from a data set.
I want to select, 70% random records from a List. I thougth set where
a good idea so I
tested this way:

c = set(range(1000))
for d in range(1000):
 print c.pop()

I was hoping to see a print out of random selected numbers from 1 to
1000
but I got an ordered count from 1 to 1000.
I also tried using a dictionary, with keys from 1 to 10, and also got
the keys in order.

Im using:
 Python 2.5.2 |EPD 2.5.2001| (r252:60911, Aug  4 2008, 13:45:20)
 [GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin

Examples in the documentation seem to work. But I cant make it.
Can some one, give me a hint on whats going on?


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Carl Banks]

On Jul 1, 2:34 pm, Mario Garcia  wrote:
> Im trying to use sets for doing statistics from a data set.
> I want to select, 70% random records from a List. I thougth set where
> a good idea so I
> tested this way:
>
> c = set(range(1000))
> for d in range(1000):
>      print c.pop()
>
> I was hoping to see a print out of random selected numbers from 1 to
> 1000
> but I got an ordered count from 1 to 1000.
> I also tried using a dictionary, with keys from 1 to 10, and also got
> the keys in order.
>
> Im using:
>  Python 2.5.2 |EPD 2.5.2001| (r252:60911, Aug  4 2008, 13:45:20)
>  [GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
>
> Examples in the documentation seem to work. But I cant make it.
> Can some one, give me a hint on whats going on?

The keys in a dict or set are not in random order, but (more or less)
they are in hash key order modulo the size of the hash.  This neglects
the effect of hash collisions.  The hash code of an integer happens to
the integer itself, so oftentimes a dict or set storing a sequence of
integers will end up with keys in order, although it's not guaranteed
to be so.

Point it, it's unsafe to rely on *any* ordering behavior in a dict or
set, even relatively random order.

Instead, call random.shuffle() on the list, and iterate through that
to get the elements in random order.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Mensanator]

On Jul 1, 4:34 pm, Mario Garcia  wrote:
> Im trying to use sets for doing statistics from a data set.
> I want to select, 70% random records from a List. I thougth set where
> a good idea so I
> tested this way:
>
> c = set(range(1000))
> for d in range(1000):
>      print c.pop()
>
> I was hoping to see a print out of random selected numbers from 1 to
> 1000
> but I got an ordered count from 1 to 1000.
> I also tried using a dictionary, with keys from 1 to 10, and also got
> the keys in order.
>
> Im using:
>  Python 2.5.2 |EPD 2.5.2001| (r252:60911, Aug  4 2008, 13:45:20)
>  [GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
>
> Examples in the documentation seem to work. But I cant make it.
> Can some one, give me a hint on whats going on?

Sets don't help. Try this:

>>> import random

>>> c = range(100)
>>> c
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87,
88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]

>>> random.shuffle(c)
>>> c
[78, 62, 38, 54, 12, 48, 24, 20, 8, 14, 1, 69, 49, 92, 41, 64, 17, 35,
88, 40, 73, 45, 5, 84, 96, 90, 98, 57, 51, 75, 99, 13, 29, 4, 97, 77,
74, 56, 91, 95, 59, 79, 89, 19, 9, 42, 31, 85, 86, 23, 27, 50, 6, 21,
15, 80, 3, 30, 87, 82, 16, 63, 2, 55, 37, 33, 10, 61, 93, 72, 60, 67,
44, 65, 11, 70, 52, 58, 47, 18, 36, 66, 94, 28, 22, 68, 32, 76, 53,
25, 83, 34, 26, 71, 39, 43, 7, 46, 0, 81]

>>> for d in c:
print c.pop(),

81 0 46 7 43 39 71 26 34 83 25 53 76 32 68 22 28 94 66 36 18 47 58 52
70 11 65 44 67 60 72 93 61 10 33 37 55 2 63 16 82 87 30 3 80 15 21 6
50 27

Notice that the numbers are coming out in the exact reverse order
of the list, because you used .pop() without an index number.
But it's ok in _THIS_ example, because the list was randomly
shuffled before popping.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why re.match()?

[Carl Banks]

On Jul 1, 10:56 am, kj  wrote:
> For a recovering Perl-head like me it is difficult to understand
> why Python's re module offers both match and search.  Why not just
> use search with a beginning-of-string anchor?  I find it particularly
> puzzling because I have this (possibly mistaken) idea that the
> Python design philosophy tends towards minimalism, a sort of Occam's
> razor, when it comes to language entities; i.e. having re.match
> along with re.search seems to me like an "unnecessary multiplication
> of entities".  What am I missing?

It always seemed redundant to me also (notwithstanding Duncan Booth's
explanation of slight semantic differences).  However, I find myself
using re.match much more often than re.search, so perhaps in this case
a "second obvious way" is justified.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with uuid package when embedding a python interpreter

[Aahz]

In article <7d5cfbf0-38d5-4505-a93a-f321d0da7...@c36g2000yqn.googlegroups.com>,
=?ISO-8859-1?Q?J=E9r=F4me_Fuselier?=   wrote:
>
>I've tried to import a script in an embedded python intrepreter but
>this script fails when it imports the uuid module. I have a
>segmentation fault in Py_Finalize().

You may want to also ask on the capi-sig mailing list.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Mensanator]

On Jul 1, 5:29 pm, Mensanator  wrote:
> On Jul 1, 4:34 pm, Mario Garcia  wrote:
>
>
>
>
>
> > Im trying to use sets for doing statistics from a data set.
> > I want to select, 70% random records from a List. I thougth set where
> > a good idea so I
> > tested this way:
>
> > c = set(range(1000))
> > for d in range(1000):
> >      print c.pop()
>
> > I was hoping to see a print out of random selected numbers from 1 to
> > 1000
> > but I got an ordered count from 1 to 1000.
> > I also tried using a dictionary, with keys from 1 to 10, and also got
> > the keys in order.
>
> > Im using:
> >  Python 2.5.2 |EPD 2.5.2001| (r252:60911, Aug  4 2008, 13:45:20)
> >  [GCC 4.0.1 (Apple Computer, Inc. build 5370)] on darwin
>
> > Examples in the documentation seem to work. But I cant make it.
> > Can some one, give me a hint on whats going on?
>
> Sets don't help. Try this:
>
> >>> import random
> >>> c = range(100)
> >>> c
>
> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
> 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
> 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
> 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
> 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87,
> 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
>
> >>> random.shuffle(c)
> >>> c
>
> [78, 62, 38, 54, 12, 48, 24, 20, 8, 14, 1, 69, 49, 92, 41, 64, 17, 35,
> 88, 40, 73, 45, 5, 84, 96, 90, 98, 57, 51, 75, 99, 13, 29, 4, 97, 77,
> 74, 56, 91, 95, 59, 79, 89, 19, 9, 42, 31, 85, 86, 23, 27, 50, 6, 21,
> 15, 80, 3, 30, 87, 82, 16, 63, 2, 55, 37, 33, 10, 61, 93, 72, 60, 67,
> 44, 65, 11, 70, 52, 58, 47, 18, 36, 66, 94, 28, 22, 68, 32, 76, 53,
> 25, 83, 34, 26, 71, 39, 43, 7, 46, 0, 81]
>
> >>> for d in c:

Oops! You don't want to iterate through c while popping it.
Your original for statement is what to use.

>
>         print c.pop(),
>
> 81 0 46 7 43 39 71 26 34 83 25 53 76 32 68 22 28 94 66 36 18 47 58 52
> 70 11 65 44 67 60 72 93 61 10 33 37 55 2 63 16 82 87 30 3 80 15 21 6
> 50 27
>
> Notice that the numbers are coming out in the exact reverse order
> of the list, because you used .pop() without an index number.
> But it's ok in _THIS_ example, because the list was randomly
> shuffled before popping.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Direct interaction with subprocess - the curse of blocking I/O

[ryles]

On Jun 29, 5:43 pm, Scott David Daniels  wrote:
> and I personally wouldn't have it any other way.  Simulating a shell
> with hooks on its I/O should be so complicated that a "script kiddie"
> has trouble writing a Trojan.

+1 QOTW
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multi thread reading a file

[Lawrence D'Oliveiro]

In message , Mag Gam 
wrote:

> I am very new to python and I am in the process of loading a very
> large compressed csv file into another format.  I was wondering if I
> can do this in a multi thread approach.

Why bother?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Idioms and Anti-Idioms Question

[Lawrence D'Oliveiro]

In message , J. Cliff 
Dyer wrote:

> If the lines got separated, a leading + could disappear into its line
> without any errors showing up.  A trailing + would raise a syntax error.

Unless, of course, it was moved onto the previous line as part of whatever 
caused the separation of the lines. How would you guard against that?

-- 
http://mail.python.org/mailman/listinfo/python-list

getting rid of —

[someone]

Hello,

how can I replace '—' sign from string? Or do split at that character?
Getting unicode error if I try to do it:

UnicodeDecodeError: 'ascii' codec can't decode byte 0x97 in position
1: ordinal not in range(128)


Thanks, Pet

script is # -*- coding: UTF-8 -*-
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: invoking a method from two superclasses

[Michele Simionato]

On Jul 1, 2:34 am, Mitchell L Model  wrote:
> I suspect we should have a Multiple Inheritance HOWTO, though details and
> recommendations would be controversial. I've accumulated lots of abstract
> examples along the lines of my question, using multiple inheritance both to
> create combination classes (the kinds that are probably best done with
> composition instead of inheritance) and mixins. I like mixins, and I like 
> abstract
> classes. And yes I understand the horrors of working with a large component
> library that uses mixins heavily, because I've experienced many of them, going
> all the way back to Lisp-Machine Lisp's window system with very many combo
> classes such as FancyFontScrollingTitledMinimizableWindow, or whatever.
> Also, I understand that properties might be better instead of multiple 
> inheritance
> for some situations. What I'm trying to do is puzzle out what the reasonable 
> uses
> of multiple inheritance are in Python 3 and how classes and methods that 
> follow
> them should be written.

You may be interest in my "Things to know about super":

http://www.artima.com/weblogs/viewpost.jsp?thread=236275
http://www.artima.com/weblogs/viewpost.jsp?thread=236278
http://www.artima.com/weblogs/viewpost.jsp?thread=237121
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Making code run in both source tree and installation path

[Robert Kern]

On 2009-07-01 01:04, Carl Banks wrote:

On Jun 29, 9:20 am, Javier Collado  wrote:

I've seen different approaches:
- distutils trick in setup.py to modify the installed script (i.e.
changing a global variable value) so that it has a reference to the
data files location.



One of my biggest complaints about distutils is that it doesn't do
this, a limitation made worse by the fact that distutils allows you to
specify an alternate data file directory, but your scripts have no way
to know what that alternate directory is installed.  Which really
limits the usefulness of that functionality.

The most common way I've seen people work around this issue is to
throw their data files into the package directories.  Yuck.


Huh. I always found that to be a more elegant solution than hardcoding the data 
location into the program at install-time.


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

logging module and binary strings

[Frank Aune]

Hello,

 snip 
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> x='\xfe\x9f\xce\xc3\xa1\x00\xff\x01'
>>> x
'\xfe\x9f\xce\xc3\xa1\x00\xff\x01'
>>> print x
���á�
>>> logging.info(x)
Traceback (most recent call last):
  File "/usr/lib/python2.6/logging/__init__.py", line 773, in emit
stream.write(fs % msg.encode("UTF-8"))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfe in position 10: 
ordinal not in range(128)
>>> logging.info(x, extra={'encoding':'utf-8'})
Traceback (most recent call last):
  File "/usr/lib/python2.6/logging/__init__.py", line 773, in emit
stream.write(fs % msg.encode("UTF-8"))
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfe in position 10: 
unexpected code byte

 snip 

Is there any way to log the above value of x "properly" using the python 
logging module? By properly I mean logging the value displayed when entering 
just x in the python shell and pressing enter. 

Regards,
Frank


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[MRAB]

sato.ph...@gmail.com wrote:

I have been going through some Python Programming exercises while
following the MIT OpenCourseWare Intro to CS syllabus and am having
some trouble with the first "If" exercise listed on this page:

http://en.wikibooks.org/wiki/Python_Programming/Conditional_Statements#If_Exercises

I have been able to make the module quit after entering a password
three times, but can't get it to quit right away after the correct one
is entered.  I know this is really basic, please forgive me.  I have
no programming experience and have just started going through these
tutorials.

My code is here:

http://python.pastebin.com/m6036b52e


raw_input() returns a string, so you don't need "guess = str(guess)".

The 'if' line should end with a colon.

Incrementing the count should probably be after the 'if' statement.

You can leave a loop prematurely with the 'break' statement.
--
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[Tim Golden]

Lawrence D'Oliveiro wrote:

Want sorted order?

sorted(tuple(the_set))


Not sure why you want the tuple () there, but
you probably knew that...

TJG
--
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[MRAB]

Lawrence D'Oliveiro wrote:
In message , João 
Valverde wrote:



Simple example usage case: Insert string into data structure in sorted
order if it doesn't exist, else retrieve it.


the_set = set( ... )

if str in the_set :
... "retrieval" case ...
else :
the_set.add(str)
#end if

Want sorted order?

sorted(tuple(the_set))

What could be simpler?


Why not just "sorted(the_set)"?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Multi thread reading a file

[Mag Gam]

Thanks for the response Gabriel.



On Wed, Jul 1, 2009 at 12:54 AM, Gabriel
Genellina wrote:
> En Tue, 30 Jun 2009 22:52:18 -0300, Mag Gam  escribió:
>
>> I am very new to python and I am in the process of loading a very
>> large compressed csv file into another format.  I was wondering if I
>> can do this in a multi thread approach.
>
> Does the format conversion involve a significant processing time? If not,
> the total time is dominated by the I/O time (reading and writing the file)
> so it's doubtful you gain anything from multiple threads.

The format does inolve significant time processing each line.
>
>> Here is the pseudo code I was thinking about:
>>
>> Let T  = Total number of lines in a file, Example 100 (1 million
>> files)
>> Let B = Total number of lines in a buffer, for example 1 lines
>>
>>
>> Create a thread to read until buffer
>> Create another thread to read buffer+buffer  ( So we have 2 threads
>> now. But since the file is zipped I have to wait until the first
>> thread is completed. Unless someone knows of a clever technique.
>> Write the content of thread 1 into a numpy array
>> Write the content of thread 2 into a numpy array
>
> Can you process each line independently? Is the record order important? If
> not (or at least, some local dis-ordering is acceptable) you may use a few
> worker threads (doing the conversion), feed them thru a Queue object, put
> the converted lines into another Queue, and let another thread write the
> results onto the destination file.

Yes, each line can be independent. The original file is a time series
file which I am placing it into a Numpy array therefore I don't think
the record order is important.  The writing is actually done when I
place a value into a "dset" object.


Let me show you what I mean.
reader=csv.reader(open("10.csv"))
for s,row in enumerate(reader):
 if s!=0 and s%bufsize==0:
  dset[s-bufsize:s] = t  #here is where I am writing the data to
the data structure. Using a range or broadcasting.

  #15 columns
 if len(row) != 15:
   break

 t[s%bufsize] = tuple(row)

#Do this all the way at the end for flushing.
if (s%bufsize != 0):
   dset[(s//bufsize)*bufsize:s]=t[0:s%bufsize]












>
> import Queue, threading, csv
>
> def convert(in_queue, out_queue):
>  while True:
>    row = in_queue.get()
>    if row is None: break
>    # ... convert row
>    out_queue.put(converted_line)
>
> def write_output(out_queue):
>  while True:
>    line = out_queue.get()
>    if line is None: break
>    # ... write line to output file
>
> in_queue = Queue.Queue()
> out_queue = Queue.Queue()
> tlist = []
> for i in range(4):
>  t = threading.Thread(target=convert, args=(in_queue, out_queue))
>  t.start()
>  tlist.append(t)
> output_thread = threading.Thread(target=write_output, args=(out_queue,))
> output_thread.start()
>
> with open("...") as csvfile:
>  reader = csv.reader(csvfile, ...)
>  for row in reader:
>    in_queue.put(row)
>
> for t in tlist: in_queue.put(None) # indicate end-of-work
> for t in tlist: t.join() # wait until finished
> out_queue.put(None)
> output_thread.join() # wait until finished
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[Charles Yeomans]

Let me offer a bit of editing.

First, using the condition count != 3 is perhaps risky.  A mistake or  
a change in logic in the loop body might result in an infinite loop.   
So instead I suggest


while count < 3...

Second, I'd suggest storing the value 3 in a variable with a name that  
describes it.


MaxAttempts = 3
while count < MaxAttempts

This suggests that you change the name 'count' to 'attemptcount'.

The variable 'guess' is used only inside the loop, so I suggest  
removing the assignment outside the loop.


Finally, I'd remove correct_password_given from the loop test, and  
replace it with a break statement when the correct password is entered.


password = "qwerty"
correct_password_given = False
attemptcount = 0
MaxAttempts = 3
while attemptcount < MaxAttempts:
  guess = raw_input("Enter your password: ")
  guess = str(guess)
  if guess != password:
  print "Access Denied"
  attemptcount = attemptcount + 1
  else:
  print "Password Confirmed"
  correct_password_given = True
  break


Charles Yeomans

On Jul 1, 2009, at 3:58 AM, sato.ph...@gmail.com wrote:


Thank you for all of the help.  With your assistance and help from the
Python Tutor mailing list I was able to come up with the following
code:

password = "qwerty"
correct_password_given = False
guess = "0"
count = 0
while count != 3 and not correct_password_given :
  guess = raw_input("Enter your password: ")
  guess = str(guess)
  if guess != password:
  print "Access Denied"
  count = count + 1
  else:
  print "Password Confirmed"
  correct_password_given = True


If I understand it correctly, the function will continue to loop until
either the count == 3 or until correct_password_give = True,
satisfying the two conditions of the assignment, which were to lock a
user out after three failed password attempts and to print "Access
Granted" and end the module if the correct password is given.  Am I
understanding this correctly?

Thanks!

On Jul 1, 12:06 am, alex23  wrote:

On Jul 1, 3:38 pm, "sato.ph...@gmail.com" 
wrote:


I have been able to make the module quit after entering a password
three times, but can't get it to quit right away after the correct  
one

is entered.


Not with the code you pasted, you haven't. There's a missing colon on
line 7 & line 9 isn't indented properly. It's always best if we're
referring to the same source when trying to help with a problem.

The problem you're having, though, has to do with the while loop and
the fact that it's only checking one condition: has the 'count'
counter been incremented to 3. What you need to do is _also_ check  
for

the success condition:

is_confirmed = False
while count != 3 or is_confirmed:
  guess = raw_input("Enter your password: ")
  guess = str(guess)
  if guess != password:
print "Access Denied"
count = count + 1
  else:
print "Password Confirmed"
is_confirmed = True

This also provides you with a nice boolean that shows whether or not
the password was confirmed, which may be useful for latter code.

However, whenever you want to loop a set number of times, it's  
usually

better to use a 'for' loop instead:

PASSWORD = 'qwerty'
MAXRETRY = 3
for attempt in xrange(MAXRETRY):
  guess = str(raw_input('Enter your password: '))
  is_complete = guess == PASSWORD
  if is_complete:
print 'Password confirmed'
break # this exits the for loop
  else:
print 'Access denied: attempt %s of %s' % (attempt+1, MAXRETRY)

This saves you from the burden of creating, incrementing & testing
your own counter.

If there's anything here that isn't clear, please don't hesitate to
ask.


--



--
http://mail.python.org/mailman/listinfo/python-list

Re: string character count

[Jean-Michel Pichavant]

MRAB wrote:

noydb wrote:

If I have a string for a file name such that I want to find the number
of characters to the left of the dot, how can that be done?

I did it this way:
x = "text12345.txt"
dot = x.find('.')
print dot

Was curious to see what method others would use - helps me learn.  I
guess I was most curious to see if it could be done in one line.


>>> print "text12345.txt".find('.')
9


And, how would a char count be done with no dot -- like if the string
were "textstringwithoutdot" or "no dot in text string"?


If there's no dot then find() returns -1.

If you wanted to know the number of characters before the dot, if
present, or in total otherwise, then you could use split():

>>> len("text12345.txt".split(".", 1)[0])
9
>>> len("textstringwithoutdot".split(".", 1)[0])
20



You may have problems with files containing multiple dots.
the os module provide nice & safe methods to parse a file path (on any 
system, mac os windows, linux).


>>>f = "/dir1/dir2/test.log"
>>>os.path.splitext(f)
('/dir1/dir2/test', '.log')

>>>os.path.splitext(os.path.basename(f))
('test', '.log')

>>>f2 = 'test.log'
>>>os.path.splitext(os.path.basename(f2))
('test', '.log')



one safe way would be then
>>> import os
>>> x = "text12345.txt"
>>> base , ext = os.path.splitext(os.path.basename(x))
>>> print len(base)
9

Jean-Michel
--
http://mail.python.org/mailman/listinfo/python-list

What are the limitations of cStringIO.StringIO() ?

[Barak, Ron]

Hi,

I think I'm up against a limitation in cStringIO.StringIO(), but could not find 
any clues on the web, especially not in 
http://docs.python.org/library/stringio.html.

What I have is the following function:

def concatenate_files(self,filename_array):
import types

f = cStringIO.StringIO()
for filename in filename_array:
log_stream = LogStream(filename)
if not isinstance(log_stream, types.NoneType):
try:
string_ = log_stream.input_file.read()
except AttributeError, e:
sys.stderr.write("AttributeError: "+str(e)+"\n")
sys.stderr.write("log_stream: "+str(log_stream)+"\n")
else:
return(None)

f.write(string_)
return (f)

And, when the list of files - in filename_array - is pointing to long/large 
enough files, I get the exception:

Traceback (most recent call last):
  File "svm_ts_tool_in_progress.py", line 244, in OnSelectCell
self.InspectorViewController(row,col)
  File "svm_ts_tool_in_progress.py", line 1216, in InspectorViewController
self.InspectorePaneGetRecords(self.req_table, rec)
  File "svm_ts_tool_in_progress.py", line 1315, in InspectorePaneGetRecords
log_stream = 
ConcatenatedLogStream(self.events_dict[req_table]["db_dict"].keys())
  File "c:\Documents and 
Settings\rbarak\rbarak_devel\dpm16\ConcatenatedLogStream.py", line 31, in 
__init__
self.input_file = self.concatenate_files(filename_array)
  File "c:\Documents and 
Settings\rbarak\rbarak_devel\dpm16\ConcatenatedLogStream.py", line 47, in 
concatenate_files
string_ = log_stream.input_file.read()
MemoryError

 1.  Anyone knows whet's the limitation on cStringIO.StringIO() objects ?
 2.  Could you suggest a better way to create a string that contains the 
concatenation of several big files ?

Thanks,
Ron.
-- 
http://mail.python.org/mailman/listinfo/python-list

Bytes, Strings, Encoding

[Eric Pruitt]

Hello,

I am working on the subprocess.Popen module for Python 2.7 and am now moving
my changes over to Python 3.1 however I am having trouble with the whole
byte situation and I can't quite seem to understand how to go back and forth
between bytes and strings. I am also looking for the Python 3k equivalent
for the Python 2.X built-in buffer class.

One version of the file with my modifications can be found here  <
http://code.google.com/p/subprocdev/source/browse/subprocess.py?spec=svn5b570f8cbfcaae859091eb01b21b183aa5221af9&r=5b570f8cbfcaae859091eb01b21b183aa5221af9>.
Lines 1 - 15 are me attempting to get around certain changes between
Python 3.0 and Python 2.7. Further down on line 916, we have the function
"send" and "recv" in which I am having the most trouble with bytes and
strings.

Any help is appreciated. Feel free to comment on my blog
http://subdev.blogspot.com/ or reply to the last.

Thanks in advance,
Eric
-- 
http://mail.python.org/mailman/listinfo/python-list

Getting input the scanf way

[Mr.SpOOn]

Hi,
I need to do some kind of interactive command line program.

I mean: I run the program, it asks me for input, I type something and
then I get the output or other questions.
I'm not sure what is the right way to achieve this.

Thanks,
Carlo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Ubigraph vs Matplotlib (dynamic plotting, event handling)

[Che M]

On Jun 30, 6:20 pm, Ala  wrote:
> Hello everyone,
>
> I intend to use python for some network graph plotting, with event
> handling (clicking on network nodes, zooming in/out etc..) and so far I
> have come across two good candidates which are Matplotlib and Ubigraph.
>
> Did anyone have any experience with either of them for dynamic plotting
> (a slider bar on a Qt interface causing the graph to be replotted for
> each value for example), as well as event handling? (it seems on first
> notice that Ubigraph will have an upperhand on that), as well as event
> handling such as mouse clicks? (on this one Matplotlib has good
> documentation showing it does achieve that while I find ubigraph's
> documentation lacking, but I'd preffere to have the opinion of those
> who have used them before).
>
> Thank you.

I have used Matplotlib for both uses.  It has a built-in toolbar that
allows for changing the plot (zoom, pan, etc), and has good support
for point picking (clicking on a datapoint to do send an event).  You
could probably connect the QT slider to the zoom function without much
trouble, and Matplotlib is embeddable in QT GUIs.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic question from pure beginner

[Scott David Daniels]

Charles Yeomans wrote:

Let me offer a bit of editing
Finally, I'd remove correct_password_given from the loop test, and 
replace it with a break statement when the correct password is entered.


password = "qwerty"
correct_password_given = False
attemptcount = 0
MaxAttempts = 3
while attemptcount < MaxAttempts:
  guess = raw_input("Enter your password: ")
  guess = str(guess)
  if guess != password:
  print "Access Denied"
  attemptcount = attemptcount + 1
  else:
  print "Password Confirmed"
  correct_password_given = True
  break



And even simpler:
PASSWORD = "qwerty"
MAXRETRY = 3
for attempt in range(MAXRETRY):
if raw_input('Enter your password: ') == PASSWORD:
print 'Password confirmed'
break # this exits the for loop
print 'Access denied: attempt %s of %s' % (attempt+1, MAXRETRY)
else:
# The else for a for statement is not executed for breaks,
# So indicates the end of testing without a match
raise SystemExit # Or whatever you'd rather do.

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: logging module and binary strings

[Peter Otten]

Frank Aune wrote:

 import logging
 logging.basicConfig(level=logging.DEBUG)
 x='\xfe\x9f\xce\xc3\xa1\x00\xff\x01'
 x
> '\xfe\x9f\xce\xc3\xa1\x00\xff\x01'

 logging.info(x)
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/logging/__init__.py", line 773, in emit
> stream.write(fs % msg.encode("UTF-8"))
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xfe in position 10:
> ordinal not in range(128)
 logging.info(x, extra={'encoding':'utf-8'})
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/logging/__init__.py", line 773, in emit
> stream.write(fs % msg.encode("UTF-8"))
> UnicodeDecodeError: 'utf8' codec can't decode byte 0xfe in position 10:
> unexpected code byte

> Is there any way to log the above value of x "properly" using the python
> logging module? By properly I mean logging the value displayed when
> entering just x in the python shell and pressing enter.

How about logging.info(repr(x))?

Peter


-- 
http://mail.python.org/mailman/listinfo/python-list

Why re.match()?

[kj]

For a recovering Perl-head like me it is difficult to understand
why Python's re module offers both match and search.  Why not just
use search with a beginning-of-string anchor?  I find it particularly
puzzling because I have this (possibly mistaken) idea that the
Python design philosophy tends towards minimalism, a sort of Occam's
razor, when it comes to language entities; i.e. having re.match
along with re.search seems to me like an "unnecessary multiplication
of entities".  What am I missing?

TIA!

kj
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Ben Finney]

Mario Garcia  writes:

> Im trying to use sets for doing statistics from a data set.
> I want to select, 70% random records from a List. I thougth set where
> a good idea so I
> tested this way:
> 
> c = set(range(1000))
> for d in range(1000):
>  print c.pop()
> 
> I was hoping to see a print out of random selected numbers from 1 to
> 1000

The ‘set’ and ‘dict’ types are unordered collections, but that doesn't
have any implication of randomness. Rather, it means that the order of
retrieval is not guaranteed to be predictable.

You can't even predict that it'll be a random order; you can't predict
(based on the Python code) *anything* about the order. Any appearance of
predictable ordering is an unreliable artefact of the implementation and
may change at any time, since the language explicitly disclaims any
specific ordering to the items.

For a random sequence, you want the ‘random’ module
http://docs.python.org/library/random>::

>>> import random
>>> for n in (random.randint(0, 999) for _ in range(10)):
... print n
...
381
4
471
624
70
979
110
932
105
262 

-- 
 \“When in doubt tell the truth. It will confound your enemies |
  `\   and astound your friends.” —Mark Twain, _Following the Equator_ |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A question about fill_free_list(void) function

[Christian Heimes]

Pedram schrieb:
> Hello community,
> I'm reading the CPython interpreter source code,
> first, if you have something that I should know for better reading
> this source code, I would much appreciate that :)
> second, in intobject.c file, I read the following code in
> fill_free_list function that I couldn't understand:
> while (--q > p)
> Py_TYPE(q) = (struct _typeobject *)(q-1);
> Py_TYPE(q) = NULL;
> 
> What does it mean?

The code is abusing the ob_type member to store a single linked, NULL
terminated list of free integer objects. It's a tricky optimization you
don't have to understand in order to understand the rest of the code.

And no, the code in ctypes.h is totally unrelated to the free list in
intobject.c.

Christian

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python/Pygame question

[Rhodri James]

On Tue, 30 Jun 2009 19:15:24 +0100, Crip  wrote:


I have been experimenting with pygame. I would like to know how to go
about placing an image of my creation as the background of the
generated window. To where should I save the image, and what should it
be saved as?


Wherever will be most useful to you (the same directory as where you
are developing your game is often good).  You can use most of the common
formats, so pick whatever's most appropriate for your image.

The documentation is at http://www.pygame.org/docs/ref/image.html
In particular, you use pygame.image.load() to load your file into a
surface, then blit it into your display.  Some resizing may be
necessary!


--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

Open Source RSS Reader in Python?

[Alex]

I am looking for an open source RSS reader (desktop, not online)
written in Python but in vain. I am not looking for a package but a
fully functional software.

Google: python "open source" (rss OR feeds) reader

Any clue ?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Noob

[Chris Rebert]

On Tue, Jun 30, 2009 at 11:02 AM, kaffeen wrote:
> Hello all, just joined this list and am just beginning to learn Python.
> Thanks to everyone for making this a great place to learn and their
> contributions.
>
> As mentioned, I'm new to Python (but have experience with other programming
> languages/scripting). I have a couple of questions...

> 2) What are the differences between IPython and IDLE, is one better than the
> other, why?

IDLE is a GUI interactive interpreter and IDE that uses the Tk GUI toolkit.
IPython is a command-line interactive interpreter with tab-completion,
advanced command history functions, and other handy extras.

I would say they have completely different purposes. IPython is a
souped-up version of the command-line Python REPL, but doesn't include
a text editor for writing a file of code. It is very good for
experimenting and exploring with though. IDLE is a quite servicable
free IDE.

Cheers,
Chris
-- 
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.5 incompatible with Fedora Core 6 - packaging problems again

[Melissa Powers]

Hi,

 

This may be a stretch but I came across this blog while searching for a
SW Packaging Engineer.  Do you know anyone in the Boston, MA area who
may be interested in this position?  The company is a start up with a
lot of funding and this is an opportunity to define the environment and
grow with a promising company.  Please4= forward to anyone you may know
of...

 

SW Packaging Engineer

* Python developer

* Red Hat Applications expert (Fedora 10) 

* They want a real Red Hat Linux expert

* Custom Build Distribution 

* Custom RPM's 

* Some Linux Server administration duties 

* Virtualization experience a big plus

 

 

Melissa Frechette Powers

Account Manager

Corsair Solutions, Inc.

2 New Pasture Rd Unit 1

Newburyport, MA 01950

978-465-0085 x 206 Phone

978-465-0068 Fax

 

http://www.corsairsolutions.com  

Please visit our NEW website: www.corsairsolutions.com
 

 

The documents included with this electronic mail transmission contain
information from the staffing firm of Corsair Solutions, Inc which is
confidential and/or privileged. This information is intended to be for
the use of the addressee only. Note that any disclosure, printing,
photocopying, distribution or use of the contents of this e-mailed
information by persons other than the addressee or an agent of the
addressee, is unauthorized and prohibited. If you have received this
electronic mail in error, please notify us via electronic mail reply to
the sender or by telephone (collect 877-626-7724) immediately.

 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: No trees in the stdlib?

[João Valverde]

Lawrence D'Oliveiro wrote:
In message , João 
Valverde wrote:


  

Simple example usage case: Insert string into data structure in sorted
order if it doesn't exist, else retrieve it.



the_set = set( ... )

if str in the_set :
... "retrieval" case ...
else :
the_set.add(str)
#end if

Want sorted order?

sorted(tuple(the_set))

What could be simpler?

  


Try putting that inside a loop with thousands of iterations and you'll 
see what the problem is.

--
http://mail.python.org/mailman/listinfo/python-list

Re: problems displaying results in ibm_db

[ptn]

On Jul 1, 3:20 pm, digz  wrote:
> result = ibm_db.exec_immediate( conn, 'SELECT * FROM TEST FETCH FIRST
> 1 ROWS ONLY')

You have the same string split into two lines, which means that what
you actually have is this:

result = ibm_db.exec_immediate( conn, 'SELECT * FROM TEST FETCH
FIRST
1 ROWS ONLY')

You need to escape that little '\n' with a backslash:

result = ibm_db.exec_immediate( conn, 'SELECT * FROM TEST FETCH FIRST\
1 ROWS ONLY')

Better yet, use implicit string concatenation:

result = ibm_db.exec_immediate( conn, 'SELECT * FROM TEST FETCH FIRST'
'1 ROWS ONLY')

Note that now you have two strings: 'SELECT * FROM TEST FETCH FIRST'
and  '1 ROWS ONLY'


Pablo Torres N.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: problem installing python 2.6.2 from tarball

[ptn]

Sounds like you untarred directly into your home dir.  You're OK if
you delete those, they are the sources, although you should do so only
after uninstalling, just in case.

To unistall, follow this thread from the list from a while ago:
http://mail.python.org/pipermail/python-list/2004-December/296269.html

I have followed it and it worked.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Open Source RSS Reader in Python?

[Simon Forman]

On Jul 1, 7:18 pm, Alex  wrote:
> I am looking for an open source RSS reader (desktop, not online)
> written in Python but in vain. I am not looking for a package but a
> fully functional software.
>
> Google: python "open source" (rss OR feeds) reader
>
> Any clue ?

There's Canto: http://codezen.org/canto/
-- 
http://mail.python.org/mailman/listinfo/python-list

Suppressing Implicit Chained Exceptions (Python 3.0)

[andrew cooke]

I have some (library) code where an exception is caught and, since the
underlying cause is rather obscure, a different exception is raised that
more clearly explains the issue to the caller.

However, when printed via format_exc(), this new exception still has the
old exception attached via the mechanism described at
http://www.python.org/dev/peps/pep-3134/ (this is Python 3.0).

Is there any simple way to stop this?  It's rather annoying and
misleading, as it exposes a lot of internal detail that the caller does
not understand or want.  This is marked as an "open issue" in the PEP
described above.

Thanks,
Andrew



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why re.match()?

[MRAB]

Carl Banks wrote:

On Jul 1, 10:56 am, kj  wrote:

For a recovering Perl-head like me it is difficult to understand
why Python's re module offers both match and search.  Why not just
use search with a beginning-of-string anchor?  I find it particularly
puzzling because I have this (possibly mistaken) idea that the
Python design philosophy tends towards minimalism, a sort of Occam's
razor, when it comes to language entities; i.e. having re.match
along with re.search seems to me like an "unnecessary multiplication
of entities".  What am I missing?


It always seemed redundant to me also (notwithstanding Duncan Booth's
explanation of slight semantic differences).  However, I find myself
using re.match much more often than re.search, so perhaps in this case
a "second obvious way" is justified.


re.match is anchored at a certain position, whereas re.search isn't. The
re module doesn't support Perl's \G anchor.
--
http://mail.python.org/mailman/listinfo/python-list

Re: logging module and binary strings

[Robert Kern]

On 2009-07-01 10:02, Frank Aune wrote:

Hello,

 snip 

import logging
logging.basicConfig(level=logging.DEBUG)
x='\xfe\x9f\xce\xc3\xa1\x00\xff\x01'
x

'\xfe\x9f\xce\xc3\xa1\x00\xff\x01'

print x

���á�

logging.info(x)

Traceback (most recent call last):
   File "/usr/lib/python2.6/logging/__init__.py", line 773, in emit
 stream.write(fs % msg.encode("UTF-8"))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfe in position 10:
ordinal not in range(128)

logging.info(x, extra={'encoding':'utf-8'})

Traceback (most recent call last):
   File "/usr/lib/python2.6/logging/__init__.py", line 773, in emit
 stream.write(fs % msg.encode("UTF-8"))
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfe in position 10:
unexpected code byte

 snip 

Is there any way to log the above value of x "properly" using the python
logging module? By properly I mean logging the value displayed when entering
just x in the python shell and pressing enter.


logging.info(repr(x))

Or, to be more descriptive in your logging message:

logging.info("Got a binary value: %r", x)

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Timeout when connecting to sybase DBS

[skip]

Gil> There's no such group as python-sybase :-(

http://sourceforge.net/mailarchive/forum.php?forum_name=python-sybase-misc

S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: getting rid of —

[Benjamin Peterson]

someone  googlemail.com> writes:

> 
> Hello,
> 
> how can I replace '—' sign from string? Or do split at that character?
> Getting unicode error if I try to do it:
> 
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x97 in position
> 1: ordinal not in range(128)


Please paste your code. I suspect that you are mixing unicode and normal 
strings.




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Timeout when connecting to sybase DBS

[skip]

Gil> Are you saying, that when you trying to connect to a sybase DBS
Gil> server and the DBS or the server is down, you get an error after a
Gil> few seconds and not after a few minutes?

Yes, though thankfully our server tends to almost always be up.

Gil> Which python version are you using?

We're still running 2.4.5 at work and have a slightly hacked very old
version of the python-sybase package (0.36).

-- 
Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/
when i wake up with a heart rate below 40, i head right for the espresso
machine. -- chaos @ forums.usms.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: deleting certain entries in numpy array

[Robert Kern]

On 2009-07-01 09:51, Sebastian Schabe wrote:

Hello everybody,

I'm new to python and numpy and have a little/special problem:


You will want to ask numpy questions on the numpy mailing list.

  http://www.scipy.org/Mailing_Lists


I have an numpy array which is in fact a gray scale image mask, e.g.:

mask =
array([[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 255, 255, 255, 0, 0, 255, 0],
[ 0, 0, 255, 255, 255, 0, 0, 255, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)

and I have another array of (y, x) positions in that mask (first two
values of each row):

pos =
array([[ 3., 2., 0., 0.],
[ 3., 4., 0., 0.],
[ 5., 2., 0., 0.],
[ 5., 4., 0., 0.],
[ 6., 2., 0., 0.],
[ 6., 7., 0., 0.],
[ 0., 0., 0., 0.],
[ 8., 8., 0., 0.]])

and now I only want to keep all lines from 2nd array pos with those
indices that are nonzero in the mask, i.e. line 3-6 (pos[2]-pos[5]).

F.e. line 4 in pos has the values (5, 4) and mask[5][4] is nonzero, so I
want to keep it. While line 2 (pos[1]) has the values (4, 6) and
mask[4][6] is zero, so shall be discarded.

I want to avoid a for loop (if possible!!!) cause I think (but don't
know) numpy array are handled in another way. I think numpy.delete is
the right function for discarding the values, but I don't know how to
build the indices.


First, convert the pos array to integers, and just the columns with indices in 
them:

  ipos = pos[:,:2].astype(int)

Now check the values in the mask corresponding to these positions:

  mask_values = mask[ipos[:,0], ipos[:,1]]

Now extract the rows from the original pos array where mask_values is nonzero:

  result = pos[mask_values != 0]

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: getting rid of —

[MRAB]

someone wrote:

Hello,

how can I replace '—' sign from string? Or do split at that character?
Getting unicode error if I try to do it:

UnicodeDecodeError: 'ascii' codec can't decode byte 0x97 in position
1: ordinal not in range(128)


Thanks, Pet

script is # -*- coding: UTF-8 -*-


It sounds like you're mixing bytestrings with Unicode strings. I can't
be any more helpful because you haven't shown the code.
--
http://mail.python.org/mailman/listinfo/python-list

Adding an object to the global namespace through " f_globals" is that allowed ?

[Stef Mientki]

hello,

I need to add an object's name to the global namespace.
The reason for this is to create an environment,
where you can add some kind of math environment,
where no need for Python knowledge is needed.

The next statement works,
but I'm not sure if it will have any dramatical side effects,
other than overruling a possible object with the name A

def some_function ( ...) :
 A = object ( ...)
 sys._getframe(1).f_globals [ Name ] = A


thanks,
Stef Mientki
--
http://mail.python.org/mailman/listinfo/python-list

Re: problem installing python 2.6.2 from tarball

[Pablo Torres N.]

On Wed, Jul 1, 2009 at 16:16, Paul Simon wrote:
> I have just finished going through the usual ./configure, make, etc. and now
> have a load of stuff in my home directory that I think doesn't belong there.
> Apparently I did the installation from my home directory (linux) and have a
> directory  in my home directory "Python2.6.2" with subdirectories like
> "build", "Demo", "Doc", "Include", "Lib", etc.  There are many files under
> /usr/bin/local/  which appear to be duplicates.
>
> This looks like a mess to me and would like some help to sort this out.
>
> Paul Simon

Sounds like you untarred directly into your home dir.  You're OK if
you delete those, they are the sources, although you should do so only
after uninstalling, just in case.
To unistall, follow this thread from the list from a while ago:
http://mail.python.org/pipermail/python-list/2004-December/296269.html
I have followed it and it worked.

Pablo Torres N.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Getting input the scanf way

[MRAB]

Mr.SpOOn wrote:

Hi,
I need to do some kind of interactive command line program.

I mean: I run the program, it asks me for input, I type something and
then I get the output or other questions.
I'm not sure what is the right way to achieve this.


Use raw_input() and print.
--
http://mail.python.org/mailman/listinfo/python-list

Re: What are the limitations of cStringIO.StringIO() ?

[Benjamin Kaplan]

On Wed, Jul 1, 2009 at 7:48 AM, Barak, Ron wrote:
> Hi,
>
> I think I'm up against a limitation in cStringIO.StringIO(), but could not
> find any clues on the web, especially not in
> http://docs.python.org/library/stringio.html.
>
> What I have is the following function:
>
>     def concatenate_files(self,filename_array):
>     import types
>
>     f = cStringIO.StringIO()
>     for filename in filename_array:
>     log_stream = LogStream(filename)
>     if not isinstance(log_stream, types.NoneType):
>     try:
>     string_ = log_stream.input_file.read()
>     except AttributeError, e:
>     sys.stderr.write("AttributeError: "+str(e)+"\n")
>     sys.stderr.write("log_stream:
> "+str(log_stream)+"\n")
>     else:
>     return(None)
>
>     f.write(string_)
>     return (f)
>
> And, when the list of files - in filename_array - is pointing to long/large
> enough files, I get the exception:
>
> Traceback (most recent call last):
>   File "svm_ts_tool_in_progress.py", line 244, in OnSelectCell
>     self.InspectorViewController(row,col)
>   File "svm_ts_tool_in_progress.py", line 1216, in InspectorViewController
>     self.InspectorePaneGetRecords(self.req_table, rec)
>   File "svm_ts_tool_in_progress.py", line 1315, in InspectorePaneGetRecords
>     log_stream =
> ConcatenatedLogStream(self.events_dict[req_table]["db_dict"].keys())
>   File "c:\Documents and
> Settings\rbarak\rbarak_devel\dpm16\ConcatenatedLogStream.py", line 31, in
> __init__
>     self.input_file = self.concatenate_files(filename_array)
>   File "c:\Documents and
> Settings\rbarak\rbarak_devel\dpm16\ConcatenatedLogStream.py", line 47, in
> concatenate_files
>     string_ = log_stream.input_file.read()
> MemoryError
>
> Anyone knows whet's the limitation on cStringIO.StringIO() objects ?
> Could you suggest a better way to create a string that contains the
> concatenation of several big files ?
>

MemoryErrors are caused because, for whatever reason, the OS won't
allocate any more memory for the process. For a 32-bit Windows
process, that maximum is 2 GB. No matter what programming language or
library you use, you can't bypass that limit. There is a way to up the
limit to 3GB but I don't know how to do it. The link below has all the
information about Windows memory limits. If you need lots of really
big strings in memory simultaneously, you'll have to get a 64-bit
system with a ton of RAM.

http://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx

> Thanks,
> Ron.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Using Python to set desktop background image under Windows XP

[ELLINGHAUS, LANCE]

Does anyone have any code that would allow setting the desktop background image 
under Windows XP?

Thank you,
lance


Lance Ellinghaus
mailto:lance.ellingh...@hp.com
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANNC] acromania-0.4

[Lee Harr]

Acromania is a word game of acronyms.

This program is a computer moderator for networked
games of acromania. It can connect to a channel on
IRC, or start a standalone server which can be
accessed much like a MUD.

http://acromania.googlecode.com/


Acromania uses Twisted and SQLite. Optionally, it can
use NLTK to generate computer opponents.

Acromania is released under GPLv3.


Changes in acromania-0.4:
- autocreate conf.py on first start
- initialize database on first start
- add contact information
- various bug fixes


Notes:
I have only played the game using the standalone
server on a secure network. If you have experience
with programming IRC bots securely, please take
a look and contact me if you see any problems.

If you decide to connect it to IRC, please let me
know, because I'd like to play :o)


_
Drag n’ drop—Get easy photo sharing with Windows Live™ Photos.

http://www.microsoft.com/windows/windowslive/products/photos.aspx
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Paul Rubin]

Mario Garcia  writes:
> Im trying to use sets for doing statistics from a data set.
> I want to select, 70% random records from a List. I thougth set where
> a good idea so I

No that's not a good idea.  When the set/dict documentation says you
get the keys in an undetermined order, it doesn't mean the ordering is
random.  It means there is a fixed ordering controlled by the
implementation and not by you.

If you want a random sample, use random.sample().  See the docs for
the random module.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Paul Rubin]

Carl Banks  writes:
> Instead, call random.shuffle() on the list, and iterate through that
> to get the elements in random order.

It's better to use random.sample() than random.shuffle().
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why re.match()?

[John Machin]

On Jul 2, 9:50 am, MRAB  wrote:
> Carl Banks wrote:
> > On Jul 1, 10:56 am, kj  wrote:
> >> For a recovering Perl-head like me it is difficult to understand
> >> why Python's re module offers both match and search.  Why not just
> >> use search with a beginning-of-string anchor?  I find it particularly
> >> puzzling because I have this (possibly mistaken) idea that the
> >> Python design philosophy tends towards minimalism, a sort of Occam's
> >> razor, when it comes to language entities; i.e. having re.match
> >> along with re.search seems to me like an "unnecessary multiplication
> >> of entities".  What am I missing?
>
> > It always seemed redundant to me also (notwithstanding Duncan Booth's
> > explanation of slight semantic differences).  However, I find myself
> > using re.match much more often than re.search, so perhaps in this case
> > a "second obvious way" is justified.
>
> re.match is anchored at a certain position, whereas re.search isn't. The
> re module doesn't support Perl's \G anchor.

re.search is obstinately determined when it comes to flogging dead
horses:

C:\junk>\python26\python -mtimeit -s"import re" "re.match('xxx',
'y'*100)"
10 loops, best of 3: 3.37 usec per loop

C:\junk>\python26\python -mtimeit -s"import re" "re.search('^xxx',
'y'*100)"
10 loops, best of 3: 7.01 usec per loop

C:\junk>\python26\python -mtimeit -s"import re" "re.match('xxx',
'y'*1000)"
10 loops, best of 3: 3.85 usec per loop

C:\junk>\python26\python -mtimeit -s"import re" "re.search('^xxx',
'y'*1000)"
1 loops, best of 3: 37.9 usec per loop

C:\junk>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multi thread reading a file

[Mag Gam]

LOL! :-)

Why not. I think I will take just do it single thread for now and if
performance is really that bad I can re investigate.

Either way, thanks everyone for your feedback! I think I like python a
lot because of the great user community and wiliness to help!



On Wed, Jul 1, 2009 at 1:07 AM, Lawrence
D'Oliveiro wrote:
> In message , Mag Gam
> wrote:
>
>> I am very new to python and I am in the process of loading a very
>> large compressed csv file into another format.  I was wondering if I
>> can do this in a multi thread approach.
>
> Why bother?
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Steven D'Aprano]

On Wed, 01 Jul 2009 17:49:21 -0700, Paul Rubin wrote:

> Carl Banks  writes:
>> Instead, call random.shuffle() on the list, and iterate through that to
>> get the elements in random order.
> 
> It's better to use random.sample() than random.shuffle().

Why?


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Carl Banks]

On Jul 1, 5:49 pm, Paul Rubin  wrote:
> Carl Banks  writes:
> > Instead, call random.shuffle() on the list, and iterate through that
> > to get the elements in random order.
>
> It's better to use random.sample() than random.shuffle().

If you're iterating through the whole list and don't need to preserve
the original order (as was the case here) random.shuffle() is better.


Aren't-absolutist-opinions-cool?-ly yr's,

Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Paul Rubin]

Carl Banks  writes:
> If you're iterating through the whole list and don't need to preserve
> the original order (as was the case here) random.shuffle() is better.

1. Random.sample avoids iterating through the whole list when it can.

2. Say you want to choose 10 random numbers between 1 and 100.

   random.sample(xrange(100), 10)

works nicely.  Doing the same with shuffle is painful.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Paul Rubin]

Carl Banks  writes:
> random.shuffle() is still better when you're iterating through the
> whole list as the OP was doing.

The OP wrote:

I want to select, 70% random records from a List. I thougth set
where a good idea so I tested this way: ...

That sounds like 70% of the list, not the whole list.  Note that
in addition to using time proportional to the size of the entire
lsit rather than the size of the sample, shuffle() also messes up
the order of the list.  

Why would shuffle ever be better?  It is designed for a different
purpose.  Using a function called "sample" when you want to sample
should be a no-brainer, and if using shuffle instead is ever
preferable, that should be treated as a misfeature in the "sample"
implementation, and fixed.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Carl Banks]

On Jul 1, 6:37 pm, Paul Rubin  wrote:
> Carl Banks  writes:
> > If you're iterating through the whole list and don't need to preserve
> > the original order (as was the case here) random.shuffle() is better.
>
> 1. Random.sample avoids iterating through the whole list when it can.
>
> 2. Say you want to choose 10 random numbers between 1 and 100.
>
>    random.sample(xrange(100), 10)
>
> works nicely.  Doing the same with shuffle is painful.

random.shuffle() is still better when you're iterating through the
whole list as the OP was doing.

Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Trying to use sets for random selection, but the pop() method returns items in order

[Carl Banks]

On Jul 1, 7:02 pm, Paul Rubin  wrote:
> Carl Banks  writes:
> > random.shuffle() is still better when you're iterating through the
> > whole list as the OP was doing.
> The OP wrote:
>
>     I want to select, 70% random records from a List. I thougth set
>     where a good idea so I tested this way: ...

I was going by his example which went through all the items in the
list.


> That sounds like 70% of the list, not the whole list.  Note that
> in addition to using time proportional to the size of the entire
> lsit rather than the size of the sample, shuffle() also messes up
> the order of the list.  
>
> Why would shuffle ever be better?

Because we were talking about different things.


Carl Banks
-- 
http://mail.python.org/mailman/listinfo/python-list