Re: Filtering out non-readable characters

[Ross]

On 15 Jul 2005 17:33:39 -0700, "MKoool" <[EMAIL PROTECTED]> wrote:

>I have a file with binary and ascii characters in it.  I massage the
>data and convert it to a more readable format, however it still comes
>up with some binary characters mixed in.  I'd like to write something
>to just replace all non-printable characters with '' (I want to delete
>non-printable characters).
>
>I am having trouble figuring out an easy python way to do this... is
>the easiest way to just write some regular expression that does
>something like replace [^\p] with ''?
>
>Or is it better to go through every character and do ord(character),
>check the ascii values?
>
>What's the easiest way to do something like this?
>
>thanks

Easiest way is open the file with EdXor (freeware editor), select all,
Format > Wipe Non-Ascii.

Ok it's not python, but it's the easiest.
-- 
http://mail.python.org/mailman/listinfo/python-list

Cross-Platform Bonjour Module

[Ross]

Can anybody point me to a Python module for using the mDNSResponder
stuff (http://developer.apple.com/networking/bonjour)?  Thanks!

-- 
http://mail.python.org/mailman/listinfo/python-list

What language to manipulate text files

[ross]

I want to do some tricky text file manipulation on many files, but have
only a little programming knowledge.

What are the ideal languages for the following examples?

1. Starting from a certain folder, look in the subfolders for all
filenames matching *FOOD*.txt Any files matching in each folder should
be copied to a new subfolder within the current folder called EATING
with a new name of *FOOD*COPY.txt

2. Process each file as follows:
Here is a simplified example of what I want as input and output.

- input
. 'several unknown lines of text file
Get apples from apples shop
Get oranges from oranges shop
Get plums from plums shop
Get pears from pears shop
Eat from apples, oranges,
  plums, pears'whitespace at start of line is unimportant
. 'more unknown lines of text file
Chapter 1
  Several lines of text about apples in here
Chapter 2
  Several lines of text about oranges in here
Chapter 3
  Several lines of text about plums in here
Chapter 4
  Several lines of text about pears in here

- output
. 'several unknown lines of text file
Get apples from apples shop
Get oranges from oranges shop
Get plums from plums shop
Get pears from pears shop
Get bagels from bagels shop  'the Get lines...
Get donuts from donuts shop  'can be in any order
Eat from apples, bagels, oranges,
  plums, donuts, pears'whitespace at start of line is unimportant
. 'more unknown lines of text file
Chapter 1
  Several lines of text about apples in here
Chapter 2
  Several lines of text about bagels in here
Chapter 3
  Several lines of text about oranges in here
Chapter 4
  Several lines of text about plums in here
Chapter 5
  Several lines of text about donuts in here
Chapter 6
  Several lines of text about pears in here

Summary:
I have added two new items to Get;
I have put them into the comma-delimited list after searching for a
particular fruit to put each one after;
The Chapters are renumbered to match their position in the
comma-delimited list.
The "several lines of text" about each new item can be pulled from a
new_foods.txt file (or a bagels.txt and a donuts.txt file).

My first objective is to process the files as described.
My second objective is to learn the best language for this sort of text
manipulation. The language should run on Windows 98, XP and Linux.

Would Python be best, or would a macro-scripting thing like AutoHotKey
work?
I thought about Perl, but think I would learn bad habits and have hard
to read code.

Thanks, Ross

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What language to manipulate text files

[ross]

Roose wrote:
> Why do people keep asking what language to use for certain things in the
> Python newsgroup?  Obviously the answer is going to biased.
>
> Not that it's a bad thing because I love Python, but it doesn't make sense
> if you honestly want an objective opinion.
>
> R

What usenet group is it best to ask in then?
Is there one where people have good knowledge of many scripting
languages?

Ross

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What language to manipulate text files

[ross]

I tried Bash on Cygwin, but did not know enough about setting up the
environment to get it working.
Instead I got an excellent answer from alt.msdos.batch which used the
FOR IN DO command.
My next job is to learn Python.
Ross

-- 
http://mail.python.org/mailman/listinfo/python-list

Managing timing in Python calls

[Ross]

I'm porting some ugly javascript managed stuff to have an equivalent 
behaviour in a standalone app. It uses events that arrive from a server, 
and various small images.  In this standalone version, the data is local 
in a file and the images in a local directory.


My AJAX code managed a timely presentation of the info, and in the 
Javascript that relied on the ugly:


myImage.onload = function(){dosomething_when_it's_finished}

structure. Also, I used the similarly unpretty:

var t = window.setTimeout( function () { do_when_timed_out}


structures which allows stuff to happen after a perscribed period.

In my python implementation my first guess is to use a thread to load my 
image into a variable


 myImage = wx.Image("aPic.gif",
wx.BITMAP_TYPE_GIF ).ConvertToBitmap()

so that it won't block processing. (Though perhaps it'll just happen so 
fast without a server involved that I won't care.)


Is there a nice equivalent of a 'setTimeout' function in python? ie to 
call a function after some time elapses without blocking my other 
processing?  I suppose just a thread with a time.sleep(x_mS) in it would 
be my first guess?


Can anyone give me some feedback on whether that's a logical path 
forward, or if there are some nicer constructs into which I might look?


Thanks for any suggests... Ross.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Managing timing in Python calls

[Ross]

bieff...@gmail.com wrote:


Python has in its standard library a timer class which actually is
implemented as a thread (I think) ...
however, when using a GUI package, I think it is better to use gui-
specific functions for event-driven programming,
to make sure that your code do not mess with GUI event loop and to
work around the lack  of thread-safety in some GUI libraries.
This applies to timer/timeouts but also to execute code when specific
I/O events occur ( e.g. the receiving of data from a socket ).

Although I'm not an expert of  pywx, a quick search pointed me to this
page:

http://wxpython.org/onlinedocs.php

from which it seams that WxTimerEvent couldbe what you need.

I agree with you that for loading images from local files a thread
should not be needed.

P.S : notice that the documentation refers to the C++ library on which
the python wrapper is built. This is often the case for
python wrapper of GUI libraries. However, the most important ones come
with a rich set of demo programs (and pywx demo suite is quite
complete) from which one can lear what he needs.

Ciao
-
FB



The wxTimerEvent does sound attractive - I'll look into that.

Thanks too for the opinion on loading images - gives me some guts to 
just give it a try without threading it and see how it goes.


I appreciate the quick input :)

Ross.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Managing timing in Python calls

[Ross]

Interesting stuff - I hadn't come across the 'with' syntax before, so 
I've learned something already.


I was briefly excited to learn about the callLater command which is just 
a convenience class for the wxTimer class.   It seems to be pretty much 
a parallel of the

var t = window.setTimeout( function () { do_when_timed_out}

sort of thing in AJAX.

However, as is well grumbled on the 'net, you can't use wxTimer from a 
non-main thread.   So that dropped off my plate.


But getting my head around my AJAX problem versus my python 
implementation, I realized my use of those javascript structures were 
really just used because javascript doesn't allow any threading at all.


With Python, just having my other processing path in a thread is enough, 
and I can use the brutish time.sleep() function, without worrying about 
blocking the processing of my mainline (UI) thread.  So I'm able to proceed.


I do want to know more about the 'with' command tho' so I'll look into that.

Thx again.

Ross.



cmdrrickhun...@yaho.com wrote:

I believe WxTimerEvent is handled using the event queue, which isn't
going to do what you want.  An event which goes through the queue does
not get processed until you return to the queue.

What you want to do is actually a rather difficult task to do
generically.  Should the task be interrupted immediately?  Or is a
tiny latency acceptable?  Should the function being terminated get to
handle its own termination?  Or should the termination be forced on
it.  What sort of overhead is acceptable for this "set_timeout"
behavior?

I would not be surprised if there isn't a built in solution, because
its so hard, but rather built in tools which can be used to do it.

If your timeouts are on the order of seconds, you might be able to
just check time.time() at the begining, and compare it to the current
time later in the function.  This could be on the main thread or on a
worker thread.

If you need better handling, you may want to look at how condition
variables and such work.

Finally, thread has a function to send a Keyboard Interrupt to the
main thread.  I believe you could do your work on the main thread, and
catch the interrupt.

"Background" tasks are not easy to implement in any language (other
than perhaps AJAX ;-) ).

Remember, Python does not support truly simultaneous threads.  It
actually does timeslices of about 100 operations.  Any solution you
choose should work given this information.

And as for a "nicer" construct, I personally just learned of how to
handle the "with" command.  I could see something like

class Timeout:
def __init__(self, t):
self.t = t
def __enter__(self):
self.start = time.time()
def __exit__(self, x, y, z):
return None
def __nonzero__(self):
return time.time() - self.start <= self.t


def doSomethingLong(timeout = True): # true guarentees bailout never
occurs
   while timeout:
   doAnIteration()

with Timeout(3) as t:
doSomethingLong(t)



and have your Timeout class have a flag which it sets when
doSomethingLong needs to bail out, using whatever method is "best" for
your particular application.  This is, of course pseudocode - I've not
run it through python msyself.  Hopefully any errors are obvious
enough that you can work around them.

--
http://mail.python.org/mailman/listinfo/python-list

Custom C Exception Subclasses

[Ross]

For a project that I am doing, it would be useful to have an exception
class that stores some additional data along with the message.
However, I want to be able to store a couple pointers to C++ classes,
so I can't just use an exception created with PyExc_NewException.  If
I were to subclass the built-in Exception type, I would need to have
access to the PyExc_ExceptionObject, but the headers only give
PyExc_Exception, the type object.  This seems like a rather
straightforward task, but I can't seem to find any documentation for
it.  Does anyone know how to do this?  Thanks!
--
http://mail.python.org/mailman/listinfo/python-list

Re: Custom C Exception Subclasses

[Ross]

On Dec 24, 9:24 am, "Gabriel Genellina" 
wrote:
> In fact you can, you could store those pointers as attributes of the
> exception object, using a PyCObject.

Excellent.  I was not aware of the PyCObject type.

> Accessing those attributes isn't as easy as doing exc->field, but I think
> it's easy enough. Inheriting from some exception type requires you to
> define the type structure and fill it right, and IMHO is a lot harder.
>
> Perhaps there is a misunderstanding here. To subclass a type, you need the
> type, not an instance of such type.

Ah yes, I probably should have been more clear.  In the docs about
subclassing, they use a PyListObject as the first field of the Shoddy
struct so that the fields are filled in correctly.

Now, how would I go about adding methods to a custom exception object?
--
http://mail.python.org/mailman/listinfo/python-list

get method

[Ross]

I am teaching myself Python by going through Allen Downing's "Think
Python." I have come across what should be a simple exercise, but I am
not getting the correct answer. Here's the exercise:

Given:

def histogram(s):
d = dict()
for c in s:
if c not in d:
d[c] = 1
else:
d[c] += 1
return d


Dictionaries have a method called get that takes a key and a default
value. If the key appears in the dictionary, get returns the
corresponding value; otherwise it returns the default value. For
example:

>>> h = histogram('a')
>>> print h
{'a': 1}
>>> h.get('a', 0)
1
>>> h.get('b', 0)
0

Use get to write histogram more concisely. You should be able to
eliminate the if statement.

Here's my code:

def histogram(s):
d = dict()
for c in s:
d[c]= d.get(c,0)
return d

This code returns a dictionary of all the letters to any string s I
give it but each corresponding value is incorrectly the default of 0.
What am I doing wrong?

--
http://mail.python.org/mailman/listinfo/python-list

Re: get method

[Ross]

On Dec 29, 8:07 pm, Scott David Daniels  wrote:
> Ross wrote:
> > ... Use get to write histogram more concisely. You should be able to
> > eliminate the if statement.
>
> > def histogram(s):
> >    d = dict()
> >    for c in s:
> >            d[c]= d.get(c,0)
> >    return d
>
> > This code returns a dictionary of all the letters to any string s I
> > give it but each corresponding value is incorrectly the default of 0.
> > What am I doing wrong?
>
> How is this code supposed to count?
>
> --Scott David Daniels
> scott.dani...@acm.org

I realize the code isn't counting, but how am I to do this without
using an if statement as the problem instructs?
--
http://mail.python.org/mailman/listinfo/python-list

formatted 'time' data in calculations

[Ross]

There seems to be no shortage of information around on how to use the 
time module, for example to use time.ctime() and push it into strftime 
and get something nice out the other side, but I haven't found anything 
helpful in going the other way.


That is, given some formatted text describing times - is there something 
that makes it easy to calculate time differences, or do I have to index 
my way through the string pulling out characters, converting to integers 
etc...


Data is formatted:

   t1 = 09:12:10
   t2 = 11:22:14

I want to calculate tdiff = t2-t1

Any suggestions? (Thanks for anything you can offer)

-Ross
--
http://mail.python.org/mailman/listinfo/python-list

Re: formatted 'time' data in calculations

[Ross]

Thanks Chris and Diez for the quick pointers... Very helpful

Ross.
--
http://mail.python.org/mailman/listinfo/python-list

Re: formatted 'time' data in calculations

[Ross]

Scott David Daniels wrote:

Ross wrote:
There seems to be no shortage of information around on how to use the 
time module, for example to use time.ctime() and push it into strftime 
and get something nice out the other side, but I haven't found 
anything helpful in going the other way.


As to a paucity of conversion formatting, there is no magic way to take
everyone's way of putting date and time information in text and convert
it to unambiguous format, in part because there are too many different
and contradictory formats.  When I write dates, I know what I intended;
when I read dates, I guess what the author intended.

Have you read the entire time module document?  If so, which functions
in that module take strings as arguments?

That is, given some formatted text describing times - is there 
something that makes it easy to calculate time differences, or do I 
have to index my way through the string pulling out characters, 
converting to integers etc...


Data is formatted:
   t1 = 09:12:10
   t2 = 11:22:14
I want to calculate tdiff = t2-t1


Do you do any work yourself?  Show us your attempts.  This looks like
a trivial exercise.  It seems that for less than four times the effort
of asking your question you might have found the answer.

Perhaps I am being too cranky this morning.
--Scott David Daniels
scott.dani...@acm.org



Jeeze, you're quite an ass aren't you?
--
http://mail.python.org/mailman/listinfo/python-list

Re: formatted 'time' data in calculations

[Ross]

Thanks Chris and Diez for the quick pointers... Very helpful

Ross.
--
http://mail.python.org/mailman/listinfo/python-list

league problem in python

[Ross]

I'm new to programming and have chosen Python as my first language.
I've gone through Allen Downey's Think Python book and I think I'm
ready to dive into a project. The first problem I've chosen to tackle
is a problem I have seen at my tennis club. Each spring/fall, the pro
puts out a sheet of paper for people to sign up for tennis leagues.
Depending on how many people sign up for a league, he'll assign a
certain number of courts each week to that league.

 After this, he makes up a schedule of who plays who on each week and
who has a bye. Unfortunately, he does this by hand and a lot of times,
some people will play certain people more than once and certain other
people never. Other problems that arise: some people have more bye
weeks than others, some people have all their bye weeks clumped
together so that they don't play for weeks on end.

I would like to create a simple program where the pro could enter in
how many people were in the league, the number of courts available,
and the number of weeks the schedule would run and then generate a
schedule where everybody played everybody else once and got the same
number of bye weeks, preferably spaced out evenly.

How should I go about starting this problem...I'm feel like this is a
really simple problem, but I'm having writer's/coder's block. Can you
guys help?
--
http://mail.python.org/mailman/listinfo/python-list

possible pairings in a set

[Ross]

I'm new to python and I'm trying to come up with a function that takes
a given number of players in a game and returns all possible unique
pairings. Here's the code I've come up with so far, but I'm not
getting the output I'd like to:

def all_pairings(players):
cleanlist = []
for i in range(players):
cleanlist.append(i)
return cleanlist
start = 0
follow = start +1
finallist = []
while follow <= len(cleanlist)-1:
for player in cleanlist:
mini = cleanlist[start],cleanlist[follow]
finallist.append(mini)
follow +=1
start+=1
return finallist

If I were to execute the function with all_pairings(4), I want to get
the output [[0,1],[0,2],[0,3],[1,2],[1,3],[2,3]. Instead, I get
[0,1,2,3] with the code I currently have. Can you guys help me out?
Also, if my code is considered ugly or redundant by this community,
can you make suggestions to clean it up?
--
http://mail.python.org/mailman/listinfo/python-list

sharing/swapping items between lists

[Ross]

I'm trying to design an iterator that produces two lists. The first
list will be a list of unique pairings and the second will be a list
of items that weren't used in the first list. After each round, the
items that weren't used in the round before will get put back in and
the second list will be populated with unique items. To clarify,
here's an example of what my output would be if I had 8 items:

First Iteration

LIST 1LEFTOVERS LIST

[(1,2),(3,4),(5,6)]   [7,8]

Second Iteration

[(1,3), (2,7),(4,8)] [5,6]

Third Iteration

[(1,5), (2,6), (7,8)][3,4]

etc

Additionally, I want the items in the "leftovers" list to be used the
same amount.

Can you guys suggest an approach to this problem...I'm trying to teach
myself python so an outline of how to approach this would probably be
more helpful to me than an explicit solution. I'll cry mercy if I
can't figure it out after your hints.
--
http://mail.python.org/mailman/listinfo/python-list

Re: sharing/swapping items between lists

[Ross]

On Apr 11, 1:10 pm, a...@pythoncraft.com (Aahz) wrote:
> In article 
> <4fd78ac3-ba83-456b-b768-3a0043548...@f19g2000vbf.googlegroups.com>,
>
> Ross   wrote:
>
> >I'm trying to design an iterator that produces two lists. The first
> >list will be a list of unique pairings and the second will be a list
> >of items that weren't used in the first list. After each round, the
> >items that weren't used in the round before will get put back in and
> >the second list will be populated with unique items.
>
> How do you specify what goes into the first list?  Based on your
> description, I would have expected that the output from the first
> iteration would be
>
> ( [(1,2),(3,4),(5,6)], [7,8] )
>
> Regardless of the actual algorithm, if you are returning items one at a
> time and maintaining state in a computation, you probably want to use a
> generator.
> --
> Aahz (a...@pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> Why is this newsgroup different from all other newsgroups?

I'm sorry...my example was probably a bad one. A better example of
output I would like would be something like [[1,2],[3,4],[5,6]] and
then for the leftovers list [7,8,9,10 etc]. What I'm trying to do is
produce some sort of round robin algorithm for tennis that is
constrained by the number of courts available each week. So if there
are only 3 courts available for a singles league and 10 people have
signed up, 4 players will have a bye each week. I want my algorithm to
produce unique matchups each week and also give each player the same
angle?
--
http://mail.python.org/mailman/listinfo/python-list

Re: sharing/swapping items between lists

[Ross]

On Apr 13, 9:08 am, a...@pythoncraft.com (Aahz) wrote:
> In article ,
>
> Ross   wrote:
>
> >I'm sorry...my example was probably a bad one. A better example of
> >output I would like would be something like [[1,2],[3,4],[5,6]] and
> >then for the leftovers list [7,8,9,10 etc]. What I'm trying to do is
> >produce some sort of round robin algorithm for tennis that is
> >constrained by the number of courts available each week. So if there
> >are only 3 courts available for a singles league and 10 people have
> >signed up, 4 players will have a bye each week. I want my algorithm to
> >produce unique matchups each week and also give each player the same
> >angle?
>
> How about Googling for "round robin algorithm python"?  ;-)
> --
> Aahz (a...@pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> Why is this newsgroup different from all other newsgroups?

I have the basic algorithm and it works fine...I'm just having trouble
adding another parameter to it that allows for court constraints and
bye weeks.
--
http://mail.python.org/mailman/listinfo/python-list

Re: sharing/swapping items between lists

[Ross]

On Apr 14, 5:57 am, a...@pythoncraft.com (Aahz) wrote:
> In article ,
>
>
>
> Ross   wrote:
> >On Apr 13, 9:08=A0am, a...@pythoncraft.com (Aahz) wrote:
> >> In article  >com>,
> >> Ross =A0 wrote:
>
> >>>I'm sorry...my example was probably a bad one. A better example of
> >>>output I would like would be something like [[1,2],[3,4],[5,6]] and
> >>>then for the leftovers list [7,8,9,10 etc]. What I'm trying to do is
> >>>produce some sort of round robin algorithm for tennis that is
> >>>constrained by the number of courts available each week. So if there
> >>>are only 3 courts available for a singles league and 10 people have
> >>>signed up, 4 players will have a bye each week. I want my algorithm to
> >>>produce unique matchups each week and also give each player the same
> >>>angle?
>
> >> How about Googling for "round robin algorithm python"? ;-)
>
> >I have the basic algorithm and it works fine...I'm just having trouble
> >adding another parameter to it that allows for court constraints and
> >bye weeks.
>
> You'll need to give us more information, then.  Why don't you start with
> the core algorithm you're using?
> --
> Aahz (a...@pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> Why is this newsgroup different from all other newsgroups?

Here's the core algorithm I'm using:

>>> def round_robin(teams,rounds):
if len(teams)%2:
teams.append(None)
mid = len(teams) //2
for i in range(rounds):
yield zip(teams[:mid], teams[mid:])
teams = teams[0:1] + teams[mid:mid+1] + teams[1:mid-1]+teams[mid
+1:]+teams[mid-1:mid]


>>> if __name__== '__main__':
rounds = 15
teams = range(16)
for round in round_robin(teams,rounds):
print round
--
http://mail.python.org/mailman/listinfo/python-list

Re: sharing/swapping items between lists

[Ross]

On Apr 14, 10:34 am, Ross  wrote:
> On Apr 14, 5:57 am, a...@pythoncraft.com (Aahz) wrote:
>
>
>
> > In article 
> > ,
>
> > Ross   wrote:
> > >On Apr 13, 9:08=A0am, a...@pythoncraft.com (Aahz) wrote:
> > >> In article 
> > >>  > >com>,
> > >> Ross =A0 wrote:
>
> > >>>I'm sorry...my example was probably a bad one. A better example of
> > >>>output I would like would be something like [[1,2],[3,4],[5,6]] and
> > >>>then for the leftovers list [7,8,9,10 etc]. What I'm trying to do is
> > >>>produce some sort of round robin algorithm for tennis that is
> > >>>constrained by the number of courts available each week. So if there
> > >>>are only 3 courts available for a singles league and 10 people have
> > >>>signed up, 4 players will have a bye each week. I want my algorithm to
> > >>>produce unique matchups each week and also give each player the same
> > >>>angle?
>
> > >> How about Googling for "round robin algorithm python"? ;-)
>
> > >I have the basic algorithm and it works fine...I'm just having trouble
> > >adding another parameter to it that allows for court constraints and
> > >bye weeks.
>
> > You'll need to give us more information, then.  Why don't you start with
> > the core algorithm you're using?
> > --
> > Aahz (a...@pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> > Why is this newsgroup different from all other newsgroups?
>
> Here's the core algorithm I'm using:
>
> >>> def round_robin(teams,rounds):
>
>         if len(teams)%2:
>                 teams.append(None)
>         mid = len(teams) //2
>         for i in range(rounds):
>                 yield zip(teams[:mid], teams[mid:])
>                 teams = teams[0:1] + teams[mid:mid+1] + 
> teams[1:mid-1]+teams[mid
> +1:]+teams[mid-1:mid]
>
> >>> if __name__== '__main__':
>
>         rounds = 15
>         teams = range(16)
>         for round in round_robin(teams,rounds):
>                 print round

fyi rounds=15 and teams =range(16) was just test code I was playing
around with...they could theoretically be anything.
--
http://mail.python.org/mailman/listinfo/python-list

Re: sharing/swapping items between lists

[Ross]

On Apr 14, 7:18 pm, Aaron Brady  wrote:
> On Apr 14, 7:01 pm, Aaron Brady  wrote:
>
>
>
> > On Apr 14, 12:37 pm, Ross  wrote:
>
> > > On Apr 14, 10:34 am, Ross  wrote:
>
> > > > On Apr 14, 5:57 am, a...@pythoncraft.com (Aahz) wrote:
>
> > > > > In article 
> > > > > ,
>
> > > > > Ross   wrote:
> > > > > >On Apr 13, 9:08=A0am, a...@pythoncraft.com (Aahz) wrote:
> > > > > >> In article 
> > > > > >>  > > > > >com>,
> > > > > >> Ross =A0 wrote:
>
> > > > > >>>I'm sorry...my example was probably a bad one. A better example of
> > > > > >>>output I would like would be something like [[1,2],[3,4],[5,6]] and
> > > > > >>>then for the leftovers list [7,8,9,10 etc]. What I'm trying to do 
> > > > > >>>is
> > > > > >>>produce some sort of round robin algorithm for tennis that is
> > > > > >>>constrained by the number of courts available each week. So if 
> > > > > >>>there
> > > > > >>>are only 3 courts available for a singles league and 10 people have
> > > > > >>>signed up, 4 players will have a bye each week. I want my 
> > > > > >>>algorithm to
> > > > > >>>produce unique matchups each week and also give each player the 
> > > > > >>>same
> > > > > >>>angle?
>
> > > > > >> How about Googling for "round robin algorithm python"? ;-)
>
> > > > > >I have the basic algorithm and it works fine...I'm just having 
> > > > > >trouble
> > > > > >adding another parameter to it that allows for court constraints and
> > > > > >bye weeks.
>
> > > > > You'll need to give us more information, then.  Why don't you start 
> > > > > with
> > > > > the core algorithm you're using?
> > > > > --
> > > > > Aahz (a...@pythoncraft.com)           <*>        
> > > > > http://www.pythoncraft.com/
>
> > > > > Why is this newsgroup different from all other newsgroups?
>
> > > > Here's the core algorithm I'm using:
>
> > > > >>> def round_robin(teams,rounds):
>
> > > >         if len(teams)%2:
> > > >                 teams.append(None)
> > > >         mid = len(teams) //2
> > > >         for i in range(rounds):
> > > >                 yield zip(teams[:mid], teams[mid:])
> > > >                 teams = teams[0:1] + teams[mid:mid+1] + 
> > > > teams[1:mid-1]+teams[mid
> > > > +1:]+teams[mid-1:mid]
>
> > > > >>> if __name__== '__main__':
>
> > > >         rounds = 15
> > > >         teams = range(16)
> > > >         for round in round_robin(teams,rounds):
> > > >                 print round
>
> > > fyi rounds=15 and teams =range(16) was just test code I was playing
> > > around with...they could theoretically be anything.
>
> > Here is an idea.  Create a list of all possible pairs, using
> > itertools.combinations.  You'll notice everyone gets equal play time
> > and equal time against each other on a pair-by-pair basis.  Then, call
> > random.shuffle until one player isn't playing on two courts in one
> > day.
>
> This might take a long time.  Not that I can guarantee that a depth-
> first-search would be any faster, or that a breadth-first-search would
> run faster *and* run in available memory.  

I have a sub-optimal solution that I'm playing with now. Since my
output is a list of tuples and looks something like this (if there
were 16 teams and 15 rounds), I could designate a each nth tuple in
each round as a bye, but since the 1st item in my list remains fixed,
it's suboptimal. For example, you could say every 4th (and/or 3rd ,
5th, etc depending on how many available cts) tuple in each round gets
a bye and pop() it from the list...:

[(0, 8), (1, 9), (2, 10), (3, 11), (4, 12), (5, 13), (6, 14), (7, 15)]
[(0, 9), (8, 10), (1, 11), (2, 12), (3, 13), (4, 14), (5, 15), (6, 7)]
[(0, 10), (9, 11), (8, 12), (1, 13), (2, 14), (3, 15), (4, 7), (5, 6)]
[(0, 11), (10, 12), (9, 13), (8, 14), (1, 15), (2, 7), (3, 6), (4, 5)]
[(0, 12), (11, 13), (10, 14), (9, 15), (8, 7), (1, 6), (2, 5), (3, 4)]
[(0, 13), (12, 14), (11, 15), (10, 7), (9, 6), (8, 5), (1, 4), (2, 3)]
[(0, 14), (13, 15), (12, 7), (11, 6), (10, 5), (9, 4), (8, 3), (1, 2)]
[(0, 15), (14, 7), (13, 6), (12, 5), (11, 4), (10, 3), (9, 2), (8, 1)]
[(0, 7), (15, 6), (14, 5), (13, 4), (12, 3), (11, 2), (10, 1), (9, 8)]
[(0, 6), (7, 5), (15, 4), (14, 3), (13, 2), (12, 1), (11, 8), (10, 9)]
[(0, 5), (6, 4), (7, 3), (15, 2), (14, 1), (13, 8), (12, 9), (11, 10)]
[(0, 4), (5, 3), (6, 2), (7, 1), (15, 8), (14, 9), (13, 10), (12, 11)]
[(0, 3), (4, 2), (5, 1), (6, 8), (7, 9), (15, 10), (14, 11), (13, 12)]
[(0, 2), (3, 1), (4, 8), (5, 9), (6, 10), (7, 11), (15, 12), (14, 13)]
[(0, 1), (2, 8), (3, 9), (4, 10), (5, 11), (6, 12), (7, 13), (15, 14)]
--
http://mail.python.org/mailman/listinfo/python-list

Removing items from a list simultaneously

[Ross]

Is there a quick way to simultaneously pop multiple items from a list?
For instance if i had the list a = [1,2,3,4,5,6,7] and I wanted to
return every odd index into a new list, my output would be new_list =
[2,4,6] or similarly if I wanted to return each index that was one
away from the midpoint in a, I would get [3,5].
--
http://mail.python.org/mailman/listinfo/python-list

complementary lists?

[Ross]

If I have a list x = [1,2,3,4,5,6,7,8,9] and another list that is a
subset of x:  y = [1,4,7] , is there a quick way that I could return
the complementary subset to y z=[2,3,5,6,8,9] ?

The reason I ask is because I have a generator function that generates
a list of tuples and I would like to divide this list into
complementary lists.
--
http://mail.python.org/mailman/listinfo/python-list

list comprehension question

[Ross]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?
--
http://mail.python.org/mailman/listinfo/python-list

yet another list comprehension question

[Ross]

I'm trying to set up a simple filter using a list comprehension. If I
have a list of tuples, a = [(1,2), (3,4), (5,None), (6,7), (8, None)]
and I wanted to filter out all tuples containing None, I would like to
get the new list b = [(1,2), (3,4),(6,7)].

I tried b = [i for i in a if t for t in i is not None]   but I get the
error that "t is not defined". What am I doing wrong?
--
http://mail.python.org/mailman/listinfo/python-list

Re: yet another list comprehension question

[Ross]

On May 2, 7:21 pm, Chris Rebert  wrote:
> On Sat, May 2, 2009 at 7:13 PM, Ross  wrote:
> > I'm trying to set up a simple filter using a list comprehension. If I
> > have a list of tuples, a = [(1,2), (3,4), (5,None), (6,7), (8, None)]
> > and I wanted to filter out all tuples containing None, I would like to
> > get the new list b = [(1,2), (3,4),(6,7)].
>
> b = [tup for tup in a if None not in tup]
>
> Cheers,
> Chris
> --http://blog.rebertia.com

Thanks I feel retarded sometimes.
--
http://mail.python.org/mailman/listinfo/python-list

Code works fine except...

[Ross]

For the past couple weeks, I've been working on an algorithm to
schedule tennis leagues given court constraints and league
considerations (i.e. whether it's a singles or a doubles league). Here
were my requirements when I was designing this algorithm:

-Each player plays against a unique opponent each week.
-Similarly, in a doubles league, each player plays with a unique
partner each week.
-Each player gets a fair number of bye weeks (i.e. the player with the
most bye weeks will have no more than one bye week than the player
with the least number of bye weeks)

I'm very close to arriving at my desired solution, but I have one
glaring flaw. When I have an even number of players sign up for my
league and there are court constraints, my current algorithm gives the
first player in my league a bye week every single week. I'll post my
code below and see how you guys think I should add to/ amend my code.

def round_robin(players, rounds):
if len(players)%2:
players.insert(0, None)
mid = len(players)//2
for i in range(rounds):
yield zip(players[:mid], players[mid:])
players = players[0:1] + players[mid:mid+1] + players[1:mid-1] +
players[mid+1:] + players[mid-1:mid]


def test_round_robin(players, rounds, courts, doubles = False):
players = range(players)
for week in round_robin(players,rounds,courts):
if doubles == True:
doubles_week = len(week)/2.0
byes = doubles_week - courts
if byes == 0:
bye_list = []
else:
bye_list = 
week[::int(round(1.072*(courts/byes)+1.08))]
playing = [u for u in week if u not in bye_list]
midd = len(playing)//2
doub_sched = zip(playing[:midd], playing[midd:])
print doub_sched, bye_list
else:
byes = len(week)- courts
if byes == 0:
bye_list = []
else:
bye_list = 
week[::int(round(1.072*(courts/byes)+1.08))]
playing = [u for u in week if u not in bye_list]
print playing, bye_list
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 3, 10:16 pm, John Yeung  wrote:
> On May 3, 11:29 pm, Chris Rebert  wrote:
>
> > Probably not the cause of the problem, but where
> > did the magic numbers 1.072 and 1.08 come from?
>
> It is perhaps not the most direct cause of the problem, in the sense
> that the magic numbers could take various values and the problem would
> still be there.  But the magic numbers appear to be used for
> "spreading out" bye selection, and that's broken.
>
> The extended slice as written will always pick the first element,
> since the step is guaranteed to be positive.  Since the first player
> (or None, when there are an odd number of players) stays put in the
> first position during the round_robin shuffle, that player will always
> be selected for a bye.
>
> Further, as written, the calculated number of byes has no bearing on
> the actual number of byes selected.
>
> I think what I would do is adjust the shuffling algorithm in such a
> way that everyone moves through the various positions in the list
> (would it be as simple as adding a shift at the end of
> round_robin???).  Then I could simply select the byes from one end of
> the list (with a regular slice instead of an extended slice).
>
> John

The "magic numbers" that everyone is wondering about are indeed used
for spreading out the bye selection and I got them by simply
calculating a line of best fit when plotting several courts: byes
ratios.

The "byes = #whatever" in my code calculate the number of tuples that
need to be placed in the bye_list.

At first glance, the suggestion of adding a simple shift at the end of
round_robin doesn't seem to work since round_robin already shifts
everything except for the first position already. Please correct me if
I'm wrong because you might have been suggesting something different.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 4, 7:01 am, Ross  wrote:
> On May 3, 10:16 pm, John Yeung  wrote:
>
>
>
> > On May 3, 11:29 pm, Chris Rebert  wrote:
>
> > > Probably not the cause of the problem, but where
> > > did the magic numbers 1.072 and 1.08 come from?
>
> > It is perhaps not the most direct cause of the problem, in the sense
> > that the magic numbers could take various values and the problem would
> > still be there.  But the magic numbers appear to be used for
> > "spreading out" bye selection, and that's broken.
>
> > The extended slice as written will always pick the first element,
> > since the step is guaranteed to be positive.  Since the first player
> > (or None, when there are an odd number of players) stays put in the
> > first position during the round_robin shuffle, that player will always
> > be selected for a bye.
>
> > Further, as written, the calculated number of byes has no bearing on
> > the actual number of byes selected.
>
> > I think what I would do is adjust the shuffling algorithm in such a
> > way that everyone moves through the various positions in the list
> > (would it be as simple as adding a shift at the end of
> > round_robin???).  Then I could simply select the byes from one end of
> > the list (with a regular slice instead of an extended slice).
>
> > John
>
> The "magic numbers" that everyone is wondering about are indeed used
> for spreading out the bye selection and I got them by simply
> calculating a line of best fit when plotting several courts: byes
> ratios.
>
> The "byes = #whatever" in my code calculate the number of tuples that
> need to be placed in the bye_list.
>
> At first glance, the suggestion of adding a simple shift at the end of
> round_robin doesn't seem to work since round_robin already shifts
> everything except for the first position already. Please correct me if
> I'm wrong because you might have been suggesting something different.

And also, as you all have pointed out, my inclusion of

> def test_round_robin(players, rounds, courts, doubles = False):
> players = range(players)
> for week in round_robin(players,rounds,courts):

should have been

> def test_round_robin(players, rounds, courts, doubles = False):
> players = range(players)
> for week in round_robin(players,rounds):

I forgot to erase that extra parameter when I was playing around with
my code.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 3, 8:29 pm, John Machin  wrote:
> On May 4, 12:36 pm, Ross  wrote:
>
>
>
> > For the past couple weeks, I've been working on an algorithm to
> > schedule tennis leagues given court constraints and league
> > considerations (i.e. whether it's a singles or a doubles league). Here
> > were my requirements when I was designing this algorithm:
>
> > -Each player plays against a unique opponent each week.
> > -Similarly, in a doubles league, each player plays with a unique
> > partner each week.
> > -Each player gets a fair number of bye weeks (i.e. the player with the
> > most bye weeks will have no more than one bye week than the player
> > with the least number of bye weeks)
>
> > I'm very close to arriving at my desired solution, but I have one
> > glaring flaw. When I have an even number of players sign up for my
> > league and there are court constraints, my current algorithm gives the
> > first player in my league a bye week every single week. I'll post my
> > code below and see how you guys think I should add to/ amend my code.
>
> > def round_robin(players, rounds):
> >     if len(players)%2:
> >         players.insert(0, None)
> >     mid = len(players)//2
> >     for i in range(rounds):
> >         yield zip(players[:mid], players[mid:])
> >         players = players[0:1] + players[mid:mid+1] + players[1:mid-1] +
> > players[mid+1:] + players[mid-1:mid]
>
> > def test_round_robin(players, rounds, courts, doubles = False):
> >     players = range(players)
>
> DON'T change the type/contents/meaning of a variable name like that.
> E.g. use "nthings" for a number of things and "things" for a
> collection of things.
>
> >     for week in round_robin(players,rounds,courts):
>
> The round_robin() function has only TWO arguments. This code won't
> even run.
>
> When you document neither your data structures nor what your functions
> are intended to do, the last hope for somebody trying to make sense of
> your code is to give meaningful names to your variables. "week" and
> "doubles_week" are NOT meaningful.
>
> >             if doubles == True:
>
> Bletch. s/ == True//
>
> >                     doubles_week = len(week)/2.0
>
> I doubt very much that using floating point is a good idea here.
>
> >                     byes = doubles_week - courts
> >                     if byes == 0:
> >                             bye_list = []
> >                     else:
> >                             bye_list = 
> > week[::int(round(1.072*(courts/byes)+1.08))]
>
> The derivation of the constants 1.072 and 1.08 is  what?
>
> >                     playing = [u for u in week if u not in bye_list]
> >                     midd = len(playing)//2
> >                     doub_sched = zip(playing[:midd], playing[midd:])
> >                     print doub_sched, bye_list
> >             else:
> >                     byes = len(week)- courts
> >                     if byes == 0:
> >                             bye_list = []
> >                     else:
> >                             bye_list = 
> > week[::int(round(1.072*(courts/byes)+1.08))]
> >                     playing = [u for u in week if u not in bye_list]
> >                     print playing, bye_list

For everybody's enlightenment, I have gone through and commented my
code so you can better understand what I'm doing. Here it is:

def round_robin(players, rounds):
# if number of players odd, insert None at first position
if len(players)%2:
players.insert(0, None)
mid = len(players)//2
for i in range(rounds):
yield zip(players[:mid], players[mid:])
players = players[0:1] + players[mid:mid+1] + players[1:mid-1] +
players[mid+1:] + players[mid-1:mid]
""" rotates players like this: 1 2  ->  3 -> 4

 /|

   5 <- 6 <-7 <- 8 """

def test_round_robin(players, rounds, courts, doubles = False):
players = range(players)
for week in round_robin(players,rounds):
if doubles == True: #for doubles pairings
doubles_week = len(week)/2.0
byes = doubles_week - courts #number of tuples to be put 
into
bye_list
if byes == 0:
bye_list = []
else: """ following formula equally spaces out tuples 
selected
for bye_list and selects appropriate number according to length of the
league""&qu

Re: Code works fine except...

[Ross]

On May 4, 12:15 pm, a...@pythoncraft.com (Aahz) wrote:
> In article <8d4ec1df-dddb-469a-99a1-695152db7...@n4g2000vba.googlegroups.com>,
>
> Ross   wrote:
>
> >def test_round_robin(players, rounds, courts, doubles = False):
> >    players = range(players)
> >    for week in round_robin(players,rounds,courts):
> >        if doubles == True:
> >                doubles_week = len(week)/2.0
> >                byes = doubles_week - courts
>
> Side note: thou shalt indent four spaces, no more, no fewer
>
> For more info, see PEP 8.
> --
> Aahz (a...@pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> "It is easier to optimize correct code than to correct optimized code."
> --Bill Harlan

Yes... I know this. Unfortunately, copy/pasting my code from the IDE
into the google group messes with indentations.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 4, 7:59 pm, John Yeung  wrote:
> On May 4, 10:01 am, Ross  wrote:
>
> > The "magic numbers" that everyone is wondering about are
> > indeed used for spreading out the bye selection and I got
> > them by simply calculating a line of best fit when plotting
> > several courts: byes ratios.
>
> But that doesn't really help you.  When you do seq[::step], step is
> evaluated once and used for the whole extended slice.  So in almost
> all cases that you are likely to encounter, step is 2, so you'll get
> seq[0], seq[2], seq[4], etc.  Even if step is some other positive
> number, seq[0] will always be chosen.
>
> > The "byes = #whatever" in my code calculate the number of
> > tuples that need to be placed in the bye_list.
>
> Fine, but that's not the number of byes that you put into bye_list.
>
> > At first glance, the suggestion of adding a simple shift
> > at the end of round_robin doesn't seem to work since
> > round_robin already shifts everything except for the first
> > position already. Please correct me if I'm wrong because
> > you might have been suggesting something different.
>
> If you read my post carefully, you would see that you HAVE to do
> something about the first player always being stuck in the first
> spot.  Either move him around, or select your byes differently.
>
> That said, I've played around with the shuffling a bit, and it does
> seem to be pretty tricky to get it to work for the general case where
> there is no prior knowledge of how many players and courts you will
> have.
>
> I can't shake the feeling that someone good at math will be able to
> figure out something elegant; but if left to my own devices, I am
> starting to lean toward just generating all the possible matches and
> somehow picking from them as needed to fill the courts, trying to keep
> the number of matches played by each player as close to equal as
> possible, and trying to keep the waiting times approximately equal as
> well.
>
> John

"But that doesn't really help you.  When you do seq[::step], step is
evaluated once and used for the whole extended slice.  So in almost
all cases that you are likely to encounter, step is 2, so you'll get
seq[0], seq[2], seq[4], etc.  Even if step is some other positive
number, seq[0] will always be chosen."

It's not true that in almost all cases the step is 2. How that is
evaluated directly depends on the number of available courts. Anyways,
you're right that seq[0] is always evaluated. That's why my algorithm
works fine when there are odd numbers of players in a league. But if
you will notice, my original question was how I could ADD TO or AMMEND
my current code to account for even number of players. I have used an
algorithm that comes up with all possible pairings and then randomly
puts people together each week and places players in a bye list
according to how many times they've previously been in the bye list
but since I was dealing with random permutations, the algorithm took
minutes to evaluate when there were more than 10 players in the
league.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 4, 7:33 pm, John Yeung  wrote:
> On May 4, 8:56 pm, Ross  wrote:
>
> > Anyways, you're right that seq[0] is always evaluated.
> > That's why my algorithm works fine when there are odd
> > numbers of players in a league.
>
> It doesn't work fine for all odd numbers of players.  For example, 15
> players on 3 courts should result in 5 byes.  But what actually
> happens with your code is that you get 4 byes or 8 byes, depending on
> whether you've got floating-point division enabled.
>
> So the way you are selecting byes does not even guarantee that you'll
> allocate the correct number of active matches for the number of
> courts, and this is due to the fact that the extended slice is too
> "coarse" a tool for the task.  Now, it may be that for you, the number
> of players and courts is always within a confined range such that
> extended slicing and your magic constants will work.  That's fine for
> you, but we weren't given that information.
>
> I haven't even begun to look at what happens for doubles.
>
> John

You're right... I only tested cases when number of people playing
outnumbered the number of byes that week. Anyways, I'm new to
programming and this has been a good learning experience. Next time
around, I'll be sure to thoroughly comment my code before I ask for
help on it. I really appreciate all the help that you've offered so
far. Right now, I'm debating whether I should try to reinvent the
round_robin generator part of the code or whether there still might be
a way to shuffle the results of the generated output so that I can
slice it effectively.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 5, 12:32 am, John Yeung  wrote:
> On May 5, 1:12 am, John Yeung  wrote:
>
> > [...] the problem may require bigger guns (either much better
> > math or much more sophisticated programming).
>
> Yes, I'm responding to myself.
>
> Well, I went ahead with the approach I mentioned earlier, generating
> all possible matches and then selecting among them as needed to fill
> up the courts, trying to keep the number of matches played by each
> player as fair as possible.  (I should mention that this, or something
> similar, was suggested earlier by someone else in a different thread,
> in response to the same question by the same OP.)
>
> I did use "bigger guns" (mainly a class for player objects, with
> custom __cmp__ method), but still didn't do anything with doubles.
>
> I haven't tested it much, but I'll post it if anyone's interested.
> (That way people can pick on me instead of the OP. ;)
>
> John

I'm interested to see what you did. From your description, it sounds
like I've tried what you've done, but when I implemented my version,
it took minutes to evaluate for bigger numbers. If that isn't the case
with yours, I'd be interested in seeing your implementation.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 5, 1:33 pm, MRAB  wrote:
> Ross wrote:
> > On May 5, 12:32 am, John Yeung  wrote:
> >> On May 5, 1:12 am, John Yeung  wrote:
>
> >>> [...] the problem may require bigger guns (either much better
> >>> math or much more sophisticated programming).
> >> Yes, I'm responding to myself.
>
> >> Well, I went ahead with the approach I mentioned earlier, generating
> >> all possible matches and then selecting among them as needed to fill
> >> up the courts, trying to keep the number of matches played by each
> >> player as fair as possible.  (I should mention that this, or something
> >> similar, was suggested earlier by someone else in a different thread,
> >> in response to the same question by the same OP.)
>
> >> I did use "bigger guns" (mainly a class for player objects, with
> >> custom __cmp__ method), but still didn't do anything with doubles.
>
> >> I haven't tested it much, but I'll post it if anyone's interested.
> >> (That way people can pick on me instead of the OP. ;)
>
> >> John
>
> > I'm interested to see what you did. From your description, it sounds
> > like I've tried what you've done, but when I implemented my version,
> > it took minutes to evaluate for bigger numbers. If that isn't the case
> > with yours, I'd be interested in seeing your implementation.
>
> Here's my approach (incomplete):
>
> def get_pair(player_list, played):
>      for first in range(len(player_list)):
>          player_1 = player_list[first]
>          for second in range(first + 1, len(player_list)):
>              player_2 = player_list[second]
>              pair = player_1, player_2
>              sorted_pair = tuple(sorted(pair))
>              if sorted_pair not in played:
>                  played.add(sorted_pair)
>                  del player_list[second]
>                  del player_list[first]
>                  return pair
>      return None
>
> def round_robin(player_list, courts, played):
>      playing = []
>      for c in range(courts):
>          pair = get_pair(player_list, played)
>          if pair is None:
>              break
>          playing.append(pair)
>      byes = player_list[:]
>      player_list[:] = byes + [player for pair in playing for player in pair]
>      yield playing, byes
>
> def test_round_robin(players, rounds, courts, doubles=False):
>      player_list = range(players)
>      played = set()
>      for r in range(rounds):
>          for playing, byes in round_robin(player_list, courts, played):
>              print playing, byes- Hide quoted text -
>
> - Show quoted text -

Looks like somewhat of an improvement, although the bye distribution
is still slightly lopsided. For example, in a singles league with 12
players, 12 rounds, and 4 courts, The first player had at least 2 less
byes than every other player and 3 less byes than the players with the
most number of byes. Thanks for your input though...I'll look into how
I can improve upon it.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 5, 10:33 am, MRAB  wrote:
> Ross wrote:
> > On May 5, 12:32 am, John Yeung  wrote:
> >> On May 5, 1:12 am, John Yeung  wrote:
>
> >>> [...] the problem may require bigger guns (either much better
> >>> math or much more sophisticated programming).
> >> Yes, I'm responding to myself.
>
> >> Well, I went ahead with the approach I mentioned earlier, generating
> >> all possible matches and then selecting among them as needed to fill
> >> up the courts, trying to keep the number of matches played by each
> >> player as fair as possible.  (I should mention that this, or something
> >> similar, was suggested earlier by someone else in a different thread,
> >> in response to the same question by the same OP.)
>
> >> I did use "bigger guns" (mainly a class for player objects, with
> >> custom __cmp__ method), but still didn't do anything with doubles.
>
> >> I haven't tested it much, but I'll post it if anyone's interested.
> >> (That way people can pick on me instead of the OP. ;)
>
> >> John
>
> > I'm interested to see what you did. From your description, it sounds
> > like I've tried what you've done, but when I implemented my version,
> > it took minutes to evaluate for bigger numbers. If that isn't the case
> > with yours, I'd be interested in seeing your implementation.
>
> Here's my approach (incomplete):
>
> def get_pair(player_list, played):
>      for first in range(len(player_list)):
>          player_1 = player_list[first]
>          for second in range(first + 1, len(player_list)):
>              player_2 = player_list[second]
>              pair = player_1, player_2
>              sorted_pair = tuple(sorted(pair))
>              if sorted_pair not in played:
>                  played.add(sorted_pair)
>                  del player_list[second]
>                  del player_list[first]
>                  return pair
>      return None
>
> def round_robin(player_list, courts, played):
>      playing = []
>      for c in range(courts):
>          pair = get_pair(player_list, played)
>          if pair is None:
>              break
>          playing.append(pair)
>      byes = player_list[:]
>      player_list[:] = byes + [player for pair in playing for player in pair]
>      yield playing, byes
>
> def test_round_robin(players, rounds, courts, doubles=False):
>      player_list = range(players)
>      played = set()
>      for r in range(rounds):
>          for playing, byes in round_robin(player_list, courts, played):
>              print playing, byes

FYI... I was testing your code further and discovered a strange
outcome... when there are 16 people for 7 courts, every 7th round your
code produces 4 byes instead of the correct 2 byes.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 6, 3:14 pm, John Yeung  wrote:
> On May 6, 3:29 am, MRAB  wrote:
>
> > I have the feeling that if the number of rounds is restricted then the
> > difference between the minimum and maximum number of byes could be 2
> > because of the requirement that players shouldn't play each other more
> > than once, meaning that the players have to be shuffled around a bit, so
> > a player might play a week earlier or later than would otherwise be the
> > case.
>
> This is the feeling that I am getting also.  All my efforts to keep
> everything as balanced as possible at all times (to be ready for the
> "season" to end suddenly at any time) result in messy jams that could
> otherwise be alleviated if I allowed temporary imbalances, knowing
> that there are more weeks later to make them up.
>
> John

If I were to set up a dictionary that counted players used in the bye
list and only allowed players to be added to the bye list if they were
within 2 of the least used player, would this be a good approach for
managing bye selection or would using a dictionary in this manner be
unnecessary/redundant?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 6, 3:14 pm, John Yeung  wrote:
> On May 6, 3:29 am, MRAB  wrote:
>
> > I have the feeling that if the number of rounds is restricted then the
> > difference between the minimum and maximum number of byes could be 2
> > because of the requirement that players shouldn't play each other more
> > than once, meaning that the players have to be shuffled around a bit, so
> > a player might play a week earlier or later than would otherwise be the
> > case.
>
> This is the feeling that I am getting also.  All my efforts to keep
> everything as balanced as possible at all times (to be ready for the
> "season" to end suddenly at any time) result in messy jams that could
> otherwise be alleviated if I allowed temporary imbalances, knowing
> that there are more weeks later to make them up.
>
> John

If I were to set up a dictionary that counted players used in the bye
list and only allowed players to be added to the bye list if they were
within 2 of the least used player, would this be a good approach for
managing bye selection or would using a dictionary in this manner be
unnecessary/redundant?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 6, 3:14 pm, John Yeung  wrote:
> On May 6, 3:29 am, MRAB  wrote:
>
> > I have the feeling that if the number of rounds is restricted then the
> > difference between the minimum and maximum number of byes could be 2
> > because of the requirement that players shouldn't play each other more
> > than once, meaning that the players have to be shuffled around a bit, so
> > a player might play a week earlier or later than would otherwise be the
> > case.
>
> This is the feeling that I am getting also.  All my efforts to keep
> everything as balanced as possible at all times (to be ready for the
> "season" to end suddenly at any time) result in messy jams that could
> otherwise be alleviated if I allowed temporary imbalances, knowing
> that there are more weeks later to make them up.
>
> John

If I were to set up a dictionary that counted players used in the bye
list and only allowed players to be added to the bye list if they were
within 2 of the least used player, would this be a good approach for
managing bye selection or would using a dictionary in this manner be
unnecessary/redundant?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Code works fine except...

[Ross]

On May 7, 1:11 am, John Yeung  wrote:
> On May 7, 12:30 am, Ross  wrote:
>
>
>
> > If I were to set up a dictionary that counted players used in the bye
> > list and only allowed players to be added to the bye list if they were
> > within 2 of the least used player, would this be a good approach for
> > managing bye selection or would using a dictionary in this manner be
> > unnecessary/redundant?
>
> I certainly have not proved it, but I think you don't need to resort
> to anything fancy if you are OK with the maximum byes being within two
> of the minimum byes.  (Maybe this needs to be larger with larger
> numbers of players.)  Try using your original shuffle but instead of
> selecting matches to "throw away" each week, just use the matches you
> need (to fill up the courts) and pick up where you left off the next
> week.  For example, with 10 players, each "round" ideally consists of
> five matches.  If you only have four courts, don't throw away the
> fifth match; save it as the first match next week.
>
> To be honest, I didn't look too carefully at your original shuffle.
> It may be good enough.  It's a little different than the "standard"
> rotation as presented on Wikipedia:
>
>  http://en.wikipedia.org/wiki/Round-robin_tournament#Scheduling_algorithm
>
> The one on Wikipedia happened to pass my casual tests with no more
> than a 2-bye difference between the most-played and least-played
> players, and didn't run into the problem of scheduling the same player
> for two matches the same week.  But I have to stress I only tried a
> few starting values, which all worked; I didn't try to break it, or
> run an extensive battery of tests.
>
> John

John,
   I really appreciate your help with this problem. Thanks to your
suggestions, I've managed to solve the problem. Here's what I did: I
used my original round_robin generator to generate each week. I then
took every week and chained them all together end to end. Then,
depending on how many courts are available, I can select that many
tuples at a time from the list. If you go in order, the discrepancy
between the player with the least amount of byes and the greatest
amount of byes is only 1. If you can make it exactly all the way
through a cycle, there will be no discrepancy. Anyways, I've done all
this by hand and it works so now I'm going to go ahead and code it up.
Again, thanks for your help.

-Ross
--
http://mail.python.org/mailman/listinfo/python-list

slice iterator?

[Ross]

I have a really long list that I would like segmented into smaller
lists. Let's say I had a list a = [1,2,3,4,5,6,7,8,9,10,11,12] and I
wanted to split it into groups of 2 or groups of 3 or 4, etc. Is there
a way to do this without explicitly defining new lists? If the above
problem were to be split into groups of 3, I've tried something like:

start = 0
stop = 3
for i in range(len(a)):
segment = a[start:stop]
print segment
start += stop
stop += stop

Unfortunately start and stop don't increment using this code. Ideally,
my outcome would be
[1,2,3]
[4,5,6]
[7,8,9]
[10,11,12]
--
http://mail.python.org/mailman/listinfo/python-list

indirectly addressing vars in Python

[Ross]

Forgive my newbieness - I want to refer to some variables and indirectly 
 alter them.  Not sure if this is as easy in Python as it is in C.


Say I have three vars: oats, corn, barley

I add them to a list: myList[{oats}, {peas}, {barley}]

Then I want to past that list around and alter one of those values. 
That is I want to increment the value of corn:


myList[1] = myList[1] + 1

Is there some means to do that?.   Here's my little session trying to 
figure this out:


>>> oats = 1
>>> peas = 6
>>> myList=[]
>>> myList
[]
>>> myList.append(oats)
>>> myList
[1]
>>> myList.append(peas)
>>> myList
[1, 6]
>>> myList[1]= myList[1]+1
>>> myList
[1, 7]
>>> peas
6
>>>

So I don't seem to change the value of peas as I wished.  I'm passing 
the values of the vars into the list, not the vars themselves, as I 
would like.


Your guidance appreciated...

Ross.
--
http://mail.python.org/mailman/listinfo/python-list

My Python / wxPython app crashing - suggestions?

[Ross]

I have a rather elaborate app on python 2.5.2 on mac osx 10.4.11. The
GUI elements are built on wxPython 2.8.10.1.   Development has gone
pretty well, but I've got an intermittent rather silent crash that
happens without spewing any error messages to my console at all.

I'm developing in Eclipse with PyDev, and I build with py2app to make
an executable app.

Whether in Eclipse/PyDev or running standalone, the crash occurs
randomly without any messages almost every run, though it takes me 10
-15min of usage before I get the fateful crash. The only data I can
get is from the CrashReporter Log directory where a MyApp.crash.log
tells me that Thread 4 crashed with:


Thread: 4

Exception:  EXC_BAD_ACCESS (0x0001)
Codes:  KERN_PROTECTION_FAILURE (0x0002) at 0x0004

and:

Thread 4 crashed with X86 Thread State (32-bit):
  eax: 0x  ebx: 0x9083d561  ecx: 0x0074  edx: 0x92e341ac
  edi: 0x14442b60  esi: 0x  ebp: 0xb0370b58  esp: 0xb0370b58
   ss: 0x001f  efl: 0x00010286  eip: 0x92e30522   cs: 0x0017
   ds: 0x001f   es: 0x001f   fs: 0x   gs: 0x0037

Given the lack of other info, this doesn't tell me much.   Any
suggestions?

One other (perhaps related issue?)  very rarely, much less common than
the crash is a spurious malloc error from deep in python somewhere.
Not sure that my code is responsible. I don't recognize this format of
complaint.

Python(9544,0xa000d000) malloc: *** error for object 0x1a05c8f0:
double free
Python(9544,0xa000d000) malloc: *** set a breakpoint in 
szone_error
to debug

Can anyone suggest somewhere to start?   If I start putting random
breakpoints and prints in trying to catch the crash, I could be here
for years as I can't yet nail down the conditions that precipitate it.

Regards,
Ross.
-- 
http://mail.python.org/mailman/listinfo/python-list

unicode compare errors

[Ross]

I've a character encoding issue that has stumped me (not that hard to
do). I am parsing a small text file with some possibility of various
currencies being involved, and want to handle them without messing up.

Initially I was simply doing:

  currs = [u'$', u'£', u'€', u'¥']
  aFile = open(thisFile, 'r')
  for mline in aFile:  # mline might be "£5.50"
 if item[0] in currs:
  item = item[1:]

But the problem was:
   SyntaxError: Non-ASCII character '\xa3' in file

The remedy was of course to declare the file encoding for my Python
module, at the start of the file I used:

# -*- coding: UTF-8 -*-

That allowed me to progress. But now when I come to line item that is
a non $ currency, I get this error:

views.py:3364: UnicodeWarning: Unicode equal comparison failed to
convert both arguments to Unicode - interpreting them as being
unequal.

…which I think means Python's unable to convert the char's in the file
I'm reading from into unicode to compare to the items in the list
currs.

I think this is saying that u'£' == '£' is false.
(I hope those chars show up okay in my post here)

Since I can't control the encoding of the input file that users
submit, how to I get past this?  How do I make such comparisons be
True?

Thanks in advance for any suggestions
Ross.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode compare errors

[Ross]

On Dec 10, 2:51 pm, Ross  wrote:

> Initially I was simply doing:
>
>   currs = [u'$', u'£', u'€', u'¥']
>   aFile = open(thisFile, 'r')
>   for mline in aFile:              # mline might be "£5.50"
>      if item[0] in currs:
>           item = item[1:]
>

Don't you love it when someone solves their own problem?  Posting a
reply here so that other poor chumps like me can get around this...

I found I could import codecs that allow me to read the file with my
desired encoding. Huzzah!

Instead of opening the file with a standard
   aFile = open(thisFile, 'r')

I instead ensure I've imported the codecs:

import codecs

... and then I used a specific encoding on the file read:

aFile = codecs.open(thisFile, encoding='utf-8')

Then all my compares seem to work fine.
If I'm off-base and kludgey here and should be doing something
differently please give me a poke.

Regards,
Ross.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unicode compare errors

[Ross]

On Dec 10, 4:09 pm, Nobody  wrote:
> On Fri, 10 Dec 2010 11:51:44 -0800, Ross wrote:
> > Since I can't control the encoding of the input file that users
> > submit, how to I get past this?  How do I make such comparisons be
> > True?
> On Fri, 10 Dec 2010 12:07:19 -0800, Ross wrote:
> > I found I could import codecs that allow me to read the file with my
> > desired encoding. Huzzah!
> > If I'm off-base and kludgey here and should be doing something
>
> Er, do you know the file's encoding or don't you? Using:
>
>     aFile = codecs.open(thisFile, encoding='utf-8')
>
> is telling Python that the file /is/ in utf-8. If it isn't in utf-8,
> you'll get decoding errors.
>
> If you are given a file with no known encoding, then you can't reliably
> determine what /characters/ it contains, and thus can't reliably compare
> the contents of the file against strings of characters, only against
> strings of bytes.
>
> About the best you can do is to use an autodetection library such as:
>
>        http://chardet.feedparser.org/

That's right I don't know what encoding the user will have used. The
use of autodetection sounds good - I'll look into that. Thx.

R.
-- 
http://mail.python.org/mailman/listinfo/python-list

Plain simple unix timestamp with an HTTP GET

[Ross]

I'd like to just quickly and with a minimum of parsing (ie no screen-
scraping) get a unix epoch timestamp (or another format if necessary).

 I thought with a quick second search on Google I'd find a URL where I
could do a simple urllib2 based HTTP  GET and have a timestamp
returned to me. I don't want to use NTP.
I need this because I want to run it on an embedded system where I
don't have a local timesource, but do have a network service. I'm very
low on memory tho.

I can set up my own service on django I suppose, and just render back
the timestamp from datetime.time() but SURELY someone else is already
doing that?

My googling has fallen flat. Any suggestions.

Thanks in advance!

-Ross.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Plain simple unix timestamp with an HTTP GET

[Ross]

No - it's not really a python specific need, it's just what I'm using
just now, and can't think of where else to ask. It's also my fav test-
bed, as it's so easy.

Your curl example is using grep and date which I don't have available.
I have no fancy libraries, just core parsing capability.

I found that NIST has some capability on various servers.

RFC 868 and 867.  I can get this

> curl http://208.66.175.36:13/
55351 10-06-04 00:24:46 50 0 0   8.3 UTC(NIST) *

But I'd have a lot of parsing to pull it together.

Apparently RFC868 provides a 32bit unformated binary response, but I
can't make much out of it. I think my TCP client library is expecting
chars and is screwed by bit-boundary expectations.
The number is supposed to be seconds since 1900, which is just as good
as seconds since 1970.

Still hunting. Tho' maybe getting a bit off topic for a python msg
board :)


On Jun 3, 8:36 pm, livibetter  wrote:
> I don't know what tools do you have on embedded system, but I really
> don't think this has to be using Python.
>
> Here is what I would do on a normal desktop using your unique way to
> set up time:
>
>   date -s "$(curl -s -Ihttp://example.com| grep Date | cut -d \  -f
> 2-)"
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Plain simple unix timestamp with an HTTP GET

[Ross]

On Jun 3, 11:20 pm, livibetter  wrote:
> This?
>
> hwclock --utc --set --date="$(datestr="$(curlhttp://208.66.175.36:13/
> 2>/dev/null | cut -d \  -f 2-3)" ; echo ${datestr//-//})"
>
> Only hwclock, curl, cut, and Bash.
>
> PS. I didn't know I can set the time via hwclock, learned from Paul's
> post, but still didn't try to see if it does work.
>
>

Thanks for the info.  Yes, I like the port 13 stuff from NIST et al
which is  RFC 867 formatted, but on the hdwe the parsing is more
work.

Found a bit of port 37  RFC 868 stuff that sounds interesting. I am
able to get a long int from it now I think (e.g. 64.236.96.53:37 in
Virginia), though it seems to be a bit mangled, and doesn't work out
to the number I'd expect for a 1900 epoch. Still, I think it's usable,
and is just a single number.

I hear NIST is gradually getting away from RFC868 stuff tho' which is
too bad. Some of us don't need pS accuracy. +/- 5min is fine.

Thx for the input!
-- 
http://mail.python.org/mailman/listinfo/python-list

unpacking vars from list of tuples

[Ross]

I'm inexperienced with some of the fancy list slicing syntaxes where
python shines.

If I have a list of tuples:

   k=[("a", "bob", "c"), ("p", "joe", "d"), ("x", "mary", "z")]

and I want to pull the middle element out of each tuple to make a new
list:

myList = ["bob", "joe", "mary"]

is there some compact way to do that?  I can imagine the obvious one
of

myList = []
for a in k:
   myList.append(a[1])

But I'm guessing Python has something that will do that in one line...

Any suggestion is appreciated...

Ross.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unpacking vars from list of tuples

[Ross]

On Sep 15, 6:00 pm, Andre Engels  wrote:
> On Tue, Sep 15, 2009 at 11:51 PM, Ross  wrote:
> > I'm inexperienced with some of the fancy list slicing syntaxes where
> > python shines.
>
> > If I have a list of tuples:
>
> >   k=[("a", "bob", "c"), ("p", "joe", "d"), ("x", "mary", "z")]
>
> > and I want to pull the middle element out of each tuple to make a new
> > list:
>
> > myList = ["bob", "joe", "mary"]
>
> > is there some compact way to do that?  I can imagine the obvious one
> > of
>
> > myList = []
Thanks both Chris and André. That's quite obvious once it's pointed
out for me.  Thanks especially for the terminology that will make
learning the related concepts a bit easier.

Ross


> > for a in k:
> >   myList.append(a[1])
>
> > But I'm guessing Python has something that will do that in one line...
>
> > Any suggestion is appreciated...
>
> You can use a list comprehension:
>
> myList = [a[1] for a in k]
>
> --
> André Engels, andreeng...@gmail.com

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unpacking vars from list of tuples

[Ross]

Thanks Tim,

That's actually the stuff I was trying to remember.

   my_list = [name for _, name, _ in k]

I recalled using some underscores for nice dense unnamed variable  
unpacking before, but couldn't recall the process.


Thanks for that.

Ross.


On 15-Sep-09, at 6:33 PM, Tim Chase wrote:


If I have a list of tuples:
   k=[("a", "bob", "c"), ("p", "joe", "d"), ("x", "mary", "z")]
and I want to pull the middle element out of each tuple to make a new
list:
myList = ["bob", "joe", "mary"]
is there some compact way to do that?  I can imagine the obvious one
of
myList = []
for a in k:
   myList.append(a[1])
But I'm guessing Python has something that will do that in one  
line...


To add some readability to the other suggested solutions, I'd use
tuple unpacking

 my_list = [name for status, name, code in k]

Not knowing what [0] and [2] are, I randomly designated them as  
"status" and "code", but you likely have your own meanings.  If you  
don't, you can always just use the "_" convention:


  my_list = [name for _, name, _ in k]
  # or
  my_list = [name for (_, name, _) in k]

As an aside, "my_list" is preferred over "myList" in common Python  
practice.  I don't know if there's a preferred convention for "with  
vs without" the parens in such a tuple-unpacking list comprehension.


-tkc









--
http://mail.python.org/mailman/listinfo/python-list

Re: unpacking vars from list of tuples

[Ross]

Cool - Now that would be some seriously dense, efficient code!   Will  
have to play with numpy sometime.



R.


On 17-Sep-09, at 12:25 PM, Chris Colbert wrote:


if you have numpy installed:


ln[12]: import numpy as np
In [13]: k = np.array([('a', 'bob', 'c'), ('p', 'joe', 'd'), ('x',
'mary', 'z')])

In [14]: k
Out[14]:
array([['a', 'bob', 'c'],
   ['p', 'joe', 'd'],
   ['x', 'mary', 'z']],
  dtype='|S4')

In [15]: k[:,1]
Out[15]:
array(['bob', 'joe', 'mary'],
  dtype='|S4')


--
http://mail.python.org/mailman/listinfo/python-list

Not this one the other one, from a dictionary

[Ross]

Learning my way around list comprehension a bit.   I wonder if  
someone has a better way to solve this issue.  I have a two element  
dictionary, and I know one of the keys but not the other, and I want  
to look up the other one.


So I have this dictionary:

aDict = {'a': 'bob', 'b': 'stu'}

I know that the dictionary contains two keys/value pairs,  but I  
don't know the values nor that the keys will be 'a' and 'b'.   I  
finally get one of the keys passed to me as variable BigOne. e.g.:


BigOne = "a"

The other key, call it  littleOne remains unknown.  It might be "b"  
but could be "c", "x", etc...   I later need to access both values...


I have something that works, with list comprehension - but wonder if  
there's a more brief/elegant way to get there:


BigValu = aDict[BigOne]
temp =  [ thing for thing in aDict if thing != BigOne ]
LittleValu = aDict[ temp[0] ]

Any thoughts?

- Ross.
 
--

http://mail.python.org/mailman/listinfo/python-list

Re: Not this one the other one, from a dictionary

[Ross]

Thanks Tim (and Ishwor) for the suggestions, those are structures  
that somewhat new to me - looks good!  I'll play with those.At  
this rate I may soon almost know what I'm doing.


Rgds
Ross.


On 18-Sep-09, at 1:19 PM, Tim Chase wrote:

Learning my way around list comprehension a bit.   I wonder if   
someone has a better way to solve this issue.  I have a two  
element  dictionary, and I know one of the keys but not the other,  
and I want  to look up the other one.


Several ways occur to me.  Of the various solutions I played with,  
this was my favorite (requires Python2.4+ for generator expressions):


  d = {'a': 'alice', 'b':'bob'}
  known = 'a'
  other_key, other_value = (
(k,v)
for k,v
in d.iteritems()
if k != known
).next()

If you just want one or the other, you can simplify that a bit:

  other_key = (k for k in d.iterkeys() if k != known).next()
  other_key = (k for k in d if k != known).next()

or

  other_value = (v for k,v in d.iteritems() if k != known).next()

If you're using pre-2.4, you might tweak the above to something like

  other_key, other_value = [
(k,v)
for k,v
in d.iteritems()
if k != known
][0]
  other_key = [k for k in d if k != known)[0]
  other_value = [k for k in d.iteritems if k != known][0]

Hope this helps,

-tkc







--
http://mail.python.org/mailman/listinfo/python-list

virtualenv doesn't see my compiled tensorflow

[ross]

With the prebuilt version of tensorflow, I did:

   virtualenv --system-site-packages  ~/tensorflow

and somehow got it working with keras. Now I've compiled tensorflow in another 
shell/directory, where to start with I did:

   virtualenv --system-site-packages  .

and I got it running with keras on my net, with a nice speedup. Then I went 
back to my previous shell, did a deactivate, then

virtualenv --system-site-packages ~/tf_compile/tensorflow

to point to the dir that was '.' above, but my prompt path did not pick up 
'(tensorflow)' as before:

% virtualenv --system-site-packages ~/tf_compile/tensorflow
New python executable in /Users/priot/tf_compile/tensorflow/bin/python
Installing setuptools, pip, wheel...done.
priot keras%

and I get:

% python prog.py
  ...
  File "/Users/ppp/anaconda/lib/python2.7/site-
  packages/keras/backend/tensorflow_backend.py", line 1, in 
import tensorflow as tf
  ImportError: No module named tensorflow

Seems inconsistent.

% virtualenv --version
15.1.0

% python --version
Python 2.7.10 :: Anaconda custom (x86_64)

OS: OSx Darwin
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: virtualenv doesn't see my compiled tensorflow

[ross]

Solution: remember to run the 'activate' script:

% source ~/tf_compile/tensorflow/bin/activate


On Friday, September 1, 2017 at 2:39:33 PM UTC-7, ro...@cgl.ucsf.edu wrote:
> With the prebuilt version of tensorflow, I did:
> 
>virtualenv --system-site-packages  ~/tensorflow
> 
> and somehow got it working with keras. Now I've compiled tensorflow in 
> another shell/directory, where to start with I did:
> 
>virtualenv --system-site-packages  .
> 
> and I got it running with keras on my net, with a nice speedup. Then I went 
> back to my previous shell, did a deactivate, then
> 
> virtualenv --system-site-packages ~/tf_compile/tensorflow
> 
> to point to the dir that was '.' above, but my prompt path did not pick up 
> '(tensorflow)' as before:
> 
> % virtualenv --system-site-packages ~/tf_compile/tensorflow
> New python executable in /Users/priot/tf_compile/tensorflow/bin/python
> Installing setuptools, pip, wheel...done.
> priot keras%
> 
> and I get:
> 
> % python prog.py
>   ...
>   File "/Users/ppp/anaconda/lib/python2.7/site-
>   packages/keras/backend/tensorflow_backend.py", line 1, in 
> import tensorflow as tf
>   ImportError: No module named tensorflow
> 
> Seems inconsistent.
> 
> % virtualenv --version
> 15.1.0
> 
> % python --version
> Python 2.7.10 :: Anaconda custom (x86_64)
> 
> OS: OSx Darwin

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: when "normal" parallel computations in CPython will be implemented at last?

[Ross Ridge]

Thomas Jollans   wrote:
>There is, of course, Stackless Python.
>http://stackless.com/

Stackless Python doesn't really address the original poster's problem
as the GIL still effectively limits Python code running in one thread
at a time.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2 + 2 = 5

[Michael Ross]

Am 04.07.2012, 21:37 Uhr, schrieb Paul Rubin :


I just came across this (https://gist.github.com/1208215):

import sys
import ctypes
pyint_p = ctypes.POINTER(ctypes.c_byte*sys.getsizeof(5))
five = ctypes.cast(id(5), pyint_p)
print(2 + 2 == 5) # False
five.contents[five.contents[:].index(5)] = 4
print(2 + 2 == 5) # True (must be sufficiently large values of 2  
there...)


Heh.  The author is apparently anonymous, I guess for good reason.



Neat.

Playing with it, i'm wondering:


This:

import sys
import ctypes
pyint_p = ctypes.POINTER(ctypes.c_byte*sys.getsizeof(5))
five = ctypes.cast(id(5), pyint_p)
five.contents[five.contents[:].index(5)] = 4

print ( 2 + 2 == 5 )
print 5
print 5 - 2

put into a script and run prints:

True
4
3

while entered at the python prompt it prints:

True
4
2

??


Regards,
Michael
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to safely maintain a status file

[Ross Ridge]

Laszlo Nagy:
> This is not a contradiction. Although the rename operation is atomic,
> the whole "change status" process is not. It is because there are two
> operations: #1 delete old status file and #2. rename the new status
> file. And because there are two operations, there is still a race
> condition. I see no contradiction here.

Christian Heimes   wrote:
>Sorry, but you are wrong. It's just one operation that boils down to
>"point name to a different inode".

For some reason you're assuming POSIX semantics, an assumption that
Laszlo Nagy did not make.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: the meaning of rﾕ.......��

[Ross Ridge]

Roy Smith   wrote:
>When I first started writing C code, it was on ASR-33s which did not 
>support curly baces.  We wrote ¥( for { and ¥) for } (although I think the 
>translation was 
>handled entirely in the TTY driver and the compiler was never in on the 
>joke).  20 or 30 years from now, people are going to look back on us 
>neanderthals and laugh about how we had to write r''.

No, it's not going to change in 20 or 30 years.  The ASR-33 Teletype
was pretty much obsolete by the time C escaped Bell Labs.  You were
programming in a 70's language using early 60's technology and suffered
accordingly.  Today, the technology to support "Unicode" operators in
programming langauges is both widespread and has existed for a long
time now.  I'm sure you've heard of APL, which both predates Unicode and
C and is almost old as the ASR-33.  If any one actually wanted another
programming language like this it would've come into existance 20 or 30
years ago not 20 or 30 years from now.

Python actually choose to go the other direction and choose to use
keywords as operators instead of symbols in a number of instances.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: the meaning of rﾕ.......ï¾

[Ross Ridge]

Steven D'Aprano   wrote:
>Hey, if the Japanese and Chinese can manage it, English speakers can 
>surely find a way to enter π or ∞ without a keyboard the size of a 
>battleship.

Japanese and Chinese programmers don't use (and don't seem to want to)
use non-ASCII characters outside of strings and comments even when the
language (supposedly) allows it.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: from future import pass_function

[Ross Ridge]

Ulrich Eckhardt wrote:
> I just had an idea, it occurred to me that the pass statement is pretty
> similar to the print statement, 
[...]
> try:
> do_something()
> except:
> pass()

Steven D'Aprano   wrote:
>What's the point of this?

Remember everything you've said about why its a good thing the that
print statement is now a function?  That.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: from future import pass_function

[Ross Ridge]

Steven D'Aprano   wrote:
>What's the point of this?
 
Ross Ridge wrote:
> Remember everything you've said about why its a good thing the that
> print statement is now a function?  That.

Steven D'Aprano   wrote:
>I can't believe I actually have to point this out explicitly, but pass is 
>not print. Apart from them both starting with the letter "P", they are 
>nothing alike. There are good reasons for making print a function, and 
>they don't apply to pass because pass doesn't do what print does.

No, they're very much alike.  That's why all your arguments for print
as function also apply just as well to pass a function.  Your arguments
had very little to do what what print actually did.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: from future import pass_function

[Ross Ridge]

Ross Ridge  wrote:
> No, they're very much alike.  That's why all your arguments for print
> as function also apply just as well to pass a function.  Your arguments
> had very little to do what what print actually did.

Chris Angelico   wrote:
>Except that print / print() is executable. Execution proceeds through
>your code, comes to a "print", and goes off to handle that, then comes
>back to your code. But "pass" doesn't have code attached to it. Why
>should it be a function?

For consistancy with print.  What it does doesn't matter any more than
what print did mattered.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Redirecting STDOUT to a Python Variable

[ross . marsden]

To capture the traceback, so to put it in a log, I use this

import traceback

def get_traceback(): # obtain and return the traceback
exc_type, exc_value, exc_traceback = sys.exc_info()
return ''.join(traceback.format_exception(exc_type, exc_value, 
exc_traceback)) 


Suppose I have a script run by the scheduler, this captures the traceback form 
any problems and emails them.



if __name__ == '__main__':
try: 
Runs_unattended()
except:
send_mail(send_from = yourEmailAddress,
  send_to = [ yourEmailAddress ], subject = 'Runs_unattended',
  text = '%s' % get_traceback(),
  files = [], server=yourLocalSMTP)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to investigate web script not running?

[Michael Ross]

On Fri, 28 Sep 2012 13:37:36 +0200, Gilles  wrote:


Hello

I'm trying to run my very first FastCGI script on an Apache shared
host that relies on mod_fcgid:
==
#!/usr/bin/python
from fcgi import WSGIServer
import cgitb

# enable debugging
cgitb.enable()

def myapp(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello World!\n']

WSGIServer(myapp).run()
==

After following a tutorial, Apache complains with the following when I
call my script:
==
Internal Server Error

The server encountered an internal error or misconfiguration and was
unable to complete your request.
==



Do it the other way around:

# cgitb before anything else
import cgitb
cgitb.enable()

# so this error will be caught
from fcgi import WSGIServer



Regards,
Michael
--
http://mail.python.org/mailman/listinfo/python-list

Re: String manipulation in python..NEED HELP!!!!

[Ross Ridge]

John Gordon wrote:
> def encode(plain):
> '''Return a substituted version of the plain text.'''
> encoded = ''
> for ch in plain:
>encoded += key[alpha.index(ch)]
> return encoded

Terry Reedy   wrote:
>The turns an O(n) problem into a slow O(n*n) solution. Much better to 
>build a list of chars and then join them.

There have been much better suggestions in this thread, but John Gordon's
code above is faster than the equivilent list and join implementation
with Python 2.6 and Python 3.1 (the newest versions I have handy).
CPython optimized this case of string concatenation into O(n) back in
Python 2.4.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tarfile and usernames

[Michael Ross]

On Sun, 30 Dec 2012 19:57:31 +0100, Nicholas Cole  wrote:Dear List,I'm hoping to use the tarfile module in the standard library to move some files between computers. I can't see documented anywhere what this library does with userids and groupids.  I can't guarantee that the computers involved will have the same users and groups, and would like the archives to be extracted so that the files are all owned by the extracting user. 
Essentially, I do *not* with to preserve the owner and groups specified in the archives.As long as you don't extract as root, doesn't extracting set the extracting user's uid/gid anyway?Regards,MichaelWhat is the right way to achieve this?
Best wishes,Nicholas
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question on for loop

[Don Ross]

I'm interested to know why you're trying this as well.  Is this something that 
would be helped by creating a class and then dynamically creating instances of 
that class?  Something like...

class Fruit:
def __init__(self, name):
self.name = name

for fruit in ['banana', 'apple', 'mango']:
varName = Fruit(fruit)
# do stuff with varName

On Thursday, January 3, 2013 2:04:03 PM UTC-6, subhaba...@gmail.com wrote:
> Dear Group,
> 
> If I take a list like the following:
> 
> 
> 
> fruits = ['banana', 'apple',  'mango']
> 
> for fruit in fruits:
> 
>print 'Current fruit :', fruit
> 
> 
> 
> Now, 
> 
> if I want variables like var1,var2,var3 be assigned to them, we may take,
> 
> var1=banana,
> 
> var2=apple,
> 
> var3=mango
> 
> 
> 
> but can we do something to assign the variables dynamically I was thinking
> 
> of 
> 
> var_series=['var1','var2','var3']
> 
> for var in var_series:
> 
>   for fruit in fruits:
> 
>print var,fruits
> 
> 
> 
> If any one can kindly suggest.
> 
> 
> 
> Regards,
> 
> Subhabrata
> 
> 
> 
> NB: Apology for some alignment mistakes,etc.

-- 
http://mail.python.org/mailman/listinfo/python-list

unittest and threading

[Ross Boylan]

Is it safe to use unittest with threads?

In particular, if a unit test fails in some thread other than the one
that launched the test, will that information be captured properly?

A search of the net shows a suggestion that all failures must be
reported in the main thread, but I couldn't find anything definitive.

If it matters, I'm using CPython 2.7.

Thanks.  If you're using email, I'd appreciate a cc.
Ross Boylan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unittest and threading

[Ross Boylan]

On Tue, 2012-01-24 at 13:54 -0800, Ross Boylan wrote:
> Is it safe to use unittest with threads?
> 
> In particular, if a unit test fails in some thread other than the one
> that launched the test, will that information be captured properly?
> 
> A search of the net shows a suggestion that all failures must be
> reported in the main thread, but I couldn't find anything definitive.
> 
> If it matters, I'm using CPython 2.7.
> 
> Thanks.  If you're using email, I'd appreciate a cc.
> Ross Boylan
> 
Steven D'Aprano wrote

> I think you need to explain what you mean here in a little more detail.
> 
> If you mean, "I have a library that uses threads internally, and I want 
> to test it with unittest", then the answer is almost certainly yes it is 
> safe.
> 
> If you mean, "I want to write unit tests which use threads as part of the 
> test", then the answer again remains almost certainly yes it is safe.
Thanks for your responses (only partially excerpted above).

The code I want to test uses threads, but that is not entirely internal
from the standpoint of the unit test framework.  The unit test will be
executing in one thread, but some of the assertions may occur in other
threads.  The question is whether that will work, in particular whether
assertion failures will be properly captured and logged by the test
framework.

Concretely, a test may exercise some code that triggers a callback; the
callback might come in a different thread, and the code that is
triggered might make various assertions.

There are two issues: whether assertions and their failures that happen
in other threads will be correctly received by the test framework, and
whether the framework is robust against several assertions being raised
"simultaneously" in different threads.  The latter seems a bit much to
hope for.

I assume that, at a minimum, the my test code will need to use locks or
other coordination mechanisms so the test doesn't end before all code
under test executes.

Finally, I'll mention two senses of threads in tests that my question
does not concern, although they are also interesting.

I am not concerned with testing the performance of my code, in the sense
of asserting  that an operation must complete before x seconds or after
y seconds.  Some potential implementations of such tests might use
threads even if the code under test was single-threaded.

The question also does not concern running lots of unit tests in
parallel.

Ross

-- 
http://mail.python.org/mailman/listinfo/python-list

Condition.wait() behavior with timeout

[Ross Boylan]

The Python 2.7 documents for the threading module says, in part,
wait([timeout])¶

Wait until notified or until a timeout occurs. If the calling
thread has not acquired the lock when this method is called, a
RuntimeError is raised.

This method releases the underlying lock, and then blocks until
it is awakened by a notify() or notifyAll() call for the same
condition variable in another thread, or until the optional
timeout occurs. Once awakened or timed out, it re-acquires the
lock and returns.

First, the documentation does not say what the return value is.  I was
hoping it was True or False depending on whether a timeout occurred, as
Event.wait().

Second, the "Once awakened or timed out, it re-acquires the lock and
returns" sounds very strange.  If there was a timeout then an attempt to
acquire the lock will block until it is released.  Since there was no
notify there almost certainly will be no release() immediately after the
tiemout.  Which would make the timeout pretty useless (since the thread
that called wait() blocks even after the timeout expires), and might
cause a race on the condition object.

I've googled around, but haven't found anything quite on topic.
http://bugs.python.org/issue1175933 from 2005 requested adding a timeout
to Condition.wait(), a proposal rejected in 2009.  Clearly it's there
now.

http://www.gossamer-threads.com/lists/python/dev/761847 complains there
is no return value from wait and so no way to determine if a timeout
occurred.   One response was

>GR> How am I supposed to know if it was notified or if it timed out? 

Normally you wouldn't have to know. The logic of your program should be 
such that you wait until a certain condition is satisfied. After each 
wait you usually check that condition anyway, like: 

http://bugs.python.org/issue1175933#msg48141 also refers to the need to
check things after returning from wait().  But both of these cases seem
to refer to a scenario in which there are many workers waiting on the
condition, not one with notify() and a single thread waiting (which is
what I'm thinking about).  The thread does say there is no return value;
it seems to me it would be useful to document that if it's still true
(or True :).

Can anyone help me understand what's going on?

Thanks.
Ross Boylan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XSLT to Python script conversion?

[Ross Ridge]

Matej Cepl   wrote:
>No, the strangness is not that bad (well, it is bad ... almost anything 
>feels bad comparing to Python, to be honest, but not the reason I would 
>give up; after all I spent couple of years with Javascript).

The XSLT language is one of the worst misuses of XML, which puts it way
beyond bad.

>The terrible debugging is one thing, and even worse, I just still cannot 
>get over rules around spaces: whitespace just jumps at me randomly in 
>random places and is erased in others.

I use explicit  nodes exclusively to avoid this problem.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: XSLT to Python script conversion?

[Ross Ridge]

Ross Ridge writes:
> The XSLT language is one of the worst misuses of XML, which puts it way
> beyond bad.

Stefan Behnel   wrote:
>Clearly a matter of opinion.

No.  There's no excuse for using XML as the syntax of a language like
XLST.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

#line in python

[Ross Boylan]

The ast module shows that elements of the syntax tree have line and
column numbers.  Would it be sensible to attempt to revise them to
achieve effects like the #line directive in C?

Context: Using noweb, a literate programming tool, which from a source
file foo.nw produces foo.py.  The lines in the two files may be in
completely different sequenes. For debugging, it is useful to receive
error reports that refer to the original line number in foo.nw.

I am not sure how such rewriting would interact with debugger commands
that set a breakpoint at a file and line number.  I'm also not sure it
would change the reported line numbers of errors.

The lack of a file name could be problematic if multiple sources
contributed to the same .py file, but that is an unlikely scenario.

As an extension or alternate, could there be a decorator like
@source_line(lineno, filename)
for classes and methods that could do the conversion on the fly?  I
don't know if there's a way to go from the function (or class) object
the decorator receives to the AST.

Comments?

Ross Boylan


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: #line in python (dirty tricks)

[Ross Boylan]

Duncan Booth wrote

> Ross Boylan  wrote:
> 
> > As an extension or alternate, could there be a decorator like
> > @source_line(lineno, filename)
> > for classes and methods that could do the conversion on the fly?  I
> > don't know if there's a way to go from the function (or class) object
> > the decorator receives to the AST.
> > 
> No [easy] way to go from bytecodes back to AST, but I see no reason why you 
> can't create a new code object with your filename and line numbers and then 
> create a new function using your modified code object.
Could you elaborate?  I don't understand what you are suggesting.
> 
> If you don't have a 1:1 correspondence of lines then you'll need to pick 
> out all the existing line numbers from the code object co_lnotab and modify 
> them: see dis.py findlinestarts() for how to do this.
> 
> Classes would be harder: the decorator doesn't run until after the class 
> body has executed, so you can't change the line numbers that way until it's 
> too late. The only thing I can think would be to put all of the generated 
> code inside a function and fix up that function with a decorator that scans 
> the bytecode to find all contained classes and fix them up.
> 
> Or you could generate a .pyc file and then fix up line numbers in the whole 
> file: see 
> http://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html for 
> some code that shows you what's in a .pyc
> 
My latest concept is to produce code that rewrites itself.  Suppose the
naive file would be
--- mycode.py (naive) 
class SomeClass:
"class comment"

def some_function(self, bar):
pass
- end ---

Protect that code by putting  an "if 0:" in front of it and indenting
each line one space.  Than prepend a bit of code to do the rewriting,
and add indicators of the original line numbers.

--- mycode.py (after wrapping) -
from detangle import detangle
detangle("mycode.py", "mycode.nw")
if 0:
 # original code goes here
 class SomeClass:
"class comment"
 #and when line numbering changes
 #line 35
def some_function(self, bar):
   pass
- end ---
I would write detangle so that it scans through the file in which it
appears (named in the first argument), rewriting so that it appears to
come from the original file (mycode.nw) given in the second argument.

The scanning would look for the "if 0:" in the file.  At that point it
would accumulate code by reading lines and stripping the leading space.
If it found a #line directive it would remember it and then remove it
from the string it was accumulating.  Finally, detangle would would
pass the string of code to ast.compile, catching any syntax errors and
rewriting the file and line number (I might rewrite columns too with an
extension) and then rethrowing them.

If compilation succeeded detangle could rewrite the AST and then exec
it.

Ross

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Chris Angelico   wrote:
>What is a string? It's not a series of bytes.

Of course it is.  Conceptually you're not supposed to think of it that
way, but a string is stored in memory as a series of bytes.

What he's asking for many not be very useful or practical, but if that's
your problem here than then that's what you should be addressing, not
pretending that it's fundamentally impossible.

        Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Ross Ridge  wr=
> Of course it is. =A0Conceptually you're not supposed to think of it that
> way, but a string is stored in memory as a series of bytes.

Chris Angelico   wrote:
>Note that distinction. I said that a string "is not" a series of
>bytes; you say that it "is stored" as bytes.

The distinction is meaningless.  I'm not going argue with you about what
you or I ment by the word "is".

>But a Python Unicode string might be stored in several
>ways; for all you know, it might actually be stored as a sequence of
>apples in a refrigerator, just as long as they can be referenced
>correctly.

But it is in fact only stored in one particular way, as a series of bytes.

>There's no logical Python way to turn that into a series of bytes.

Nonsense.  Play all the semantic games you want, it already is a series
of bytes.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Steven D'Aprano   wrote:
>The right way to convert bytes to strings, and vice versa, is via 
>encoding and decoding operations.

If you want to dictate to the original poster the correct way to do
things then you don't need to do anything more that.  You don't need to
pretend like Chris Angelico that there's isn't a direct mapping from
the his Python 3 implementation's internal respresentation of strings
to bytes in order to label what he's asking for as being "silly".

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Tim Chase   wrote:
>Internally, they're a series of bytes, but they are MEANINGLESS 
>bytes unless you know how they are encoded internally.  Those 
>bytes could be UTF-8, UTF-16, UTF-32, or any of a number of other 
>possible encodings[1].  If you get the internal byte stream, 
>there's no way to meaningfully operate on it unless you also know 
>how it's encoded (or you're willing to sacrifice the ability to 
>reliably get the string back).

In practice the number of ways that CPython (the only Python 3
implementation) represents strings is much more limited.  Pretending
otherwise really isn't helpful.

Still, if Chris Angelico had used your much less misleading explaination,
then this could've been resolved much quicker.  The original poster
didn't buy Chris's bullshit for a minute, instead he had to find out on
his own that that the internal representation of strings wasn't what he
expected to be.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Evan Driscoll   wrote:
>So yes, you can say that pretending there's not a mapping of strings to 
>internal representation is silly, because there is. However, there's 
>nothing you can say about that mapping.

I'm not the one labeling anything as being silly.  I'm the one labeling
the things as bullshit, and that's what you're doing here.  I can in
fact say what the internal byte string representation of strings is any
given build of Python 3.  Just because I can't say what it would be in
an imaginary hypothetical implementation doesn't mean I can never say
anything about it.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Re: Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Evan Driscoll   wrote:
>People like you -- who write to assumptions which are not even remotely
>guaranteed by the spec -- are part of the reason software sucks.
...
>This email is a bit harsher than it deserves -- but I feel not by much.

I don't see how you could feel the least bit justified.  Well meaning,
if unhelpful, lies about the nature Python strings in order to try to
convince someone to follow what you think are good programming practices
is one thing.  Maliciously lying about someone else's code that you've
never seen is another thing entirely.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Re: Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Chris Angelico   wrote:
>Actually, he is justified. It's one thing to work in C or assembly and
>write code that depends on certain bit-pattern representations of data
>(although even that causes trouble - assuming that
>sizeof(int)=3D=3Dsizeof(int*) isn't good for portability), but in a high
>level language, you cannot assume any correlation between objects and
>bytes. Any code that depends on implementation details is risky.

How does that in anyway justify Evan Driscoll maliciously lying about
code he's never seen?

        Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Steven D'Aprano   wrote:
>Your reaction is to make an equally unjustified estimate of Evan's 
>mindset, namely that he is not just wrong about you, but *deliberately 
>and maliciously* lying about you in the full knowledge that he is wrong. 

No, Evan in his own words admitted that his post was ment to be harsh,
"a bit harsher than it deserves", showing his malicious intent.  He made
accusations that where neither supported by anything I've said in this
thread nor by the code I actually write.  His accusation about me were
completely made up, he was not telling the truth and had no reasonable
basis to beleive he was telling the truth.  He was malicously lying and
I'm completely justified in saying so.

Just to make it clear to all you zealots.  I've not once advocated writing
any sort "risky code" in this thread.  I have not once advocated writing
any style of code in thread.  Just because I refuse to drink the "it's
impossible to represent strings as a series of bytes" kool-aid does't mean
that I'm a heretic that must oppose against everything you believe in.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "convert" string to bytes without changing data (encoding)

[Ross Ridge]

Ross Ridge wrote:
> Just because I refuse to drink the
> "it's impossible to represent strings as a series of bytes" kool-aid

Terry Reedy   wrote:
>I do not believe *anyone* has made that claim. Is this meant to be a 
>wild exaggeration? As wild as Evan's?

Sorry, it would've been more accurate to label the flavour of kool-aid
Chris Angelico was trying to push as "it's impossible ... without
encoding":

What is a string? It's not a series of bytes. You can't convert
it without encoding those characters into bytes in some way.

>In my first post on this thread, I made three truthful claims.

I'm not objecting to every post made in this thread.  If your post had
been made before the original poster had figured it out on his own,
I would've hoped he would have found it much more convincing than what
I quoted above.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie questions on import & cmd line run

[Ross Ridge]

Steven D'Aprano   wrote:
>#! ("hash-bang") lines currently do nothing on Windows machines, they are 
>just comments. However, on Unix and Linux machines (and Macintosh?) they 
>are interpreted by the shell (equivalent to cmd.exe or command.com), in 
>order to tell the shell what interpreter to use to execute the program if 
>you run it directly.

They're actually interpreted by the kernel so that they'll work when
run from any program.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Smallest/cheapest possible Python platform?

[Ross Ridge]

Roy Smith   wrote:
>What's the smallest/cheapest/lowest-power hardware platform I can run 
>Python on today?

Not counting the Rasberry Pi, then probably a wireless router or one of
those cheap media streaming boxes running custom firmware.

>Performance requirements are minimal.  I need to monitor a few switches, 
>control a couple of LEDs and relays, and keep time over about a 30 
>minute period to 1/10th second accuracy.  Nice-to-have (but not 
>essential) would be a speech synthesizer with a vocabulary of maybe 50 
>words.

Unfortunately I don't think any of these devices would have the GPIO
pins you'd want for such a project.

>The Rasberry Pi certainly looks attractive, but isn't quite available 
>today.  Can you run Python on an Arduino?  Things like 
>http://www.embeddedarm.com/products/board-detail.php?product=TS-7250 are 
>more than I need, and the $129 price probably busts my budget.

The Arduino uses an 8-bit micro-controller, so probably not.  (The ARM
emulator based port of Linux probably doesn't meet your performance
requirements.)

I think you may need to either wait for the Rasberry Pi to become
generally available or increase your budget.  You should also consider
whether any of these devices have Python bindings to interface with
their GPIO pins.  If not you'll probably have to end up writing some C
code anyways.

    Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: English version for M�mento Python 3 (draft, readers needed)

[Alec Ross]

In message <4fcf9108$0$16471$426a7...@news.free.fr>, Laurent Pointal 
 writes

Ulrich Eckhardt wrote:


Am 05.06.2012 19:32, schrieb Laurent Pointal:


...




"see verso for string formatting..." - what is "verso"?


Modified with Paul indications (its the "other side" in french - from
latin).



FWIW, English idiomatic usage includes "see overleaf", and "see over", 
for the obverse side of a page/sheet, i.e, the following page; and "see 
facing page", w/ the obvious meaning.


Alec
--
Alec Ross
--
http://mail.python.org/mailman/listinfo/python-list

Re: can I distribute Microsoft.VC90.CRT files?

[Ross Ridge]

"J.O. Aho"  wrote:
> Read the EULA that comes with Microsoft Visual C++ Redistributable Package.

Chaz  wrote:
> PLEASE RESPOND WITH AN ANSWER NEXT TIME

READ THE EULA THAT COMES WITH THE MICROSOFT VISUAL C++ REDISTRIBUTABLE PACKAGE.

I hope that helps.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [OT] Free software versus software idea patents (was: Python benefits over Cobra)

[Ross Ridge]

Steven D'Aprano   wrote:
>Perhaps what you mean is, none of the licences granted are *irrevocable*. 
>But the same applies to the GPL -- break the GPL's (generous) terms, and 
>you too could find that your licence is revoked.

Actually, you could argue since the GPL doesn't meet the legal definition
of a contract, it can be revoked unilateraly (but not retroactively)
by the copyright holder at any time for any reason.

Ross Ridge

-- 
 l/  //   Ross Ridge -- The Great HTMU
[oo][oo]  rri...@csclub.uwaterloo.ca
-()-/()/  http://www.csclub.uwaterloo.ca/~rridge/ 
 db  //   
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mutable defaults

[Ross Wilson]

On Thu, 11 Feb 2564 BE at 12:52 Grant Edwards 
wrote:

> On 2021-02-11, J. Pic  wrote:
>
> > I just meant removing the whole "default value mutating" story, not
> > removing mutable variables. Really, I was wondering if there was a use
> case
> > where this actually turns to an advantage,
>
> I've seen people show how it can be used to provide function-scope
> persistent storage -- the equivalent of declaring a static variable in
> a C function. I don't think I've seen that done in the wild, though.


Not sure this qualifies as "use in the wild", but the memoized naive
Fibonacci is very nice when written this way:

def fib(n, memo={0: 0, 1: 1}):
if n not in memo:
memo[n] = fib(n-1) + fib(n-2)
return memo[n]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call for Assistance

[Charles Ross]

> CC-BY-NC-SA is not a license for free (as in speech) content. Is that
> what you want?

I really appreciate all the conversation about the license. Perhaps I made a 
mistake with requiring noncommercial in the license, I don’t know, but I’ll 
look at the links provided about free documentation and free software and see 
if I can understand the issues more and perhaps make a better decision.

Honestly, I have no expectation that this book will ever be anything more than 
a (hopefully) useful resource for others like myself, as described in the 
current introduction. I might have a remote hope that it would be something 
that would receive attention from a traditional publisher, but that remote hope 
is in the same category as a remote hope that a distant rich uncle will someday 
name me in his well. So, given the minuscule odds of that happening, I’ll 
probably remove the NC portion. I’ll just do a bit of research first with those 
links.

Thanks,
Chuck
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call for Assistance

[Charles Ross]

Well, I’ve been convinced. The license for the book is now Creative Commons 
Attribution-ShareAlike.

https://github.com/chivalry/meta-python/blob/master/LICENSE.md

Thanks,
Chuck

> On Aug 9, 2016, at 3:00 PM, Charles Ross  wrote:
> 
>> CC-BY-NC-SA is not a license for free (as in speech) content. Is that
>> what you want?
> 
> I really appreciate all the conversation about the license. Perhaps I made a 
> mistake with requiring noncommercial in the license, I don’t know, but I’ll 
> look at the links provided about free documentation and free software and see 
> if I can understand the issues more and perhaps make a better decision.
> 
> Honestly, I have no expectation that this book will ever be anything more 
> than a (hopefully) useful resource for others like myself, as described in 
> the current introduction. I might have a remote hope that it would be 
> something that would receive attention from a traditional publisher, but that 
> remote hope is in the same category as a remote hope that a distant rich 
> uncle will someday name me in his well. So, given the minuscule odds of that 
> happening, I’ll probably remove the NC portion. I’ll just do a bit of 
> research first with those links.
> 
> Thanks,
> Chuck
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Meta Python Chapter 3 Available

[Charles Ross]

I’ve completed a first draft of Chapter 3: Modules (and Packages) of the Meta 
Python book. Feedback from novices and experts alike would be welcome.

https://github.com/chivalry/meta-python 


Thanks,
Chuck
-- 
https://mail.python.org/mailman/listinfo/python-list

How to get Read the Docs to generate py-modindex.html?

[Charles Ross]

Since I appear to be experiencing one of the few times that Stack Overflow 
hasn’t provided any quick answers to a problem 
(http://stackoverflow.com/questions/39197037/how-to-get-read-the-docs-to-generate-py-modindex-html
 
),
 I’ll try posting my question here and see if anyone has a suggestion.

I'm trying to get Read the Docs to generate the `py-modindex.html` file. 
Research into a related question lead me to the following setup:

- `setup.py` in the project directory has the following contents, which were 
the minimum needed to get `pytest` to work and haven't been changed since I got 
that part of my project working:

import setuptools

setuptools.setup(
name='polygons',
packages=setuptools.find_packages(),
)

- `docs/requirements.txt` contains a single line:

sphinx-autodoc-annotation

- The Read the Docs repository URL points to my repository 
(https://github.com/chivalry/polygons).
- The RtD setting for "Install your project inside a virtualenv using setup.py 
install" is checked.
- The RtD setting for "Requirements file" points to `docs/requirements.txt`.

The "Module Index" link gets included in `index.html` 
(http://polygons.readthedocs.io/en/latest/), but the `py-modindex.html` file is 
missing.

My understanding is that with the virtualenv setting above, RtD will use the 
`setup.py` file to install the project so that Sphinx can read the 
documentation found in the Python docstrings. I'm using function annotations 
and would like `sphyinx-autodoc-annotation` to make use of those when creating 
the built docs files. All of this works splendidly on my local machine when I 
run `make html` while in the `docs` folder. Now I'm trying to get it to work on 
Read the Docs.

Thanks,
Chuck

-- 
https://mail.python.org/mailman/listinfo/python-list

how to control formatting of a namedtuple in a list

[Boylan, Ross]

Even after defining custom __str__ and __format__ methods they don't affect the 
display of objects when they are in a list.  Is there a way to change that, 
other than explicitly converting each list element to a string?

The last line of output below shows that when I format the list I get standard 
formatting of my objects instead of my custom format.

Code
#! /usr/bin/python3
from collections import namedtuple

class Foo(namedtuple("Foo", "x")):
__slots__ = ()

def __str__(self):
  return "foolish({})".format(self.x)

def __format__(self, spec):
  return self.__str__()

f=Foo(4)
print(f)
print(str(f))
print("{}".format(f))
print("{}".format([f]))  # a list with one f

Output
foolish(4)
foolish(4)
foolish(4)
[Foo(x=4)]

I'm running Python 3.4.

Thanks.
Ross
-- 
https://mail.python.org/mailman/listinfo/python-list