[Tutor] Another mention on xkcd...

[Marc Tompkins]

I don't know whether there are any other xkcd fans here, but I am, and I was
very happy to see Python mentioned again...

http://xkcd.com/413/

Sometimes it really does seem that easy.

-- 
www.fsrtechnologies.com
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Recursion doubt

[Anshu Raval]

Hi,
 
Thank you so much for your response. I followed the 2 reasonings you gave,
 
--
You can look at find_all as a master routine that keeps calling find_path until 
find_path gives up. If the search space has not been exhausted (here there are 
still other nodes connected to 'start') take a suitable candidate call 
find_path from there until no more paths from there can be found. Repeat until 
no more suitable candidates.
-
(in this case find_path executes a depth-first search on the graph - 
represented as a dictionary & find_all combines the 2 searches, first the depth 
search & when it quits, the breadth search takes over.)

 
But my question would again be how do you know to put square brackets around 
path in if start == end: return [path] in find_all_paths. I am 
still puzzled by this.
 
Thanks & Regards,
Anshu


Date: Sun, 20 Apr 2008 03:21:03 +0200From: [EMAIL PROTECTED]: [EMAIL 
PROTECTED]: Re: [Tutor] Recursion doubtCC: tutor@python.org
Hi,
 
you are tackling 3 "heavy" subjects in just 1 go!
 
graphs :a triving math society would approve your choice. But you might start 
with the *slightly* less difficult challenge: trees.
I do not know your math/programming background, so the following link can 
perhaps enlighten you: http://www.brpreiss.com/books/opus7/
 
Trees, graphs, queues and what more implemented in several languages. Python 
included.
Also, most CS starters have chapters devoted to (some of) them.
(E.g. http://www.greenteapress.com/thinkpython/)
 
Backtracking & recursion will surely be topics too.

Reading them will help getting a handle on it
 
The 3 chosen topics are advanced or even rather advanced even for most of CS 
people.
 
 
To return to your question...
Basically, find_path takes a node connected to 'start', if not at the 'end', 
make the current node 'start' and see if find_path does return a path. As soon 
as a path is foud find_path quits searching.
 
You can look at find_all as a master routine that keeps calling find_path until 
find_path gives up. If the search space has not been exhausted (here there are 
still other nodes connected to 'start') take a suitable candidate call 
find_path from there until no more paths from there can be found. Repeat until 
no more suitable candidates.
 
As you have gone through the whole search space, all paths from 'start' to 
'end' were found.
 
Recursion makes backtracking easier: the program/routine will by itself keep 
track of the steps taken & where to go back if a search fails.
 
Backtracking is the powerful technique of remembering the steps already taken & 
to retrace them till the first non-explored sidetrack when needed.
 
Choosing the next non-explored sidetrack though can be complex involving other 
searches, ...
It all depends on what you look for, how the data is represented.
 
A related topic that will help you understand things is breadth-first & 
depth-first searches on trees.
 
(in this case find_path executes a depth-first search on the graph - 
represented as a dictionary & find_all combines the 2 searches, first the depth 
search & when it quits, the breadth search takes over.)
 
Greetz 
On 4/15/08, Anshu Raval <[EMAIL PROTECTED]> wrote: 


Hi, At the url http://www.python.org/doc/essays/graphs.html there is some code 
by Guido Van Rossum for computing paths through a graph - I have pasted it 
below for reference - 
Let's write a simple function to determine a path between two nodes. It takes a 
graph and the start and end nodes as arguments. It will return a list of nodes 
(including the start and end nodes) comprising the path. When no path can be 
found, it returns None. The same node will not occur more than once on the path 
returned (i.e. it won't contain cycles). The algorithm uses an important 
technique called backtracking: it tries each possibility in turn until it finds 
a solution. 
def find_path(graph, start, end, path=[]): path = path + [start]
 if start == end: return path if not 
graph.has_key(start): return None for node in graph[start]: 
if node not in path: newpath = find_path(graph, 
node, end, path) if newpath: return newpath return None 
*** He then says 
It is simple to change this function to return a list of all paths (without 
cycles) instead of the first path it finds: 
def find_all_paths(graph, start, end, path=[]): path = path + 
[start] if start == end: return [path] if not 
graph.has_key(start): return [] paths = [] for node 
in graph[start]: if node not in path: newpaths = 
find_all_paths(graph, node, end, path) for newpath in newpaths: 
paths.append(newpath) return paths 
*** I couldn't understand how it was simple to change the function find paths 
to find all paths. How would you thin

[Tutor] knowing when a list is updated by a thread

[Vaibhav.bhawsar]

i have this code to print every new element in a list only when the list
length changes (while the list is updated by a thread running elsewhere)...I
was wondering if there is a pythonic way to do this? how does one know when
there is a new element in the list?

prevlength = 0
while geoCode.running:
places = geoCode.getResults() #returns a list with most up to date
elements..the list grows as the thread that updates it
if len(places) > prevlength:
print places[prevlength]
prevlength = len(places)


thank you!

-- 
Vaibhav Bhawsar
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[linuxian iandsd]

if i only wanted the last element ofthe list, i would  change the
"geoCode.getResults()' to return to me both the complete list & the new
elements in the list. it should be pretty easy from there. if what you need
is to sort the list & see whether a value have changed or not then you will
be better off working with dictionaries.



On Wed, Apr 23, 2008 at 6:26 AM, Vaibhav.bhawsar <[EMAIL PROTECTED]> wrote:

> i have this code to print every new element in a list only when the list
> length changes (while the list is updated by a thread running elsewhere)...I
> was wondering if there is a pythonic way to do this? how does one know when
> there is a new element in the list?
>
> prevlength = 0
> while geoCode.running:
> places = geoCode.getResults() #returns a list with most up to date
> elements..the list grows as the thread that updates it
> if len(places) > prevlength:
> print places[prevlength]
> prevlength = len(places)
>
>
> thank you!
>
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[Kent Johnson]

Vaibhav.bhawsar wrote:
i have this code to print every new element in a list only when the list 
length changes (while the list is updated by a thread running 
elsewhere)...I was wondering if there is a pythonic way to do this? how 
does one know when there is a new element in the list?


prevlength = 0
while geoCode.running:
places = geoCode.getResults() #returns a list with most up to 
date elements..the list grows as the thread that updates it

if len(places) > prevlength:
print places[prevlength]
prevlength = len(places)


Ouch. This loop will chew up CPU time unless there is some kind of delay 
in getResults(). If the intent of the loop is to print every new value 
placed in the list, there is no guarantee that it will work.


If you are using the list to communicate results between two threads you 
should see if queue.Queue meets your needs. It is thread safe and has 
blocking access so you don't have to use a busy loop.


Otherwise you could use a threading.Condition to communicate between the 
threads.


Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Recursion doubt

[Kent Johnson]

Anshu Raval wrote:
But my question would again be how do you know to put square brackets 
around path in

if start == end:
return [path]
in find_all_paths. I am still puzzled by this.


find_all_paths() returns a *list* of paths, even when the result is a 
single path. Without the brackets, it would sometimes return a list and 
sometimes a single path.


Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[bob gailer]

Kent Johnson wrote:

Vaibhav.bhawsar wrote:
i have this code to print every new element in a list only when the 
list length changes (while the list is updated by a thread running 
elsewhere)...I was wondering if there is a pythonic way to do this? 
how does one know when there is a new element in the list?


prevlength = 0
while geoCode.running:
places = geoCode.getResults() #returns a list with most up to 
date elements..the list grows as the thread that updates it

if len(places) > prevlength:
print places[prevlength]
prevlength = len(places)


Ouch. This loop will chew up CPU time unless there is some kind of 
delay in getResults(). If the intent of the loop is to print every new 
value placed in the list, there is no guarantee that it will work.


If you are using the list to communicate results between two threads 
you should see if queue.Queue meets your needs. It is thread safe and 
has blocking access so you don't have to use a busy loop.


Evey time someone recommends Queue I think "oh boy this will really help 
me". Then I go to the Library Reference, read the Queue docs and think 
"oh boy who can help me understand this". Even the sample code is 
confusing.


Is there some other documentation or example?


--
Bob Gailer
919-636-4239 Chapel Hill, NC

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[Hansen, Mike]

 

> Evey time someone recommends Queue I think "oh boy this will 
> really help 
> me". Then I go to the Library Reference, read the Queue docs 
> and think 
> "oh boy who can help me understand this". Even the sample code is 
> confusing.
> 
> Is there some other documentation or example?
> 
> 
> -- 
> Bob Gailer

http://effbot.org/librarybook/queue.htm

Here's a direct link to the queue module.

Mike
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[Hansen, Mike]

 

> Evey time someone recommends Queue I think "oh boy this will 
> really help 
> me". Then I go to the Library Reference, read the Queue docs 
> and think 
> "oh boy who can help me understand this". Even the sample code is 
> confusing.
> 
> Is there some other documentation or example?
> 
> 
> -- 
> Bob Gailer

http://effbot.org/zone/librarybook-index.htm

That might help. It looks like it has some example code. Look at the
Threads and Processes link and search for queue.

Mike
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using Queue

[Kent Johnson]

On Wed, Apr 23, 2008 at 9:46 AM, bob gailer <[EMAIL PROTECTED]> wrote:

> Evey time someone recommends Queue I think "oh boy this will really help
> me". Then I go to the Library Reference, read the Queue docs and think "oh
> boy who can help me understand this". Even the sample code is confusing.


Can you say what is confusing about it? Do you have a specific use in mind?
Do you understand threading? Queue is used to facilitate communication
between threads, so any Queue example includes multiple threads.

In the simplest case, you have producer and consumer threads, and a single
shared instance of Queue. Each producer thread adds items to the queue using
Queue.put(). Consumer threads remove items with Queue.get(). The benefits:
- get() and put() are thread safe
- get() blocks if the queue is empty - no need for a polling loop
- put() blocks if the queue is full

Additionally if the consumers call Queue.task_done() for each consumed item,
Queue.join() will block until all tasks are complete, then continue.

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using Queue

[bob gailer]

Kent Johnson wrote:
On Wed, Apr 23, 2008 at 9:46 AM, bob gailer <[EMAIL PROTECTED] 
> wrote:


Evey time someone recommends Queue I think "oh boy this will
really help me". Then I go to the Library Reference, read the
Queue docs and think "oh boy who can help me understand this".
Even the sample code is confusing.


Can you say what is confusing about it?


"The Queue module implements a multi-producer, multi-consumer FIFO queue.

I understand producer, comsumer, FIFO.

I don't understand multi-

"It is especially useful in threads programming when information must be 
exchanged safely between multiple threads. "


I understand threads. I've written some (to me fairly sophisticated) 
programs using Threading and conditions.


I understand that threads might want to exchange information.

I guess that queue supports the exchange by receiving and releasing 
items. Is that true?


I don't know what "safely" means.

"The Queue class in this module implements all the required locking 
semantics." I have no idea what that means nor does any of the ensuing 
documentation explain.

Do you have a specific use in mind?


I have an application that uses Threading. It is not a producer, 
consumer application, just a bunch of threads that are started at the 
same time. And they do not communicate with each other, just with the 
main thread. But that seems to be true of Queue also.
Queue is used to facilitate communication between threads, so any 
Queue example includes multiple threads.




Regarding the sample code:
I must add (by trial and error) at least these 3 lines to the main program:

from threading import *
from Queue import Queue
num_worker_threads = 3

Then it bombs:
File "J:\python\queue.py", line 17, in 
   for item in source():
NameError: name 'source' is not defined

No idea what source is supposed to be. A callable object? And where in 
this example are the producer threads?


--
Bob Gailer
919-636-4239 Chapel Hill, NC

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using Queue

[Marc Tompkins]

I for one would like to thank Bob for asking the question!  So far I hadn't
had the guts.
-- 
www.fsrtechnologies.com
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using Queue

[Tim Golden]

bob gailer wrote:

Kent Johnson wrote:
On Wed, Apr 23, 2008 at 9:46 AM, bob gailer <[EMAIL PROTECTED] 
> wrote:


Evey time someone recommends Queue I think "oh boy this will
really help me". Then I go to the Library Reference, read the
Queue docs and think "oh boy who can help me understand this".
Even the sample code is confusing.


Can you say what is confusing about it?


"The Queue module implements a multi-producer, multi-consumer FIFO queue.

I understand producer, comsumer, FIFO.

I don't understand multi-


That means -- and this is perhaps Queue's raison d'etre -- that
several threads can read from and write to it "simultaneously"
(from their perspective) without having to worry about locks
and the usual problems of a memory structure shared between
threads.

"It is especially useful in threads programming when information must be 
exchanged safely between multiple threads. "


I understand threads. I've written some (to me fairly sophisticated) 
programs using Threading and conditions.


I understand that threads might want to exchange information.

I guess that queue supports the exchange by receiving and releasing 
items. Is that true?


It supports it be letting, say, thread tA write to queue qA, which
thread tB reads from. (Using put & get, optionally with timeouts).


I don't know what "safely" means.


If you've done some thread programming, I assume you're familiar with
the potential pitfalls, including race conditions, deadlocks and so
on? "Safely" means "without your having to worry about those things".

"The Queue class in this module implements all the required locking 
semantics." I have no idea what that means nor does any of the ensuing 
documentation explain.


Normally if you want to have a thread access some data which *might*
be written to by another thread at, effectively, the same time, the
normal thing is to have each thread lock the data: get hold of a flag
which grants it exclusive access to the data until the lock is released. 
(This is sometimes called a critical section). Without that lock, it's

possible for one thread to be half-way through its update when it is
switched out and the other thread switched in. Now the data is in an
intermediate state (which you don't want).


Do you have a specific use in mind?


I have an application that uses Threading. It is not a producer, 
consumer application, just a bunch of threads that are started at the 
same time. And they do not communicate with each other, just with the 
main thread. But that seems to be true of Queue also.


Yes, that's a typical setup for Queue: the main thread creates a
Queue and passes it to each thread it creates. Say you had an example
where you had a pool of worker threads, each one performing a
calculation. A common approach is to have two Queues: request &#
response. The code would look something like this (entirely untested):


import threading
import Queue

class CalcThread (threading.Thread):
  def __init__ (self, requests, responses):
threading.Thread.__init__ (self)
self.setDaemon (True)
self.requests = requests
self.responses = responses
  def run (self):
while True:
  a, b = self.requests.get ()
  self.responses.put (a + b)

if __name__ == '__main__':
  requests = Queue.Queue ()
  responses = Queue.Queue ()
  for i in range (5):
CalcThread (requests, responses).start ()

  requests.put ((1, 5))
  requests.put ((2, 6))

  print responses.get ()
  print responses.get ()



As it stands this code is fairly stunted, but
I'm trying to keep it simple enough to understand
easily.

TJG
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] knowing when a list is updated by a thread

[Vaibhav.bhawsar]

Some good ideas! Kent I looked at the queue module and that looks like
something I would use. will post my results.

Thanks

On Wed, Apr 23, 2008 at 9:52 AM, Hansen, Mike <[EMAIL PROTECTED]> wrote:

>
>
> > Evey time someone recommends Queue I think "oh boy this will
> > really help
> > me". Then I go to the Library Reference, read the Queue docs
> > and think
> > "oh boy who can help me understand this". Even the sample code is
> > confusing.
> >
> > Is there some other documentation or example?
> >
> >
> > --
> > Bob Gailer
>
> http://effbot.org/zone/librarybook-index.htm
>
> That might help. It looks like it has some example code. Look at the
> Threads and Processes link and search for queue.
>
> Mike
> ___
> Tutor maillist  -  Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>



-- 
Vaibhav Bhawsar
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using Queue

[Kent Johnson]

On Wed, Apr 23, 2008 at 1:17 PM, bob gailer <[EMAIL PROTECTED]> wrote:

> "The Queue module implements a multi-producer, multi-consumer FIFO queue.
>
> I understand producer, comsumer, FIFO.
>
> I don't understand multi-


More than one producer and/or consumer

>
>
> "It is especially useful in threads programming when information must be
> exchanged safely between multiple threads. "
>
> I understand threads. I've written some (to me fairly sophisticated)
> programs using Threading and conditions.
>
> I understand that threads might want to exchange information.
>
> I guess that queue supports the exchange by receiving and releasing items.
> Is that true?


Yes. Producers put things in the queue, consumers pull them out. Basically a
FIFO where the two ends are used by different threads.

>
> I don't know what "safely" means.
>
> "The Queue class in this module implements all the required locking
> semantics." I have no idea what that means nor does any of the ensuing
> documentation explain.


Ok, do you understand thread safety, or conversely, what kind of problems
can happen when two threads share data without any kind of locking?

For example, suppose you want a queue with a limited number of items allowed
in it, and producers should block if the queue is full. You need an atomic
"check for full and push" operation; otherwise you can have race conditions
between two threads. If the queue has one empty slot, you have to protect
against
- thread A checks for queue full - not full
- thread B checks for queue full - not full
- thread A pushes - queue is now full
- thread B pushes - overflow

queue.Queue uses locks to prevent this and other problems.

> Do you have a specific use in mind?
>

I have an application that uses Threading. It is not a producer, consumer
> application, just a bunch of threads that are started at the same time. And
> they do not communicate with each other, just with the main thread. But that
> seems to be true of Queue also.


Possibly you could use a Queue here. The main thread would be the only
consumer; the rest of the threads would be producers.

>
>  Queue is used to facilitate communication between threads, so any Queue
> > example includes multiple threads.
> >
> >
> >  Regarding the sample code:
> I must add (by trial and error) at least these 3 lines to the main
> program:
>
> from threading import *
> from Queue import Queue
> num_worker_threads = 3
>
> Then it bombs:
> File "J:\python\queue.py", line 17, in 
>   for item in source():
> NameError: name 'source' is not defined
>
> No idea what source is supposed to be. A callable object? And where in
> this example are the producer threads?


Yes, the example is really just a sketch. source is a callable object that
returns an iterable object whose items represent some kind of unit of work.

The threads created by calling Thread(target=worker) are the consumer
threads. The main thread is the producer - it  puts items into the queue in
the for loop.

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Using Queue

[Kent Johnson]

On Wed, Apr 23, 2008 at 1:36 PM, Tim Golden <[EMAIL PROTECTED]> wrote:

> Yes, that's a typical setup for Queue: the main thread creates a
> Queue and passes it to each thread it creates. Say you had an example
> where you had a pool of worker threads, each one performing a
> calculation. A common approach is to have two Queues: request &#
> response. The code would look something like this (entirely untested):
>

There are many threadpool examples in the online Python cookbook, for
example this one (picked mostly at random - see also links from there, the
'threading' topic of the cookbook, or search for 'pool'):
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/435883

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Needing help with variables

[Cain, Steven]

I have the code below and I am in need of a little help with variables.
What I am trying to do is substitute a variable where it says insert
variable. I actually need to place 3 variables. 1 for date, 1 for time
and 1 for a string of text.

Any help would be greatly appreciated.

Steve

def BackwardsReader(file, BLKSIZE = 4096):
#"""Read a file line by line, backwards"""

buf = ""
file.seek(-1,2)
lastchar = file.read(1)
trailing_newline = (lastchar == "\n")

while 1:
newline_pos = buf.rfind("\n")
pos = file.tell()
if newline_pos != -1:
# Found a newline
line = buf[newline_pos+1:]
buf = buf[:newline_pos]
if pos or newline_pos or trailing_newline:
line += "\n"
yield line
elif pos:
# Need to fill buffer
toread = min(BLKSIZE, pos)
file.seek(-toread, 1)
buf = file.read(toread) + buf
file.seek(-toread, 1)
if pos == toread:
buf = "\n" + buf
else:
# Start-of-file
return


for line in BackwardsReader(open('stopServer.log')):
if line[0] == '[':
print line[1:8],line[9:16],line[62:93] ( insert Variable )
break

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Needing help with variables

[Kent Johnson]

On Wed, Apr 23, 2008 at 2:55 PM, Cain, Steven <[EMAIL PROTECTED]> wrote:
> I have the code below and I am in need of a little help with variables.
>  What I am trying to do is substitute a variable where it says insert
>  variable. I actually need to place 3 variables. 1 for date, 1 for time
>  and 1 for a string of text.
>
>  Any help would be greatly appreciated.
>
>  Steve
>
>  def BackwardsReader(file, BLKSIZE = 4096):

Presumably the files you are trying to read are too large to fit in
memory? Otherwise this could be implemented as
  return reversed(file)

>  for line in BackwardsReader(open('stopServer.log')):
> if line[0] == '[':
> print line[1:8],line[9:16],line[62:93] ( insert Variable )

I'm not too sure what you want to do here. Do you want to create three
variables with the three substrings? That would be
  date, time, text = line[1:8],line[9:16],line[62:93]

or of course you could use three separate assignments:
  date = line[1:8]
  time = lline[9:16]
  text = line[62:93]

Kent
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Open Source Hero my @$$!

[Marc Tompkins]

Just thought I'd share something I found quite humorous...  a while back, I
heard about Microsoft's "Open Source Hero" program, and being a curious type
I signed up.  Hey, I love me some Open Source, and I wanna be a hero, right?

My Hero Pack arrived a few days ago. It contains:
  a 90-day trial of Visual Studio 2008
and
  a 120-day trial of Windows Server 2008.

Now don't get me wrong - I'm not ungrateful; I fully intend to try out
Windows Server so I can see what I'll be facing over the next few months.
But Visual Studio?  Open Source?  Am I missing the joke here?

Any other Heros here?  Anybody else find this as funny as I did?

-- 
www.fsrtechnologies.com
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor