Re: A small quiz
On Thursday, January 23, 2020 at 3:54:56 AM UTC-5, Z wrote: > what is PLR? PLR: Private Label Rights (https://en.wikipedia.org/wiki/Private_label_rights) -- https://mail.python.org/mailman/listinfo/python-list
Re: Friday Finking: Enum by gum
On 24/01/2020 04:26, DL Neil via Python-list wrote: Questions: Have I made proper sense of this? (please don't laugh too much) Um. My basic take on enums is that they provide a convenient namespace to collect together related constants. API flags and opcodes are an obvious example of related constants you would want to keep together, but Python enums can be more complex than that both as structured data and associated functions. Is the above 'interfacing' an appropriate use of enum-s; or is it really 'overkill' or posturing? I think it's an entirely appropriate use of enums. In fact it's pretty much my only use of them. One caveat: when you care about the value of a constant, set it explicitly. Don't rely on auto() to come up with the right number for you, because the documentation really gives you no guarantees. Python v3.6+ brings Flag-enums into the game. These seem better for interface use, as described. Do you prefer them to the 'plain-vanilla' enum, and for what reason? I prefer them when the interface calls for flags, because they behave like flags. I use plain enums when I don't require that behaviour. It's horses for courses, as ever. -- Rhodri James *-* Kynesim Ltd -- https://mail.python.org/mailman/listinfo/python-list
Threading
I am using this example for threading in Python: from threading import Thread def start_test( address, port ): print address, port sleep(1) for line in big_list: t = Thread(target=start_test, args=(line, 80)) t.start() But say big_list has thousands of items and I only want to have a maximum of 10 threads open. How do work my way through the big_list with only 10 threads for example? -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
On Sat, Jan 25, 2020 at 7:35 AM Matt wrote: > > I am using this example for threading in Python: > > from threading import Thread > > def start_test( address, port ): > print address, port > sleep(1) > > for line in big_list: > t = Thread(target=start_test, args=(line, 80)) > t.start() > > But say big_list has thousands of items and I only want to have a > maximum of 10 threads open. How do work my way through the big_list > with only 10 threads for example? First off, it is high time you move to Python 3, as the older versions of Python have reached end-of-life. The best way is to create your ten threads, and have each one request "jobs" (for whatever definition of job you have) from a queue. Once the queue is exhausted, the threads terminate cleanly, and then you can join() each thread to wait for the entire queue to be completed. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
On Sat, 25 Jan 2020 07:43:49 +1100 Chris Angelico wrote: > On Sat, Jan 25, 2020 at 7:35 AM Matt wrote: > > > > I am using this example for threading in Python: > > > > from threading import Thread > > > > def start_test( address, port ): > > print address, port > > sleep(1) > > > > for line in big_list: > > t = Thread(target=start_test, args=(line, 80)) > > t.start() > > > > But say big_list has thousands of items and I only want to have a > > maximum of 10 threads open. How do work my way through the big_list > > with only 10 threads for example? > > First off, it is high time you move to Python 3, as the older versions > of Python have reached end-of-life. > > The best way is to create your ten threads, and have each one request > "jobs" (for whatever definition of job you have) from a queue. Once > the queue is exhausted, the threads terminate cleanly, and then you > can join() each thread to wait for the entire queue to be completed. Or use a thread pool: https://docs.python.org/3/library/concurrent.futures.html Dan -- “Atoms are not things.” – Werner Heisenberg Dan Sommers, http://www.tombstonezero.net/dan -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
So I would create 10 threads. And each would pop items off list like so? def start_test(): while big_list: list_item = big_list.pop() print list_item, port sleep(1) port = 80 for i = 1 to 10 t = Thread(target=start_test) t.start() t.join() Would that be thread safe? On Fri, Jan 24, 2020 at 2:44 PM Chris Angelico wrote: > > On Sat, Jan 25, 2020 at 7:35 AM Matt wrote: > > > > I am using this example for threading in Python: > > > > from threading import Thread > > > > def start_test( address, port ): > > print address, port > > sleep(1) > > > > for line in big_list: > > t = Thread(target=start_test, args=(line, 80)) > > t.start() > > > > But say big_list has thousands of items and I only want to have a > > maximum of 10 threads open. How do work my way through the big_list > > with only 10 threads for example? > > First off, it is high time you move to Python 3, as the older versions > of Python have reached end-of-life. > > The best way is to create your ten threads, and have each one request > "jobs" (for whatever definition of job you have) from a queue. Once > the queue is exhausted, the threads terminate cleanly, and then you > can join() each thread to wait for the entire queue to be completed. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
On 2020-01-24 21:33, Matt wrote: So I would create 10 threads. And each would pop items off list like so? def start_test(): while big_list: list_item = big_list.pop() print list_item, port sleep(1) port = 80 for i = 1 to 10 t = Thread(target=start_test) t.start() t.join() Not quite. 1. Create a list of threads. 2. Put the items into a _queue_, not a list. 3. Start the threads. 4. Iterate over the list of threads, using .join() on each. If you're going to start the threads before you've put all of the items into the queue, you can also put a sentinel such as None into the queue after you've finished putting the items into it. When a thread sees the sentinel, it knows there are no more items to come. You could have one sentinel for each thread, or have only one and have each thread put it back when it sees it, for the other threads to see. Would that be thread safe? On Fri, Jan 24, 2020 at 2:44 PM Chris Angelico wrote: On Sat, Jan 25, 2020 at 7:35 AM Matt wrote: > > I am using this example for threading in Python: > > from threading import Thread > > def start_test( address, port ): > print address, port > sleep(1) > > for line in big_list: > t = Thread(target=start_test, args=(line, 80)) > t.start() > > But say big_list has thousands of items and I only want to have a > maximum of 10 threads open. How do work my way through the big_list > with only 10 threads for example? First off, it is high time you move to Python 3, as the older versions of Python have reached end-of-life. The best way is to create your ten threads, and have each one request "jobs" (for whatever definition of job you have) from a queue. Once the queue is exhausted, the threads terminate cleanly, and then you can join() each thread to wait for the entire queue to be completed. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
Created this example and it runs. import time import threading big_list = [] for i in range(1, 200): big_list.append(i) def start_test(): while big_list: #is this list_item = big_list.pop() #and this thread safe print list_item, port time.sleep(1) print "Creating Threads..." port = 80 for i in range(1, 10): t = threading.Thread(target=start_test) t.start() print "Waiting on Threads..." t.join() print "Finished..." On Fri, Jan 24, 2020 at 2:44 PM Chris Angelico wrote: > > On Sat, Jan 25, 2020 at 7:35 AM Matt wrote: > > > > I am using this example for threading in Python: > > > > from threading import Thread > > > > def start_test( address, port ): > > print address, port > > sleep(1) > > > > for line in big_list: > > t = Thread(target=start_test, args=(line, 80)) > > t.start() > > > > But say big_list has thousands of items and I only want to have a > > maximum of 10 threads open. How do work my way through the big_list > > with only 10 threads for example? > > First off, it is high time you move to Python 3, as the older versions > of Python have reached end-of-life. > > The best way is to create your ten threads, and have each one request > "jobs" (for whatever definition of job you have) from a queue. Once > the queue is exhausted, the threads terminate cleanly, and then you > can join() each thread to wait for the entire queue to be completed. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
On Sat, Jan 25, 2020 at 9:05 AM Matt wrote: > > Created this example and it runs. > > import time > import threading > > big_list = [] > > for i in range(1, 200): > big_list.append(i) > > def start_test(): > while big_list: #is this > list_item = big_list.pop() #and this thread safe > print list_item, port > time.sleep(1) > > print "Creating Threads..." > > port = 80 > for i in range(1, 10): > t = threading.Thread(target=start_test) > t.start() > > print "Waiting on Threads..." > > t.join() > > print "Finished..." > Please don't top-post. Also, switch to a Python 3 interpreter before you end up filling your code with Py2isms and make your job harder later. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
First come remarks, then a different suggestion about your capacity problem (no more than 10). Details below. On 24Jan2020 16:03, Matt wrote: Created this example and it runs. [...] big_list = [] for i in range(1, 200): big_list.append(i) You can just go: big_list.extend(range(1,200)) def start_test(): while big_list: #is this This tests if big_list is nonempty. In Python, most "containers" (lists, tuples, dicts, sets etc) are "falsey" if empty and "true" if nonempty. list_item = big_list.pop() #and this thread safe Don't think so. print list_item, port Please use Python 3; Python 2 is end of life. So: print(list_item, port) time.sleep(1) print "Creating Threads..." port = 80 for i in range(1, 10): t = threading.Thread(target=start_test) t.start() This starts only 10 Threads. print "Waiting on Threads..." t.join() This waits for only the last Thread. My suggestion for your capacity thing: use a Semaphore, which is a special thread safe counter which cannot go below zero. from threading import Semaphore def start_test(sem, args...): sem.acquire() ... do stuff with args ... sem.release() sem = Semaphore(10) threads = [] for item in big_list: t = Thread(target=start_test, args=(sem, item)) t.start() threads.append(t) ... wait for all the threads here ... This version starts many threads, but only 10 at a time will do "work" because they stall until they can acquire the Semaphore. The first 10 acquire it immediately, then the later only stall until an earlier Thread releases the Semaphore. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: Threading
> Not quite.
>
> 1. Create a list of threads.
>
> 2. Put the items into a _queue_, not a list.
>
> 3. Start the threads.
>
> 4. Iterate over the list of threads, using .join() on each.
>
> If you're going to start the threads before you've put all of the items
> into the queue, you can also put a sentinel such as None into the queue
> after you've finished putting the items into it. When a thread sees the
> sentinel, it knows there are no more items to come. You could have one
> sentinel for each thread, or have only one and have each thread put it
> back when it sees it, for the other threads to see.
>
Is the list not thread safe and I need to use Queue instead or is it
just that using a Queue is more efficient? I think I have everything
else you mentioned changed. Even in python3 now though I still need to
work in python2 in places for time being. Thanks.
import time
import datetime
import threading
import random
big_list = []
def date_stamp():
return "[" + datetime.datetime.now().strftime('%Y-%m-%d
%H:%M:%S:%f')[:-3] + "] "
for i in range(1, 5000):
big_list.append(i)
def start_test(id):
while big_list:
list_item = big_list.pop()
print(date_stamp(), "Thread", id, ":", list_item, port)
time.sleep(random.random())
print(date_stamp(), "Thread", id, "done...")
print(date_stamp(), "Creating Threads...")
port = 80
threads = []
for i in range(1, 10):
t = threading.Thread(target=start_test, args=(i,))
print(date_stamp(), "Starting Thread:", i)
t.start()
threads.append(t)
print(date_stamp(), "Waiting on Threads...")
for t in threads:
t.join()
print(date_stamp(), "Finished...")
--
https://mail.python.org/mailman/listinfo/python-list
Re: A small quiz
On Thu, Jan 23, 2020 at 2:57 AM Z wrote: > > what is PLR? Python Language Reference? https://docs.python.org/3/reference/index.html Or, perhaps, Python Library Reference? https://docs.python.org/3/library/index.html -- boB -- https://mail.python.org/mailman/listinfo/python-list
