[ python-Bugs-1454855 ] Explanation of pow() in lib
Bugs item #1454855, was opened at 2006-03-20 20:54 Message generated for change (Comment added) made by cito You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1454855&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Not a Bug Status: Closed Resolution: Works For Me Priority: 2 Submitted By: Christoph Zwerschke (cito) Assigned to: Nobody/Anonymous (nobody) Summary: Explanation of pow() in lib Initial Comment: The Python Lib Reference explains the pow() function in section 2.1 like that: >>> pow(x, y[, z]) Return x to the power y; if z is present, return x to the power y, modulo z (computed more efficiently than pow(x, y) % z). The arguments must have numeric types. With mixed operand types, the coercion rules for binary arithmetic operators apply. For int and long int operands, the result has the same type as the operands (after coercion) unless the second argument is negative; in that case, all arguments are converted to float and a float result is delivered. For example, 10**2 returns 100, but 10**-2 returns 0.01. <<< The problem is here that the notation 10**2 is used in the example without prior explanation that it is equivalent to pow(10,2). A newbie reading the docs in linear order may not know this here (many other languages write x^y instead of x**y). The notation x**y is only introduced later in section 2.3.4. I recommend adding a short remark to this paragraph explaining that instead of writing pow(x,y) you can also write x**y. -- >Comment By: Christoph Zwerschke (cito) Date: 2006-03-21 11:42 Message: Logged In: YES user_id=193957 Now I understand why the Lib Reference suddenly uses the ** notation when it speaks about the pow() function: The passage has been simply copied over to the Lib Reference from section 5.4 of the Lang Reference about the ** operator. By the way, this section 5.4 in the Lang reference mentions the pow() function as well. So I think it would be only consequent if the Lib reference about pow() reciprocates that and refers to the power operator in turn. Anyway, somebody has changed it in the SVN already; I think it is now ok and hope you agree. -- Comment By: Christoph Zwerschke (cito) Date: 2006-03-20 23:23 Message: Logged In: YES user_id=193957 I'm not sure about that. You are thinking too much from an expert/insider point of view, not from a newbie/learner point of view. For newbies, this is not so clear. See for example: http://groups.google.com/group/comp.lang.python/browse_frm/thread/c82eb3c3a6068b7d Even if the reader understands what 10**2 is, it is still not clear at this place whether this is a Python expression or simply a mathematical notation, and whether pow(x,y) behaves exactly like x**y or not. I still think a short note like "Note that you can also write x**y as an abbreviation for pow(x,y)." makes all of this clear and may be a help or a hint for newbies and non-mathematicians. Your argument that the Lib reference is not meant to be read like a novel is true, but the introduction says explicitely that reading it as a novel *is* an option, and encourages people to read at least chapter 2 which is actually one of the core parts of the whole Python documentation. So I think many newbies will read the Tutorial (where the equivalence of x**y and pow(x,y) is not mentioned either) and then at least chapter 2 from the Lib reference. And even if you take it only as a reference and just read the explanation of pow() it may be a good reminder that pow(x,y) is actually the same as x**y. -- Comment By: Raymond Hettinger (rhettinger) Date: 2006-03-20 21:07 Message: Logged In: YES user_id=80475 The Lib Reference is like an encylopaedia; it does not purport to avoid forward references and be read linearly start to finish. Also, in this case the meaning is 100% clear from the context. IOW, given a discussion about x raised to the y power and an expression "10**2 returns 100, but 10**-2 returns 0.01", the meaning of the operator is self-evident. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1454855&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455357 ] Incompatible size for long (int) in Python Win and Linux?
Bugs item #1455357, was opened at 2006-03-21 22:08
Message generated for change (Settings changed) made by perky
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455357&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.4
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Michael Yanowitz (yanowitz)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incompatible size for long (int) in Python Win and Linux?
Initial Comment:
There appears to be a problem with
packing/unpacking long (and int) values when sending
data from Windows to Linux.
I am not sure if it is a bug or just due to
incompatibleness. Normally we think of long as being 4
bytes. However, on Linux, it might be 8 bytes (in
Python).
Here are good results in Windows, packing and
unpacking a long:
$ python
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for
more information.
>>> import struct
>>> p = struct.pack("L", 12345)
>>> p
'90\x00\x00'
>>> u = struct.unpack("L",p)
>>> u
(12345L,)
However, on Linux, I get slightly different results:
Python 2.4.1 (#1, May 16 2005, 15:15:14)
[GCC 4.0.0 20050512 (Red Hat 4.0.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for
more information>>> import struct
>>> p = struct.pack("L", 12345)
>>> p
'90\x00\x00\x00\x00\x00\x00'
>>> u = struct.unpack("L",p)
>>> u
(12345L,)
This is annoying, because I want to be able to
send data from Windows to Linux (and vice-versa)
over a socket. I am new to Python, and the only
way I know of of doing that is by packing and
unpacking the data.
Please email me [EMAIL PROTECTED] if this
isn't a bug or there is a known work-around for
this problem.
Thanks in advance:
Michael Yanowitz
--
>Comment By: Hye-Shik Chang (perky)
Date: 2006-03-21 22:30
Message:
Logged In: YES
user_id=55188
In C, the size of long depends on platform ABI. If you are
on a 64bit platform, long may be 8 bytes while you got 4
bytes in 32bit platforms. It's not Python's problem.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455357&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455357 ] Incompatible size for long (int) in Python Win and Linux?
Bugs item #1455357, was opened at 2006-03-21 08:08
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455357&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Michael Yanowitz (yanowitz)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incompatible size for long (int) in Python Win and Linux?
Initial Comment:
There appears to be a problem with
packing/unpacking long (and int) values when sending
data from Windows to Linux.
I am not sure if it is a bug or just due to
incompatibleness. Normally we think of long as being 4
bytes. However, on Linux, it might be 8 bytes (in
Python).
Here are good results in Windows, packing and
unpacking a long:
$ python
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for
more information.
>>> import struct
>>> p = struct.pack("L", 12345)
>>> p
'90\x00\x00'
>>> u = struct.unpack("L",p)
>>> u
(12345L,)
However, on Linux, I get slightly different results:
Python 2.4.1 (#1, May 16 2005, 15:15:14)
[GCC 4.0.0 20050512 (Red Hat 4.0.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for
more information>>> import struct
>>> p = struct.pack("L", 12345)
>>> p
'90\x00\x00\x00\x00\x00\x00'
>>> u = struct.unpack("L",p)
>>> u
(12345L,)
This is annoying, because I want to be able to
send data from Windows to Linux (and vice-versa)
over a socket. I am new to Python, and the only
way I know of of doing that is by packing and
unpacking the data.
Please email me [EMAIL PROTECTED] if this
isn't a bug or there is a known work-around for
this problem.
Thanks in advance:
Michael Yanowitz
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455357&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455357 ] Incompatible size for long (int) in Python Win and Linux?
Bugs item #1455357, was opened at 2006-03-21 08:08
Message generated for change (Comment added) made by tim_one
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455357&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.4
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Michael Yanowitz (yanowitz)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incompatible size for long (int) in Python Win and Linux?
Initial Comment:
There appears to be a problem with
packing/unpacking long (and int) values when sending
data from Windows to Linux.
I am not sure if it is a bug or just due to
incompatibleness. Normally we think of long as being 4
bytes. However, on Linux, it might be 8 bytes (in
Python).
Here are good results in Windows, packing and
unpacking a long:
$ python
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for
more information.
>>> import struct
>>> p = struct.pack("L", 12345)
>>> p
'90\x00\x00'
>>> u = struct.unpack("L",p)
>>> u
(12345L,)
However, on Linux, I get slightly different results:
Python 2.4.1 (#1, May 16 2005, 15:15:14)
[GCC 4.0.0 20050512 (Red Hat 4.0.0-5)] on linux2
Type "help", "copyright", "credits" or "license" for
more information>>> import struct
>>> p = struct.pack("L", 12345)
>>> p
'90\x00\x00\x00\x00\x00\x00'
>>> u = struct.unpack("L",p)
>>> u
(12345L,)
This is annoying, because I want to be able to
send data from Windows to Linux (and vice-versa)
over a socket. I am new to Python, and the only
way I know of of doing that is by packing and
unpacking the data.
Please email me [EMAIL PROTECTED] if this
isn't a bug or there is a known work-around for
this problem.
Thanks in advance:
Michael Yanowitz
--
>Comment By: Tim Peters (tim_one)
Date: 2006-03-21 13:07
Message:
Logged In: YES
user_id=31435
I agree with closing this report, but we should point out
that Python _does_ solve most of this problem: don't use
"native mode" pack/unpack, use "standard mode" pack/unpack,
to transport values across architectures. Read the `struct`
docs for details. For example, struct.pack("!L", 12345)
uses 4-byte big-endian ("network") format regardless of the
platform it runs on.
--
Comment By: Hye-Shik Chang (perky)
Date: 2006-03-21 08:30
Message:
Logged In: YES
user_id=55188
In C, the size of long depends on platform ABI. If you are
on a 64bit platform, long may be 8 bytes while you got 4
bytes in 32bit platforms. It's not Python's problem.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455357&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455641 ] pyport.h freebsd inconsistent
Bugs item #1455641, was opened at 2006-03-21 15:43 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455641&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Jim Jewett (jimjjewett) Assigned to: Nobody/Anonymous (nobody) Summary: pyport.h freebsd inconsistent Initial Comment: pyport.h redefines isalnum and friends #if __FreeBSD_version > 500039 but the comment just above says not to do so in FreeBSD 6 (which is now available). Add an extra guard clause? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455641&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455676 ] Simplify use of Queues with consumer threads
Bugs item #1455676, was opened at 2006-03-21 16:36 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Simplify use of Queues with consumer threads Initial Comment: When Queues are used to communicate between producer and consumer threads, there is often a need to determine when all of the enqueued tasks have been completed. With this small patch, determining when all work is done is as simple as adding q.task_done() to each consumer thread and q.join() to the main thread. Without the patch, the next best approach is to count the number of puts, create a second queue filled by the consumer when a task is done, and for the main thread to call successive blocking gets on the result queue until all of the puts have been accounted for: def worker(): while 1: task = tasks_in.get() do_work(task) tasks_out.put(None) tasks_in = Queue() tasks_out = Queue() for i in range(num_worker_threads): Thread(target=worker).start() n = 0 for elem in source(): n += 1 tasks_in.put(elem) # block until tasks are done for i in range(n): tasks_out.get() That approach is not complicated but it does entail more lines of code and tracking some auxiliary data. This becomes cumersome and error-prone when an app has multiple occurences of q.put() and q.get(). The patch essentially encapsulates this approach into two methods, making it effortless to use and easy to graft on to existing uses of Queue. So, the above code simplies to: def worker(): while 1: task = q.get() do_work(task) q.task_done() q = Queue() for i in range(num_worker_threads): Thread(target=worker).start() for elem in source(): q.put(elem) # block until tasks are done q.join() The put counting is automatic, there is no need for a separate queue object, the code readably expresses its intent with clarity. Also, it is easy to inpect for accuracy, each get() followed by a task_done(). The ease of inspection remains even when there are multiple gets and puts scattered through the code (a situtation which would become complicated for the two Queue approach). If accepted, will add docs with an example. Besides being a fast, lean, elegant solution, the other reason to accept the patch is that the underlying problem appears again and again, requiring some measure to invention to solve it each time. There are a number of approaches but none as simple, fast, or as broadly applicable as having the queue itself track items loaded and items completed. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455676 ] Simplify using Queues with consumer threads
Bugs item #1455676, was opened at 2006-03-21 16:36 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) >Assigned to: Tim Peters (tim_one) >Summary: Simplify using Queues with consumer threads Initial Comment: When Queues are used to communicate between producer and consumer threads, there is often a need to determine when all of the enqueued tasks have been completed. With this small patch, determining when all work is done is as simple as adding q.task_done() to each consumer thread and q.join() to the main thread. Without the patch, the next best approach is to count the number of puts, create a second queue filled by the consumer when a task is done, and for the main thread to call successive blocking gets on the result queue until all of the puts have been accounted for: def worker(): while 1: task = tasks_in.get() do_work(task) tasks_out.put(None) tasks_in = Queue() tasks_out = Queue() for i in range(num_worker_threads): Thread(target=worker).start() n = 0 for elem in source(): n += 1 tasks_in.put(elem) # block until tasks are done for i in range(n): tasks_out.get() That approach is not complicated but it does entail more lines of code and tracking some auxiliary data. This becomes cumersome and error-prone when an app has multiple occurences of q.put() and q.get(). The patch essentially encapsulates this approach into two methods, making it effortless to use and easy to graft on to existing uses of Queue. So, the above code simplies to: def worker(): while 1: task = q.get() do_work(task) q.task_done() q = Queue() for i in range(num_worker_threads): Thread(target=worker).start() for elem in source(): q.put(elem) # block until tasks are done q.join() The put counting is automatic, there is no need for a separate queue object, the code readably expresses its intent with clarity. Also, it is easy to inpect for accuracy, each get() followed by a task_done(). The ease of inspection remains even when there are multiple gets and puts scattered through the code (a situtation which would become complicated for the two Queue approach). If accepted, will add docs with an example. Besides being a fast, lean, elegant solution, the other reason to accept the patch is that the underlying problem appears again and again, requiring some measure to invention to solve it each time. There are a number of approaches but none as simple, fast, or as broadly applicable as having the queue itself track items loaded and items completed. -- >Comment By: Raymond Hettinger (rhettinger) Date: 2006-03-21 17:27 Message: Logged In: YES user_id=80475 Tim, do you have a chance to look at this? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1448060 ] gettext.py breaks on plural-forms header (PATCH)
Bugs item #1448060, was opened at 2006-03-12 00:20
Message generated for change (Comment added) made by dsegan
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1448060&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Danilo Segan (dsegan)
Assigned to: Nobody/Anonymous (nobody)
Summary: gettext.py breaks on plural-forms header (PATCH)
Initial Comment:
See http://bugzilla.gnome.org/show_bug.cgi?id=334256
The problem is a PO file like the following:
test.po:
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"#-#-#-#-# plo.po (PACKAGE VERSION) #-#-#-#-#\n"
(these kinds of entries are sometimes inserted by
msgmerge, so they're somewhat common)
Any Python program trying to access this breaks:
$ python test.py
Traceback (most recent call last):
File "test.py", line 7, in ?
gt = gettext.GNUTranslations(file)
File "/usr/lib/python2.4/gettext.py", line 177, in
__init__
self._parse(fp)
File "/usr/lib/python2.4/gettext.py", line 300, in _parse
v = v.split(';')
AttributeError: 'list' object has no attribute 'split'
test.py is simple:
#!/usr/bin/env python
import gettext
file = open("test.mo", "rb")
if file:
gt = gettext.GNUTranslations(file)
The problem is the corner-case: plural-forms precedes
this kind of comment, so "v" is split (v=v.split(";")).
In the next instance, lastk is "plural-forms", yet the
entry doesn't contain ":", so it tries to set plural
forms to v.split(";") again, which fails since v is
already a list.
The attached simple patch fixes this.
--
>Comment By: Danilo Segan (dsegan)
Date: 2006-03-22 00:28
Message:
Logged In: YES
user_id=219596
No. And based on what Bruno said, it's obviously not
supported (and since it's a GNU thingy, Bruno would probably
know best ;).
[btw, we need no plural forms in documentation at all, at
least not in static content translation; Yelp's
gnome-doc-utils stylesheets provide plural forms for where
they are appropriate]
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-03-20 08:50
Message:
Logged In: YES
user_id=21627
dsegan: Can you give a real-world (i.e. non-documentation)
example of a PO file with a multi-line plural formula?
--
Comment By: Danilo Segan (dsegan)
Date: 2006-03-20 05:07
Message:
Logged In: YES
user_id=219596
Agreed on all points, except the "summary": multi-line
plural forms are actually supported and widely used.
Anyway, gettext.py should fail gracefully in case of any
problem in the header, instead of running into exception.
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-03-19 12:51
Message:
Logged In: YES
user_id=21627
Several things seem to be going on here:
1. gettext.py is clearly wrong; it shouldn't break on that file.
2. it is trying to process multi-line fields here. So the
patch is also wrong, as it just sets k and v to None.
3. I believe that the PO file presented is also wrong. I
believe the intention of the header is that it should have
the RFC822 style syntax, which doesn't allow for # comment
lines. The tool should use a syntax like
X-Filename: plo.po; package=PACKAGE; version=VERSION;
To summarize, I think the attempt to process multi-line
fields in the header is misguided, and gettext.py should
just fetch the first line of content-type and plural-forms.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1448060&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455676 ] Simplify using Queues with consumer threads
Bugs item #1455676, was opened at 2006-03-21 16:36 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Tim Peters (tim_one) Summary: Simplify using Queues with consumer threads Initial Comment: When Queues are used to communicate between producer and consumer threads, there is often a need to determine when all of the enqueued tasks have been completed. With this small patch, determining when all work is done is as simple as adding q.task_done() to each consumer thread and q.join() to the main thread. Without the patch, the next best approach is to count the number of puts, create a second queue filled by the consumer when a task is done, and for the main thread to call successive blocking gets on the result queue until all of the puts have been accounted for: def worker(): while 1: task = tasks_in.get() do_work(task) tasks_out.put(None) tasks_in = Queue() tasks_out = Queue() for i in range(num_worker_threads): Thread(target=worker).start() n = 0 for elem in source(): n += 1 tasks_in.put(elem) # block until tasks are done for i in range(n): tasks_out.get() That approach is not complicated but it does entail more lines of code and tracking some auxiliary data. This becomes cumersome and error-prone when an app has multiple occurences of q.put() and q.get(). The patch essentially encapsulates this approach into two methods, making it effortless to use and easy to graft on to existing uses of Queue. So, the above code simplies to: def worker(): while 1: task = q.get() do_work(task) q.task_done() q = Queue() for i in range(num_worker_threads): Thread(target=worker).start() for elem in source(): q.put(elem) # block until tasks are done q.join() The put counting is automatic, there is no need for a separate queue object, the code readably expresses its intent with clarity. Also, it is easy to inpect for accuracy, each get() followed by a task_done(). The ease of inspection remains even when there are multiple gets and puts scattered through the code (a situtation which would become complicated for the two Queue approach). If accepted, will add docs with an example. Besides being a fast, lean, elegant solution, the other reason to accept the patch is that the underlying problem appears again and again, requiring some measure to invention to solve it each time. There are a number of approaches but none as simple, fast, or as broadly applicable as having the queue itself track items loaded and items completed. -- >Comment By: Tim Peters (tim_one) Date: 2006-03-21 20:42 Message: Logged In: YES user_id=31435 Yup, I'll try to make time tomorrow (can't today). _Offhand_ it sounds like a nice addition to me. -- Comment By: Raymond Hettinger (rhettinger) Date: 2006-03-21 17:27 Message: Logged In: YES user_id=80475 Tim, do you have a chance to look at this? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455676 ] Simplify using Queues with consumer threads
Bugs item #1455676, was opened at 2006-03-21 16:36 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Tim Peters (tim_one) Summary: Simplify using Queues with consumer threads Initial Comment: When Queues are used to communicate between producer and consumer threads, there is often a need to determine when all of the enqueued tasks have been completed. With this small patch, determining when all work is done is as simple as adding q.task_done() to each consumer thread and q.join() to the main thread. Without the patch, the next best approach is to count the number of puts, create a second queue filled by the consumer when a task is done, and for the main thread to call successive blocking gets on the result queue until all of the puts have been accounted for: def worker(): while 1: task = tasks_in.get() do_work(task) tasks_out.put(None) tasks_in = Queue() tasks_out = Queue() for i in range(num_worker_threads): Thread(target=worker).start() n = 0 for elem in source(): n += 1 tasks_in.put(elem) # block until tasks are done for i in range(n): tasks_out.get() That approach is not complicated but it does entail more lines of code and tracking some auxiliary data. This becomes cumersome and error-prone when an app has multiple occurences of q.put() and q.get(). The patch essentially encapsulates this approach into two methods, making it effortless to use and easy to graft on to existing uses of Queue. So, the above code simplies to: def worker(): while 1: task = q.get() do_work(task) q.task_done() q = Queue() for i in range(num_worker_threads): Thread(target=worker).start() for elem in source(): q.put(elem) # block until tasks are done q.join() The put counting is automatic, there is no need for a separate queue object, the code readably expresses its intent with clarity. Also, it is easy to inpect for accuracy, each get() followed by a task_done(). The ease of inspection remains even when there are multiple gets and puts scattered through the code (a situtation which would become complicated for the two Queue approach). If accepted, will add docs with an example. Besides being a fast, lean, elegant solution, the other reason to accept the patch is that the underlying problem appears again and again, requiring some measure to invention to solve it each time. There are a number of approaches but none as simple, fast, or as broadly applicable as having the queue itself track items loaded and items completed. -- >Comment By: Raymond Hettinger (rhettinger) Date: 2006-03-22 01:02 Message: Logged In: YES user_id=80475 Thanks. There are two particular areas for extra attention. First, should the waiter acquire/release pairs be in a try/finally (iow, is there some behavior in notify() or release() that potentially needs to be trapped)? Second, should the notify() in task_done() really be a notifyAll() (iow, does it make sense that multiple joins may be pending)? Thanks again. -- Comment By: Tim Peters (tim_one) Date: 2006-03-21 20:42 Message: Logged In: YES user_id=31435 Yup, I'll try to make time tomorrow (can't today). _Offhand_ it sounds like a nice addition to me. -- Comment By: Raymond Hettinger (rhettinger) Date: 2006-03-21 17:27 Message: Logged In: YES user_id=80475 Tim, do you have a chance to look at this? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455676&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1452697 ] broken string on mbcs
Bugs item #1452697, was opened at 2006-03-18 05:07
Message generated for change (Comment added) made by ocean-city
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1452697&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.4
>Status: Deleted
Resolution: None
Priority: 7
Submitted By: ocean-city (ocean-city)
Assigned to: M.-A. Lemburg (lemburg)
Summary: broken string on mbcs
Initial Comment:
Hello. I noticed unicode conversion from mbcs was
sometimes broken. This happened when I used
codecs.open("foo", "r", "mbcs") as iterator.
# It's OK if I use "shift_jis" or "cp932".
I'll attach the script and text file to reproduce the
problem. I'm using Win2000SP4(Japanese).
Thank you.
--
>Comment By: ocean-city (ocean-city)
Date: 2006-03-22 16:14
Message:
Logged In: YES
user_id=1200846
I'll move this to "Patches" tracker.
--
Comment By: ocean-city (ocean-city)
Date: 2006-03-19 11:08
Message:
Logged In: YES
user_id=1200846
I updated the patch. Compared to version1...
* [bug] consumed should be 0 if the length of string is 0
* [enhancement] use IsDBCSLeadByte to detect incomplete
buffer termination instead of trying MultiByteToWideChar
with MB_ERR_INVALID_CHARS. This could cause performance
hit if string contains invalid chars in early part.
--
Comment By: ocean-city (ocean-city)
Date: 2006-03-18 13:17
Message:
Logged In: YES
user_id=1200846
Probably this patch will fix the problem. (for release24-maint)
Cause: MultiByteToWideChar returns non zero value for
incomplete multibyte character. (ex: if buffer terminates
with leading byte, MultiByteToWideChar returns 1 (not 0) for
it. It should return 0, otherwise result will be broken.
Solution: Set flag MB_ERR_INVALID_CHARS to avoid incorrect
handling of trailing incomplete multibyte part. If error
occurs, removes the trailing byte and tries again.
Caution: I have not tested this so intensibly.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1452697&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1455641 ] pyport.h freebsd inconsistent
Bugs item #1455641, was opened at 2006-03-21 20:43 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455641&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Jim Jewett (jimjjewett) >Assigned to: Hye-Shik Chang (perky) Summary: pyport.h freebsd inconsistent Initial Comment: pyport.h redefines isalnum and friends #if __FreeBSD_version > 500039 but the comment just above says not to do so in FreeBSD 6 (which is now available). Add an extra guard clause? -- >Comment By: Georg Brandl (gbrandl) Date: 2006-03-22 07:54 Message: Logged In: YES user_id=849994 perky, you added that comment to pyport.h. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1455641&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
