Re: Help with implementing callback functions using ctypes
On Wed, 08 May 2013 04:19:07 -0700, jamadagni wrote: > I have the below C program spiro.c (obviously a simplified testcase) > which I compile to a sharedlib using clang -fPIC -shared -o libspiro.so > spiro.c, sudo cp to /usr/lib and am trying to call from a Python script > spiro.py using ctypes. However it would seem that the first call to the > callback results in a segfault. > # call the C function > spiro_to_bezier_strict ( src, len ( src ), bc ) This line should be: spiro_to_bezier_strict ( src, len ( src ), byref(bc) ) Without the byref(...), it will try to pass a copy of the structure rather than passing a pointer to it. -- http://mail.python.org/mailman/listinfo/python-list
Re: object.enable() anti-pattern
On Thu, 09 May 2013 05:23:59 +, Steven D'Aprano wrote: > There is no sensible use-case for creating a file without opening it. > What would be the point? Any subsequent calls to just about any method > will fail. Since you have to open the file after creating the file object > anyway, why make them two different calls? As a counterpoint, some OSes (e.g. Plan 9) allow you to get a "handle" to a file without opening it. This can then be used for deleting, renaming or stat()-type operations without either the risk of race conditions (if another process renames files between operations, the operations may be performed on different files) or the side-effects of actually opening the file (particularly for device files, e.g. opening a tape drive may rewind the tape). Python's file model doesn't allow for this, so there isn't really anything meaningful that you can do on a file object which isn't open (although these actually exist; any file object on which the .close() method has been called will be in this state). However: there are situations where it is useful to be able to separate the simple task of creating an object from more invasive actions such as system calls. Particularly in multi-threaded or real-time code (although the latter is a non-starter in Python for many other reasons). -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie question about evaluating raw_input() responses
On Thu, 23 May 2013 17:20:19 +1000, Chris Angelico wrote: > Aside: Why was PHP's /e regexp option ever implemented? Because it's a stupid idea, and that's the only requirement for a feature to be implemented in PHP. -- http://mail.python.org/mailman/listinfo/python-list
Re: Short-circuit Logic
On Sun, 26 May 2013 04:11:56 -0700, Ahmed Abdulshafy wrote:
> I'm having a hard time wrapping my head around short-circuit logic that's
> used by Python, coming from a C/C++ background; so I don't understand why
> the following condition is written this way!>
>
> if not allow_zero and abs(x) < sys.float_info.epsilon:
> print("zero is not allowed")
>
> The purpose of this snippet is to print the given line when allow_zero is
> False and x is 0.
I don't understand your confusion. The above is directly equivalent to the
following C code:
if (!allow_zero && fabs(x) < DBL_EPSILON)
printf("zero is not allowed\n");
In either case, the use of short-circuit evaluation isn't necessary here;
it would work just as well with a strict[1] "and" operator.
Short-circuit evaluation is useful if the second argument is expensive to
compute, or (more significantly) if the second argument should not be
evaluated if the first argument is false; e.g. if x is a pointer then:
if (x && *x) ...
relies upon short-circuit evaluation to avoid dereferencing a null pointer.
On an unrelated note: the use of the "epsilon" value here is
almost certainly wrong. If the intention is to determine if the result of
a calculation is zero to within the limits of floating-point accuracy,
then it should use a value which is proportional to the values used in
the calculation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Short-circuit Logic
On Mon, 27 May 2013 13:11:28 -0700, Ahmed Abdulshafy wrote: > On Sunday, May 26, 2013 2:13:47 PM UTC+2, Steven D'Aprano wrote: > >> What the above actually tests for is whether x is so small that (1.0+x) >> cannot be distinguished from 1.0, which is not the same thing. It is >> also quite arbitrary. Why 1.0? Why not (0.0001+x)? Or (0.0001+x)? >> Or (1.0+x)? > > That may be true for integers, What may be true for integers? > but for floats, testing for equality is not always precise And your point is? What Steven wrote is entirely correct: sys.float_info.epsilon is the smallest value x such that 1.0 and 1.0+x have distinct floating-point representations. It has no relevance for comparing to zero. -- http://mail.python.org/mailman/listinfo/python-list
Re: Short-circuit Logic
On Thu, 30 May 2013 12:07:40 +0300, Jussi Piitulainen wrote: > I suppose this depends on the complexity of the process and the amount > of data that produced the numbers of interest. Many individual > floating point operations are required to be within an ulp or two of > the mathematically correct result, I think, and the rounding error > when parsing a written representation of a number should be similar. Elementary operations (+, -, *, /, %, sqrt) are supposed to be within +/- 0.5 ULP (for round-to-nearest), i.e. the actual result should be the closest representable value to the exact result. Transcendental functions should ideally be within +/- 1 ULP, i.e. the actual result should be one of the two closest representable values to the exact result. Determining the closest value isn't always feasible due to the "table-maker's dilemma", i.e. the fact that regardless of the number of digits used for intermediate results, the upper and lower bounds can remain on opposite sides of the dividing line. -- http://mail.python.org/mailman/listinfo/python-list
Re: Short-circuit Logic
On Thu, 30 May 2013 19:38:31 -0400, Dennis Lee Bieber wrote: > Measuring 1 foot from the 1000 foot stake leaves you with any error > from datum to the 1000 foot, plus any error from the 1000 foot, PLUS any > azimuth error which would contribute to shortening the datum distance. First, let's ignore azimuthal error. If you measure both distances from the same origin, and you have a measurement error of 0.1% (i.e. 1/1000), then the 1000' measurement will actually be between 999' and 1001', while the 1001' measurement will be between 1000' and 1002' (to the nearest whole foot). Meaning that the distance from the 1000' stake to the 1001' stake could be anywhere between -1' and 3' (i.e. the 1001' stake could be measured as being closer than the 1000' stake). This is why technical drawings which include regularly-spaced features will normally specify the positions of features relative to their neighbours instead of (or as well as) relative to some origin. When you're dealing with relative error, the obvious question is "relative to what?". -- http://mail.python.org/mailman/listinfo/python-list
Re: Create a file in /etc/ as a non-root user
On Fri, 31 May 2013 02:12:58 -0700, BIBHU DAS wrote:
> I am a python novice;request all to kindly bear with me.
>
> fd = open('/etc/file','w')
> fd.write('jpdas')
> fd.close()
>
>
> The above snippet fails with:
> IOError: [Errno 13] Permission denied: '/etc/file'
As it should.
> Any Idea how to create a file in /etc as non-root user?
This should not be possible. The language used is irrelevant.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Changing filenames from Greeklish => Greek (subprocess complain)
On Sat, 01 Jun 2013 08:44:36 -0700, Νικόλαος Κούρας wrote: > CalledProcessError: Command '/home/nikos/public_html/cgi-bin/files.py' > returned non-zero exit status 1 > args = (1, '/home/nikos/public_html/cgi-bin/files.py') > cmd = '/home/nikos/public_html/cgi-bin/files.py' > output = b'Content-type: text/html; charset=utf-8\n\n 74: surrogates not allowed\n\n-->\n\n' > returncode = 1 > with_traceback = object> The traceback indicates that files.py terminated with a non-zero exit code, indicating an error. And that's *all* that can be determined from the information which you have posted. If you want to solve the problem, you'll need to make files.py generate useful error messages, then capture them. -- http://mail.python.org/mailman/listinfo/python-list
Re: Changing filenames from Greeklish => Greek (subprocess complain)
On Mon, 03 Jun 2013 23:28:21 -0700, nagia.retsina wrote: > I can't believe Chrome whcih by default uses utf8 chosed iso-8859-1 to > presnt the filenames. Chrome didn't choose ISO-8859-1, the server did; the HTTP response says: Content-Type: text/html;charset=ISO-8859-1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Changing filenames from Greeklish => Greek (subprocess complain)
On Tue, 04 Jun 2013 00:58:42 -0700, Νικόλαος Κούρας wrote: > Τη Τρίτη, 4 Ιουνίου 2013 10:39:08 π.μ. UTC+3, ο > χρήστης Nobody έγραψε: > >> Chrome didn't choose ISO-8859-1, the server did; the HTTP response says: >> Content-Type: text/html;charset=ISO-8859-1 > > From where do you see this $ wget -S -O - http://superhost.gr/data/apps/ --2013-06-04 14:00:10-- http://superhost.gr/data/apps/ Resolving superhost.gr... 82.211.30.133 Connecting to superhost.gr|82.211.30.133|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Server: ApacheBooster/1.6 Date: Tue, 04 Jun 2013 13:00:19 GMT Content-Type: text/html;charset=ISO-8859-1 Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding X-Cacheable: YES X-Varnish: 2000177813 Via: 1.1 varnish age: 0 X-Cache: MISS -- http://mail.python.org/mailman/listinfo/python-list
Re: how to detect the character encoding in a web page ?
On Thu, 06 Jun 2013 03:55:11 +1000, Chris Angelico wrote: > The HTTP header is completely out of band. This is the best way to > transmit encoding information. Otherwise, you assume 7-bit ASCII and start > parsing. Once you find a meta tag, you stop parsing and go back to the > top, decoding in the new way. Provided that the meta tag indicates an ASCII-compatible encoding, and you haven't encountered any decode errors due to 8-bit characters, then there's no need to go back to the top. > "ASCII-compatible" covers a huge number of > encodings, so it's not actually much of a problem to do this. With slight modifications, you can also handle some almost-ASCII-compatible encodings such as shift-JIS. Personally, I'd start by assuming ISO-8859-1, keep track of which bytes have actually been seen, and only re-start parsing from the top if the encoding change actually affects the interpretation of any of those bytes. And if the encoding isn't even remotely ASCII-compatible, you aren't going to be able to recognise the meta tag in the first place. But I don't think I've ever seen a web page encoded in UTF-16 or EBCDIC. Tools like chardet are meant for the situation where either no encoding is specified or the specified encoding can't be trusted (which is rather common; why else would web browsers have a menu to allow the user to select the encoding?). -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Game Development?
On Fri, 07 Jun 2013 08:53:03 -0700, letsplaysforu wrote: > I was planning on making a small 2D game in Python. Are there any > libraries for this? I know of: [snip] There's also Pyglet. -- http://mail.python.org/mailman/listinfo/python-list
Re: A few questiosn about encoding
On Sun, 09 Jun 2013 03:44:57 -0700, Νικόλαος Κούρας wrote: >>> Since 1 byte can hold up to 256 chars, why not utf-8 use 1-byte for >>> values up to 256? > >>Because then how do you tell when you need one byte, and when you need >>two? If you read two bytes, and see 0x4C 0xFA, does that mean two >>characters, with ordinal values 0x4C and 0xFA, or one character with >>ordinal value 0x4CFA? > > I mean utf-8 could use 1 byte for storing the 1st 256 characters. I > meant up to 256, not above 256. But then you've used up all 256 possible bytes for storing the first 256 characters, and there aren't any left for use in multi-byte sequences. You need some means to distinguish between a single-byte character and an individual byte within a multi-byte sequence. UTF-8 does that by allocating specific ranges to specific purposes. 0x00-0x7F are single-byte characters, 0x80-0xBF are continuation bytes of multi-byte sequences, 0xC0-0xFF are leading bytes of multi-byte sequences. This scheme has the advantage of making UTF-8 non-modal, i.e. if a byte is corrupted, added or removed, it will only affect the character containing that particular byte; the encoder can re-synchronise at the beginning of the following character. OTOH, with encodings such as UTF-16, UTF-32 or ISO-2022, adding or removing a byte will result in desyncronisation, with all subsequent characters being corrupted. > A surrogate pair is like itting for example Ctrl-A, which means is a > combination character that consists of 2 different characters? Is this > what a surrogate is? a pari of 2 chars? A surrogate pair is a pair of 16-bit codes used to represent a single Unicode character whose code is greater than 0x. The 2048 codepoints from 0xD800 to 0xDFFF inclusive aren't used to represent characters, but "surrogates". Unicode characters with codes in the range 0x1-0x10 are represented in UTF-16 as a pair of surrogates. First, 0x1 is subtracted from the code, giving a value in the range 0-0xF (20 bits). The top ten bits are added to 0xD800 to give a value in the range 0xD800-0xDBFF, while the bottom ten bits are added to 0xDC00 to give a value in the range 0xDC00-0xDFFF. Because the codes used for surrogates aren't valid as individual characters, scanning a string for a particular character won't accidentally match part of a multi-word character. > 'a' to be utf8 encoded needs 1 byte to be stored ? (since ordinal = 65) > 'α΄' to be utf8 encoded needs 2 bytes to be stored ? (since ordinal is > > 127 ) 'a chinese ideogramm' to be utf8 encoded needs 4 byte to be > stored ? (since ordinal > 65000 ) Most Chinese, Japanese and Korean (CJK) characters have codepoints within the BMP (i.e. <= 0x), so they only require 3 bytes in UTF-8. The codepoints above the BMP are mostly for archaic ideographs (those no longer in normal use), mathematical symbols, dead languages, etc. > The amount of bytes needed to store a character solely depends on the > character's ordinal value in the Unicode table? Yes. UTF-8 is essentially a mechanism for representing 31-bit unsigned integers such that smaller integers require fewer bytes than larger integers (subsequent revisions of Unicode cap the range of possible codepoints to 0x10, as that's all that UTF-16 can handle). -- http://mail.python.org/mailman/listinfo/python-list
RE: Popen and reading stdout in windows
On Tue, 11 Jun 2013 01:50:07 +, Joseph L. Casale wrote: > I am using Popen to run the exe with communicate() and I have sent stdout > to PIPE without luck. Just not sure what is the proper way to iterate over > the stdout as it eventually makes its way from the buffer. The proper way is: p = subprocess.Popen(..., stdout=subprocess.PIPE) for line in p.stdout: # use 'line' p.wait() If the program uses stdin, matters get more complicated. -- http://mail.python.org/mailman/listinfo/python-list
Re: A few questiosn about encoding
On Wed, 12 Jun 2013 14:23:49 +0300, Νικόλαος Κούρας wrote: > So, how many bytes does UTF-8 stored for codepoints > 127 ? U+..U+007F 1 byte U+0080..U+07FF 2 bytes U+0800..U+ 3 bytes >=U+1 4 bytes So, 1 byte for ASCII, 2 bytes for other Latin characters, Greek, Cyrillic, Arabic, and Hebrew, 3 bytes for Chinese/Japanese/Korean, 4 bytes for dead languages and mathematical symbols. The mechanism used by UTF-8 allows sequences of up to 6 bytes, for a total of 31 bits, but UTF-16 is limited to U+10 (slightly more than 20 bits). -- http://mail.python.org/mailman/listinfo/python-list
Re: A few questiosn about encoding
On Thu, 13 Jun 2013 12:01:55 +1000, Chris Angelico wrote: > On Thu, Jun 13, 2013 at 11:40 AM, Steven D'Aprano > wrote: >> The *mechanism* of UTF-8 can go up to 6 bytes (or even 7 perhaps?), but >> that's not UTF-8, that's UTF-8-plus-extra-codepoints. > > And a proper UTF-8 decoder will reject "\xC0\x80" and "\xed\xa0\x80", even > though mathematically they would translate into U+ and U+D800 > respectively. The UTF-16 *mechanism* is limited to no more than Unicode > has currently used, but I'm left wondering if that's actually the other > way around - that Unicode planes were deemed to stop at the point where > UTF-16 can't encode any more. Indeed. 5-byte and 6-byte sequences were originally part of the UTF-8 specification, allowing for 31 bits. Later revisions of the standard imposed the UTF-16 limit on Unicode as a whole. -- http://mail.python.org/mailman/listinfo/python-list
Re: "Don't rebind built-in names*" - it confuses readers
On Thu, 13 Jun 2013 01:23:27 +, Steven D'Aprano wrote: > Python does have a globally-global namespace. It is called "builtins", and > you're not supposed to touch it. Of course, being Python, you can if you > want, but if you do, you are responsible for whatever toes you shoot off. > > Modifying builtins will effect *all modules*. That's normally too much, > although it can very, very, very occasionally be useful. For a specific example, gettext.install() adds the _() function to the builtins namespace. -- http://mail.python.org/mailman/listinfo/python-list
Re: Eval of expr with 'or' and 'and' within
On Fri, 14 Jun 2013 18:16:05 +0300, Nick the Gr33k wrote: > My question is why the expr (name and month and year) result in the > value of the last variable whic is variable year? For much the same reason that an OR expression returns the first true value. "or" and "and" only evaluate as many arguments are required in order to determine the correct result (aka "short-circuit evaluation"). If the first argument of "or" is true, or the first argument of "and" is false, the second argument isn't evaluated (this is important if evaluation can have side effects). The operators can be expressed as: True or X = True False or X = X False and X = False True and X = X Note that in the short-circuit case, the result has the same sense (true or false) as the first argument, while in the other case the result has the same sense as the second argument. Python implements these operators by returning the actual value which determined the result of the expression rather than simply True or False. If the result is known after evaluating the first argument, the first argument is returned. If it has to evaluate the second argument, the second argument is returned (by that point it has already forgotten the value of the first argument). -- http://mail.python.org/mailman/listinfo/python-list
Re: Eval of expr with 'or' and 'and' within
On Fri, 14 Jun 2013 19:30:27 +, Grant Edwards wrote: > 2. Returning one the objects that result from the evaluation of the > operands instead of returning True or False. > > This is what seems to be confusing him. This is much less common > than short-circuit evaluation. FWIW, Lisp also does this. But Lisp is slightly simpler as the only false value is "nil", while everything else is true (including integer zero). Although Python's any() and all() (which are closer to Lisp's "and" and "or" insofar as they all work with any number of values, including zero) always return True or False rather than the final value. -- http://mail.python.org/mailman/listinfo/python-list
Re: Eval of expr with 'or' and 'and' within
On Fri, 14 Jun 2013 16:49:11 +, Steven D'Aprano wrote: > Unlike Javascript though, Python's idea of truthy and falsey is actually > quite consistent: Beyond that, if a user-defined type implements a __nonzero__() method then it determines whether an instance is true or false. If it implements a __len__() method, then an instance is true if it has a non-zero length. -- http://mail.python.org/mailman/listinfo/python-list
Re: Eval of expr with 'or' and 'and' within
On Sat, 15 Jun 2013 03:56:28 +1000, Chris Angelico wrote:
> With a few random oddities:
>
bool(float("nan"))
> True
>
> I somehow expected NaN to be false. Maybe that's just my expectations
> that are wrong, though.
In general, you should expect the behaviour of NaN to be the opposite of
what you expect.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Popen in Python3
On Wed, 19 Jun 2013 23:03:05 +, Joseph L. Casale wrote: > I am trying to invoke a binary that requires dll's in two places all of > which are included in the path env variable in windows. When running this > binary with popen it can not find either, passing env=os.environ to open > made no difference. > > Anyone know what might cause this or how to work around this? Do any of the DLLs have dependencies of their own which need to be found? Do DLLs with the same name exist in directories which are searched before %PATH%? Directories listed in %PATH% are searched after all other options have failed. The relevant MSDN page appears to be: http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx -- http://mail.python.org/mailman/listinfo/python-list
Re: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte
On Thu, 04 Jul 2013 13:38:09 +0300, Νίκος wrote: > So you are also suggesting that what gesthostbyaddr() returns is not > utf-8 encoded too? The gethostbyaddr() OS function returns a byte string with no specified encoding. Python 3 will doubtless try to decode that to a character string using some (probably unspecified) encoding. Names obtained from DNS should consist entirely of ASCII characters (gethostbyname shouldn't attempt to decode internationalised names which use IDN, it should return the raw data). Names obtained by other means (e.g. /etc/hosts or Active Directory) could contain anything, but if you use non-ASCII hostnames you're asking for trouble. -- http://mail.python.org/mailman/listinfo/python-list
Re: Concurrent writes to the same file
On Wed, 10 Jul 2013 22:57:09 -0600, Jason Friedman wrote: > Other than using a database, what are my options for allowing two processes > to edit the same file at the same time? When I say same time, I can accept > delays. What do you mean by "edit"? Overwriting bytes and appending bytes are simple enough, but inserting or deleting bytes such that subsequent bytes change their offsets is harder. > I considered lock files, Well, you shouldn't have, unless you're targeting a platform which doesn't support file locks (are there any left?). > but I cannot conceive of how I avoid race conditions. By using locks. E.g. fcntl.lockf() or msvcrt.locking(). -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I get the OS System Font Directory(Cross-Platform) in python?
On Thu, 11 Jul 2013 08:32:34 -0700, Metallicow wrote: > How do I get the OS System Font Directory(Cross-Platform) in python? What makes you think the system *has* a system font directory? In the traditional X11 model, the only program which needs fonts is the X server, and that can be configured to get its fonts from a font server rather than from a local directory. Even if it doesn't use a font server, the font directory will be on the system running the X server, not the one running the client. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple Python script as SMTP server for outgoing e-mails?
On Mon, 22 Jul 2013 14:19:57 +0200, Gilles wrote: > Incidently, how do ISP MTAs find whether the remote MTA is legit or > running on some regular user's computer? Look up the IP address in a database. If they don't have a database, perform a reverse DNS lookup and reject anything which looks like a typical auto-generated name for a consumer DSL/cable connection. FWIW, I've been running sendmail on my home system (ADSL with static IP) for years, and have had very few problems with mail being rejected. -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginner. 2d rotation gives unexpected results.
On Tue, 23 Jul 2013 15:11:43 +0200, Peter Otten wrote: > The conversion to int introduces a rounding error that accumulates over > time. Most floating point calculations introduce a rounding error. If the calculations are iterated, the error will accumulate. In general, you want to avoid accumulating entire transformations. E.g. if you want a spinning object, maintain the cumulative rotation angle and rotate the original points each frame. If you must accumulate transformations, you need to actively work to maintain any desired invariants. E.g. if a transformation is supposed to be orthonormal (all axes perpendicular and of unit length), you should renormalise it periodically, otherwise the lengths and angles will change over time. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to interact with Windows cmd?
On Wed, 04 Jul 2012 20:10:47 -0700, self.python wrote: > 2. after this, I typed like "cd .." but I/O is already closed so I > can't do another things.. Don't use .communicate() if you want to keep the child process alive. Write to p.stdin and read p.stdout and p.stderr. In general, you'll need to use a separate thread for each stream, otherwise you risk deadlock. Look at the source code for the .communicate() method for an example. Also, unless you specifically need stdout and stderr to be separated, use "stderr=subprocess.STDOUT". The problem with separating them is that there's no way to determine the order in which data was written, so there's no way to reconstruct the output as it would have appeared on the console. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
On Sun, 08 Jul 2012 22:57:56 +0200, Laszlo Nagy wrote: > Yes, this is much better. Almost perfect. Don't forget to consult your > system documentation, and check if the rename operation is atomic or not. > (Most probably it will only be atomic if the original and the renamed file > are on the same physical partition and/or mount point). On Unix, rename() is always atomic, and requires that source and destination are on the same partition (if you want to "move" a file across partitions, you have to copy it then delete the original). > But even if the rename operation is atomic, there is still a race > condition. Your program can be terminated after the original status file > has been deleted, and before the temp file was renamed. In this case, you > will be missing the status file (although your program already did > something just it could not write out the new status). In the event of abnormal termination, losing some data is to be expected. The idea is to only lose the most recent data while keeping the old copy, rather than losing everything. Writing to a temp file then rename()ing achieves that. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to print stdout before writing stdin using subprocess module in Python?
On Mon, 23 Jul 2012 06:01:23 -0700, Sarbjit singh wrote:
> proc = subprocess.Popen("cp -i a.txt b.txt", shell=True,
> stdin=subprocess.PIPE, stdout=subprocess.PIPE,
> stderr=subprocess.STDOUT,)
> stdout_val, stderr_val = proc.communicate()
> print stdout_val b.txt?
>
> proc.communicate("y")
>
> Now in this example if i read only stdout/stderr and prints it, later on
> if i try to write "y" or "n" based on user's input, i got an error that
> channel is closed. Can some one please help me on achieving this
> behavior in python such that i can print stdout first, then should take
> user input and write stdin later on.
You can't use .communicate() if you want an interact with the child
process. Any text given as an argument is sent to the child, then the
child's stdin is closed. Then it reads the child's stdout and/or stderr
until the child terminates.
If you want to interact with the child, you need to write to proc.stdin
and read from proc.stdout and/or proc.stderr. And you need to do this
asynchronously, i.e. you either need to use non-blocking I/O or multiple
threads, otherwise you risk deadlock. Look at the source code for the
.communicate() method in subprocess.py for examples (the Unix version uses
non-blocking I/O, the Windows version uses threads).
Even that may not be enough, as the child process may behave differently
if std{in,out,err} are pipes rather than a tty. Typically, stdin and
stdout are line-buffered when associated with a tty but block-buffered
otherwise (e.g. when associated with a pipe).
If pipes don't work, the only solution may be to run the child process on
a pseudo-tty (pty); see the pty module or the os.openpty() function.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Calling Values
On Fri, 03 Aug 2012 04:49:46 -0700, Subhabrata wrote: > I am trying to call the values of one function in the another function > in the following way: > def func1(): > num1=10 > num2=20 > print "The Second Number is:",num2 > return > > def func2(): > num3=num1+num2 > num4=num3+num1 A function's local variables only exist while that function is being executed[1]. It's meaningless to try to access them from outside the function. [1] There is an exception (closures), but it doesn't have any bearing on this particular problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: Deciding inheritance at instantiation?
On Fri, 03 Aug 2012 13:48:08 -0700, Tobiah wrote:
> I have a bunch of classes from another library (the html helpers
> from web2py). There are certain methods that I'd like to add to
> every one of them. So I'd like to put those methods in a class,
> and pass the parent at the time of instantiation. Web2py has
> a FORM class for instance. I'd like to go:
>
> my_element = html_factory(FORM)
>
> Then my_element would be an instance of my class, and also
> a child of FORM.
You can use type() to create classes dynamically. E.g.:
class my_base_class(object):
# extra methods
subclasses = {}
def html_factory(cls, *args, **kwargs):
name = "my_" + cls.__name__
if name not in subclasses:
subclasses[name] = type(name, (cls, my_base_class), {})
return subclasses[name](*args, **kwargs)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pickle file and send via socket
On Mon, 06 Aug 2012 06:32:13 -0700, S.B wrote: > Does anyone know if it's possible to pickle and un-pickle a file across > a network socket. i.e: First host pickles a file object and writes the > pickled file object to a client socket. Second host reads the pickled > file object from the server socket and un-pickles it. Yes, provided that the socket is a stream socket (e.g. TCP, Unix), not a datagram socket (e.g. UDP). > Can anyone provide a simple code example of the client and server sides? Pickling via a socket is no different to pickling via a file. For a socket, the .makefile() method will return a suitable file object. -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for a neat solution to a nested loop problem
On Mon, 06 Aug 2012 17:52:31 +0200, Tom P wrote: > consider a nested loop algorithm - > > for i in range(100): > for j in range(100): > do_something(i,j) > > Now, suppose I don't want to use i = 0 and j = 0 as initial values, but > some other values i = N and j = M, and I want to iterate through all > 10,000 values in sequence - is there a neat python-like way to this? for i in range(N,N+100): for j in range(M,M+100): do_something(i,j) Or did you mean something else? Alternatively: import itertools for i, j in itertools.product(range(N,N+100),range(M,M+100)): do_something(i,j) This can be preferable to deeply-nested loops. Also: in 2.x, use xrange() in preference to range(). -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for a neat solution to a nested loop problem
On Mon, 06 Aug 2012 21:02:33 -0700, Larry Hudson wrote: >> for i in range(N,N+100): >> for j in range(M,M+100): >> do_something(i % 100 ,j % 100) >> >> Emile > > How about... > > for i in range(100): > for j in range(100): > do_something((i + N) % 100, (j + M) % 100) Both of these approaches move the modifications to the sequence into the body of the loop. It may be preferable to iterate over the desired sequence directly. E.g. for i in ((N + ii) % 100 for ii in xrange(100)): for j in ((M + jj) % 100 for jj in xrange(100)): do_something(i, j) -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python remember the initial directory?
On Sun, 19 Aug 2012 14:01:15 -0700, Giacomo Alzetta wrote: > You can obtain the working directory with os.getcwd(). Maybe. On Unix, it's possible that the current directory no longer has a pathname. As with files, directories can be "deleted" (i.e. unlinked) even while they're still in use. Similarly, a directory can be renamed while it's in use, so the current directory's pathname may have changed even while the current directory itself hasn't. -- http://mail.python.org/mailman/listinfo/python-list
Re: "convert" string to bytes without changing data (encoding)
On Wed, 29 Aug 2012 19:39:15 -0400, Piet van Oostrum wrote: >> Reading from stdin/a file gets you bytes, and not a string, because >> Python cannot automagically guess what format the input is in. >> > Huh? Oh, it can certainly guess (in the absence of any other information, it uses the current locale). Whether or not that guess is correct is a different matter. Realistically, if you want sensible behaviour from Python 3.x, you need to use an ISO-8859-1 locale. That ensures that conversion between str and bytes will never fail, and an str-bytes-str or bytes-str-bytes round-trip will pass data through unmangled. -- http://mail.python.org/mailman/listinfo/python-list
Re: Moving folders with content
On Sat, 15 Sep 2012 04:36:00 +, jyoung79 wrote: > I am working in both OS X Snow Leopard and Lion (10.6.8 and 10.7.4). > I'm simply wanting to move folders (with their content) from various > servers to the hard drive and then back to different directories on the > servers. > > I want to be careful not to remove any metadata or resource forks from > the files in the directories. I did a bit of researching on shutil, and > looks like it is similar to using "cp -p" and copystat(), which I believe > will keep the resource fork, etc. I don't think so. The shutil documentation says: Warning Even the higher-level file copying functions (copy(), copy2()) can’t copy all file metadata. On POSIX platforms, this means that file owner and group are lost as well as ACLs. On Mac OS, the resource fork and other metadata are not used. This means that resources will be lost and file type and creator codes will not be correct. On Windows, file owners, ACLs and alternate data streams are not copied. The macostools module has functions which can copy the resource fork, but they aren't available in 64-bit builds and have been removed in Python 3.0. -- http://mail.python.org/mailman/listinfo/python-list
Re: Moving folders with content
On Sun, 16 Sep 2012 12:40:18 +0200, Hans Mulder wrote: > But you should get into the habit of using shell=False whenever > possible, because it is much easier to get it right. More accurately, you should get into the habit of passing a list as the first argument, rather than a string. On Unix-like systems (including Mac OS X), this effectively requires shell=False. Passing a list with shell=True has behaviour which is well-defined, but rarely useful (the first element of the list will be executed as a shell command, the remaining elements will be available via the shell variables $1, $2, etc within that command). On Windows, the list is converted to a command string using the same quoting rules regardless of the value of the shell= parameter. The difference is that shell=False requires the "executable" to actually be a binary executable, while shell=True allows it to be some other type of file (e.g. a batch file, Python script, etc). -- http://mail.python.org/mailman/listinfo/python-list
Re: portable way of locating an executable (like which)
On Thu, 20 Sep 2012 23:06:46 +0200, Gelonida N wrote: > I'd like to implement the equivalent functionality of the unix command > /usr/bin/which > > The function should work under Linux and under windows. Note that "which" attempts to emulate the behaviour of execvp() etc. The exec(3) manpage will explain the precise algorithm used (e.g. they skip files for which the process lacks execute permission). Also, note that the shell has built-in commands, functions, and aliases in addition to programs. The "type" built-in command performs a similar function to "which" but using the shell's semantics. On some systems, the default configuration may alias "which" to "type". On Windows, there's a host of different "execute program" interface, all with subtly different semantics: which extensions they will run, which extensions can be omitted, which paths are used (e.g. %PATH%, paths from the registry, current directory). -- http://mail.python.org/mailman/listinfo/python-list
Re: Exact integer-valued floats
On Fri, 21 Sep 2012 17:29:13 +, Steven D'Aprano wrote: > The question is, what is the largest integer number N such that every > whole number between -N and N inclusive can be represented as a float? > > If my tests are correct, that value is 9007199254740992.0 = 2**53. > > Have I got this right? Is there a way to work out the gap between one > float and the next? CPython's "float" type uses C's "double". For a system where C's "double" is IEEE-754 double precision, N=2**53 is the correct answer. An IEEE-754 double precision value consists of a 53-bit integer whose first bit is a "1", multiplied or divided by a power of two. http://en.wikipedia.org/wiki/IEEE_754-1985 The largest 53-bit integer is 2**53-1. 2**53 can be represented as 2**52 * 2**1. 2**53+1 cannot be represented in this form. 2**53+2 can be represented as (2**52+1) * 2**1. For values x where 2**52 <= x < 2**53, the the interval between representable values (aka Unit in the Last Place or ULP) is 1.0. For 2**51 <= x < 2**52, the ULP is 0.5. For 2**53 <= x < 2**54, the ULP is 2.0. And so on. -- http://mail.python.org/mailman/listinfo/python-list
Re: Exact integer-valued floats
On Fri, 21 Sep 2012 15:23:41 -0700, Paul Rubin wrote: > Steven D'Aprano writes: >> Have I got this right? Is there a way to work out the gap between one >> float and the next? > > Yes, 53-bit mantissa as people have mentioned. That tells you what ints > can be exactly represented. But, arithmetic in some situations can have a > 1-ulp error. So I wonder if it's possible that if n is large enough, you > might have something like n+1==n even if the integers n and n+1 have > distinct floating point representations. Not for IEEE-754. Or for any sane implementation, for that matter. OTOH, you can potentially get n != n due to the use of extended precision for intermediate results. For IEEE-754, addition, subtraction, multiplication, division, remainder and square root are "exact" in the sense that the result is as if the arithmetic had been performed with an infinite number of bits then rounded afterwards. For round-to-nearest, the result will be the closest representable value to the exact value. Transcendental functions suffer from the "table-maker's dilemma", and the result will be one of the two closest representable values to the exact value, but not necessarily *the* closest. -- http://mail.python.org/mailman/listinfo/python-list
Re: what is the difference between st_ctime and st_mtime one is the time of last change and the other is the time of last modification, but i can not understand what is the difference between 'change'
On Fri, 28 Sep 2012 06:12:35 -0700, 陈伟 wrote: > what is the difference between st_ctime and st_mtime one is the time of > last change and the other is the time of last modification, but i can > not understand what is the difference between 'change' and 'modification'. st_mtime is updated when the file's contents change. st_ctime is updated when the file's metadata (owner, group, permissions, link count, etc) changes. -- http://mail.python.org/mailman/listinfo/python-list
Re: what is the difference between st_ctime and st_mtime one is the time of last change and the other is the time of last modification, but i can not understand what is the difference between 'change'
On Fri, 28 Sep 2012 11:48:23 -0600, Kristen J. Webb wrote: > NOTE: I am a C programmer and new to python, so can anyone comment > on what the st_ctime value is when os.stat() is called on Windows? The documentation[1] says: st_ctime - platform dependent; time of most recent metadata change on Unix, or the time of creation on Windows) [1] http://docs.python.org/library/os#os.stat -- http://mail.python.org/mailman/listinfo/python-list
Re: Insert item before each element of a list
On Mon, 08 Oct 2012 12:28:43 -0700, mooremathewl wrote:
import itertools
x = [1, 2, 3]
y = list(itertools.chain.from_iterable(('insertme', x[i]) for i in
range(len(x y
> ['insertme', 1, 'insertme', 2, 'insertme', 3]
>>> [i for j in [1,2,3] for i in ('insertme', j)]
['insertme', 1, 'insertme', 2, 'insertme', 3]
--
http://mail.python.org/mailman/listinfo/python-list
Re: Checking for dlls in ctypes
On Fri, 12 Oct 2012 12:28:17 -0400, Dave Angel wrote: > Using bare excepts is almost never a good idea. If it "works" you get no > clues what went wrong. For example, a typo in source code can trigger a > bare exception, as can a user typing Ctrl-C. So when you're using bare > excepts, you have robbed the user of any way to terminate the program. If you want to catch any error, use "except StandardError:". That covers all errors but not things like KeyboardInterrupt (Ctrl-C) or SystemExit (sys.exit()). In situations such as this, where you try multiple candidates until one succeeds, there's no reason to be any more specific than that. In any case, Python's lack of formal interfaces makes it hard to reliably be more specific. However: you should bear in mind that loading the wrong DLL may just result in an OS-level exception (e.g. segfault), which can't be caught. It's preferable to allow the DLL to be explicitly selected e.g. in a configuration file. -- http://mail.python.org/mailman/listinfo/python-list
Re: overriding equals operation
On Tue, 16 Oct 2012 08:51:46 -0500, Pradipto Banerjee wrote:
> I am trying to define class, where if I use a statement a = b, then
> instead of "a" pointing to the same instance as "b", it should point to a
> copy of "b", but I can't get it right.
It cannot be done.
Name binding ("variable = value") is a language primitive; it is not
delegated to the object on the LHS (if there even is one; the name doesn't
have to exist at the point that the statement is executed).
You'll have to change the syntax to something which is delegated to
objects, e.g. "a[:] = b" will call "a.__setitem__(slice(None), b)"
--
http://mail.python.org/mailman/listinfo/python-list
Re: Watching output and put back in background
On Thu, 18 Oct 2012 14:05:58 +0100, andrea crotti wrote: > Maybe a solution would be to redirect the stderr to file and watch that > instead.. > > Or otherwise I could use a thread for each shell command, but I would like > to avoid head-aches with possible race-conditions.. If you're running multiple concurrent commands, and you have redirected their output streams to pipes, something needs to keep reading those pipes if you don't want the commands to hang. Rather than having a separate thread for each process, you could have a single thread which manages all "background" processes using select(), poll() or non-blocking I/O, but that's easier to do on Unix than on Windows (Popen.communicate() uses a pair of threads on Windows). Redirecting output to files then reading them upon completion is the simplest solution, but you can't easily monitor progress that way (there's no easy way to get notification when more output is written). -- http://mail.python.org/mailman/listinfo/python-list
Re: locking files on Linux
On Thu, 18 Oct 2012 14:44:27 +0100, andrea crotti wrote: > Uhh I see thanks, I guess I'll use the good-old .lock file (even if it > might have some problems too). In which case, you don't see. A lock file is also advisory, i.e. it only affects applications which explicitly check for a lock file. Historically, the advantage of lock files was that they worked on NFS implementations which didn't implement locking (it's a long-standing Unix joke that "NFS" stands for "Not a FileSystem", because it failed to conform to established filesystem semantics). Nowadays, NFS implementations which don't support locking are sufficiently rare that they can safely be ignored. So lock files don't offer any advantages, and one fairly obvious disadvantage (the possibility of a "stale" lock file if the program terminates unexpectedly without removing the lock file). For any form of advisory locking, the one thing which matters is that all progams which access the file agree on the mechanism used, i.e. whether to use lockf(), fcntl(), flock() (locks created by one mechanism may or may not be recognised by the others), or lock files, and in the case of lock files, the naming convention which is used. If the file is specific to a particular program, and you just need to protect against multiple instances of that program, you can use whichever mechanism you wish, and would be strongly advised to use kernel locks (fcntl() is the most portable, followed by lockf(); flock() is a BSD-ism). -- http://mail.python.org/mailman/listinfo/python-list
Re: Preserving unicode filename encoding
On Sat, 20 Oct 2012 13:43:16 -0700, Julien Phalip wrote:
> I've noticed that the encoding of non-ascii filenames can be inconsistent
> between platforms when using the built-in open() function to create files.
>
> For example, on a Ubuntu 10.04.4 LTS box, the character u'ş' (u'\u015f')
> gets encoded as u'ş' (u's\u0327'). Note how the two characters look
> exactly the same but are encoded differently. The original character uses
> only one code (u'\u015f'), but the resulting character that is saved on
> the file system will be made of a combination of two codes: the letter 's'
> followed by a diacritical cedilla (u's\u0327'). (You can learn more about
> diacritics in [1]). On the Mac, however, the original encoding is always
> preserved.
>
> This issue was also discussed in a blog post by Ned Batchelder [2].
You are conflating two distinct issues here: representation (how a
given "character" is represented as a Unicode string) and encoding (how a
given Unicode string is represented as a byte string).
E.g. you state:
> For example, on a Ubuntu 10.04.4 LTS box, the character u'ş' (u'\u015f')
> gets encoded as u'ş' (u's\u0327').
which is incorrect.
The latter isn't an "encoding" of the former. They are alternate Unicode
representations of the same character. The former uses a pre-composed
character (LATIN SMALL LETTER S WITH CEDILLA) while the latter uses a
letter 's' with a combining accent (COMBINING CEDILLA).
Unlike the Mac, neither Unix nor Windows will automatically normalise
Unicode strings. A Unix filename is a sequence of bytes, nothing more and
nothing less. This is part of the reason why Unix filenames are case
sensitive: case applies to characters, and the kernel doesn't know which
characters, if any, those bytes are meant to represent.
Python will convert a Unicode string to a sequence of bytes using the
filesystem encoding. If the encoding is UTF-8, then u'\u015f'
will be encoded as b'\xc5\x9f', while u's\u0327' will be encoded as
b's\xcc\xa7'.
If you want to convert a Unicode string to a given normalisation, you can
use unicodedata.normalize(), e.g.:
> unicodedata.normalize('NFC', 's\u0327')
'\u015f'
> unicodedata.normalize('NFD', '\u015f')
's\u0327'
However: if you want to access an existing file, you must use the filename
as it appears on disc. On Unix and Windows, it's perfectly possible to
have two files named e.g. '\u015f.txt' and 's\u0327.txt' in the same
directory. Which one gets opened depends upon the exact sequence of
Unicode codepoints passed to open().
The situation is different on the Mac, where system libraries
automatically impose a specific representation on filenames, and will
normalise Unicode strings to that representation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: a.index(float('nan')) fails
On Thu, 25 Oct 2012 22:04:52 -0400, Terry Reedy wrote: > Containment of nan in collection is tested by is, not ==. AFAICT, it isn't specific to NaN. The test used by .index() and "in" appears to be equivalent to: def equal(a, b): return a is b or a == b IOW, it always checks for object identity before equality. Replacing NaN with an instance of a user-defined class with a non-reflexive __eq__() method supports this: > class Foo(object): = def __eq__(self, other): = return False = > a = Foo() > b = Foo() > a in [1,2,a,3,4] True > b in [1,2,a,3,4] False > [1,2,a,3,4].index(a) 2 > [1,2,a,3,4].index(b) Traceback (most recent call last): File "", line 1, in ValueError: <__main__.Foo object at 0x7fa7055b0550> is not in list -- http://mail.python.org/mailman/listinfo/python-list
Re: a.index(float('nan')) fails
On Sat, 27 Oct 2012 08:56:16 +0200, Thomas Rachel wrote: > Am 27.10.2012 06:48 schrieb Dennis Lee Bieber: > >> I don't know about the more modern calculators, but at least up >> through my HP-41CX, HP calculators didn't do (binary) "floating >> point"... They did a form of BCD with a fixed number of significant >> /decimal/ digits > > Then, what about sqrt(x)**2 or arcsin(sin(x))? Did that always return > the original x? I'd be impressed if it managed the latter, i.e. arcsin(sin(0))==0 while arcsin(sin(pi))==pi ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes free memory which is allocated in C DLL
On Sat, 27 Oct 2012 07:42:01 -0700, zlchen.ken wrote: > I have a DLL which written in C language, one of the function is to > allocate a structure, fill the members and then return the pointer of > the structure. > > After Python called this function, and done with the returned structure, > I would like to free the returned structure. How can I achieve this ? > Basically, I tried that I wrote a corresponding free interface in the > DLL, it works, but calling the libc.free in Python doesn't work. > > my_dll.return_structure_ptr.restypes = POINTER(Dummy) res_ptr = > my_dll.return_structure_ptr() windll.msvcrt.free(res_ptr) < doesn't > work, memory violation my_dll.free_dummy_struture(res_ptr) <== This > works. On Windows, a process may have multiple versions of the MSVCRT DLL (which provides malloc/free). If an executable or DLL is linked against multiple DLLs, each DLL could be using a different version of MSVCRT. Different versions of MSVCRT may have separate heaps, so anything which is allocated with malloc() (or calloc() or realloc()) from a specific version of MSVCRT must be passed to free() from the same version of MSVCRT. windll.msvcrt refers to the version of MSVCRT against which the Python DLL is linked, which isn't necessarily the version against which my_dll is linked. If a function in a DLL returns a pointer to memory which it allocated with malloc(), that DLL must also provide a function which can be used to free that memory. It can't leave it to the application (or higher-level DLL) to call free(), because the application may not be using the same version of MSVCRT as the DLL. -- http://mail.python.org/mailman/listinfo/python-list
Re: Avoiding defunct processes
On Thu, 01 Nov 2012 19:16:17 -0700, Richard wrote: > I create child processes with subprocess.Popen(). > Then I either wait for them to finish or kill them. > Either way these processes end up as defunct until the parent process > completes: > $ ps e > 6851 pts/5Z+ 1:29 [python] You need to either call the .wait() method, or keep calling the .poll() method until the .returncode attribute is not None. -- http://mail.python.org/mailman/listinfo/python-list
Re: Subprocess puzzle and two questions
On Wed, 14 Nov 2012 20:49:19 -0500, Roy Smith wrote: >> I'm slightly surprised that there's no way with the Python stdlib to >> point a DNS query at a specific server > > Me too, including the "only slightly" part. The normal high-level C > resolver routines (getaddrinfo/getnameinfo, or even the old > gethostbyname series), don't expose any way to do that. That's because the high-level routines aren't tied to DNS. gethostbyname() and getaddrinfo() use the NSS (name-service switch) mechanism, which is configured via /etc/nsswitch.conf. Depending upon configuration, hostnames can be looked up via a plain text file (/etc/hosts), Berkeley DB files, DNS, NIS, NIS+, LDAP, WINS, etc. DNS is just one particular back-end, which may or may not be used on any given system. If you specifically want to perform DNS queries, you have to use a DNS-specific interface (e.g. the res_* functions described in the resolver(3) manpage), or raw sockets, rather than a high-level interface such as gethostbyname() or getaddrinfo(). -- http://mail.python.org/mailman/listinfo/python-list
Re: Subprocess puzzle and two questions
On Thu, 15 Nov 2012 20:07:38 -0500, Roy Smith wrote: >>> gethostbyname() and getaddrinfo() use the NSS (name-service switch) >> mechanism, which is configured via /etc/nsswitch.conf. Depending upon >> configuration, hostnames can be looked up via a plain text file >> (/etc/hosts), Berkeley DB files, DNS, NIS, NIS+, LDAP, WINS, etc. > > Gethostbyname() long predates NSS. Before NSS there was host.conf, which provided similar functionality except that the set of mechanisms was fixed (they were built into libc rather than being dynamically-loaded libraries) and it only applied to hostnames (NSS is also used for getpwent(), getprotoent(), etc). > For that matter, I think it even predates DNS (i.e. back to the days > when /etc/hosts was the *only* way to look up a hostname). > > But, that's a nit. Indeed; the main point is that gethostbyname() has never been specific to DNS. -- http://mail.python.org/mailman/listinfo/python-list
Re: latin1 and cp1252 inconsistent?
On Fri, 16 Nov 2012 13:44:03 -0800, buck wrote: > When a user agent [browser] would otherwise use a character encoding given > in the first column [ISO-8859-1, aka latin1] of the following table to > either convert content to Unicode characters or convert Unicode characters > to bytes, it must instead use the encoding given in the cell in the second > column of the same row [windows-1252, aka cp1252]. It goes on to say: The requirement to treat certain encodings as other encodings according to the table above is a willful violation of the W3C Character Model specification, motivated by a desire for compatibility with legacy content. [CHARMOD] IOW: Microsoft's "embrace, extend, extinguish" strategy has been too successful and now we have to deal with it. If HTML content is tagged as using ISO-8859-1, it's more likely that it's actually Windows-1252 content generated by someone who doesn't know the difference. Given that the only differences between the two are for code points which are in the C1 range (0x80-0x9F), which should never occur in HTML, parsing ISO-8859-1 as Windows-1252 should be harmless. If you need to support either, you can parse it as ISO-8859-1 then explicitly convert C1 codes to their Windows-1252 equivalents as a post-processing step, e.g. using the .translate() method. -- http://mail.python.org/mailman/listinfo/python-list
Re: latin1 and cp1252 inconsistent?
On Sat, 17 Nov 2012 08:56:46 -0800, buck wrote: >> Given that the only differences between the two are for code points >> which are in the C1 range (0x80-0x9F), which should never occur in HTML, >> parsing ISO-8859-1 as Windows-1252 should be harmless. > > "should" is a wish. The reality is that documents (and especially URLs) > exist that can be decoded with latin1, but will backtrace with cp1252. In which case, they're probably neither ISO-8859-1 nor Windows-1252, but some other (unknown) encoding which has acquired the ISO-8859-1 label "by default". In that situation, if you still need to know the encoding, you need to resort to heuristics such as those employed by the chardet library. -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting a seeded value from a list
On Mon, 19 Nov 2012 21:45:55 -0800, frednotbob wrote: > What I'm trying to do is set a persistent state for the levels generated > by make_map(), so the player can move between floors without generating a > totally new randomized floor each time. You need to distinguish between immutable data (e.g. walls) and mutable data (monsters, treasure). The former can be generated from the seed each time you enter the level; the latter must be generated the first time then stored. If the number of levels is reasonable, you can use a PRNG to generate a list of random numbers at game start, which are the seeds used to generate each level. Alternatively, you can generate a single random "game id" at game start then generate the seed for each level from a hash of the level number and the game id. When it comes to random level generation, a single sequential PRNG isn't always the best option, as it requires that you always use all of the generated numbers in the same order, even if you only want a subset of the level's data. This is particularly significant for large levels. It's sometimes better to use a hierarchy of PRNGs, where the values generated by higher-level PRNGs are used as seeds for the lower-level PRNGs. This way, if you need to generate a subset of the data, you only need to run the PRNGs for the specific subset, and their ancestors. E.g. if you want a 256x256 grid of random numbers, you could just generate 65536 random numbers and use them in top-to-bottom, left-to-right order. But if you only want the bottom-right tile, you still need to generate all of the preceding numbers to ensure that you get the correct value. A hierarchical approach would generate 4 numbers which are used as the seeds for the 4 128x128 quarters. Each quarter uses its seed to generate 4 more numbers for the 4 64x64 quarters. And so on until you get down to a 2x2 region, at which point the 4 numbers generated are the actual values used. This way, calculating a single cell only requires 4*log(n) (e.g. 4*8=32) values, while generating the entire grid only requires 33% more numbers for the higher levels (1 + 1/4 + 1/16 + 1/64 + ... = 1+1/3). -- http://mail.python.org/mailman/listinfo/python-list
Re: Encoding conundrum
On Wed, 21 Nov 2012 03:24:01 -0800, danielk wrote: >> >>> import sys >> >>> sys.stdout.encoding >> 'cp437' > > Hmmm. So THAT'S why I am only able to use 'cp437'. I had (mistakenly) > thought that I could just indicate whatever encoding I wanted, as long as > the codec supported it. sys.stdout.encoding determines how Python converts unicode characters written to sys.stdout to bytes. If you want the correct characters to be shown, this has to match the encoding which the console window uses to convert those bytes back to unicode characters. You can tell Python to use whichever encoding you want, but often you only get to control one side of the equation, in which case there's only one "right" answer. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pass class instance to a method?
On Sun, 25 Nov 2012 04:11:29 -0800, ALeX inSide wrote:
> How to "statically type" an instance of class that I pass to a method of
> other instance?
Python isn't statically typed. You can explicitly check for a specific
type with e.g.:
if not isinstance(arg, SomeType):
raise TypeError('expected SomeType but got %s' % type(arg))
But this defeats duck typing. If you do this a lot, you're using the wrong
language.
> I suppose there shall be some kind of method decorator to treat an
> argument as an instance of class?
>
> Generally it is needed so IDE (PyCharm) can auto-complete instance's
> methods and properties.
You have it backwards.
In a dynamically-typed language such as Python, the set of acceptable
types for an argument is determined by the operations which the function
performs on it. This is in direct contrast to a statically-typed language,
where the set of acceptable operations on an argument is determined by the
type of the argument.
--
http://mail.python.org/mailman/listinfo/python-list
Re: os.popen and the subprocess module
On Thu, 29 Nov 2012 10:09:44 +0100, Thomas Rachel wrote:
> The variant with shell=True is more os.popen()-like, but has security
> flaws (e.g., what happens if there are spaces or, even worse, ";"s in the
> command string?
I think that you're conflating the shell= option with whether the command
is a given as a list or a string.
Attempting to construct a command string risks introducing security flaws
(or other bugs). Wherever possible, the first argument should be a list. A
string should only be used if that's what you're given (e.g. via a
configuration file), in which case it should be used literally, without
any attempt to substitute filenames or other parameters.
On Windows, list-versus-string and shell= are orthogonal. A list will
always be converted to a string, as that's what the underlying
CreateProcess() function requires. shell=True prepends "cmd /c " ("cmd" is
replaced by the value of %comspec% if that is defined); this allows
execution of batch files, scripts, etc based upon their associations.
On Unix, passing a list with shell=True is rarely useful. It just prepends
['/bin/sh', '-c'] to the list, so the first item is the shell command
while subsequent items provide the values for the shell variables $1, $2,
etc.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Imaging libraries in active development?
On Wed, 28 Nov 2012 04:30:25 -0800, Alasdair McAndrew wrote: > What I want to know is - what are the current "standard" libraries for > image processing in Python which are in active development? NumPy/SciPy. PIL is fine for loading/saving image files (although if you're using a GUI toolkit, that probably has its own equivalents). But for any non-trivial processing, I normally end up using either NumPy or (if speed is an issue) PyOpenGL/GLSL. -- http://mail.python.org/mailman/listinfo/python-list
Re: forking and avoiding zombies!
On Tue, 11 Dec 2012 13:25:36 +, andrea crotti wrote: > But actually why do I need to move away from the current directory of the > parent process? It's not required, it's just "best practice". Often, the current directory is simply whichever directory it happened to inherit from the shell which spawned it. So long as that directory continues to be used as the daemon's current directory, the filesystem on which it resides cannot be unmounted. So daemons normally change to the root directory (or to some other directory, e.g. one which they actually need to use) in order to "release" the directory from which they were initially started. -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On Wed, 02 Jan 2013 00:49:36 +0100, someone wrote: > In [11]: del format > --- > NameError Traceback (most recent call last) > in () > > 1 del format > > NameError: name 'format' is not defined > > > What does this mean? Why does it say 'format" cannot be deleted after I > did the wildcard import ? You can't delete built-in names. It has nothing to do with the wildcard import. The PyOpenGL modules delete "format" from the module's variables as soon as they are finished with it, so the set of names created by the wildcard import doesn't include "format". -- http://mail.python.org/mailman/listinfo/python-list
Re: python sys.stdout and C++ iostreams::cout
On Thu, 17 Jan 2013 07:02:24 -0800, Utpal Sarkar wrote:
> I was assuming that sys.stdout would be referencing the same physical
> stream as iostreams::cout running in the same process, but this doesn't
> seem to be the case.
At startup, it refers to the same FILE* as C's stdout. This initially
shares a buffer with C++'s std::cout, but that can be changed via
std::ios_base::sync_with_stdio().
However ...
> The following code, which makes a call to a C++
> function with a python wrapper called "write", that writes to cout:
>
> from cStringIO import StringIO
> import sys
> orig_stdout = sys.stdout
> sys.stdout = stringout = StringIO()
> write("cout") # wrapped C++ function that writes to cout print "-" * 40
> print "stdout"
> sys.stdout = orig_stdout
> print stringout.getvalue()
This code changes sys.stdout so that it refers to something other than C's
stdout. C's stdout is still the same FILE*, C++'s std::count is still the
same std::ostream, and the synchronisation between the two hasn't changed.
> immediately writes "cout" to the console, then the separator "---...", and
> finally, as the return value of stringout.getvalue(), the string "stdout".
> My intention was to capture in stringout also the string written to cout
> from C++. Does anyone know what is going on, and if so, how I can capture
> what is written to cout in a python string?
Changing sys.stdout doesn't (and cannot) have any effect upon how C or C++
code behaves. sys.stdout is just a Python variable.
If you want to capture output from C or C++ code, you'll have to do so via
other means, e.g. freopen() or dup2().
--
http://mail.python.org/mailman/listinfo/python-list
Re: Windows subprocess.call problem
On Mon, 21 Jan 2013 07:25:06 -0400, Tom Borkin wrote: > It opens the first song and hangs on subsequent songs. It doesn't open the > next song or execute the print until I have closed the first one. I want it > to open all in the list, one after another, so I have all those songs > available. Please advise. If you want to be able to keep track of the child process (e.g. to determine when it has finished), use subprocess.Popen(). If you just want to "fire and forget", use the "start" shell command, e.g.: subprocess.call(['start', 'notepad.exe', '%s.txt' % song], shell=True) -- http://mail.python.org/mailman/listinfo/python-list
Inherent asyncore.dispatcher_with_send exception
Hi, I have a class ClientHandler(asyncore.dispatcher_with_send), it was running fine without calling any of my own classes. But it got following exception when I called my own class GetMyResponse inside the def handle_read(self). Not sure why it causes disturbance to asyncore.dispatcher_with_send __init__ when calling an extra class? Appreciate any clues and tips. class GetMyResponse: def __init__(self, message): class ClientHandler(asyncore.dispatcher_with_send): def handle_read(self): handleResponse = GetMyResponse(data) self.send(handleResponse.getResponse()) error: uncaptured python exception, closing channel <__main__.ClientHandler connected 127.0.0.1:42383 at 0x7f3b638b6758> (:__init__() takes exactly 4 arguments (3 given) [/usr/lib64/python2.6/asyncore.py|read|78] [/usr/lib64/python2.6/asyncore.py|handle_read_event|428] -- http://mail.python.org/mailman/listinfo/python-list
sockobj.connect Errno 13 Permission denied
Hi,
I have a client program Client.py which has a statement of sockobj.connect(),
the port number 6 is used, so no problem from port permission.
I am puzzled because I can run Client.py from command line in my user account
or apache user account without any problems.
But if I run it from a web page http://localhost/client.php, the client.php
called exec("Client.py"), then it got an exception of sockobj.connect Errno 13
Permission denied.
Why it can run from command line, but cannot make connection from a web file?
Appreciate any tips and clues.
Thank you.
Kind regards.
--
http://mail.python.org/mailman/listinfo/python-list
Re: question about function pointer
On Fri, 17 Feb 2012 16:53:00 +0900, Zheng Li wrote:
> def method1(a = None):
> print a
>
> i can call it by
> method1(*(), **{'a' : 1})
>
> I am just curious why it works and how it works?
> and what do *() and **{'a' : 1} mean?
In a function call, an argument consisting of * followed by an expression
of tuple type inserts the tuple's elements as individual positional
arguments. An argument consisting of ** followed by an expression of
dictionary type inserts the dictionary's elements as individual keyword
arguments.
So if you have:
a = (1,2,3)
b = {'a': 1, 'b': 2, 'c': 3}
then:
func(*a, **b)
is equivalent to:
func(1, 2, 3, a = 1, b = 2, c = 3)
> when I type *() in python shell, error below happens
>
> File "", line 1
> *()
> ^
> SyntaxError: invalid syntax
The syntax described above is only valid within function calls.
There is a similar syntax for function declarations which does the reverse:
> def func(*a, **b):
print a
print b
> func(1, 2, 3, a = 1, b = 2, c = 3)
(1, 2, 3)
{'a': 1, 'c': 3, 'b': 2}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Generating class definitions at runtime in memory from XSD or JSON
On Thu, 16 Feb 2012 17:15:59 -0800, Stodge wrote:
> Does anyone know of a library to generate class definitions in memory,
> at runtime, from XSD or JSON? I know about PyXB, generateDS and some
> others, but they all rely on generating python source files at the
> command line, and then using those to parse XML.
You don't need a library to generate classes. If the type() function is
called with 3 arguments, it creates and returns a new class. The first
argument is the name of the class, the second argument a tuple of base
classes, the third argument is the class' dictionary. E.g.:
class Foo(Bar, Baz):
def __init__(self):
pass
could be written as:
def foo_init(self):
pass
Foo = type('Foo', (Bar, Baz), {'__init__': foo_init})
If you want to generate the function bodies from the contents of the
JSON or XML file, use exec().
--
http://mail.python.org/mailman/listinfo/python-list
Re: question about file handling with "with"
On Wed, 28 Mar 2012 11:31:21 +0200, Jabba Laci wrote: > Is the following function correct? Is the input file closed in order? > > def read_data_file(self): > with open(self.data_file) as f: > return json.loads(f.read()) Yes. The whole point of being able to use a file as a context manager is so that the file will be closed immediately upon leaving the with statement, whether by falling off the end, "return", an exception, or whatever. IOW, it's like calling .close() immediately after the "with" block, only more so, i.e. it will also handle cases that an explicit .close() misses. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple rsa from javascript to python
On Mon, 02 Apr 2012 16:19:05 -0700, Astan Chee wrote:
> and I'm trying to convert this into python and I'm rather stuck with
> pycrypto as there is no example on how to make the public key with a mod
> and exponent (or I've probably missed it).
from Crypto.PublicKey import RSA
mod = long("B99808B881F3D8A...", 16) # truncated for clarity
exp = long("010001", 16)
rsa = RSA.construct((mod, exp))
--
http://mail.python.org/mailman/listinfo/python-list
Re: No os.copy()? Why not?
On Wed, 04 Apr 2012 08:14:18 -0400, Roy Smith wrote: >> And sparse files are really hard to reproduce, at least on Unix: on >> Linux even the system's cp doesn't guarantee sparseness of the copy (the >> manual mentions a "crude heuristic"). > > I imagine the heuristic is to look for blocks of all zeros. Yes. Although it's not really accurate to describe it as a "heuristic". With --sparse=always, it will try to make the output sparse regardless of whether the input was sparse, replacing any all-zeros block with a hole. The default of --sparse=auto will only create a sparse file if the input itself is sparse, i.e. if the length of the file rounded up to the nearest block exceeds its disk usage. Regardless of the --sparse= setting and whether the input was sparse, if it tries to create a sparse file it will create holes wherever possible rather than attempting to preserve the exact pattern of holes in a sparse input file. -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Live Output from a Subprocess
On Thu, 05 Apr 2012 23:57:49 -0700, bunslow wrote: > Okay, I've been trying for days to figure this out, posting on forums, > Googling, whatever. I have yet to find a solution that has worked for me. > (I'm using Python 3.2.2, Ubuntu 11.04.) Everything I've tried has led to > buffered output being spat back all at once after the subprocess > terminates. In all probability, this is because the child process (pypy) is buffering its stdout, meaning that the data doesn't get passed to the OS until either the buffer is full or the process terminates. If it doesn't get passed to the OS, then the OS can't pass it on to whatever is on the read end of the pipe. In that situation, there is nothing that the parent process can do about it. > I've heard that the Pexpect module works wonders, If it does, that would confirm the buffering hypothesis. Pexpect causes the child process' stdout to be associated with a pty. The "stdout" stream created by the C library (libc) is initially line-buffered if it is associated with a tty (according to the isatty() function) and fully-buffered otherwise. The program can change the buffering with e.g. setvbuf(), or it can explicitly fflush(stdout) after each line, but this is up to the program, and is not something which can be controlled externally (short of "hacking" the child process with techniques such as ptrace() or LD_PRELOAD). While the libc behaviour is occasionally inconvenient, it is mandated by the C standard (7.19.3p7): As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device. It's up to individual programs to force line-buffered output where appropriate (e.g. GNU grep has the --line-buffered switch, GNU sed has the -u switch, etc). -- http://mail.python.org/mailman/listinfo/python-list
Re: escaping/encoding/formatting in python
On Thu, 05 Apr 2012 22:28:19 -0700, rusi wrote: > All this mess would vanish if the string-literal-starter and ender > were different. You still need an escape character in order to be able to embed an unbalanced end character. Tcl and PostScript use mirrored string delimiters (braces for Tcl, parentheses for PostScript), which results in the worst of both worlds: they still need an escape character (backslash, in both cases) but now you can't match tokens with a regexp/DFA. -- http://mail.python.org/mailman/listinfo/python-list
Re: escaping/encoding/formatting in python
On Fri, 06 Apr 2012 06:22:13 -0700, rusi wrote:
> But are not such cases rare?
They exist, therefore they have to be supported somehow.
> For example code such as:
> print '"'
> print str(something)
> print '"'
>
> could better be written as
> print '"%s"' % str(something)
Not if the text between the delimiters is large.
Consider:
print 'static const char * const data[] = {'
for line in infile:
print '\t"%s",' % line.rstrip()
print '};'
Versus:
text = '\n'.join('\t"%s",' % line.rstrip() for line in infile)
print 'static const char * const data[] = {\n%s\n};' % text
C++11 solves the problem to an extent by providing raw strings with
user-defined delimiters (up to 16 printable characters excluding
parentheses and backslash), e.g.:
R"delim(quote: " backslash: \ rparen: ))delim"
evaluates to the string:
quote: " backslash: \ rparen: )
The only sequence which cannot appear in such a string is )delim" (i.e. a
right parenthesis followed by the chosen delimiter string followed by a
double quote). The delimiter can be chosen either by analysing the string
or by choosing something a string at random and relying upon a collision
being statistically improbable.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Reading Live Output from a Subprocess
On Fri, 06 Apr 2012 12:21:51 -0700, Dubslow wrote:
> It's just a short test script written in python, so I have no idea how
> to even control the buffering
In Python, you can set the buffering when opening a file via the third
argument to the open() function, but you can't change a stream's buffering
once it has been created. Although Python's file objects are built on the
C stdio streams, they don't provide an equivalent to setvbuf().
On Linux, you could use e.g.:
sys.stdout = open('/dev/stdout', 'w', 1)
Other than that, if you want behaviour equivalent to line buffering, call
sys.stdout.flush() after each print statement.
> (and even if I did, I still can't modify the subprocess I need to use in
> my script).
In which case, discussion of how to make Python scripts use line-buffered
output is beside the point.
> What confuses me then is why Perl is able to get around this just fine
> without faking a terminal or similar stuff.
It isn't. If a program sends its output to the OS in blocks, anything
which reads that output gets it in blocks. The language doesn't matter;
writing the parent program in assembler still wouldn't help.
> I take it then that setting Shell=True will not be fake enough for
> catching output live?
No. It just invokes the command via /bin/sh or cmd.exe. It doesn't affect
how the process' standard descriptors are set up.
On Unix, the only real use for shell=True is if you have a "canned" shell
command, e.g. from a file, and you need to execute it. In that situation,
args should be a string rather than a list. And you should never try to
construct such a string dynamically in order to pass arguments; that's an
injection attack waiting to happen.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pass a list of variables to a procedure
On Sat, 07 Apr 2012 14:15:09 -0700, KRB wrote: > I would like to be able to pass a list of variables to a procedure, and > have the output assigned to them. Use a dictionary or an object. If the variables are globals (i.e. attributes of the current module), you can pass the result of globals() into the function, or have the function return a dictionary which the caller merges into globals(). There isn't no way to do anything similar for local variables. The parser has to "see" the name being used a local variable in order for it to actually exist as a local variable. > otherwise I have to do: > x=somefunction(1) > y=somefunction(2) > z=somefunction(3) Or you could do: x, y, z = [somefunction(i) for i in [1,2,3]] -- http://mail.python.org/mailman/listinfo/python-list
Re: f python?
On Sun, 08 Apr 2012 04:11:20 -0700, Xah Lee wrote: > Ok no problem. My sloppiness. After all, my implementation wasn't > portable. So, let's fix it. After a while, discovered there's the > os.sep. Ok, replace "/" to os.sep, done. Then, bang, all hell > went lose. Because, the backslash is used as escape in string, so any > regex that manipulate path got fucked majorly. So, now you need to > find a quoting mechanism. if os.altsep is not None: sep_re = '[%s%s]' % (os.sep, os.altsep) else: sep_re = '[%s]' % os.sep But really, you should be ranting about regexps rather than Python. They're convenient if you know exactly what you want to match, but a nuisance if you need to generate the expression based upon data which is only available at run-time (and re.escape() only solves one very specific problem). -- http://mail.python.org/mailman/listinfo/python-list
Re: can I overload operators like "=>", "->" or something like that?
On Thu, 19 Apr 2012 12:28:50 -0700, dmitrey wrote: > can I somehow overload operators like "=>", "->" or something like that? > (I'm searching for appropriate overload for logical implication "if a then > b") You cannot create new operators, but you can control how existing operators work on types which you define. IOW, you can't define "->" or "=>", but you could define ">=" or ">>". -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Mon, 23 Apr 2012 10:01:24 -0700, Paul Rubin wrote: >> I can't think of a single case where 'is' is ill-defined. > > If I can't predict the output of > > print (20+30 is 30+20) # check whether addition is commutative print > (20*30 is 30*20) # check whether multiplication is commutative > > by just reading the language definition and the code, I'd have to say "is" > is ill-defined. If anything is ill-defined, then it's "+" and "*", i.e. it's unspecified whether the value which they return is a unique object or a reference to some other object. More accurately, the existence of "is", "is not" and "id" cause many other constructs to have "ill-defined" behaviour. >> "a is b" is true iff 'a' and 'b' are the same object. Why should 'is' >> lie to the user? > > Whether a and b are the same object is implementation-dependent. And what's wrong with that? If you want a language which precisely specifies all observable behaviour, you're going to end up with a rather useless language. For a start, it can't have a time() function. For similar reasons, you can't have networking or any form of preemptive concurrency (which includes any form of inter-process communication on an OS which uses preemptive multi-tasking). -- http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
On Thu, 26 Apr 2012 11:31:39 -0700, John Nagle wrote: > I would suggest that "is" raise ValueError for the ambiguous cases. > If both operands are immutable, "is" should raise ValueError. That's the > case where the internal representation of immutables shows through. This breaks one of the most common uses of "is", i.e. "x is None". And it doesn't prevent a programmer from consfusing "is" and "==" with mutable types. > If this breaks a program, it was broken anyway. It will > catch bad comparisons like > > if x is 1000 : > ... > > which is implementation dependent. The only way to completely eliminate bugs caused by the programmer relying upon implementation-dependent behaviour is to eliminate implementation- dependent behaviour altogether, which is throwing the baby out with the bath water, IMHO. All practical languages have some implementation-defined behaviour, often far more problematic than Python's. -- http://mail.python.org/mailman/listinfo/python-list
Re: ctype C library call always returns 0 with Python3
On Sat, 19 May 2012 11:30:46 +0200, Johannes Bauer wrote:
> import ctypes
> libc = ctypes.cdll.LoadLibrary("/lib64/libc-2.14.1.so")
> print(libc.strchr("abcdef", ord("d")))
In 3.x, a string will be passed as a wchar_t*, not a char*. IOW, the
memory pointed to by the first argument to strchr() will consist mostly of
NUL bytes.
Either use a "bytes" instead of a string:
> print(libc.strchr(b"abcdef", ord("d")))
198291
or specify the argument types to force a conversion:
> libc.strchr.argtypes = [c_char_p, c_int]
> print(libc.strchr("abcdef", ord("d")))
1984755787
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help doing it the "python way"
On Thu, 24 May 2012 13:22:43 -0700, Scott Siegler wrote: > is there a way to do something like: > [(x,y-1), (x,y+1) for zzz in coord_list] > or something along those lines? [(xx,yy) for x, y in coord_list for xx, yy in [(x,y-1),(x,y+1)]] or: [(x,yy) for x, y in coord_list for yy in [y-1,y+1]] Not to be confused with: [[(xx,yy) for xx, yy in [(x,y-1),(x,y+1)]] for x, y in coord_list] or: [[(x,yy) for yy in (y-1,y+1)] for x, y in coord_list] which will produce the same pairs in the same order but as a list of lists rather than as a single flat list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Smallest/cheapest possible Python platform?
On Sat, 26 May 2012 11:34:19 -0400, Roy Smith wrote: > The Rasberry Pi certainly looks attractive, but isn't quite available > today. Can you run Python on an Arduino? Things like > http://www.embeddedarm.com/products/board-detail.php?product=TS-7250 are > more than I need, and the $129 price probably busts my budget. You can't run Python on an Arduino. There are a number of boards with ARM or PIC32 processors which can use Arduino daughter-boards (shields), but all of the ones which I've seen have rather limited RAM. -- http://mail.python.org/mailman/listinfo/python-list
Re: Compare 2 times
On Wed, 06 Jun 2012 05:50:02 -0700, loial wrote: > I have a requirement to test the creation time of a file with the current > time and raise a message if the file is more than 15 minutes old. > > Platform is Unix. > > I have looked at using os.path.getctime for the file creation time and > time.time() for the current time, but is this the best approach? Most Unix filesystems don't store a "creation" time. -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Thu, 02 Jun 2011 09:54:30 +, Steven D'Aprano wrote: >> Exceptions allow you to write more natural code by ignoring the awkward >> cases. E.g. writing "x * y + z" rather than first determining whether "x >> * y" is even defined then using a conditional. > > You've quoted me out of context. I wasn't asking for justification for > exceptions in general. There's no doubt that they're useful. We were > specifically talking about NAN == NAN raising an exception rather than > returning False. It's arguable that NaN itself simply shouldn't exist in Python; if the FPU ever generates a NaN, Python should raise an exception at that point. But given that NaNs propagate in almost the same manner as exceptions, you could "optimise" this by treating a NaN as a special-case implementation of exceptions, and turn it into a real exception at the point where you can no longer use a NaN (e.g. when using a comparison operator). This would produce the same end result as raising an exception immediately, but would reduce the number of isnan() tests. >> NaN itself is an exceptional condition which arises when a result is >> undefined or not representable. When an operation normally returns a >> number but a specific case cannot do so, it returns not-a-number. > > I'm not sure what "not representable" is supposed to mean, Consider sqrt(-1). This is defined (as "i" aka "j"), but not representable as a floating-point "real". Making root/log/trig/etc functions return complex numbers when necessary probably be inappropriate for a language such as Python. > but if you "undefined" you mean "invalid", then correct. I mean undefined, in the sense that 0/0 is undefined (I note that Python actually raises an exception for "0.0/0.0"). >> The usual semantics for NaNs are practically identical to those for >> exceptions. If any intermediate result in a floating-point expression is >> NaN, the overall result is NaN. > > Not necessarily. William Kahan gives an example where passing a NAN to > hypot can justifiably return INF instead of NAN. Hmm. Is that still true if the NaN signifies "not representable" (e.g. known but complex) rather than undefined (e.g. unknown value but known to be real)? > While it's certainly > true that *mostly* any intermediate NAN results in a NAN, that's not a > guarantee or requirement of the standard. A function is allowed to > convert NANs back to non-NANs, if it is appropriate for that function. > > Another example is the Kronecker delta: > > def kronecker(x, y): > if x == y: return 1 > return 0 > > This will correctly consume NAN arguments. If either x or y is a NAN, it > will return 0. (As an aside, this demonstrates that having NAN != any > NAN, including itself, is useful, as kronecker(x, x) will return 0 if x > is a NAN.) How is this useful? On the contrary, I'd suggest that the fact that kronecker(x, x) can return 0 is an argument against the "NaN != NaN" axiom. A case where the semantics of exceptions differ from those of NaN is: def cond(t, x, y): if t: return x else: return y as cond(True, x, nan()) will return x, while cond(True, x, raise()) will raise an exception. But this is a specific instance of a more general problem with strict languages, i.e. strict functions violate referential transparency. This is why even strict languages (i.e. almost everything except for a handful of functional languages which value mathematical purity, e.g. Haskell) have non-strict conditionals. If you remove the conditional from the function and write it in-line, then: if True: return x else: raise() behaves like NaN. Also, note that the "convenience" of NaN (e.g. not propagating from the untaken branch of a conditional) is only available for floating-point types. If it's such a good idea, why don't we have it for other types? > Equality comparison is another such function. There's no need for > NAN == NAN to fail, because the equality operation is perfectly well > defined for NANs. The definition is entirely arbitrary. You could just as easily define that (NaN == NaN) is True. You could just as easily define that "1 + NaN" is 27. Actually, "NaN == NaN" makes more sense than "NaN != NaN", as the former upholds the equivalence axioms and is consistent with the normal behaviour of "is" (i.e. "x is y" => "x == y", even if the converse isn't necessarily true). If you're going to argue that "NaN == NaN" should be False on the basis that the values are sentinels for unrepresentable values (which may be *different* unrepresentable values), it follows that "NaN != NaN" should also be False for the same reason. >> But only the floating-point types have a NaN value, while >> bool doesn't. However, all types have exceptions. > > What relevance does bool have? The result of comparisons is a bool. >> Why should there be a correct answer? What does NaN actual
Re: Something is rotten in Denmark...
On Fri, 03 Jun 2011 11:43:54 +1200, Gregory Ewing wrote: >> But going against generally accepted semantics should at least >> be clearly indicated. Lambda is one of the oldest computing abstraction, >> and they are at the core of any functional programming language. > > Yes, and Python's lambdas behave exactly the *same* way as > every other language's lambdas in this area. Changing it to > do early binding would be "going against generally accepted > semantics". In Lisp, it depends upon whether the free variable is bound: $ clisp [1]> (setq f (lambda (x) (+ x i))) # [2]> (setq i 7) 7 [3]> (apply f (list 4)) 11 [4]> (setq i 12) 12 [5]> (apply f (list 4)) 16 ^D $ clisp [1]> (let ((i 7)) (setq f (lambda (x) (+ x i # [2]> (apply f (list 4)) 11 [3]> (setq i 12) 12 [4]> (apply f (list 4)) 11 ^D If the variable is bound, then the lambda creates a closure. And Python behaves the same way: > f = (lambda i: lambda x: x + i)(7) > f(4) 11 > i = 12 > f(4) 11 # If you really want a "let", this syntax is closer: # f = (lambda i = 7: lambda x: x + i)() The original semantics (from the lambda calculus) don't deal with the concept of mutable state. > If anything should be changed here, it's the for-loop, not lambda. Right. The for loop should bind the variable rather than set it. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 04:30:46 +, Chris Torek wrote: >>I'm not sure what you mean by "full 16-bit Unicode string"? Isn't >>unicode inherently 32 bit? > > Well, not exactly. As I understand it, Python is normally built > with a 16-bit "unicode character" type though It's normally 32-bit on platforms where wchar_t is 32-bit (e.g. Linux). -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Fri, 03 Jun 2011 02:58:24 +, Chris Torek wrote: > Python might be penalized by its use of Unicode here, since a > Boyer-Moore table for a full 16-bit Unicode string would need > 65536 entries (one per possible ord() value). However, if the > string being sought is all single-byte values, a 256-element > table suffices; re.compile(), at least, could scan the pattern > and choose an appropriate underlying search algorithm. The table can be truncated or compressed at the cost of having to map codepoints to table indices. Or use a hash table instead of an array. -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Fri, 03 Jun 2011 14:52:39 +, Grant Edwards wrote: >> It's arguable that NaN itself simply shouldn't exist in Python; if >> the FPU ever generates a NaN, Python should raise an exception at >> that point. > > Sorry, I just don't "get" that argument. I depend on compliance with > IEEE-754, and I find the current NaN behavior very useful, and > labor-saving. If you're "fluent" in IEEE-754, then you won't find its behaviour unexpected. OTOH, if you are approach the issue without preconceptions, you're likely to notice that you effectively have one exception mechanism for floating-point and another for everything else. >> But given that NaNs propagate in almost the same manner as >> exceptions, you could "optimise" this by treating a NaN as a >> special-case implementation of exceptions, and turn it into a real >> exception at the point where you can no longer use a NaN (e.g. when >> using a comparison operator). >> >> This would produce the same end result as raising an exception >> immediately, but would reduce the number of isnan() tests. > > I've never found the number of isnan() checks in my code to be an > issue -- there just arent that many of them, and when they are there, > it provides an easy to read and easy to maintain way to handle things. I think that you misunderstood. What I was saying here was that, if you wanted exception-on-NaN behaviour from Python, the interpreter wouldn't need to call isnan() on every value received from the FPU, but rely upon NaN-propagation and only call it at places where a NaN might disappear (e.g. comparisons). >> I mean undefined, in the sense that 0/0 is undefined > > But 0.0/0.0 _is_ defined. It's NaN. ;) Mathematically, it's undefined. >> (I note that Python actually raises an exception for "0.0/0.0"). > > IMHO, that's a bug. IEEE-754 states explicit that 0.0/0.0 is NaN. > Pythons claims it implements IEEE-754. Python got it wrong. But then IEEE-754 considers integers and floats to be completely different beasts, while Python makes some effort to maintain a unified "numeric" interface. If you really want IEEE-754 to-the-letter, that's undesirable, although I'd question the choice of Python in such situations. >> The definition is entirely arbitrary. > > I don't agree, but even if was entirely arbitrary, that doesn't make > the decision meaningless. IEEE-754 says it's True, and standards > compliance is valuable. True, but so are other things. People with a background in mathematics (as opposed to arithmetic and numerical methods) would probably consider following the equivalence axioms to be valuable. Someone more used to Python than IEEE-754 might consider following the "x is y => x == y" axiom to be valuable. As for IEEE-754 saying that it's True: they only really had two choices: either it's True or it's False. NaNs provide "exceptions" even if the hardware or the language lacks them, but that falls down once you leave the scope of floating-point. It wouldn't have been within IEEE-754's ambit to declare that comparing NaNs should return NaB (Not A Boolean). >> Actually, "NaN == NaN" makes more sense than "NaN != NaN", as the >> former upholds the equivalence axioms > > You seem to be talking about reals. We're talking about floats. Floats are supposed to approximate reals. They're also a Python data type, and should make some effort to fit in with the rest of the language. >> If anything, it has proven to be a major nuisance. It takes a lot of >> effort to create (or even specify) code which does the right thing in >> the presence of NaNs. > > That's not been my experience. NaNs save a _huge_ amount of effort > compared to having to pass value+status info around throughout complex > caclulations. That's what exceptions are for. NaNs probably save a huge amount of effort in languages which lack exceptions, but that isn't applicable to Python. In Python, they result in floats not "fitting in". Let's remember that the thread started with an oddity relating to using floats as dictionary keys, which mostly works but fails for NaN because of the (highly unusual) property that "x == x" is False for NaNs. Why did the Python developers choose this behaviour? It's quite likely that they didn't choose it, but just overlooked the fact that NaN creates this corner-case which breaks code which works for every other primitive type except floats and even every other float except NaN. In any case, I should probably re-iterate at this point that I'm not actually arguing *for* exception-on-NaN or NaN==NaN or similar, just pointing out that IEEE-754 is not the One True Approach and that other approaches are not necessarily heresy and may have some merit. To go back to the point where I entered this thread: >>> The correct answer to "nan == nan" is to raise an exception, >>> because you have asked a question for which the answer is nether True >>> nor False. >> >> Wrong. > > That's overstating it. -- http://mail.python.org/mailman/listin
Re: except KeyError, everywhere
On Fri, 03 Jun 2011 22:08:16 +0200, Wilbert Berendsen wrote: > I find myself all over the place associating objects with each other using > dicts as caches: > Are there other peoply using things like this? Is there a solution like > this in the standard lib that I'm overlooking? The general concept is called "memoization". There isn't an implementation in the standard library, but you'll find plenty of examples, e.g. (from the first page of Google hits for "python memoization"): http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize http://code.activestate.com/recipes/52201-memoizing-cacheing-function-return-values/ http://code.activestate.com/recipes/577219-minimalistic-memoization/ -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Sat, 04 Jun 2011 00:52:17 -0700, rusi wrote: >> If you're "fluent" in IEEE-754, then you won't find its behaviour >> unexpected. OTOH, if you are approach the issue without preconceptions, >> you're likely to notice that you effectively have one exception mechanism >> for floating-point and another for everything else. > > Three actually: None, nan and exceptions None isn't really an exception; at least, it shouldn't be used like that. Exceptions are for conditions which are in some sense "exceptional". Cases like dict.get() returning None when the key isn't found are meant for the situation where the key not existing is unexceptional. If you "expect" the key to exist, you'd use dict[key] instead (and get an exception if it doesn't). -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Sat, 04 Jun 2011 13:41:33 +1200, Gregory Ewing wrote: >> Python might be penalized by its use of Unicode here, since a >> Boyer-Moore table for a full 16-bit Unicode string would need >> 65536 entries > > But is there any need for the Boyer-Moore algorithm to > operate on characters? > > Seems to me you could just as well chop the UTF-16 up > into bytes and apply Boyer-Moore to them, and it would > work about as well. No, because that won't care about alignment. E.g. on a big-endian architecture, if you search for '\u2345' in the string '\u0123\u4567', it will find a match (at an offset of 1 byte). -- http://mail.python.org/mailman/listinfo/python-list
Re: how to avoid leading white spaces
On Sat, 04 Jun 2011 05:14:56 +, Steven D'Aprano wrote: > This fails to support non-ASCII letters, and you know quite well that > having to spell out by hand regexes in both upper and lower (or mixed) > case is not support for case-insensitive matching. That's why Python's re > has a case insensitive flag. I find it slightly ironic that you pointed out the ASCII limitation while overlooking the arbitrariness of upper/lower-case equivalence. Case isn't the only type of equivalence; it's just the only one which affects ASCII. Should we also have flags to treat half-width and full-width characters as equivalent? What about traditional and simplified Chinese, hiragana and katakana, or the various stylistic variants of the Latin and Greek alphabets in the mathematical symbols block (U+1D400..U+1D7FF)? -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Sun, 05 Jun 2011 07:21:10 +, Steven D'Aprano wrote: > Returning a sentinel meaning "an exceptional event occurred" is hardly > unusual, even in Python. str.find() does is, as does re.search() and > re.match(). These are not "exceptional" conditions; if they were, an exception would be used. E.g. dict supports both d.get(key) and d[key] for lookups. The former returns a sentinel, the latter raises an exception. The latter makes sense if you "expect" the key to be present, the former if you don't. >> As for IEEE-754 saying that it's [NAN == NAN] True: they only really >> had two choices: either it's True or it's False. > > Incorrect. They could have specified that it was an error, like dividing > by zero, but they didn't. Specifying an error doesn't remove the requirement to also specify a result. E.g. dividing a finite value by zero produces a result of infinity. In languages which lack exceptions, errors only matter if the code bothers to check for them (if such checks are even possible; C89 lacks ). -- http://mail.python.org/mailman/listinfo/python-list
Re: float("nan") in set or as key
On Mon, 06 Jun 2011 00:55:18 +, Steven D'Aprano wrote: > And thus we come back full circle. Hundreds of words, and I'm still no > closer to understanding why you think that "NAN == NAN" should be an > error. Well, you could try improving your reading comprehension. Counselling might help. AFAICT, your main problem is that you can't distinguish between not agreeing with a particular argument and being unable to even recognise the existence of the argument. A really big clue is here: > why you think that "NAN == NAN" should be an error Given that my very first comment in the thread was: > > Wrong. > > That's overstating it. There's a good argument to be made for ... I never said that it /should/ be an error, and later explicitly stated that I wasn't arguing for it but pointing out that it's /arguable/. But you appear unable to comprehend such distinctions. Don't agree with the argument, don't accept the argument, don't recognise that there is an argument; these all appear to be the same thing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Validating string for FDQN
On Mon, 06 Jun 2011 17:40:29 -0700, Eric wrote: > Is there a library or regex that can determine if a string is a fqdn > (fully qualified domain name)? I'm writing a script that needs to add > a defined domain to the end of a hostname if it isn't already a fqdn > and doesn't contain the defined domain. Try socket.getfqdn() or socket.gethostbyname_ex(). With one exception[1], you can't reliably do it just by examining the string; you have to ask the resolver. [1] If a hostname ends with a dot, it's fully qualified. -- http://mail.python.org/mailman/listinfo/python-list
