Re: [Python-Dev] graphics maths types in python core?
2009/4/5 Antoine Pitrou : > Nick Coghlan gmail.com> writes: >> >> That's largely because parts 2 and 3 are somewhat use case challenged: >> the key motivation behind PEP 3118 was so that libraries like NumPy, PIL >> and the like would have a common standard for data interchange. > > If I understand correctly, one of the motivations behind memoryview() is to > replace buffer() as a way to get cheap slicing without memory copies (it's > used > e.g. in the C IO library). I don't know whether the third-party types > mentioned > above could also benefit from that. Well, PEP 3118 is useful because it would be nice having e.g. the possibility of opening an image with PIL, manipulate it directly with NumPy and saving it to file with PIL. Right now this is possible only if the PIL image is first converted (and copied) to a new NumPy array and then the array is converted back to an image. BTW, while PEP 3118 provides a common C API for this, the related PEP 368 proposes a standard "image protocol" on the Python side that should be compatible with the image classes of PIL, wxPython and pygame, and (mostly) with NumPy arrays. I started an implementation of PEP 368 at: http://code.google.com/p/pyimage/ Both the PEP and the implementation need updates (pyimage already includes an IEEE 754r compatible half-precision floating point type, aka float16, that's not yet in the PEP), but if someone is interested and willing to help I may start again working on them. Also note that the subjects "vec2, vec3, quaternion, etc" (PEP 3141) and "multi-dimensional arrays" (PEP 3118) are mostly unrelated. -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-ideas] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)
2009/4/12 Mart Sõmermaa : > The bad thing about reasoning about query strings is that there is no > comprehensive documentation about their meaning. Both RFC 1738 and RFC 3986 > are rather vague in that matter. FYI the HTML5 spec (http://whatwg.org/html5 ) may have a better contact with reality than the RFCs. >From a quick scan, two sections that may be relevant are "4.10.16.3 Form submission algorithm": <http://www.whatwg.org/specs/web-apps/current-work/multipage/forms.html#form-submission-algorithm> and "4.10.16.4 URL-encoded form data": <http://www.whatwg.org/specs/web-apps/current-work/multipage/forms.html#url-encoded-form-data> -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping bytes "support" in json
2009/4/13 Daniel Stutzbach :
> print("Content-Type: application/json; charset=utf-8")
Please don't do that! According to RFC 4627 the "charset" parameter is
not allowed for the application/json media type.
Just use "Content-Type: application/json", the charset is only
misleading because even if you specify, e.g., ISO-8859-1 a
standard-compliant receiver will probably still try to interpret the
content as UTF-8/16/32.
OTOH a charset can be used if you send JSON with an
application/javascript MIME type.
--
Lino Mastrodomenico
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces
2009/4/22 "Martin v. Löwis" : > To convert non-decodable bytes, a new error handler "python-escape" is > introduced, which decodes non-decodable bytes using into a private-use > character U+F01xx, which is believed to not conflict with private-use > characters that currently exist in Python codecs. Why not use U+DCxx for non-UTF-8 encodings too? Overall I like the PEP: I think it's the best proposal so far that doesn't put an heavy burden on applications that only want to do simple things with the API. -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 (again)
2009/4/28 Thomas Breuel : > If we follow PEP 383, you will get lots of errors anyway because those > strings, when encoded in utf-8b, will result in an error when you try to > write them on a Windows file system or any other system that doesn't allow > the byte sequences that the utf-8b encodes. I'm not sure if when you say "write them on a Windows FS" you mean from within Windows itself or a filesystem mounted on another OS, so I'll cover both cases. Let's suppose that I use Python 2.x or something else to create a file with name b'\xff'. My (Linux) system has a sane configuration and the filesystem encoding is UTF-8, so it's an invalid name but the kernel will blindly accept it anyway. With this PEP, Python 3.1 listdir() will convert b'\xff' to the string '\udcff'. Now if this string somehow ends up in a Python 3.1 program running on Windows and it tries to create a file with this name, it will work (no exception will be raised). The Windows GUI will display the standard "invalid character" symbol (an empty box) when listing this file, but this seems reasonable since the original file was displayed as "?" by the Linux console and with another invalid character symbol by the GNOME file manager. OTOH if I write the same file on a Windows filesystem mounted on another OS, there will be in place an automatic translation (probably done by the OS kernel) from the user-visible filesystem encoding (see e.g. the "iocharset" or "utf8" mount options for vfat on Linux) to UTF-16. Which means that the write will fail with something like: IOError: [Errno 22] invalid filename: b'/media/windows_disk/\xff' (The "problem" is that a vfat filesystem mounted with the "utf8" option on Linux will only accept byte sequences that are valid UTF-8, or at least reasonably similar: e.g. b'\xed\xb3\xbf' is accepted.) Again this seems reasonable since it already happens in Python 2 and with pretty much any other software, including GNU cp. I don't see how Martin can do better than this. Well, ok, I guess he could break into my house and rename the original file to something sane... -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces
2009/4/28 Glenn Linderman : > The switch from PUA to half-surrogates does not resolve the issues with the > encoding not being a 1-to-1 mapping, though. The very fact that you think > you can get away with use of lone surrogates means that other people might, > accidentally or intentionally, also use lone surrogates for some other > purpose. Even in file names. It does solve this issue, because (unlike e.g. U+F01FF) '\udcff' is not a valid Unicode character (not a character at all, really) and the only way you can put this in a POSIX filename is if you use a very lenient UTF-8 encoder that gives you b'\xed\xb3\xbf'. Since this byte sequence doesn't represent a valid character when decoded with UTF-8, it should simply be considered an invalid UTF-8 sequence of three bytes and decoded to '\udced\udcb3\udcbf' (*not* '\udcff'). Martin: maybe the PEP should say this explicitly? Note that the round-trip works without ambiguities between '\udcff' in the filename: b'\xed\xb3\xbf' -> '\udced\udcb3\udcbf' -> b'\xed\xb3\xbf' and b'\xff' in the filename, decoded by Python to '\udcff': b'\xff' -> '\udcff' -> b'\xff' -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces
2009/4/28 Hrvoje Niksic :
> Lino Mastrodomenico wrote:
>>
>> Since this byte sequence [b'\xed\xb3\xbf'] doesn't represent a valid
>> character when
>> decoded with UTF-8, it should simply be considered an invalid UTF-8
>> sequence of three bytes and decoded to '\udced\udcb3\udcbf' (*not*
>> '\udcff').
>
> "Should be considered" or "will be considered"? Python 3.0's UTF-8 decoder
> happily accepts it and returns u'\udcff':
>
>>>> b'\xed\xb3\xbf'.decode('utf-8')
> '\udcff'
Only for the new utf-8b encoding (if Martin agrees), while the
existing utf-8 is fine as is (or at least waaay outside the scope of
this PEP).
--
Lino Mastrodomenico
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
2009/5/3 "Martin v. Löwis" : > With issue 3672 resolved, it is now unnecessary to introduce > an utf-8b codec, since the utf-8 codec will properly report errors > for all byte sequences invalid in UTF-8, including lone surrogates. > Therefore, utf-8b can be implemented solely through the error handler. That's even nicer. One minor detail though, in the sentence: "non-decodable bytes >128 will be represented as lone half surrogate" ">" should be ">=". -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
2009/5/5 Stephen J. Turnbull : > Third, it is not clear to me why non-decodable ASCII should be an > error. The PEP originally allowed the conversion to U+DCxx of bytes below 128 that cannot be decoded by the encoding used, but this creates potential security problems. See: <http://mail.python.org/pipermail/python-dev/2009-April/089102.html> -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 383 update: utf8b is now the error handler
2009/5/6 Antoine Pitrou : > By the way, what are the ASCII characters that are not suppported by > Shift-JIS? > Not many I suppose? (if I read the Wikipedia entry correctly, it's only the > backslash and the tilde). The biggest problem with Shift-JIS is that a perfectly valid unicode character above 127 can be encoded to a byte sequence that includes bytes in range(128). E.g. the character 掛 (a.k.a. '\u639b') when encoded with Shift-JIS becomes the two bytes sequence b'\x8a|'. Notice that the second byte is 124, which on POSIX is usually interpreted as the pipe character and can have security implications. It's a know problem with Shift-JIS and was fixed in UTF-8. -- Lino Mastrodomenico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] SoC proposal: multimedia library
Hello everyone, I would like to participate as a student in google Summer of Code and I'm interested in feedback on a multimedia library for Python. The library I propose should have the following features: * the capability to extract and decompress video and audio from a few common multimedia file format; * and, vice versa, it can create a compressed video file from a series of still images and from uncompressed audio; * it should have an interface as simple and pythonic as possible; * it must be cross-platform, easy to extend and not be limited to a single file container format or to some specific audio or video compression format; * at least a subset of the supported formats should be available without any external dependency. Why? I think this can be an useful addition to the standard library: the goal isn't to reimplement a full fledged media player like mplayer or VLC, but to offer, e.g., a very simple way to add multimedia animations to a Tkinter application. Or, for an application that generates animations, a simple way to save them in a common file format that can be played by existing media players. How? I know that this may sound like a project way too big for SoC, but I think it may be feasible: I plan to split it in two parts. A low level part responsible for splitting encoded audio and video data from multimedia files (demuxing) and putting it together in new multimedia files (muxing). I plan to initially only support AVI and MOV/Quicktime files because they are relatively simple and well documented (and I have already written Python programs for manipulating these two formats, so I know them pretty well). New container formats can be added later and, if the library interface meets the requirements, existing application will be able to use them without any modification. The second part of the library is a collection of codecs for compressing and decompressing audio and video streams. I don't plan to invest too much time here: the idea is to write only a few ctypes wrappers around existing libraries for common formats (e.g. MPEG4/XVID and Theora for video and MP3 and Vorbis for audio) and/or use existing third-part Python bindings. Again, new codecs can be added later. Similar libraries. There are already a number of multimedia libraries both inside and outside of the standard library, but AFAIK they all have limitations: * a lot of them are specific for a single container format or codec (e.g. aifc, wave, oggpy, python-vorbis...); * they aren't cross-platform (e.g. videoreader). Is there any interest in a library of this kind (inside or outside of the stdlib)? Suggestions? Criticism? -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SoC proposal: multimedia library
2007/3/25, Terry Reedy <[EMAIL PROTECTED]>: > There is a Python wrapping of the cross-platform Simple Directmedia Library > (C) that is a major part of PyGame. Are you familiar with both and how > does your proposal improve on the current situation. Yes I know both Pygame and SMPEG (the SDL MPEG library used by Pygame). The differences with my proposal are: * SMPEG is not a general purpose library: it only supports MPEG-1 video in a MPEG system stream container (and the corresponding audio); * it only supports decoding, not encoding; * the Pygame wrapper is strictly tied to Pygame itself and direct playback; e.g.: AFAIK there is no way to access the uncompressed audio track of a MPEG file, you can only "play" a movie; * it's CPython specific: my plan is to use only Python and ctypes, to make it easily portable to PyPy. The nearest existing library to what I propose that I was able to find is PyMedia (http://www.pymedia.org/ ). But it still isn't what I want: their interface is too low level and not exactly pythonic and their implementation is written in C and C++ (a very old fork of the ffmpeg library). 2007/3/25, Josiah Carlson <[EMAIL PROTECTED]>: > For decoding, not many packages can currently match VLC. It has wrappers > for most major GUI toolkits, and seems to be easily accessable via > ctypes. There are also bindings for various media players in wxPython, > though they are strictly play only - no screenshots, etc. Yes, but I don't want to implement yet another media player, my proposal is for a library that supports both encoding and decoding and doesn't force the use of a specific output layer. E.g. you should be able to get the uncompressed data of a video frame and pass it to Tkinter.Image, or PIL (the Python Imaging Library), or do fancy things to it with numpy... > For encoding, MediaCoder exists, but seems to be Windows only. Well, technically it works under Linux too, with Wine. ;-) But, yes, I want something a bit more cross-platform than that. > I would > personally suggest limiting support and going for ffmpeg (via ctypes > would be quickest, though a SWIG wrapping could make it more convenient). Yes, I thought about simply writing a wrapper around ffmpeg, but I think it would be preferable to write at least the (de)muxing part of the library in Python since this should guarantee that at least a subset of the functionality (e.g. an AVI file with MJPEG video and u-law sound) is always available without any dependecy outside of the standard library and of libraries that are pretty much always available on Linux/MacOSX/Windows. Of course the CPU intensive work of encoding and decoding audio and video will be done by existing C libraries. As I said before I have a slight preference for using ctypes instead of the CPython APIs because this means that the library will (probably) be usable unmodified by PyPy, but I can change my mind if the community prefers something else. -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)
2007/3/26, Pete Shinners <[EMAIL PROTECTED]>: > My main question is what is the image and sound container passed back to > Python? > This single thing along would be worth a SoC if it could be implemented across > all libraries. > > Will your image objects be transferrable to PIL, Pygame, PyOpengl, Numpy, > Pythonmagick, Pycairo, wxPython, etc? It would be best if this could avoid the > "fromstring/tostring" mojo that always requires and extra copy of the data for > each transfer. I agree. I withdrew my original "multimedia library" idea and submitted a proposal for the creation of two standard Image and Sound classes. If the PSF accepts my proposal and if Google grants a student slot for me, I will try to implement an interface for the two classes that can work well for everyone and be easily usable from both the Python and the C side. I guess this will require a PEP, if this is the case I will volunteer to write it. And, most important, I will create patches for the appropriate standard library modules and for a lot of external libraries, to try to make the use of the two new classes really interoperable. Finally, as Greg suggested, adding the new buffer interface to the two classes can be a really good idea. I'm following with interest that discussion. Keeping fingers crossed... -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)
2007/3/27, Anthony Baxter <[EMAIL PROTECTED]>: > On Tuesday 27 March 2007 11:03, Lino Mastrodomenico wrote: > > I agree. I withdrew my original "multimedia library" idea and > > submitted a proposal for the creation of two standard Image and > > Sound classes. > > Ideally you'd hook this into the standard library's existing sound > file handling modules. Yes, in my SoC application I listed the following stdlib modules (for both sound and images): Tkinter, Tix, wave, ossaudiodev, winsound, aifc, audioop, sunau, sunaudiodev and imageop. There are also a few Mac modules that can probably use the new classes: videoreader, PixMapWrapper and some Carbon modules. IRIX goodness: al, imgfile, jpeg and gl, but I can't test these. BTW, will they be still here in Python 3.0? -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)
2007/3/28, Lino Mastrodomenico <[EMAIL PROTECTED]>: > IRIX goodness: al, imgfile, jpeg and gl, but I can't test these. BTW, > will they be still here in Python 3.0? Ok, I found the answer: PEP 3108. Sorry for the noise. -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New Super PEP
2007/4/29, Gustavo Carneiro <[EMAIL PROTECTED]>: > On 29/04/07, James Y Knight <[EMAIL PROTECTED]> wrote: > > Since calling super with any > > arguments other than the exact same arguments you have received is > > nearly always wrong, > > Erm. Excuse me, but are you saying this code is wrong? > > class Rectangle: > def __init__(self, width, height): > self.width = width > self.height = height > > class Square: > def __init__(self, side): > Rectangle.__init__(self, side, side) You probably mean "class Square(Rectangle):". Anyway it's not wrong, but it isn't multiple-inheritance-friendly either. -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 238 - The // operator should truncate instead of floor
2007/8/29, dany <[EMAIL PROTECTED]>: > For some reason, the integer division behaves unexpectedly for negative > integers. > > Looking deeper in the python PEPs, I saw that division on integers is > defined as: idiv(a, b) = floor(fdiv(a, b)). > This non-quotient division leads to some odd results, e.g. Python seems > to think -3/2+3/2 = -1. This is clearly, and correct me if I'm mistaken > - wrong. FAQ: Why does -22 / 10 return -3? http://www.python.org/doc/faq/programming/#why-does-22-10-return-3 Past discussion: <http://mail.python.org/pipermail/python-list/2005-February/thread.html#306577> Also, FYI, TCL and Ruby behave the same way. -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
