[issue10785] parser: store the filename as an unicode object

Changes by STINNER Victor : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue10785] parser: store the filename as an unicode object

Roundup Robot added the comment: New changeset 6e9dc970ac0e by Victor Stinner in branch 'default': Issue #10785: Store the filename as Unicode in the Python parser. http://hg.python.org/cpython/rev/6e9dc970ac0e -- nosy: +python-dev ___ Python tracker

[issue10785] parser: store the filename as an unicode object

STINNER Victor added the comment: @Benjamin: You told me that you don't want two versions of pgen, but I don't remember why. As my work on #3080 is mostly done, I now plan to patch the Python parser to store the filename as Unicode. So could you please review the patch attached to this issue?

[issue10785] parser: store the filename as an unicode object

Changes by Antoine Pitrou : -- nosy: +benjamin.peterson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue10785] parser: store the filename as an unicode object

Changes by STINNER Victor : Removed file: http://bugs.python.org/file20184/parser_filename_obj-2.patch ___ Python tracker ___ ___ Python-bugs-

[issue10785] parser: store the filename as an unicode object

Changes by STINNER Victor : Removed file: http://bugs.python.org/file20180/parser_filename_obj.patch ___ Python tracker ___ ___ Python-bugs-li

[issue10785] parser: store the filename as an unicode object

STINNER Victor added the comment: Version 3 of the patch to fix also #9319. -- Added file: http://bugs.python.org/file20271/parser_filename_obj-3.patch ___ Python tracker ___ __

[issue10785] parser: store the filename as an unicode object

STINNER Victor added the comment: err_clear() should set err->filename to NULL. -- versions: -Python 3.2 ___ Python tracker ___ ___

[issue10785] parser: store the filename as an unicode object

STINNER Victor added the comment: Version 2 of the patch: - remove filename attribute from perrdetail and tok_state structure in PGEN mode, and add a comment to explain why - rename filename_obj to filename - indenterror() no longer print the input filename in PGEN mode -- Added fil

[issue10785] parser: store the filename as an unicode object

STINNER Victor added the comment: When testing my patch, I found and fixed two bugs in pgen: - r87557: PGEN was not defined to compile pgenmain.c and printgrammar.c - r87558: pgen error was ignored on "make Parser/pgen.stamp" (when executing pgen to compile the grammar) -- _

[issue10785] parser: store the filename as an unicode object

STINNER Victor added the comment: > maybe a more elegant solution than changing the type conditional > on PGEN can be found In pgen, the filename is only used to display the following warning, in indenterror(): : inconsistent use of tabs and spaces in indentation In pratical, this warnin

[issue10785] parser: store the filename as an unicode object

Alexander Belopolsky added the comment: I like the idea, but I don't like the trend that parser code continues to diverge from pgen. I understand that most of the Python runtime is not available to pgen, but maybe a more elegant solution than changing the type conditional on PGEN can be foun

[issue10785] parser: store the filename as an unicode object

Changes by STINNER Victor : Added file: http://bugs.python.org/file20180/parser_filename_obj.patch ___ Python tracker ___ ___ Python-bugs-list

[issue10785] parser: store the filename as an unicode object

Changes by STINNER Victor : Removed file: http://bugs.python.org/file20179/parse_filename_obj.patch ___ Python tracker ___ ___ Python-bugs-lis

[issue10785] parser: store the filename as an unicode object

New submission from STINNER Victor : The Python parser stores the filename as a byte string. But it decodes the filename on error because most Python functions now use unicode strings. Instead of decoding the filename at error, which may raise a new error, I propose to decode the filename on t