[issue31184] Fix data descriptor detection in inspect.getattr_static
David Halter added the comment: This is not a duplicate. It is related to https://bugs.python.org/issue26103, because __get__ is not required anymore for an object to be a data descriptor. The current code on master (of inspect.getattr_static) still thinks a descriptor has both __get__ and __set__ set. Since issue #26103 has been fixed, it's now clear that my patch is slightly wrong, but I'm happy to fix that if someone is actually going to review it. -- resolution: duplicate -> status: closed -> open versions: +Python 3.10, Python 3.11, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue31184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40360] Deprecate lib2to3 (and 2to3) for future removal
David Halter added the comment: I'm the maintainer of parso. Feel free to addd me to the Nosy List if we have these discussions in the future. Parso is indeed a lib2to3 fork with error recovery, round tripping and incremental parsing as its features. Most pgen2 code has been rewritten since for various reasons, but it's essentially still LL(1). We're currently trying to think how to proceed with Non-LL(1). For very simple cases a few hacks could suffice, but for larger stuff we will probably need to implement some form of PEG parsing. I'm mostly worried about incremental parsing breaking if we replace the PEG parser, not about writing the PEG parser. But I guess we'll find a way, because I don't want to abandon Jedi (which depends on parso). -- nosy: +davidhalter ___ Python tracker <https://bugs.python.org/issue40360> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40360] Deprecate lib2to3 (and 2to3) for future removal
David Halter added the comment: Parso's incremental parser is a terrible idea. It also works and is pretty fast, but the design is pretty terrible (it took me a lot of fuzzing to make sure that it works decently well). The basic problem is that it's reusing nodes in a mutable way. If I were to redo it, I would probably choose a similar approach to Roslyn's red/green trees. It's probably also possible to use these approaches in Python, but they might be quite a bit slower than what I'm using (because recreating nodes can be quite expensive). I imagine that one of the biggest issues with parsing PEG in parso would be to do it with error recovery AND incremental parsing. That combination can be quite annoying, but it's definitely still possible. I'm not really sure about the future of parso with PEG. I'm definitely going to have to find a way to parse 3.10+ (so Jedi is going to keep working), however I feel like that it's hard to achieve a fast parser in pure Python. Parso is like 20% faster, but still more than ten times slower than the CPython parser... -- ___ Python tracker <https://bugs.python.org/issue40360> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40360] Deprecate lib2to3 (and 2to3) for future removal
David Halter added the comment: @gvanrossum > Does parso have to be pure Python? If not, we could generate C code like we > do for CPython's parser. I would rather write the parser either in C or Rust. So no, parso does not need to be pure Python. > Now, that doesn't work for incremental parsing, but I did an alternative > implementation that uses a stack machine, also in C, that's only 2x slower > than the PEG parser. Maybe that could be adapted to incremental parsing > (because it's a stack machine). Error recovery is still a research project > (at least for me -- I'm actually reading papers :-). Makes sense! I was also thinking about GLL parsing. Obviously GLL does not cover all cases where PEG could potentially work, but I doubt that Python ever moves to a place where GLL would not be sufficient. I'm also doing a bit of research on Rust parsers and trying to find a solution for my parsing needs in the future. (I'd rather have a Rust parser than a C one, because I like the language better and both should still work in Python). Please let me know if you're making a lot of progress with PEG parsers and error recovery/incremental parsing. I'm definitely interested in copying an approach if it works :). -- ___ Python tracker <https://bugs.python.org/issue40360> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41506] Inclusion or documentation of extended with syntax in 3.9
Change by David Halter : -- nosy: +davidhalter ___ Python tracker <https://bugs.python.org/issue41506> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41506] Inclusion or documentation of extended with syntax in 3.9
David Halter added the comment: I second this. I just see people complaining about this not working in a lot of tools. This is really not necessary for a feature that should not be used anyway and puts work onto the greater Python ecosystem for no good reason. -- ___ Python tracker <https://bugs.python.org/issue41506> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31184] Fix data descriptor detection in inspect.getattr_static
New submission from David Halter: inspect.getattr_static is currently not identifying data descriptors the right way. Data descriptors are defined by having a __get__ attribute and at least one of the __set__ and __delete__ attributes. Implementation detail: Both __delete__ and __get__ set the same slot called tp_descr_set in CPython. I have attached a patch that fixes the issue IMO. -- files: 0001-Fix-data-descriptor-detection-in-inspect.getattr_sta.patch keywords: patch messages: 300170 nosy: davidhalter priority: normal severity: normal status: open title: Fix data descriptor detection in inspect.getattr_static versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file47078/0001-Fix-data-descriptor-detection-in-inspect.getattr_sta.patch ___ Python tracker <http://bugs.python.org/issue31184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33754] f-strings should be part of the Grammar
New submission from David Halter :
Currently f-strings are a bit of a hack. They certainly work very well for
users, but they are implemented in ast.c and therefore not part of the Python
grammar and the tokenizer.
I want to change this. I wrote an alternative implementation of f-strings in
parso (http://parso.readthedocs.io/en/latest/). The idea I have is to modify
the Python grammar slightly
(https://github.com/davidhalter/parso/blob/master/parso/python/grammar37.txt#L149):
fstring: FSTRING_START fstring_content* FSTRING_END
fstring_content: FSTRING_STRING | fstring_expr
fstring_conversion: '!' NAME
fstring_expr: '{' testlist [ fstring_conversion ] [ fstring_format_spec ] '}'
fstring_format_spec: ':' fstring_content*
We would push most of the hard work to the tokenizer. This obviously means that
we have to add a lot of code there. I wrote a tokenizer in Python for parso
here: in
https://github.com/davidhalter/parso/blob/master/parso/python/tokenize.py. It
is definitely working well. The biggest difference to the current tokenizer.c
is that you have to work with stacks and be way more context-sensitive.
There were attempts to change the Grammar of f-strings like
https://www.python.org/dev/peps/pep-0536/. It hasn't caught on, because it
tried to change the semantics of f-strings. The implementation in parso has not
changed the semantics of f-strings.
In a first step I would like to get this working for CPython and not
tokenize.py. Modifying tokenize.py will not be part of my initial work here.
I have discussed this with Łukasz Langa, so if you guys have no objections I
will start working on it. Please let me know if you support this or not.
--
components: Interpreter Core
messages: 318547
nosy: davidhalter
priority: normal
severity: normal
status: open
title: f-strings should be part of the Grammar
type: enhancement
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue33754>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33754] f-strings should be part of the Grammar
David Halter added the comment: As I wrote before, I'm not trying to change anything about the f-string behavior. It is a refactoring. If anyone wants to change the behavior, I feel like they should probably write a PEP anyway. I personally don't like that f-strings get parsed multiple times. It just smells bad. Also f-strings are IMO not just strings. They should maybe look like strings for other tools to parse them. But they are more or less statements that get executed. The code in ast.c is not bad. Don't get me wrong. I just think that it's the wrong approach. Regarding the edge cases: I don't think there are that many. In the end the ast output will look similar anyway. All the backslashes, string literals and comments can be checked and rejected in the tokenizer already. -- ___ Python tracker <https://bugs.python.org/issue33754> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19002] ``dir`` function does not work correctly with classes.
New submission from David Halter: I recently stumbled over the ``dir`` function not working correctly with classes. This means that ``dir(object)`` won't list ``object.__bases__`` or ``object.__subclasses`` (and many other useful methods). I think that we should change that. The C function ``type_dir`` (in ``Objects/typeobject.c``) has a documentation entry which states: "We deliberately don't suck up its __class__, as methods belonging to the metaclass would probably be more confusing than helpful." I think that that's not true. The current behaviour of ``dir`` is way more confusing, since ``dir`` is typically the tool how we find out about magic methods. I wrote about it in more details here: http://jedidjah.ch/code/2013/9/8/wrong_dir_function/ -- components: Interpreter Core messages: 197471 nosy: davidhalter priority: normal severity: normal status: open title: ``dir`` function does not work correctly with classes. type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker <http://bugs.python.org/issue19002> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15846] literal_eval raises SystemError instead of SyntaxError, because of illegal escape sequence
New submission from David Halter: Hi! I'm the developer of Jedi: https://github.com/davidhalter/jedi, which is an auto-completion library. I try to support Python 2.5-3.2. The bug just appears in Python 3.2. Parsing string literals with literal_eval works normally without a problem, but in this certain constellation, it raises a SystemError (instead of a SyntaxError). The error line is simple: literal_eval(r"'\U'") But: It only raises the error in that particular place: https://github.com/davidhalter/jedi/blob/master/parsing.py#L157 To try and test it: git clone git://github.com/davidhalter/jedi.git cd jedi # uncomment the line after TODO in parsing.Scope.add_docstr python3 test/run.py array The error message: Traceback (most recent call last): File "./run.py", line 51, in run_definition_test result = defs(line_nr, len(line)) File "./run.py", line 49, in defs return set(functions.get_definitions(source, line_nr, indent, path)) File "./functions.py", line 253, in get_definitions scopes = _prepare_goto(source, pos, source_path, f, goto_path) File "./functions.py", line 227, in _prepare_goto scopes = evaluate.follow_statement(stmt) File "./helpers.py", line 23, in __call__ if self.push_stmt(stmt): File "./helpers.py", line 31, in push_stmt self.current = RecursionNode(stmt, self.current) File "./helpers.py", line 75, in __init__ or (self.script == builtin.Builtin.scope) File "./builtin.py", line 408, in scope return self._builtins.parser.module File "./builtin.py", line 42, in parser self._load_module() File "./builtin.py", line 50, in _load_module self._parser = parsing.PyFuzzyParser(source, self.path or self.name) File "./parsing.py", line 1047, in __init__ self.parse() File "./parsing.py", line 1600, in parse stmt, tok = self._parse_statement(self.current) File "./parsing.py", line 1386, in _parse_statement self.scope.add_docstr(self.last_token[1]) File "./parsing.py", line 157, in add_docstr literal_eval(r"'\U'") File "/usr/lib/python3.2/ast.py", line 48, in literal_eval node_or_string = parse(node_or_string, mode='eval') File "/usr/lib/python3.2/ast.py", line 36, in parse return compile(source, filename, mode, PyCF_ONLY_AST) SystemError: ../Objects/tupleobject.c:126: bad argument to internal function Cheers! David -- messages: 169687 nosy: davidhalter priority: normal severity: normal status: open title: literal_eval raises SystemError instead of SyntaxError, because of illegal escape sequence versions: Python 3.2 ___ Python tracker <http://bugs.python.org/issue15846> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16791] itertools.chain.from_iterable doesn't stop
New submission from David Halter: The following lines run infinitely: b = [1] import itertools b += itertools.chain.from_iterable([c] for c in b) In my opinion this is a bug. Clearly you could say that this is an implementation detail. But afaik this is not documented and very misleading. BTW: In PyPy b would be [1, 1]. -- messages: 178296 nosy: davidhalter priority: normal severity: normal status: open title: itertools.chain.from_iterable doesn't stop type: behavior versions: Python 2.7, Python 3.2, Python 3.3 ___ Python tracker <http://bugs.python.org/issue16791> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12855] linebreak sequences should be better documented
David Halter added the comment: I would vote for the inclusion of that patch. I just stumbled over this. -- nosy: +davidhalter ___ Python tracker <http://bugs.python.org/issue12855> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22221] ast.literal_eval confused by coding declarations
Changes by David Halter : -- nosy: +davidhalter ___ Python tracker <http://bugs.python.org/issue1> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
