[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
Larry Hastings writes: > On 4/22/22 19:36, Terry Reedy wrote: > > How about a 'regular' class statement with a special marker of some > > sort. Example: 'body=None'. > > It's plausible. I take it "body=None" would mean the declaration would > not be permitted to have a colon and a class body. So now we have two > forms of the "class" statement, and which syntax you're using is > controlled by a named parameter argument. > > I think this "body=None" argument changing the syntax of the "class" > statement is clumsy. How about another predefined keyword in class definitions? class ForwardDeclared(*bases, metaclass=Meta, declaration=True): pass AIUI, in your scheme you can't have a body in a forward declaration, so in this one you could error on a non-pass body, or just ignore it. I don't see how you can avoid the 'continue class' statement, if you want the actual argument to be an arbitrary expression that evaluates to an incomplete class. Then the symmetry of the 'forward class' statement is appealing. If you restrict it to be a name, though, you could use class SomeModule.ForwardDeclared(declaration=False): # body goes here I think. What's the use case for arbitrary expressions vs. a (possibly qualified) name? A class factory that produces forward declarations? Do you have a use case in mind? I dunno, but all this seems really complicated, unless you go the proxy object route (which is basically what we have now, except that str is extremely limited as a proxy, you could call it a PINO :-). Sure, there are risks if the temporary thing produced by forward class' escapes, but if both the "thing" and the proxy are limited enough, there will be little incentive to do that. Consenting adults. Just add the FAFO warning label. Add features when they're called for. A separate question: There are going to be metaclasses that define __new__ whose instances cannot be used in forward declarations because the class object can't be created until the class body is seen, and therefore cannot be split into __new_forward__ and __new_continue__, I suppose. Is that true? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SQB63XJWE67NVOT4XT6I3RSU4HS73AGL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/24/2022 5:42 AM, Stephen J. Turnbull wrote: What's the use case for arbitrary expressions vs. a (possibly qualified) name? A class factory that produces forward declarations? Do you have a use case in mind? It's: x.py: --8< forward class A() --8< x_impl.py --8< import X continue class X.A: # class body here --8< It needs to be an expression because it's not defining a name, it referring to an existing name. You could use "from X import A" here and avoid a dotted expression, but it still needs to be an expression referring to an existing "forward class". Even if you restrict it to not having dots, it's logically an expression, not a name binding. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VV53MWDTYQCHBCAQGXLZ4SFCKUEHPQ2K/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
Hi Larry, On Sat, Apr 23, 2022 at 1:53 AM Larry Hastings wrote: > But rather than speculate further, perhaps someone who works on one of the > static type analysis checkers will join the discussion and render an informed > opinion about how easy or hard it would be to support "forward class" and > "continue class". I work on a Python static type checker. I think a major issue with this proposal is that (in the separate-modules case) it requires monkey-patching as an import side effect, which is quite hard for both humans and static analysis tools to reason effectively about. Imagine we have a module `foo` that contains `forward class Bar`, a separate module `foo.impl` that contains `continue class Bar: ...`, and then a module `baz` that contains `import foo`. What type of object is `foo.Bar` during the import of `baz`? Will it work for the module body of `baz` to create a singleton instance `my_bar = foo.Bar()`? The answer is that we have no idea. `foo.Bar` might be a non-instantiable "forward class declaration" (or proxy object, in your second variation), or it might be a fully-constituted class. Which one it is depends on accidents of import order anywhere else in the codebase. If any other module happens to have imported `foo.impl` before `baz` is imported, then `foo.Bar` will be the full class. If nothing else has imported `foo.impl`, then it will be a non-instantiable declaration/proxy. This question of import order potentially involves any other module in the codebase, and the only way to reliably answer it is to run the entire program; neither a static type checker nor a reader of the code can reliably answer it in the general case. It will be very easy to write a module `baz` that does `import foo; my_bar = foo.Bar()` and have it semi-accidentally work initially, then later break mysteriously due to a change in imports in a seemingly unrelated part of the codebase, which causes `baz` to now be imported before `foo.impl` is imported, instead of after. There is another big problem for static type checkers with this hypothetical module `baz` that only imports `foo`. The type checker cannot know the shape of the full class `Bar` unless it sees the right `continue Bar: ...` statement. When analyzing `baz`, it can't just go wandering the filesystem aimlessly in hopes of encountering some module with `continue Bar: ...` in it, and hope that's the right one. (Even worse considering it might be `continue snodgrass: ...` or anything else instead.) So this means a type checker must require that any module that imports `Bar` MUST itself import `foo.impl` so the type checker has a chance of understanding what `Bar` actually is. This highlights an important difference between this proposal and languages with real forward declarations. In, say, C++, a forward declaration of a function or class contains the full interface of the function or class, i.e. everything a type checker (or human reader) would need to know in order to know how it can use the function or class. In this proposal, that is not true; lots of critical information about the _interface_ of the class (what methods and attributes does it have, what are the signatures of its methods?) are not knowable without also seeing the "implementation." This proposal does not actually forward declare a class interface; all it declares is the existence of the class (and its inheritance hierarchy.) That's not sufficient information for a type checker or a human reader to make use of the class. Taken together, this means that every single `import foo` in the codebase would have to be accompanied by an `import foo.impl` right next to it. In some cases (if `foo.Bar` is not used in module-level code and we are working around a cycle) it might be safe for the `import foo.impl` to be within an `if TYPE_CHECKING:` block; otherwise it would need to be a real runtime import. But it must always be there. So every single `import foo` in the codebase must now become two or three lines rather than one. There are of course other well-known problems with import-time side effects. All the imports of `foo.impl` in the codebase would exist only for their side effect of "completing" Bar, not because anyone actually uses a name defined in `foo.impl`. Linters would flag these imports as unused, requiring extra cruft to silence the linter. Even worse, these imports would tend to appear unused to human readers, who might remove them and be confused why that breaks the program. All of these import side-effect problems can be resolved by dis-allowing module separation and requiring `forward class` and `continue class` to appear in the same module. But then the proposal no longer helps with resolving inter-module cycles, only intra-module ones. Because of these issues (and others that have been mentioned), I don't think this proposal is a good solution to forward references. I think PEP 649, with some tricks that I've mentioned elsewhere to allow introspecting annotations u
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
I am not worried about the bikeshed part of which syntax to use - and more worried with the possible breaking of a lot of stuff, unless we work with creation of a non-identical "forward object" that is rebound, as in plain name binding, when the second part is declared. I've stated that amidst my ramblings, but Nick Coghlan managed to keep it suscint at https://mail.python.org/archives/list/python-dev@python.org/message/DMITVTUIQKJW6RYVOPQXHD54VSYE7QHA/ """ Something worth considering: whether forward references need to be *transparent* at runtime. If they can just be regular ForwardRef objects then much of the runtime complexity will go away (at the cost of some potentially confusing identity check failures between forward references and the actual class objects). ForwardRef's constructor could also potentially be enhanced to accept a "resolve" callable, and a "resolve()" method added to its public API, although the extra complexity that would bring may not be worth it. Cheers, Nick. """ For that matter, syntax wise, it could just be: break class XX() ... continue class XX: ... It gains at once no backwards incompatibility, and reuse concepts that are already somewhat "grouped together". On Sun, Apr 24, 2022 at 11:25 AM Eric V. Smith wrote: > On 4/24/2022 5:42 AM, Stephen J. Turnbull wrote: > > What's the use case for arbitrary expressions vs. a (possibly > > qualified) name? A class factory that produces forward declarations? > > Do you have a use case in mind? > > It's: > > x.py: > > --8< > forward class A() > --8< > > x_impl.py > > --8< > import X > > continue class X.A: > # class body here > --8< > > It needs to be an expression because it's not defining a name, it > referring to an existing name. You could use "from X import A" here and > avoid a dotted expression, but it still needs to be an expression > referring to an existing "forward class". Even if you restrict it to not > having dots, it's logically an expression, not a name binding. > > Eric > > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VV53MWDTYQCHBCAQGXLZ4SFCKUEHPQ2K/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WBXIZLNKWTJGAQLTFJOOPFJ7VCD42U6D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] What is Modules/rotatingtree.c for?
I've just noticed Modules/rotatingtree.{h,c}, which don't seem to be
used anywhere. Are they just dead code? If so, is there a reason they
haven't been removed?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/46D6DD3OEZEOH4CUPFKOSLVUG6MHV7BU/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sun, Apr 24, 2022 at 10:20 AM Joao S. O. Bueno wrote: > > I am not worried about the bikeshed part of which syntax to use - > and more worried with the possible breaking of a lot of stuff, unless > we work with creation of a non-identical "forward object" that is > rebound, as in plain name binding, when the second part > is declared. I've stated that amidst my ramblings, > but Nick Coghlan managed to keep it suscint at > https://mail.python.org/archives/list/python-dev@python.org/message/DMITVTUIQKJW6RYVOPQXHD54VSYE7QHA/ I don't think name rebinding works. That means that if we have `forward class Bar` in module `foo` and `continue class Bar: ...` in module `foo.impl`, if module `baz` does `from foo import Bar`, it will forever have either the forward reference object or the real class, and which one it has is entirely unpredictable (depends on import ordering accidents of the rest of the codebase.) If `baz` happens to be imported before `foo.impl`, the name `Bar` in the `baz` namespace will never be resolved to the real class, and isn't resolvable to the real class without some outside intervention. > """ > Something worth considering: whether forward references need to be > *transparent* at runtime. If they can just be regular ForwardRef objects > then much of the runtime complexity will go away (at the cost of some > potentially confusing identity check failures between forward references > and the actual class objects). > > ForwardRef's constructor could also potentially be enhanced to accept a > "resolve" callable, and a "resolve()" method added to its public API, > although the extra complexity that would bring may not be worth it. > """ I'm not sure how this `resolve()` method is even possible under the proposed syntax. If `forward class Bar` and `continue class Bar` are in different modules, then how can `forward class Bar` (which must create the "forward reference" object) possibly know which module `continue class Bar: ...` exists in? How can it know how to resolve itself? Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WLZRZIMPRST52UMINB5VB57TOIQVTYQH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: What is Modules/rotatingtree.c for?
On Sun, Apr 24, 2022 at 9:24 AM Patrick Reader <_...@pxeger.com> wrote:
> I've just noticed Modules/rotatingtree.{h,c}, which don't seem to be
> used anywhere. Are they just dead code? If so, is there a reason they
> haven't been removed?
>
grep -R rotatingtree ; grep -R _lsprof
rotatingtree is used by the _lsprof module which is the internal
implementation behind cProfile.
-gps
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/46D6DD3OEZEOH4CUPFKOSLVUG6MHV7BU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/W45DZ2RU75HOKELTMRZY7YL3JH7KS7V5/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
I guess Carl's messages put an end to this proposal as it is. >From "the language side", I can think of nothing short of actually transforming _in place_ (like it is possible with instances of ordinary classes when you assign then a new "__class__") the "forward referenced object" into the complete classes. This is not "unfeasable", but would be hackish as never before done, and there are practical problems: the final size of the class object itself cannot be known before the final committing of the class-body namespace into the final-namespace (slots, and inheritance of built-in classes with different layouts, etc...) Then, it looks like it only would be remotely practical for a static type checker if the class continuation is on the same file as its header. (I think I did not even consider they being in separate files - of course the name-rebinding would not work in that case) On the other hand, if it is practical to declare as header the complete interface of a class, with the intent of putting the actual methods with code later on, that is already possible, with typing.protocols ; whoever would take the time to split a class in two due to annotations, forward referencing, interface declaration separated from implementation, can just as well declare a protocol, as it exists today, I am not sure if typing.Protocol works nicely with inheritance, in the sense that it could inherit from other protocols or concrete classes, and present everything inherited as part of its interface, if it does not, then _that_ is the improvement needed so that what this proto-pep would allow could be done with them. best regards, js -><- On Sun, Apr 24, 2022 at 1:33 PM Carl Meyer wrote: > On Sun, Apr 24, 2022 at 10:20 AM Joao S. O. Bueno > wrote: > > > > I am not worried about the bikeshed part of which syntax to use - > > and more worried with the possible breaking of a lot of stuff, unless > > we work with creation of a non-identical "forward object" that is > > rebound, as in plain name binding, when the second part > > is declared. I've stated that amidst my ramblings, > > but Nick Coghlan managed to keep it suscint at > > > https://mail.python.org/archives/list/python-dev@python.org/message/DMITVTUIQKJW6RYVOPQXHD54VSYE7QHA/ > > I don't think name rebinding works. That means that if we have > `forward class Bar` in module `foo` and `continue class Bar: ...` in > module `foo.impl`, if module `baz` does `from foo import Bar`, it will > forever have either the forward reference object or the real class, > and which one it has is entirely unpredictable (depends on import > ordering accidents of the rest of the codebase.) If `baz` happens to > be imported before `foo.impl`, the name `Bar` in the `baz` namespace > will never be resolved to the real class, and isn't resolvable to the > real class without some outside intervention. > > > """ > > Something worth considering: whether forward references need to be > > *transparent* at runtime. If they can just be regular ForwardRef objects > > then much of the runtime complexity will go away (at the cost of some > > potentially confusing identity check failures between forward references > > and the actual class objects). > > > > ForwardRef's constructor could also potentially be enhanced to accept a > > "resolve" callable, and a "resolve()" method added to its public API, > > although the extra complexity that would bring may not be worth it. > > """ > > I'm not sure how this `resolve()` method is even possible under the > proposed syntax. If `forward class Bar` and `continue class Bar` are > in different modules, then how can `forward class Bar` (which must > create the "forward reference" object) possibly know which module > `continue class Bar: ...` exists in? How can it know how to resolve > itself? > > Carl > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/US6OOS7OJMEZKQQC456LDRQKC2HRP72P/ Code of Conduct: http://python.org/psf/codeofconduct/
