Re: [Python-Dev] PEP 246, redux
Alex, This is wonderful work, thank you for keeping the ball in the air; I'm honored to keep my name as a co-author -- kinda like a free lunch. Phillip, Once again, thank you! Without PyProtocols and your advocacy, this proposal might have been buried in the historical bit-bucket. On Mon, Jan 10, 2005 at 12:43:44PM -0500, Phillip J. Eby wrote: | -1 if this introduces a performance penalty to a wide range of | adaptations (i.e. those using abstract base classes), just to support | people who want to create deliberate Liskov violations. I personally | don't think that we should pander to Liskov violators, especially since | Guido seems to be saying that there will be some kind of interface | objects available in future Pythons. I particularly like Alex's Liskov violation error; although it is not hugely common, it does happen, and there should be a way for a class to indicate that it's only being used for implementation. Perhaps... if the class doesn't have a __conform__ method, then its adaptation is automatic (that is, only the class can raise this case). The rationale for only enabling one of the two paths is that the base class would have been in-place before the derived class was created; therefore, it is highly unlikely that __adapt__ would ever be of help. Therefore, there might be a performance penalty, but it'd be really small, simply checking to see if the slot is filled in. Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Mon, Jan 10, 2005 at 01:34:59PM -0500, Phillip J. Eby wrote: | The performance penalty I was talking about was for using an abstract | base class, in a subclass with a __conform__ method for conformance to | other protocols. In this case, __conform__ will be uselessly called | every time the object is adapted to the abstract base class. *nod* If this proposal was "packaged" with an "interface" mechanism, would this address your concern? In this scenerio, there are two cases: - Older classes will most likely not have a __conform__ method. - Newer classes will use the 'interface' mechanism. In this scenerio, there isn't a performance penalty for the usual case; and for migration purposes, a flag could be added to disable the checking. Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Tue, Jan 11, 2005 at 12:54:36PM -0500, Phillip J. Eby wrote: | * Replacing LiskovViolation is possible by dropping type/isinstance | checks from adapt(), and adding an isinstance check to | object.__conform__; Liskov violators then override __conform__ in their | class to return None when asked to conform to a protocol they wish to | reject, and return super().__conform__ for all other cases. This | achieves your use case while simplifying both the implementation and the | usage. I'd rather not assume that class inheritance implies substitutability, unless the class is "marked" as an interface (assuming that one doesn't have interfaces). I'd like it to be explicit -- a bit of a nudge to remind a developer to verify substitutability is a good thing. In this scenerio, a LiskovViolation exception isn't needed (aside, I don't see the rationale for the exception: to prevent third party adapters?). Could we make a boilerplate __conform__ which enables class-based substitutability a well-known decorator? | * In my experience, incorrectly deriving an interface from another is the | most common source of unintended adaptation side-effects, not adapter | composition It'd be nice if interfaces had a way to specify a test-suite that could be run against a component which claims to be compliant. For example, it could provide invalid inputs and assert that the proper errors are returned, etc. Best, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Tue, Jan 11, 2005 at 03:30:19PM -0500, Phillip J. Eby wrote: | Clark said he didn't want to assume substitutability; I was pointing out | that he could choose to not assume that, if he wished, by implementing an | appropriate __conform__ at the base of his hierarchy. Oh, that's sufficient. If someone making a base class wants to assert that derived classes should check compliance (rather than having it automagic), then they can do this. Good enough! | I don't agree with Clark's use case, but my | proposal supports it as a possibility, and yours does not. It was a straw-man; and I admit, not a particularly compelling one. | To implement a Liskov violation with my proposal, you do exactly the same | as with your proposal, *except* that you can simply return None instead | of raising an exception, and the logic for adapt() is more | straightforward. I think I prefer just returning None rather than raising a specific exception. The semantics are different: None implies that other adaptation mechanisms (like a registry) could be tried, while LiskovException implies that processing halts and no further adaptation techniques are to be used. In this case, None is the better choice for this particular case since it would enable third-parties to register a wrapper. Overall, I think both you and Alex are now proposing essentially the same thing... no? Best, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Wed, Jan 12, 2005 at 10:16:14AM -0800, Guido van Rossum wrote: | But now, since I am still in favor of automatic "combined" adaptation | *as a last resort*, I ask you to consider that Python is not C++, and | that perhaps we can make the experience in Python better than it was | in C++. Perhaps allowing more control over when automatic adaptation | is acceptable? | | For example, inteface B (or perhaps this should be a property of the | adapter for B->C?) might be marked so as to allow or disallow its | consideration when looking for multi-step adaptations. We could even | make the default "don't consider", so only people who have to deal | with the multiple A's and/or multiple C's all adaptable via the same B | could save themselves some typing by turning it on. How about not allowing transitive adaptation, by default, and then providing two techniques to help the user cope: - raise a AdaptIsTransitive(AdaptationError) exception when an adaptation has failed, but there exists a A->C pathway using an intermediate B - add a flag to adapt, allowTransitive, which defaults to False This way new developers don't accidently shoot their foot off, as Alex warns; however, the price for doing this sort of thing is cheap. The AdaptIsTransitive error could even explain the problem with a dynamic error message like: "You've tried to adapt a LDAPName to a FirstName, but no direct translation exists. There is an indirect translation using FullName: LDAPName -> FullName -> FirstName. If you'd like to use this intermediate object, simply call adapt() with allowTransitive = True" Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Wed, Jan 12, 2005 at 04:07:37PM -0600, Ian Bicking wrote:
| A two-step adaptation encodes specific intention that it seems transitive
| adaption would be blind to.
Exactly. Nice example Ian. To parrot your example a bit more
concretely, the problem happens when you get two different
adaptation paths.
String -> PathName -> File
String -> StringIO -> File
Originally, Python may ship with the String->StringIO and
StringIO->File adapters pre-loaded, and if my code was reliant upon
this transitive chain, the following will work just wonderfully,
def parse(file: File):
...
parse("helloworld")
by parsing "helloworld" content via a StringIO intermediate object. But
then, let's say a new component "pathutils" registers another adapter pair:
String->PathName and PathName->File
This ambiguity causes a few problems:
- How does one determine which adapter path to use?
- If a different path is picked, what sort of subtle bugs occur?
- If the default path isn't what you want, how do you specify
the other path?
I think Phillip's suggestion is the only resonable one here, ambiguous
cases are an error; ask the user to register the adapter they need, or
do a specific cast when calling parse().
| As I think these things through, I'm realizing that registered
| adaptators really should be 100% accurate (i.e., no information loss,
| complete substitutability), because a registered adapter that seems
| pragmatically useful in one place could mess up unrelated code, since
| registered adapters have global effects.
I think this isn't all that useful; it's unrealistic to assume that
adapters are always perfect. If transitive adaptation is even
permitted, it should be unambiguous. Demanding that adaption is
100% perfect is a matter of perspective. I think String->StringIO
and StringIO->File are perfectly pure.
| Perhaps transitivity seems dangerous because that has the potential to
| dramatically increase the global effects of those registered adapters.
I'd prefer,
1. adaptation to _not_ be transitive (be explicit)
2. a simple mechanism for a user to register an explicit
adaptation path from a source to a destination:
adapt.path(String,PathName,File)
to go from String->File, using PathName as an intermediate.
3. an error message, AdaptationError, to list all possible
adaptation paths:
Could not convert 'String' object to 'File' beacuse
there is not a suitable adapter. Please consider an
explicit conversion, or register a composite adapter
with one of the following paths:
adapt.path(String,PathName,File)
adapt.path(String,StringIO,File)
3. raise an exception when _registering_ a 'path' which would
conflict with any existing adapter:
"Could not complete adapt.path(String,PathName,File)
since an existing direct adapter from String to Path
already exists."
"Could not complete adapt.path(String,PathName,File)
since an existing path String->StringIO->File is
already registered".
I'd rather have the latter error occur when "importing" modules
rather than at run-time. This way, the exception is pinned on
the correct library developer.
Best,
Clark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Son of PEP 246, redux
Phillip, In my mind, the driving use-case for PEP 246 was to allow causual programmers to plug components together and have it 'just work'; it does this by enabling the component vendors to carry on a discussion via __adapt__ and __conform__ to work together. I was not picturing that your average developer would be using this sort of thing. On Wed, Jan 12, 2005 at 09:57:07PM -0500, Phillip J. Eby wrote: | First, adapter abuse is *extremely* attractive to someone new to the | concept -- so from here on out I'm going to forget about the idea that we | can teach people to avoid this solely by telling them "the right way to | do it" up front. | | The second, much subtler point I noticed from your posts, was that | *adapter abuse tends to sooner or later result in adapter diamonds*. However, I'd like to assert that these cases emerge when you have a registry /w automatic transitive adaptation. These problems can be avoided quite easily by: - not doing transitive adaptation automatically - making it an error to register more than one adapter from A to Z at any given time; in effect, ban diamonds from ever being created - make it easy for a user to construct and register an adapter from A to Z, via an intermediate X, adapt.registerTransitive(A,X,Z) - if an adaptation from A to Z isn't possible, give a very meaningful error listing the possible pathways that one could build a 'transitive adaption', adaptation path', perhaps even showing the command that will do it: adapt.registerTransitive(A,B,C,Z) adapt.registerTranstive(A,Q,Z) adapt.registerTranstive(A,X,Z) The results of this operation: - most component vendors will use __adapt__ and __conform__ rather than use the 'higher-precedent' registry; therefore, transitive adaption isn't that common to start with - if two libraries register incompatible adpater chains during the 'import' of the module, then it will be an error that the casual developer will associate with the module, and not with their code - casual users are given a nice message, like "Cannot automatically convert a String to a File. Perhaps you should do a manual conversion of your String to a File. Alternatively, there happen to be two adaptation paths which could do this for you, but you have to explicitly enable the pathway which matches your intent: To convert a String to a File via StringIO, call: adapt.registerTranstive(String,StringIO,File) To convert a String to a File via FileName, call: adapt.registerTranstive(String,FileName,File)" | What that suggests to me is that it might well be safe enough in practice | to let new users of adaptation whack their hand with the mallet now and | then, given that *now* it's possible to give a much better explanation of | "as a" than it was before. By disabling (the quite dangerous?) transitive adaptation, one could guide the user along to the result they require without having them shoot themselves in the foot first. | What this also suggests to me is that maybe adaptation and interfaces are | the wrong solution to the problems we've been trying to solve with them | -- adding more objects to solve the problems created by having lots of | objects. :) I didn't see how your remaining post, in particular, Dylan's protocol was much different from an mixin/abstract-base-class. Regardless, getting back to the main goal I had when writing PEP 246 -- your alternative proposal still doesn't seem to provide a mechanism for component developers to have a dialogue with one another to connect components without involving the application programmer. Cheers! Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Thu, Jan 13, 2005 at 10:35:39AM +, Paul Moore wrote: | One thing I feel is key is the fact that adaptation is a *tool*, and | as such will be used in different ways by different people. That is | not a bad thing, even if it does mean that some people will abuse the tool. | | Now, a lot of the talk has referred to "implicit" adaptation. I'm | still struggling to understand how that concept applies in practice, | beyond the case of adaptation chains - at some level, all adaptation | is "explicit", insofar as it is triggered by an adapt() call. The 'implicit' adaptation refers to the automagical construction of composite adapters assuming that a 'transitive' property holds. I've seen nothing in this thread to explain why this is so valueable, why it shouldn't be explicit, and on the contrary, most of the "problems with adapt()" seem to stem from this aggressive extension of what was proposed: Automatic construction of adapter chains is _not_ part of the original PEP 246 and I hope it remains that way. I've outlined in several posts how this case could be made easy for a application developer to do: - transitive adapters should always be explicit - it should be an error to have more than one adapter from A to Z in the registry - when adaptation fails, an informative error message can tell the application developer of possible "chains" which could work - registration of transitive adapters can be simple command application developers use: adapt.transitive(from=A,to=Z,via=M) error message can tell an application developer | James Knight's example (which seemed to get lost in the discussion, or | at least no-one commented on it) brought up a new point for me, namely | the fact that it's the library writer who creates interfaces, and | calls adapt(), but it's the library *user* who says what classes | support (can be adapted to) what interface. I hadn't focused on the | different people involved before this point. I'd say the more common pattern is three players. The framework builder, the component budiler, and the application designer. Adapt provides a mechansim for the framework builder (via __adapt__) and the component builder (via __conform__) to work together without involving the application designer. The 'registry' idea (which was not explored in the PEP) emerges from the need, albeit limited, for the application developer who is plugging a component into a framework, to have some say in the process. I think that any actions taken by the user, by registering an adapter, should be explicit. The 'diamond' problem discussed by Phillip has only confirmed this belief. You don't want the adapt() system going around assuming transitivity. However, if the application developer is certain that a conversion path from A to Z going through B, and/or Y will work, then it should be easy for them to specify this adaptation path. | Now, if we have a transitive case A->B->C, where A is written by "the | user", and C is part of "the library" and library code calls | adapt(x,C) where x is a variable which the user supplies as an object | of type A, then WHO IS RESPONSIBLE FOR B And does it matter, and | if it does, then what are the differences? Great question. But I'd like to rephrase that C is probably a framework, A and B are probably components; and we assume that either the framework or component developers have enabled A->B and B->C. If the user wishes to make an adapter from A->C assuming no (or acceptable for his purposes) information loss from A->C through B, then this is his/her choice. However, it shouldn't be done by the framework or component developers unless it is a perfect adaptation, and it certainly shouldn't be automagic. I don't think who owns B is particularly more important than A or C. | As I write this, being careful *not* to talk interms of "interfaces" | and "classes", I start to see Philip's point - in my mind, A (written | by the user) is a class, and C (part of the library) is an | "interface". So the answer to the question above about B is that it | depends on whether B is an interface or a class - and the sensible | transitivity rules could easily (I don't have the experience to | decide) depend on whether B is a class or an interface. I'd like to say that _any_ transitivity rule should be explicit; there is a point where you make it easy for the programmer, but for heavens sake, let's not try to do their job. | BUT, and again, Philip has made this point, I can't reason about | interfaces in the context of PEP 246, because interfaces aren't | defined there. So PEP 246 can't make a clear statement about | transitivity, precisely because it doesn't define interfaces. B
Re: [Python-Dev] PEP 246, redux
On Fri, Jan 14, 2005 at 02:03:57AM +1000, Nick Coghlan wrote: | Carlos Ribeiro wrote: | > On Thu, 13 Jan 2005 10:08:10 -0500, Phillip J. Eby wrote: | > > With the previous PEP, people could create all sorts of subtle problems | > > in their code (with or without transitivity!) and have no direct | > > indicator of a problem. Clark and Ian made me realize this with their | > > string/file/path discussions -- *nobody* is safe from implicit | > > adaptation if adaptation actually creates new objects with independent | > > state! An adapter's state needs to be kept with the original object, | > > or not at all, and most of the time "not at all" is the correct answer. | > | >+1, specially for the last sentence. An adapter with local state is | >not an adapter anymore! It's funny how difficult it's to get this... | >but it's obvious once stated. | | +lots -1 There is nothing wrong with an adapter from String to File, one which adds the current read position in its state. No adapter is a perfect translation -- or you woudn't need them in the first place. An adapter, by default, does just that, it wraps the object to make it compliant with another interface. To disallow it from having local state is, like taking the wheels off a car and expecting it to be useful (in somecases it is, for ice fishing, but that's another and rather oblique story). Ian stated the issue properly: adapters bring with them an intent, which cannot, in general way, be expressed in code. Therefore, combining adapters haphazardly will, of course, get you into trouble. The solution is simple -- don't do that. PEP 246 should at the very least remain silent on this issue, it should not encourage or specify automagic transitive adaptation. If a user blows their foot off, by their own actions, they will be able to track it down and learn; if the system shots their foot off, by some automatic transitive adaption; well, that's another issue. | Now that it's been stated, I think this is similar to where implicit type | conversions in C++ go wrong, and to the extent that PEP 246 aligns with | those. . . *shudder*. PEP 246 doesn't align at all with this problem. Best, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Fri, Jan 14, 2005 at 12:30:17AM +1000, Nick Coghlan wrote: | Anyway, I'd like to know if the consensus I think you've reached is the | one the pair of you think you've reached :) This stated position is not the current PEP status, and it's not a concensus position I share. | That is, with A being our starting class, C being a target class, and F | being a target interface, the legal adaptation chains are: | # Class to class | A->C | # Class to interface, possibly via other interfaces | A(->F)*->F PEP246 should not talk about legal or illegal adaption chains, all adaption chains should be explicit, and if they are explicit, the programmer who specified them has made them legal. | With a lookup sequence of: | 1. Check the global registry for direct adaptations | 2. Ask the object via __conform__ | 3a. Check using isinstance() unless 2 raised LiskovViolation | 3b. Nothing, since object.__conform__ does an isinstance() check | 4. Ask the interface via __adapt__ These are OK up to 4. | 5. Look for transitive chains of interfaces in the global registry. No! No! No! Perhaps... 5. Raise a AdaptionFailed error, which includes the protocol which is being asked for. This error message _could_ also include a list of possible adaptation chains from the global registry, but this is just a suggestion. | 3a & 3b are the current differing answers to the question of who should | be checking for inheritance - the adaptation machinery or the __conform__ | method. Correct. I think either method is OK, and perfer Phillip's approach. Best, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Wed, Jan 12, 2005 at 01:15:20PM -0800, Guido van Rossum wrote: | [Clark] | > - add a flag to adapt, allowTransitive, which defaults to False | | That wouldn't work very well when most adapt() calls are invoked | implicitly through signature declarations (per my blog's proposal). Understood. This was a side-suggestion -- not the main thrust of my response. I'm writing to convince you that automatic "combined" adaptation, even as a last resort, is a bad idea. It should be manual, but we can provide easy mechanisms for application developers to specify combined adapters easily. On Wed, Jan 12, 2005 at 02:57:11PM -0500, Clark C. Evans wrote: | On Wed, Jan 12, 2005 at 10:16:14AM -0800, Guido van Rossum wrote: | | But now, since I am still in favor of automatic "combined" adaptation | | *as a last resort* A few problems with automatic "combined" adaptation: 1. Handling the case of multiple adaptation pathways is one issue; how do you choose? There isn't a good cost algorithem since the goodness of an adapter depends largely on the programmer's need. 2. Importing or commenting out the import of a module that may seem to have little bearing on a given chunk of code could cause subtle changes in behavior or adaptation errors, as a new path becomes available, or a previously working path is disabled. 3. The technique causes people to want to say what is and isn't an adapter -- when this choice should be soly up to the appropriate developers. I'd rather not have to standardize that FileName -> File is a _bad_ adaption, but File -> String is a good adaption. Or whatever is in vogue that year. 4. It's overly complicated for what it does. I assert that this is a very minor use case. When transitive adaptation is needed, an explicit registration of an adapter can be made simple. My current suggestion to make 'transitive adaption' easy for a application builder (one putting togeher components) has a few small parts: - If an adaptation is not found, raise an error, but list in the error message two additional things: (a) what possible adaptation paths exist, and (b) how to register one of these paths in their module. - A simple method to register an adaption path, the error message above can even give the exact line needed, adapt.registerPath(from=A,to=C,via=B) - Make it an error to register more than one adapter from A to C, so that conflicts can be detected. Also, registrations could be 'module specific', or local, so that adapters used by a library need necessarly not be global. In general, I think registries suffer all sorts of namespace and scoping issues, which is why I had proposed __conform__ and __adapt__ Extending registry mechanism with automatic 'transitive' adapters makes things even worse. Cheers, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246, redux
On Thu, Jan 13, 2005 at 06:27:08PM +0100, Alex Martelli wrote: | >The 'implicit' adaptation refers to the automagical construction of | >composite adapters assuming that a 'transitive' property holds. I've | >seen nothing in this thread to explain why this is so valueable, why | | Let me play devil's advocate: I _have_ seen explanations of why | transitive adaptation can be convenient -- the most direct one being an | example by Guido which came in two parts, the second one a | clarification which came in response to my request about the first one. hypothetical pseudocode ;) | To summarize it: say we have N concrete classes A1, A2, ... AN which | all implement interface I. | Now we want to integrate into the system function f1, which requires an | argument with interface J1, i.e. | def f1(x): | x = adapt(x, J1) | ... | or in Guido's new notation equivalently | def f1(x: J1): | ... | and also f2, ..., fM, requiring an argument with interface J2, ..., JM | respectively. | | Without transitivity, we need to code and register M*N adapters. Are you _sure_ you have M*N adapters here? But even so, for j in (J1,J2,J3,J4,...,JM) for i in (I1,I2,...,IN): register(j,i) | WITH transitivity, we only need M: I->J1, I->J2, ..., I->JM. Without transitivity, a given programmer, in a given module will probably only use a few of these permutations; and in each case, you can argue that the developer should be aware of the 'automatic' conversions that are going on. Imagine an application developer plugging a component into a framework and getting this error: """Adaption Error Could not convert A1 to a J1. There are two adaption pathways which you could register to do this conversion for you: # A1->I1 followed by I1->J1 adapt.registerPath((A1,I1),(I1,J1)) # A1->X3 followed by X3 -> PQ follwed by PQ -> J1 adapt.registerPath((A1,X3),(X3,PQ),(PQ,J1)) """ The other issue with registries (and why I avoided them in the origional PEP) is that they often require a scoping; in this case, the path taken by one module might be different from the one needed by another. | The convenience of this is undeniable; and (all other things being | equal) convenience raises productivity and thus is valuable. It also hides assumptions. If you are doing adaptation paths | James Knight gave a real-life example, although, since no combinatorial | explosion was involved, the extra convenience that he missed in | transitivity was minor compared to the potential for it when the N*M | issue should arise. Right. And that's more like it. | >it shouldn't be explicit, | | On this point I'm partly with you: I do not see any real loss of | convenience in requiring that an adapter which is so perfect and | lossless as to be usable in transitivity chains be explicitly so | registered/defined/marker. E.g., provide a | registerAdapter_TRANSITIVITY_SUITABLE(X, Y) | entry in addition to the standard registerAdapter which does not supply | transitivity (or equivalently an optional suitable_for_transitivity | argument to registerAdapter defaulting to False, etc, etc). Ok. I just think you all are solving a problem that doesn't exist, and in the process hurting a the more common use case: A component developer X and a framework developer Y both have stuff that an application developer A is putting together. The goal is for A to not worry about _how_ the components and the framework fit; to automatically "find" the glue code. The assertion that you can layer glue... is well, tenuous at best. | In terms of "should" as opposed to convenience, though, the argument is | that interface to interface adapters SHOULD always, inherently be | suitable for transitive chains because there is NO reason, EVER, under | ANY circumstances, to have such adapters be less than perfect, | lossless, noiseless, etc, etc. I strongly disagree; the most useful adapters are the ones that discard unneeded information. The big picture above, where you're plugging components into the framework will in most cases be lossy -- or the frameworks / components would be identical and you woudn't want to hook them up. Frankly, I think the whole idea of "perfect adapters" is just, well, arrogant. | > and on the contrary, most of the "problems | >with adapt()" seem to stem from this aggressive extension of what | >was proposed: Automatic construction of adapter chains is _not_ part | | Fair enough, except that it's not just chains of explicitly registered | adapters: interface inheritance has just the same issues, indeed, in | PJE's experience, MORE so, because no code is interposed -- if by | inheriting an interface you're asserting 100% no-problem | substitutability, the resulting "transitivity" may thus well give | problems (PJE and I even came close to agreeing that MS COM's | QueryInterface idea that interface inheritance does NOT implic
Re: [Python-Dev] PEP 246, redux
trying to make is that automatically constructing adapters isn't a great idea unless you have someone who can vouche for the usefulness. In other words, I picture this as a physics story problem, where a bunch of numbers are given with units. While the units may keep you "in check", just randomly combining figures with the right units can give you the wrong answer. | >| So, are you willing to do that round of editing to PEP 246...? I'll | >| then to the NEXT one which will still undoubtedly be needed... | > | >I could make a wack at it this weekend. | | Great! I assume you have copies of all relevant mails since they all | went around this mailing list, but if you need anything just holler, | including asking me privately about anything that might be unclear or | ambiguous or whatever -- I'll be around all weekend except Sunday night | (italian time -- afternoon US time;-). Ok. Best, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 246: lossless and stateless
Ok. I think we have identified two sorts of restrictions on the sorts of adaptations one may want to have: `stateless' the adaptation may only provide a result which does not maintain its own state `lossless' the adaptation preserves all information available in the original object, it may not discard state If we determined that these were the 'big-ones', we could possibly allow for the signature of the adapt request to be parameterized with these two designations, with the default to accept any sort of adapter: adapt(object, protocol, alternative = None, stateless = False, lossless = False) __conform__(self, protocol, stateless, lossless) __adapt__(self, object, stateless, lossless) Then, Guido's 'Optional Static Typing', def f(X: Y): pass would be equivalent to def f(X): X = adapt(Y, True, True) In other words, while calling adapt directly would allow for any adapter; using the 'Static Typing' short-cut one would be asking for adapters which are both stateless and lossless. Since __conform__ and __adapt__ would sprout two new arguments, it would make those writing adapters think a bit more about the kind of adapter that they are providing. Furthermore, perhaps composite adapters can be automatically generated from 'transitive' adapters (that is, those which are both stateless and lossless). But adaptations which were not stateless and lossless would not be used (by default) in an automatic adapter construction. Your thoughts? Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Thu, Jan 13, 2005 at 08:08:43PM -0500, Bob Ippolito wrote: | >Ok. I think we have identified two sorts of restrictions on the | >sorts of adaptations one may want to have: | > | > `stateless' the adaptation may only provide a result which | > does not maintain its own state | > | > `lossless' the adaptation preserves all information available | > in the original object, it may not discard state | > | >If we determined that these were the 'big-ones', we could possibly | >allow for the signature of the adapt request to be parameterized with | >these two designations, with the default to accept any sort of adapter: | > | > adapt(object, protocol, alternative = None, | > stateless = False, lossless = False) | > | >Then, Guido's 'Optional Static Typing', | > | > def f(X: Y): | > pass | > | > would be equivalent to | > | > def f(X): | > X = adapt(X,Y, stateless = True, lossless = True) .. | | In some cases, such as when you plan to consume the whole thing in one | function call, you wouldn't care so much if it's stateless. etrepum, True False statelessadapter may not add adapter may have its state beyond that alreadyown state, if it wishes provided by the object but additional state is not required lossless adapter must preserve andadapter may discard give all information which information if it wishes the underlying object has So, in this case, if your consumer doesn't care if the adapter is stateless or not, just call adapt(), which defaults to the case that you wish. Is this a better explanation? Or is this whole idea too convoluted? Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Thu, Jan 13, 2005 at 11:50:37PM -0500, Phillip J. Eby wrote: | 'lossless' isn't really a good term for non-noisy. The key is that a | "noisy" adapter is one that alters the precision of the information it | provides, by either claiming greater precision than is actually present, | or by losing precision that was present in the meaning of the data. Noisy doesn't cut it -- my PC fan is noisy. In computer science, noisy usually refers to a flag on an object that tells it to spew debug output... | 'statelessness', on the other hand, is primarily useful as a guide to | whether what you're building is really an "as-a" adapter. If an adapter | has per-adapter state, it's an extremely good indication that it's | actually a *decorator* (in GoF pattern terminology). GoF is very nice, but I'm using a much broader definition of 'adapt': To make suitable to or fit for a specific use or situation By this definition, decorators, facade are both kinds of adapters. | Anyway, for type declaration, IMO statelessness is the key criterion. | Type declaration "wants" to have true adapters (which can maintain object | identity), not decorators (which are distinct objects from the things | they add functionality to). Stateful adapters are very useful, and the value of PEP 246 is significantly reduced without alowing them. | Unfortunately, in practice this will just lead to people ignoring the | arguments, because 1) it's easier and 2) it will make their code work | with type declarations! So, it won't actually produce any useful effect. Hmm. Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Thu, Jan 13, 2005 at 08:54:33PM -0500, Raymond Hettinger wrote: | > Since __conform__ and __adapt__ | > would sprout two new arguments, it would make those writing adapters | > think a bit more about the kind of adapter that they are providing. | | Using optional arguments may not be the most elegant or extensible | approach. Perhaps a registry table or adapter attributes would fare | better. I'm not sure how either of these would work since the adapt() function could return `self`. Adapter attributes wouldn't work in that case (or would they?), and since adapters could be given dynamically by __adapt__ or __conform__ a registry isn't all that appropriate. Perhaps we could just pass around a single **kwargs? Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Fri, Jan 14, 2005 at 12:11:10AM -0500, Phillip J. Eby wrote: | Clark's proposal isn't going to solve this issue for PEP 246, alas. In | order to guarantee safety of adaptive type declarations, the | implementation strategy *must* be able to guarantee that 1) adapters do | not have state of their own, and 2) adapting an already-adapted object | re-adapts the original rather than creating a new adapter. 1. Following Raymond's idea for allowing adaption to reflect more arbitrary properties (which can be used to provide restrictions on the kinds of adapters expected): adapt(object, protocol, default = False, **properties) Request adaptation, where the result matches a set of properties, such as 'lossless', 'stateless'. __conform__(self, protocol, **properties) __adapt__(self, object, **properties) Conform/adapt but optionally parameterized by a set of restrictions. The **properties can be used to inform the adaptation. register(from, to, adapter = None, predicate = None) Register an adaptation path from one protocol to another, optionally providing an adapter. If no adapter is provided, then adapt(from,to,**properties) is used when adapting. If a predicate is provided, then the adaptation path is available only if predicate(**properties) returns True. 2. Perhaps if we just provide a mechanism for an adapter to specify that it's OK to be used "implicitly" via the declaration syntax? def fun(x: Y): ... is equivalent to, def fun(x): x = adapt(x, Y, declaration = True) On Thu, Jan 13, 2005 at 05:52:10PM -0800, Guido van Rossum wrote: | This may solve the current raging argument, but IMO it would | make the optional signature declaration less useful, because | there's no way to accept other kind of adapters. I'd be happier | if def f(X: Y) implied X = adapt(X, Y). Ideally, yes. However, some adapters may want to explicitly disable their usage in this context -- so some differentiation is warranted. This 'revised' proposal puts the burden on the adapter (or its registration) to specify that it shouldn't be used in this context. I'm carefully using 'declaration' as the restriction, not 'stateless'. One may have a stateful adapter which is most appropriate to be used in declarations (see Armin's insightful post). Furthermore, the 'full' version of adapt() where argument 'restrictions' can be specified could be done via a decorator syntax: @adapt(x, Y, **properties) I hope this helps. P.S. Clearly there is much of information to be captured in this thread and put into the PEP (mostly as appendix material); keep posting good ideas, problems, opinions, whatever -- I will summarize over this weekend. -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Fri, Jan 14, 2005 at 09:47:15AM +, Armin Rigo wrote:
| In my opinion a user-defined class or interface mixes two notions: a
| "concept" meaningful for the programmer that the instances
| represent, and the "interface" provided to manipulate it.
...
| This suggests that only concrete objects which are expected to
| encode a *single* concept should be used for adaptation.
So, in this view of the world, the adapter from FileName to File _is_
appropriate, but the adapter from String to FileName isn't?
def checkSecurity(filename: FileName):
...
Hmm. I'd like to be able to pass in a String here, and use that
String->FileName adapter. So, there isn't a problem yet; although
String is vague in a sense, it doesn't hurt to specialize it
in the context that I have in mind.
def checkContent(file: File):
... look for well known viruses ...
def checkSecurity(filename: FileName):
... look for nasty path information ...
return checkContent(filename)
Even this is _ok_ since the conceptual jump is specified by the
programmer between the two stages. The problem happens when
one does...
checkContent("is-this-a-filename-or-is-this-content")
This is where we run into issues. When an adapter which 'specializes'
the content is used implicitly in a trasitive adaption chain.
| Note that it may be useful to be able to register some adapaters
| in "local" registeries instead of the single global one, to avoid
| all kinds of unexpected global effects.
Nice...
Best,
Clark
--
Clark C. Evans Prometheus Research, LLC.
http://www.prometheusresearch.com/
o office: +1.203.777.2550
~/ , mobile: +1.203.444.0557
//
(( Prometheus Research: Transforming Data Into Knowledge
\\ ,
\/- Research Exchange Database
/\- Survey & Assessment Technologies
` \ - Software Tools for Researchers
~ *
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Fri, Jan 14, 2005 at 04:39:00PM +, Armin Rigo wrote: | I'm trying to reserve the usage of "interface" to something more | concrete: the concrete ways we have to manipulate a given object | (typically a set of methods including some unwritten expectations). I'd say that a programmer interface intends to encapsulates both the 'concept' and the 'signature'. The concept is indicated by the names of the function delcarations and fields, the signature by the position and type of arguments. | Adaptation should make [passing data between conceptually equivalent | interfaces?] more automatic, and nothing more. Ideally, both the caller | and the callee know (and write down) that the function's argument is a | "reference to some kind of file stuff", a very general concept; then they | can independently specify which concrete object they expect and provide, | e.g. "a string naming a file", "a file-like object", "a string containing | the data". But it is quite difficult to know when two interfaces are conceptually equivalent... | What I see in most arguments about adaptation/conversion/cast is some kind | of confusion that would make us believe that the concrete interface (or | even worse the formal one) fully defines what underlying concepts they | represent. It is true only for end-user application-specific classes. It seems your distinction comes down to defining 'best pratice' for when you define an adapter... and when you don't. Perhaps we don't need to qualify the adapters that exist, as much as make them transparent to the programmer. A bad adapter will most likely be detected _after_ a weird bug has happened. Perhaps the adapt() framework can provide meaningful information in these cases. Imagine enhancing the stack-trace with additional information about what adaptations were made; Traceback (most recent call last): File "xxx", line 1, in foo Adapting x to File File "yyy", line 384, in bar Adapting x to FileName etc. | In the above example, there is nothing in the general concept that helps | the caller to guess how a plain string will be interpreted, or | symmetrically that helps the callee to guess what an incoming plain | string means. In my opinion this should fail, in favor of something | more explicit. It's already a problem without any third party. How can we express your thoughts so that they fit into a narrative describing how adapt() should and should not be used? If you could respond by re-posting your idea with the 'average python programmer' as your audience it would help me quite a bit when summarizing your contribution to the thread. Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Fri, Jan 14, 2005 at 12:28:16PM -0500, Phillip J. Eby wrote: | At 08:32 AM 1/14/05 -0800, Guido van Rossum wrote: | >I have no desire to add syntax | >complexities like this to satisfy some kind of theoretically nice | >property. | | Whether it's syntax or a decorator, it allows you to create stateless | adapters without needing to write individual adapter *classes*, or even | having an explicit notion of an "interface" to adapt to. That is, it | makes it very easy to write a "good" adapter; you can do it without even | trying. The point isn't to make it impossible to write a "bad" adapter, | it's to make it more attractive to write a good one. Phillip, May I suggest that you write this up as a PEP? Being dead in the water isn't always fatal. Right now you're ideas are still very fuzzy and by forcing yourself to come up with a narrative, semantics section, minimal implementation, and examples, you will go along way to both refining your idea and also allowing others to better understand what you're proposing. Cheers, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 246: lossless and stateless
On Fri, Jan 14, 2005 at 10:02:39AM -0800, Michel Pelletier wrote: | Phillip J. Eby wrote: | > The result is that you generate a simple adapter class whose | > only state is a read-only slot pointing to the adapted object, | > and descriptors that bind the registered implementations to that object. it has only the functions in the interface, plus the adaptee; all requests through the functions are forwarded on to their equivalent in the adaptee; sounds alot like the adapter pattern ;) | I get it! Your last description didn't quite sink in but this one does | and I've been thinking about this quite a bit, and I like it. I'm | starting to see how it nicely sidesteps the problems discussed in | the thread so far. I'm not sure what else this mechanism provides; besides limiting adapters so that they cannot maintain their own state. | Does anyone know of any other languages that take this "operational" | aproach to solving the substitutability problem? Microsoft's COM? | I also think this is easier for beginners to understand, instead of | "you have to implement this interface, look at it over here, | that's the "file" interface, now you implement that in your object | and you better do it all right" you just tell them "call your | method 'read' and say its 'like file.read' and your thing will work | where any file can be read. A tangable example would perhaps better explain... Looking forward to the PEP, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 246, Feedback Request
w this issue? - topic: adaptee (aka origin) overview: There was discussion as to how to get back to the original object from an adapter. Is this in scope of PEP 246? proposal: - we specify an __adaptee__ property, to be optionally implemented by an adapter that provides a reference adaptee - the adapt.register method has an optional argument, 'adaptee', that defaults to False; if it is True, adapt() calls will stuff away into a weak-reference mapping from adapter to adaptee. - an adapt.adaptee(adaptor) function which returns the given adaptee for the adaptor; this first checks the weak-reference table, and then checks for an __adaptee_ feedback: Is this useful, worth the complexity? - topic: sticky overview: Sticky adapters, that is, ones where there is only one instance per adaptee is a common use case. Should the registry of PEP 246 provide this feature? proposal: - the adapt.register method has an optional argument, 'sticky', that defaults to False - if the given adapter factory is marked sticky, then a call to adapt() will first check to see if a given adapter (keyed by protocol) has been created for the adaptee; if so, then that adapter is returned, otherwise the factory is asked to produce an adapter and that adapter is cashed. feedback: Is this useful, worth the complexity? It seems like an easy operation. The advantage to this approach (over each factory inheriting from a StickyFactory) is that registry queries can be done, to list sticky adapters and other bookkeeping chores. Ok. That's it. Cheers, Clark -- Clark C. Evans Prometheus Research, LLC. http://www.prometheusresearch.com/ o office: +1.203.777.2550 ~/ , mobile: +1.203.444.0557 // (( Prometheus Research: Transforming Data Into Knowledge \\ , \/- Research Exchange Database /\- Survey & Assessment Technologies ` \ - Software Tools for Researchers ~ * ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
+1 Excellent Change +1 Minimal Backward Compatibility Difficulties I think this would also help quite a bit with newbie adoption of Python. I've had to explain this un-feature on numerous occassions and it given how smart Python is, I've wondered why it has this requirement. If you look in various open source packages, you'll find that 95% of these __init__.py files are empty. The ones at my work actually say: # stupid Python requirement, don't remove this file Why? Someone decided to remove files of length 0 in our repository without realizing the consequences. Since it had the __init__.pyc file around, it still worked... till one brought down a fresh copy of the repository and then it just "stopped" working. Quite a bit of hair pulling that one caused us. The only case where this might cause a problem is with "resource" directories that only contain ".html", ".jpg" and other files. So, perhpas this feature would only turn a directory into a package if it didn't have any .py files. It could also trigger only when the package is explicitly imported? Good luck /w the pitch-fork wielding users and telling the old-timers where they can keep their backward compatibility. Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate __ private (was Re: PEP 8 updates/clarifications)
Interesting discussion. I've been thinking the opposite; that I should start using __attribute more often for "undocumented, private" member variables that are implementation details and clearly not part of the public interface. I'm curious what people have against it? On Sun, Dec 11, 2005 at 09:18:04PM -0500, Tim Peters wrote: | That wasn't quite it. The original motivation was to help avoid name | collisions under inheritance period, and especially when writing a | base class intended for subclassing by other parties ... | It's even more unreasonable for A's author to have to | promise, after A's first release, never to change the name of, or | introduce any new, attribute (A's author dare not, lest the new name | conflict with a name someone else's subclass used). About one year ago, I was updating a "shared module" that I wrote about 6-9 months prior. I added a member variable, and a few days later one of my applications started to mysteriously fail. This was a bugger to track down... name collision problem. I've since become very sensitive about "from xx import *" as well, for the same reason -- it tends to cause very nasty bugs when the module xx changes to introduce a few more methods, etc. Best, Clark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
