[Python-Dev] Function Prototypes

[asleep . cult]

Hello

In thread about PEP 677, I (with the help of another proposal) came up with an 
alternative to typing.Callable called function prototypes. The basic concept is 
to create a function prototype object when the body of a function is omitted.

The thread can be found here: 
https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/

Mark Shannon initially proposed that functions be used as types and provided 
this example:

@Callable
def IntToIntFunc(a:int)->int:
 pass

def flat_map(
 l: list[int],
 func: IntToIntFunc
) -> list[int]:
 

I further proposed that we make the body of a function non-mandatory and create 
a function prototype if it is omitted. I provided these examples:

import typing

@typing.Callable
def IntToIntFunc(a: int) -> int

def flat_map(
l: list[int],
func: IntToIntFunc
) -> list[int]:
...

import ctypes

@ctypes.CFUNCTYPE
def f(x: int) -> bool

I have since taken it upon myself to implement this in a fork of cpython: 
https://github.com/asleep-cult/cpython

To remain consistent with function definitions, I have also added an 
alternative lambda syntax that allows you to annotate arguments and the return 
type. The syntax requires you to parenthesize the argument list:

lambda (a: int, b: int) -> int: ...

This new lambda syntax also allows you to create a function prototype by 
omitting the body. The original example can be rewritten as follows:

def flat_map(
l: list[int],
func: lambda (a: int) -> int
) -> list[int]:
...

Problems:
- It is not possible to use ParamSpec with this
- The lambda prototype syntax might be jarring
- Some people might accidentally forget the function body

Here is some feedback that I have already collected:

"Yeah, making the body optional (without looking at decorators) is not
acceptable either. Too easy to do by mistake (I still do this All. The.
Time. :-)" - Guido van Rossum

"i would strongly prefer this over the existing pep" - Ronny Pfannschmidt

"I find this unnecessary and unreadable.

Python isn't C or Java." - BundleOfJoysticks (Reddit)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B6VKYV5TKD2VSK6D2CUN77Q6MI5VIBU5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Christopher Barker]

How about instead of omitting the body, it contains a single expression,
say:

def int(x: float): -> float
typing.Prototype

It doesn’t totally break protocol to have a function complie differently
depending on its content— that’s done with generator functions.

-CHB

On Thu, Dec 23, 2021 at 7:20 AM  wrote:

> Hello
>
> In thread about PEP 677, I (with the help of another proposal) came up
> with an alternative to typing.Callable called function prototypes. The
> basic concept is to create a function prototype object when the body of a
> function is omitted.
>
> The thread can be found here:
> https://mail.python.org/archives/list/python-dev@python.org/thread/OGACYN2X7RX2GHAUP2AKRPT6DP432VCN/
>
> Mark Shannon initially proposed that functions be used as types and
> provided this example:
>
> @Callable
> def IntToIntFunc(a:int)->int:
>  pass
>
> def flat_map(
>  l: list[int],
>  func: IntToIntFunc
> ) -> list[int]:
>  
>
> I further proposed that we make the body of a function non-mandatory and
> create a function prototype if it is omitted. I provided these examples:
>
> import typing
>
> @typing.Callable
> def IntToIntFunc(a: int) -> int
>
> def flat_map(
> l: list[int],
> func: IntToIntFunc
> ) -> list[int]:
> ...
>
> import ctypes
>
> @ctypes.CFUNCTYPE
> def f(x: int) -> bool
>
> I have since taken it upon myself to implement this in a fork of cpython:
> https://github.com/asleep-cult/cpython
>
> To remain consistent with function definitions, I have also added an
> alternative lambda syntax that allows you to annotate arguments and the
> return type. The syntax requires you to parenthesize the argument list:
>
> lambda (a: int, b: int) -> int: ...
>
> This new lambda syntax also allows you to create a function prototype by
> omitting the body. The original example can be rewritten as follows:
>
> def flat_map(
> l: list[int],
> func: lambda (a: int) -> int
> ) -> list[int]:
> ...
>
> Problems:
> - It is not possible to use ParamSpec with this
> - The lambda prototype syntax might be jarring
> - Some people might accidentally forget the function body
>
> Here is some feedback that I have already collected:
>
> "Yeah, making the body optional (without looking at decorators) is not
> acceptable either. Too easy to do by mistake (I still do this All. The.
> Time. :-)" - Guido van Rossum
>
> "i would strongly prefer this over the existing pep" - Ronny Pfannschmidt
>
> "I find this unnecessary and unreadable.
>
> Python isn't C or Java." - BundleOfJoysticks (Reddit)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/B6VKYV5TKD2VSK6D2CUN77Q6MI5VIBU5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PFN5BYFLWA75R3ZDYVKEP37VRCPLAVM3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Steven D'Aprano]

On Thu, Dec 23, 2021 at 03:00:03PM -, asleep.c...@gmail.com wrote:

> Mark Shannon initially proposed that functions be used as types and provided 
> this example:
> 
> @Callable
> def IntToIntFunc(a:int)->int:
>  pass
> 
> def flat_map(
>  l: list[int],
>  func: IntToIntFunc
> ) -> list[int]:
>  

I have to admit, when Mark Shannon initially proposed that as an 
improvement over both the current and the proposed syntax, I was so 
taken aback that I initially thought he was being sarcastic and had 
to read it a second time to realise he was serious :-(

The status quo is an in-place declaration using an anonymous type:

func: Callable[[int], list[int]]

# proposed equivalent
func: (int) -> list[int]

The anonymous type using Callable requires requires 26 characters, 
including 7 punctuation characters.

The PEP 677 proposal cuts that down to 18 chars (6 punctuation chars) 
while increasing readability: the arrow syntax is "executable pseudo 
code". As far back as PEP 484 in 2014 this same syntax was used in 
type-comments for Python2 straddling code:

https://www.python.org/dev/peps/pep-0484/#id50

Extending that to refer to the signature of a function is an obvious 
step. Many other languages have converged on the same, or very similar, 
syntax.

I've been using similar `param -> result` pseudo-syntax when sketching 
out code using pencil and paper, or on a whiteboard, for years, and 
nobody has failed to understand it.

In comparison, Mark's version:

@Callable
def IntToIntFunc(a:int)->int:
pass

# in the type declaration
func: IntToIntFunc

uses 54 characters, plus spaces and newlines (including 7 punctuation 
characters); it takes up three extra lines, plus a blank line. As 
syntax goes it is double the size of Callable.

It separates the type declaration from the point at which it is used, 
potentially far away from where it is used.

It adds a new name to the global namespace, bloating the output of 
introspection tools like dir(), help() etc.

And it *requires* a named (non-anonymous) type where an anonymous type 
is all that is needed or wanted.

Being able to name types using an alias when it helps readability is 
good. Being required to name them even at the cost of hurting 
readability is not. Naming is hard, and bad names are worse than no 
names.

Consider Mark's name for the function: "IntToIntFunc", which tells us 
nothing that the signature (int)->int doesn't already tell us. It is the 
naming equivalent of the comment:

x += 1  # Add 1 to x.

Your proposal is slightly more compact than Mark's: you drop the ending 
colon and the body ("pass"), saving one line and five characters out of 
the 54. But it suffers from the same major flaws:

- verbose and relatively heavy on vertical space;
- bloats the output of introspection tools;
- separating the definition of the type from where it is used;
- requiring a name for something which doesn't need a name.


If I had the choice between using the current syntax with Callable[] and 
the proposed PEP 677 arrow syntax, I would almost always use the arrow 
syntax. It matches the pseudo-syntax I already use when writing pseudo- 
code on paper.

If I had the choice between Callable[] and this proposed function-as-a- 
type syntax, I would stick to Callable. If I wanted to give the type a 
name, for some reason, I would still use Callable, and just write an 
alias. I cannot imagine any scenario where I would prefer this function- 
as-a-type syntax over the other two alternatives.


> I further proposed that we make the body of a function non-mandatory 
> and create a function prototype if it is omitted. I provided these 
> examples:
> 
> import typing
> 
> @typing.Callable
> def IntToIntFunc(a: int) -> int

What do you get when the inevitable occurs, and you forget the 
decorator? If I just write this:

def IntToIntFunc(a: int) -> int

it will create what sort of object?


[...]
> This new lambda syntax also allows you to create a function prototype 
> by omitting the body. The original example can be rewritten as 
> follows:

At least that brings back the ability to write it as an anonymous type, 
but at the cost of adding a totally unnecessary keyword "lambda" and an 
unused, redundant parameter name:

func: (int) -> int
func: lambda (a: int) -> int


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SW4NYIYWIX7VBYTGIBCTOVMDWFQSTFB4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[asleep . cult]

Hello and thank you for the much needed feedback.

One thing that you must consider is that function prototypes
have a few implications beyond typing but it seems like you're
only looking at it as a tool for type hinting. The interpreter will
create a function prototype object regardless of if you forget your
decorator, it needs to pass something to the decorator after all.

After reading through your reply, I am seeing that the main concern
is the bloat added by the lambda keyword. My decision to use lambda
instead of introducing a special syntax was one that required heavy
deliberation. I ultimately decided to stick with lambda because it was
consistent with the prototype statement form. The fact that lambda
is hard to type has been felt by almost everyone who has ever used
Python, this isn't just a problem that would be introduced by
function prototypes. PEP 677 has taken the lazy approach to solving
this issue and has prioritized type hinting over functionality. PEP 667
also suggests the usage of => for lambdas which would likely
never be accepted because of the confusion it would cause.
As someone who has used typing with Python, I do think that a new
callable syntax is needed, but I truly believe that PEP 677 is taking the
wrong approach.

So what if we broke every Python program in existence by creating a
new lambda syntax, how would it look? This question is particularly
hard to answer because the body and annotations both need to be
optional. Our best bet is an augmented form of the PEP 677 syntax
that allows you to add a body. Here is an example:

(a: int) -> int: a ** 2

But of course this causes ambiguity when the return annotation and
body are both omitted. One thing that I did consider is simply treating
it like a tuple if there is no return annotation AND there is no body, but
that might lead to confusion. Another thing that I considered is a different
prefix than lambda:

$(a: int) -> int: a ** 2
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JJZGZ6EK5CY4RADPSQPRDWOB6GXQSSJ3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[MRAB]

On 2021-12-23 19:04, asleep.c...@gmail.com wrote:

Hello and thank you for the much needed feedback.

One thing that you must consider is that function prototypes
have a few implications beyond typing but it seems like you're
only looking at it as a tool for type hinting. The interpreter will
create a function prototype object regardless of if you forget your
decorator, it needs to pass something to the decorator after all.

After reading through your reply, I am seeing that the main concern
is the bloat added by the lambda keyword. My decision to use lambda
instead of introducing a special syntax was one that required heavy
deliberation. I ultimately decided to stick with lambda because it was
consistent with the prototype statement form. The fact that lambda
is hard to type has been felt by almost everyone who has ever used
Python, this isn't just a problem that would be introduced by
function prototypes. PEP 677 has taken the lazy approach to solving
this issue and has prioritized type hinting over functionality. PEP 667
also suggests the usage of => for lambdas which would likely
never be accepted because of the confusion it would cause.
As someone who has used typing with Python, I do think that a new
callable syntax is needed, but I truly believe that PEP 677 is taking the
wrong approach.

So what if we broke every Python program in existence by creating a
new lambda syntax, how would it look? This question is particularly
hard to answer because the body and annotations both need to be
optional. Our best bet is an augmented form of the PEP 677 syntax
that allows you to add a body. Here is an example:

(a: int) -> int: a ** 2

But of course this causes ambiguity when the return annotation and
body are both omitted. One thing that I did consider is simply treating
it like a tuple if there is no return annotation AND there is no body, but
that might lead to confusion. Another thing that I considered is a different
prefix than lambda:

$(a: int) -> int: a ** 2


Usually the suggestion is to use 'def':

def (a: int) -> int: a ** 2
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AKS4IHJJFDV24QZNMVMFZR72H5ORU3TB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Joao S. O. Bueno]

My eyes are bleeding with these incomplete function definitions.
If you are using decorators and "def", then, please, there is  no
need for special syntax that would just be a syntax error
in "normal"Python., Just add ": pass" to the end.

If eyes bleeding is not enough of an argument for you:
the new syntax would only be possible be usable in code
that would run on Python 3.11 and above. While using
the decorator + function declaration syntax, you have the
annotations that could readly be checked and code compatible with
existing all supported Python versions, right out of the box.

On Thu, 23 Dec 2021 at 16:31, MRAB  wrote:

> On 2021-12-23 19:04, asleep.c...@gmail.com wrote:
> > Hello and thank you for the much needed feedback.
> >
> > One thing that you must consider is that function prototypes
> > have a few implications beyond typing but it seems like you're
> > only looking at it as a tool for type hinting. The interpreter will
> > create a function prototype object regardless of if you forget your
> > decorator, it needs to pass something to the decorator after all.
> >
> > After reading through your reply, I am seeing that the main concern
> > is the bloat added by the lambda keyword. My decision to use lambda
> > instead of introducing a special syntax was one that required heavy
> > deliberation. I ultimately decided to stick with lambda because it was
> > consistent with the prototype statement form. The fact that lambda
> > is hard to type has been felt by almost everyone who has ever used
> > Python, this isn't just a problem that would be introduced by
> > function prototypes. PEP 677 has taken the lazy approach to solving
> > this issue and has prioritized type hinting over functionality. PEP 667
> > also suggests the usage of => for lambdas which would likely
> > never be accepted because of the confusion it would cause.
> > As someone who has used typing with Python, I do think that a new
> > callable syntax is needed, but I truly believe that PEP 677 is taking the
> > wrong approach.
> >
> > So what if we broke every Python program in existence by creating a
> > new lambda syntax, how would it look? This question is particularly
> > hard to answer because the body and annotations both need to be
> > optional. Our best bet is an augmented form of the PEP 677 syntax
> > that allows you to add a body. Here is an example:
> >
> > (a: int) -> int: a ** 2
> >
> > But of course this causes ambiguity when the return annotation and
> > body are both omitted. One thing that I did consider is simply treating
> > it like a tuple if there is no return annotation AND there is no body,
> but
> > that might lead to confusion. Another thing that I considered is a
> different
> > prefix than lambda:
> >
> > $(a: int) -> int: a ** 2
> >
> Usually the suggestion is to use 'def':
>
>  def (a: int) -> int: a ** 2
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/AKS4IHJJFDV24QZNMVMFZR72H5ORU3TB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BGQUJQCBW72WDGRKXT5JXKS2AEBS36OJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Guido van Rossum]

Without decorator too (that was Lukasz’ idea). Why bother with the
decorator (*if* we were to go there)?

On Thu, Dec 23, 2021 at 13:16 Joao S. O. Bueno 
wrote:

> My eyes are bleeding with these incomplete function definitions.
> If you are using decorators and "def", then, please, there is  no
> need for special syntax that would just be a syntax error
> in "normal"Python., Just add ": pass" to the end.
>
> If eyes bleeding is not enough of an argument for you:
> the new syntax would only be possible be usable in code
> that would run on Python 3.11 and above. While using
> the decorator + function declaration syntax, you have the
> annotations that could readly be checked and code compatible with
> existing all supported Python versions, right out of the box.
>
> On Thu, 23 Dec 2021 at 16:31, MRAB  wrote:
>
>> On 2021-12-23 19:04, asleep.c...@gmail.com wrote:
>> > Hello and thank you for the much needed feedback.
>> >
>> > One thing that you must consider is that function prototypes
>> > have a few implications beyond typing but it seems like you're
>> > only looking at it as a tool for type hinting. The interpreter will
>> > create a function prototype object regardless of if you forget your
>> > decorator, it needs to pass something to the decorator after all.
>> >
>> > After reading through your reply, I am seeing that the main concern
>> > is the bloat added by the lambda keyword. My decision to use lambda
>> > instead of introducing a special syntax was one that required heavy
>> > deliberation. I ultimately decided to stick with lambda because it was
>> > consistent with the prototype statement form. The fact that lambda
>> > is hard to type has been felt by almost everyone who has ever used
>> > Python, this isn't just a problem that would be introduced by
>> > function prototypes. PEP 677 has taken the lazy approach to solving
>> > this issue and has prioritized type hinting over functionality. PEP 667
>> > also suggests the usage of => for lambdas which would likely
>> > never be accepted because of the confusion it would cause.
>> > As someone who has used typing with Python, I do think that a new
>> > callable syntax is needed, but I truly believe that PEP 677 is taking
>> the
>> > wrong approach.
>> >
>> > So what if we broke every Python program in existence by creating a
>> > new lambda syntax, how would it look? This question is particularly
>> > hard to answer because the body and annotations both need to be
>> > optional. Our best bet is an augmented form of the PEP 677 syntax
>> > that allows you to add a body. Here is an example:
>> >
>> > (a: int) -> int: a ** 2
>> >
>> > But of course this causes ambiguity when the return annotation and
>> > body are both omitted. One thing that I did consider is simply treating
>> > it like a tuple if there is no return annotation AND there is no body,
>> but
>> > that might lead to confusion. Another thing that I considered is a
>> different
>> > prefix than lambda:
>> >
>> > $(a: int) -> int: a ** 2
>> >
>> Usually the suggestion is to use 'def':
>>
>>  def (a: int) -> int: a ** 2
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/AKS4IHJJFDV24QZNMVMFZR72H5ORU3TB/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/BGQUJQCBW72WDGRKXT5JXKS2AEBS36OJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KE55W466H433ECZ4GSSSW2AXNFHZFBC4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Steven D'Aprano]

On Thu, Dec 23, 2021 at 02:09:50PM -0800, Guido van Rossum wrote:

> Without decorator too (that was Lukasz’ idea). Why bother with the
> decorator (*if* we were to go there)?

So that 

def func(params): pass

creates a function object, and

def func(params)

makes a Callable type object?

I'm going to *love* explaining the difference to beginners when they 
accidently do this and try to call func.

Python is not Scratch and is not intended as a teaching language for 
kids, but some proposals are actively hostile to beginners, and I think 
this is one. Have we considered how this would effect something as 
simple as the meaning of keywords?

- `class` creates classes, also known as types;

- `def` creates functions, and also types, but not types you can use
  for anything except type-checking.

Given that annotations are optional everywhere else, what happens if you 
leave out the annotations in the type definition?

def SomethingFunction(x)

Is that the same as Callable[[Any], Any] or Callable[[None], None]?

Or are we going to create a rule that annotations are mandatory in some 
`def`s but optional in others?

-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5GXCY64PHTNBXXUHL4ULDVSECX2DWWIC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Guido van Rossum]

On Thu, Dec 23, 2021 at 3:24 PM Steven D'Aprano  wrote:

> On Thu, Dec 23, 2021 at 02:09:50PM -0800, Guido van Rossum wrote:
>
> > Without decorator too (that was Lukasz’ idea). Why bother with the
> > decorator (*if* we were to go there)?
>
> So that
>
> def func(params): pass
>
> creates a function object, and
>
> def func(params)
>
> makes a Callable type object?
>

No, no, no. That syntax has already been discredited.

Mark's proposal was
```
@Callable
def func(params): pass
```
My question is, why does it need `@Callable`? Lukasz proposed just using
any (undecorated) function, with the convention being that the body is
`...` (to which I would add the convention that the function *name* be
capitalized, since it is a type). My question (for Mark, or for anyone who
supports `@Callable`) is why bother with the decorator. It should be easy
to teach a type checker about this:

```
def SomeFn(x: float) -> int:
...

def twice(f: SomeFn) -> SomeFn:
return lambda x: f(f(x))
```


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZCKXJJOADSBH7FLZQ3KGHTMDHFI7NCHA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Barry Warsaw]

On Dec 23, 2021, at 17:09, Guido van Rossum  wrote:
> 
> Mark's proposal was
> ```
> @Callable
> def func(params): pass
> ```
> My question is, why does it need `@Callable`? Lukasz proposed just using any 
> (undecorated) function, with the convention being that the body is `...` (to 
> which I would add the convention that the function *name* be capitalized, 
> since it is a type). My question (for Mark, or for anyone who supports 
> `@Callable`) is why bother with the decorator. It should be easy to teach a 
> type checker about this:
> 
> ```
> def SomeFn(x: float) -> int:
> ...
> 
> def twice(f: SomeFn) -> SomeFn:
> return lambda x: f(f(x))
> ```

That seems pretty intuitive to me.  The conventions you mention would be just 
that though, right?  I.e. `pass` could be used, but whatever the body is it 
would be ignored for type checking `twice()` in this case, right?

-Barry




signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BRYQ26TIYPITNDOYYYGBDFS5EYEOZVUA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Steven D'Aprano]

On Thu, Dec 23, 2021 at 07:04:17PM -, asleep.c...@gmail.com wrote:

> One thing that you must consider is that function prototypes

Just to be clear, by "function prototype", do you mean what PEP 677 
calls a Callable Type?


> have a few implications beyond typing but it seems like you're
> only looking at it as a tool for type hinting. The interpreter will
> create a function prototype object regardless of if you forget your
> decorator, it needs to pass something to the decorator after all.

What sort of implications beyond typing?

Some questions:

1. What kind of object does `def Func(a:int)->int` create, if you leave 
out the decorator? Is that what you are calling a "function prototype 
object"?

2. What does it need the @Callable decorator for?

3. What can you do with it, apart from typing?

4. What happens if you leave out the annotations and just say
   `def Func(a)` alone?

5. Do you have any response to my other criticisms about this syntax?


> After reading through your reply, I am seeing that the main concern
> is the bloat added by the lambda keyword.

Perhaps you should re-read my reply. The lambda keyword is the *least* 
part of my dislike of this proposal.


> The fact that lambda is hard to type has been felt by almost everyone 
> who has ever used Python,

Such exaggerations don't help your case. I have never felt the need to 
type a lambda expression, and doubt I'm so very unusual.

Annotations have only existed for less than half of Python's existence. 
Even today, I doubt that as many as half of Python's user base are heavy 
users of typing. Or even casual users of typing. And many of them would 
not use lambda, or at least not in places where adding typing to it 
would add any value.

In any case, allowing type-hints in lambda expressions is independent of 
the question of using the lambda keyword to declare a function 
prototype. I have no objection in principle to allowing annotations in 
lambda expressions if such a thing would actually be useful. But that 
doesn't mean I want to see lambda used as function prototype syntax:

def map(func: lambda (obj: Any) -> Any, items: Sequence[Any]) -> 
Sequence[Any]

especially not in preference to just using arrow syntax:

def map(func: (Any)-> Any, items: Sequence[Any]) -> Sequence[Any]


> this isn't just a problem that would be introduced by
> function prototypes. PEP 677 has taken the lazy approach to solving
> this issue

What does that mean? What is lazy about it?


> and has prioritized type hinting over functionality. PEP 667
> also suggests the usage of => for lambdas which would likely
> never be accepted because of the confusion it would cause.

Syntactic sugar for lambda is not part of PEP 667, it merely references 
the fact that people have suggested using => as shorthand for a lambda.

For what its worth, I was skeptical about using two different arrows 
(one for declaring callable types, one for functions) when I first heard 
the idea (I think it was Guido who mentioned it?). But I've come to 
believe that whatever confusion there might be in using two arrows

"do I use -> or => here? I never remember which is which"

will be less, not more, than the confusion due to using the same arrow 
for both contexts. That is, I think, the experience from other 
languages. (Kotlin if I remember correctly? Maybe not.)

But that's a discussion for when somebody writes a PEP for lambda 
shortcut syntax.


> As someone who has used typing with Python, I do think that a new
> callable syntax is needed, but I truly believe that PEP 677 is taking the
> wrong approach.
> 
> So what if we broke every Python program in existence by creating a
> new lambda syntax, how would it look?

Creating new syntax is backwards compatible: it doesn't break existing 
code that is syntactically correct. Only removing, or changing the 
meaning of, existing syntax will break "every Python program in 
existence".

I doubt the Steering Council would accept such breakage.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MTWBA5T5ISPCL7S4PM6KE77AIEMPFDTT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Guido van Rossum]

Yes, and yes. (Or the PEP could say it has to be ‘…’ and static checkers
could enforce it. But checkers already carry everything they need to check
this with them, for checking calls.)

Downside of this idea is that it requires you to invent a name for every
callable type you use.


On Thu, Dec 23, 2021 at 17:37 Barry Warsaw  wrote:

> On Dec 23, 2021, at 17:09, Guido van Rossum  wrote:
> >
> > Mark's proposal was
> > ```
> > @Callable
> > def func(params): pass
> > ```
> > My question is, why does it need `@Callable`? Lukasz proposed just using
> any (undecorated) function, with the convention being that the body is
> `...` (to which I would add the convention that the function *name* be
> capitalized, since it is a type). My question (for Mark, or for anyone who
> supports `@Callable`) is why bother with the decorator. It should be easy
> to teach a type checker about this:
> >
> > ```
> > def SomeFn(x: float) -> int:
> > ...
> >
> > def twice(f: SomeFn) -> SomeFn:
> > return lambda x: f(f(x))
> > ```
>
> That seems pretty intuitive to me.  The conventions you mention would be
> just that though, right?  I.e. `pass` could be used, but whatever the body
> is it would be ignored for type checking `twice()` in this case, right?
>
> -Barry
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/BRYQ26TIYPITNDOYYYGBDFS5EYEOZVUA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WW6QAPAZ2RTCWSSK2XJIJP3VWVHCOL7Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Steven D'Aprano]

On Thu, Dec 23, 2021 at 05:09:18PM -0800, Guido van Rossum wrote:

> > def func(params)
> >
> > makes a Callable type object?
> >
> 
> No, no, no. That syntax has already been discredited.

It has? Where? Have I missed something?

This thread is about using that syntax as an alternative to Mark's
proposal. If it is already ruled out, then somebody should mention it 
to asleep.cult (the original poster of this thread).

I'm curious: what objections to asleep.cult's proposal don't equally 
apply to Mark's proposal? From what I can see, Mark's original proposal 
has all the same disadvantages, plus it is even more verbose.

Dropping the need for the @Callable decorator reduces the level of 
verbosity somewhat (one less line, nine fewer characters) but all the 
other negatives remain.

Honestly, I cannot see a single positive to using `def` statements. 
Sure, we can do it, and an experienced programmer could infer the 
meaning of it, but given the choice of writing an anonymous type in 
place where you want it, versus having to pre-declare a function 
prototype with a name that adds nothing to the readability of the code 
before using it, why would I prefer the `def` version?

This is not a rhetorical question.

The fact that the existing feature (Callable) and the PEP 677 arrow 
syntax are anonymous, and can be written in place rather than needing to 
be pre-declared with a name, are positives. *Requiring* a name to use 
this `def` syntax is a point against it.

If I need a named type alias, I can already create one, and name it:

IntToIntFunc = Callable[[int], int]

and while I can see that there are complicated signatures where a named 
alias would be useful:

FileOpener = Callable[ ... ] # complicated signature

we can already do that, so the `def` syntax adds nothing. For simple 
cases we don't need a name. The name IntToIntFunc adds nothing that 
isn't just as clear, if not more so, in the signature itself. It is like 
the comment:

x += 1  # add one to x


> Mark's proposal was
> ```
> @Callable
> def func(params): pass
> ```

Indeed, and I already wrote a criticism of that proposal.

Removing the decorator saves one line and nine characters, but the other 
criticisms remain.


> My question is, why does it need `@Callable`? Lukasz proposed just using
> any (undecorated) function, with the convention being that the body is
> `...` (to which I would add the convention that the function *name* be
> capitalized, since it is a type).

But without the Callable decorator, it isn't a type, its a function.
You're just using it as a type (or to be precise, a function prototype). 

I'm okay with naming conventions reflecting usage (we sort of already do 
that, with int, float, etc) but we should be clear about what's really 
happening. `def` creates a function.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VTOOJLPU2HPIW6TQBBYTW64W4DFGCQEG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Chris Angelico]

On Fri, Dec 24, 2021 at 1:36 PM Steven D'Aprano  wrote:
> > My question is, why does it need `@Callable`? Lukasz proposed just using
> > any (undecorated) function, with the convention being that the body is
> > `...` (to which I would add the convention that the function *name* be
> > capitalized, since it is a type).
>
> But without the Callable decorator, it isn't a type, its a function.
> You're just using it as a type (or to be precise, a function prototype).
>
> I'm okay with naming conventions reflecting usage (we sort of already do
> that, with int, float, etc) but we should be clear about what's really
> happening. `def` creates a function.
>

My reading of this is that a function IS the type of a function with
that signature, just like how None means the type NoneType. Is that
correct?

Or putting it another way: is this (silly) example legal?

def repeat_string(s: str, times: int) -> str:
return s * times

def decimate_strings(lines: Iterable[str], xfrm: repeat_string) -> List[str]:
return [xfrm(l, 10) for l in lines]

If def creates a function, and def creates the type of a function, it
stands to reason that a function is the type of a function.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KJFQEMO73IP7UIIN2UX74OKKHBAIDN6X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Steven D'Aprano]

On Fri, Dec 24, 2021 at 01:28:44PM +1100, Steven D'Aprano wrote:

> Honestly, I cannot see a single positive to using `def` statements. 
...
> This is not a rhetorical question.

Hmm, I think I may have come up with one.

If we did teach type checkers to use actual functions as prototypes,
that would allow the same function object to do double duty as the
type-hint (as a prototype) and as an implementation of that prototype.

So if you wanted a function that has the same signature as builtin
`print`, you could just say something like:

def traverse_graph(
g: Graph,
maxdepth: int = -1,
# Same signature as print, defaults to print
visitor: print = print,
) -> None:

instead of having to replicate print's signature.

The duplication `: print = print` is a bit on the nose, but not too 
much. And maybe type-checkers could infer that if a parameter defaults 
to a function, its type-hint should clearly be the same as the default?

This would require builtins to gain annotations, of course. Which 
they don't currently have :-(

And for the avoidance of doubt, I am not suggesting this be limited to 
only builtin functions. Any function with annotations would work.

So to my mind, that moves Mark's proposal into the category of an 
independent new feature separate to PEP 677, rather than a competitor 
or alternative proposal:

* Teach type-checkers to use functions made with `def` as function
  prototypes (regardless of what the body is).

* Likewise for classes (use the `__call__` method's signature as the
  prototype).

while PEP 677 remains as a nice-looking shorter syntax for in-place 
anonymous Callable Types.

I remain strongly -1 on `def ...` as an alternative to PEP 677, I 
don't think that the `def` syntax makes a good alternative to either 
Callable or the proposed arrow syntax. But being able to use an 
existing function, complete with implementation in the body, as a 
prototype, yeah, I'll buy that.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YDQOLLRRWKYJHCTFKQIJDXCJEOEOFLZJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Steven D'Aprano]

On Fri, Dec 24, 2021 at 01:54:35PM +1100, Chris Angelico wrote:

> My reading of this is that a function IS the type of a function with
> that signature, just like how None means the type NoneType. Is that
> correct?

That's not the status quo, but I think the idea is that it will be.

Except that I think that the preferred terminology is that it is a 
*function prototype* rather than a type, since it is a prototype, not an 
actual class you can instantiate. A template, if you will:

"Any callable with the same signature (modulo parameter names) as this 
template / protocol is acceptable."


> Or putting it another way: is this (silly) example legal?
[snip example]

I think maybe it should be, but right now, mypy reports it as illegal:

[steve ~]$ mypy ~/func_prototype.py 
/home/steve/func_prototype.py:7: error: Function 
"func_prototype.repeat_string" is not valid as a type

If we allow this, I see no reason why functions used as prototypes 
should be required to have an empty body. (Ellipsis or pass.) I think 
that actual, useful functions with an implementation should be allowed, 
as in your example.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JU2JCG5TLKA35AGD636OJH7NWQ75HFMX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[asleep . cult]

> Just to be clear, by "function prototype", do you mean what PEP 677 
> calls a Callable Type?
A "function prototype" would be a concrete type that exists alongside
the function type. The objects would hold the arguments, defaults,
annotations and name of a function.

> What sort of implications beyond typing?
Here are a few places where "function prototypes" would be useful:

import ctypes

@ctypes.CFUNCTYPE
def f(a: ctypes.c_int) -> ctypes.c_int

import abc

class Model(abc.ABC):
def get_id(self) -> int

By returning a concrete object, I leave it up to the implementation
to define its meaning.

> What kind of object does `def Func(a:int)->int` create, if you leave 
> out the decorator? Is that what you are calling a "function prototype 
> object"?
Yes, it would create a "function prototype" object.

> What does it need the @Callable decorator for?
The @Callable decorator is not needed, it's just something leftover from
Mark Shannon's proposal. Interpretation is entirely up to type checkers.

> What happens if you leave out the annotations and just say
> `def Func(a)` alone?
The same thing that happens when you create a function with no annotations.

> Do you have any response to my other criticisms about this syntax?
Maybe I'm misinterpreting your reply, but I don't see any other criticisms
about the syntax besides the one I addressed. The rest seem to be
geared towards Mark Shannon's proposal.

> What does that mean? What is lazy about it?
I called it lazy because it fixes the callable syntax with little to no care
about how it affects the future (if any) of the lambda syntax. I firmly
believe that fixing the callable syntax and revising the lambda syntax
should be done at the same time, or at least done in a way that doesn't
make a new lambda syntax impractical.

> Creating new syntax is backwards compatible: it doesn't break existing 
> code that is syntactically correct.
I meant this in the context of my proposal where the lambda syntax and
"callable" syntax are tightly coupled. Apologies if that wasn't clear.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4NIP24ZM4SKQVE3KCJA3B2GXMN2SKW43/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Guido van Rossum]

There were multiple threads about this (or maybe the thread was split by
mailers) and I already stated that the colon-free (and hence body-less)
syntax with def is too confusing to consider. Please look it up in the
archives. Happy Holidays!

On Thu, Dec 23, 2021 at 18:36 Steven D'Aprano  wrote:

> On Thu, Dec 23, 2021 at 05:09:18PM -0800, Guido van Rossum wrote:
>
> > > def func(params)
> > >
> > > makes a Callable type object?
> > >
> >
> > No, no, no. That syntax has already been discredited.
>
> It has? Where? Have I missed something?
>
> This thread is about using that syntax as an alternative to Mark's
> proposal. If it is already ruled out, then somebody should mention it
> to asleep.cult (the original poster of this thread).
>
> I'm curious: what objections to asleep.cult's proposal don't equally
> apply to Mark's proposal? From what I can see, Mark's original proposal
> has all the same disadvantages, plus it is even more verbose.
>
> Dropping the need for the @Callable decorator reduces the level of
> verbosity somewhat (one less line, nine fewer characters) but all the
> other negatives remain.
>
> Honestly, I cannot see a single positive to using `def` statements.
> Sure, we can do it, and an experienced programmer could infer the
> meaning of it, but given the choice of writing an anonymous type in
> place where you want it, versus having to pre-declare a function
> prototype with a name that adds nothing to the readability of the code
> before using it, why would I prefer the `def` version?
>
> This is not a rhetorical question.
>
> The fact that the existing feature (Callable) and the PEP 677 arrow
> syntax are anonymous, and can be written in place rather than needing to
> be pre-declared with a name, are positives. *Requiring* a name to use
> this `def` syntax is a point against it.
>
> If I need a named type alias, I can already create one, and name it:
>
> IntToIntFunc = Callable[[int], int]
>
> and while I can see that there are complicated signatures where a named
> alias would be useful:
>
> FileOpener = Callable[ ... ] # complicated signature
>
> we can already do that, so the `def` syntax adds nothing. For simple
> cases we don't need a name. The name IntToIntFunc adds nothing that
> isn't just as clear, if not more so, in the signature itself. It is like
> the comment:
>
> x += 1  # add one to x
>
>
> > Mark's proposal was
> > ```
> > @Callable
> > def func(params): pass
> > ```
>
> Indeed, and I already wrote a criticism of that proposal.
>
> Removing the decorator saves one line and nine characters, but the other
> criticisms remain.
>
>
> > My question is, why does it need `@Callable`? Lukasz proposed just using
> > any (undecorated) function, with the convention being that the body is
> > `...` (to which I would add the convention that the function *name* be
> > capitalized, since it is a type).
>
> But without the Callable decorator, it isn't a type, its a function.
> You're just using it as a type (or to be precise, a function prototype).
>
> I'm okay with naming conventions reflecting usage (we sort of already do
> that, with int, float, etc) but we should be clear about what's really
> happening. `def` creates a function.
>
>
> --
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VTOOJLPU2HPIW6TQBBYTW64W4DFGCQEG/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O4RHT7JW7VGWUQSQ3QFVVP6XYQGJKVQI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Function Prototypes

[Carl Meyer]

On Thu, Dec 23, 2021 at 7:38 PM Barry Warsaw  wrote:
>
> On Dec 23, 2021, at 17:09, Guido van Rossum  wrote:
> >
> > Mark's proposal was
> > ```
> > @Callable
> > def func(params): pass
> > ```
> > My question is, why does it need `@Callable`? Lukasz proposed just using 
> > any (undecorated) function, with the convention being that the body is 
> > `...` (to which I would add the convention that the function *name* be 
> > capitalized, since it is a type). My question (for Mark, or for anyone who 
> > supports `@Callable`) is why bother with the decorator. It should be easy 
> > to teach a type checker about this:
> >
> > ```
> > def SomeFn(x: float) -> int:
> > ...
> >
> > def twice(f: SomeFn) -> SomeFn:
> > return lambda x: f(f(x))
> > ```
>
> That seems pretty intuitive to me.  The conventions you mention would be just 
> that though, right?  I.e. `pass` could be used, but whatever the body is it 
> would be ignored for type checking `twice()` in this case, right?

I think this was briefly mentioned in another thread, but it seems to
have been lost in the discussion here, so I want to mention it again
because I think it's important: aside from the verbosity and hassle of
needing to always define callable types out-of-line and give them a
name, another significant downside to the function-as-type approach is
that generally Python signatures are too specific for intuitive use as
a callback type. Note that in the example immediately above, a
typechecker should error on this call:

```
def float_to_int(y: float) -> int:
return int(y)

twice(float_to_int)
```

The intent of the programmer was probably that `twice` should accept
any function taking a single float argument and returning an int, but
in fact, given the possibility of keyword calls, the names of
non-positional-only parameters are part of the function signature too.
Since the body of `twice` could call `f(x=...)`, a typechecker must
error on the use of `float_to_int` as the callback, since its
parameter is not named `x`.

In order to correctly express their intent, the programmer must
instead ensure that their callable type takes a positional-only
argument:

```
def SomeFn(_x: float, /) -> int:
...
```

The need to almost always use the extra `/` in the callable type
signature in order to get the desired breadth of signature, and the
likelihood of forgetting it the first time around until someone tries
to pass a second callback value and gets a spurious error, is in my
mind a major negative to functions-as-callable-type.

So I agree with Steven: this issue, plus the verbosity and
out-of-line-ness, mean that Callable would continue to be preferable
for many cases, meaning we'd end up with two ways to do it, neither
clearly preferable to the other. I don't think out-of-line
function-as-type could ever be an acceptable full replacement for
Callable, whereas I think PEP 677 immediately would be.

Carl
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4QPK23NPIZ6GNV47J27S3ZPSM52LE3PD/
Code of Conduct: http://python.org/psf/codeofconduct/