[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Attached is a patch against current trunk merging Argument Clinic. I already got the go-ahead to merge (took a poll in python-dev, got lots of +1s), but I still wanted to post it here to let it soak a little, even only if it's just a day. Unless something am

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Nick Coghlan added the comment: Stefan's draft is now published: http://www.python.org/dev/peps/pep-0437/ -- ___ Python tracker ___ __

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Larry has requested privately that I send the counter proposal PEP and additional information, so here it is: I've send the PEP draft to Nick. The patch that I uploaded contains DSL examples, an ml-yacc grammar and token specifications. Two prototype tools are av

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Stefan Krah : Added file: http://bugs.python.org/file29397/printsemant ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Stefan Krah : Added file: http://bugs.python.org/file29396/preprocess ___ Python tracker ___ ___ Python-bugs-list mailing list Unsu

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Stefan Krah : -- keywords: +patch Added file: http://bugs.python.org/file29395/issue16612-alternative-dsl.diff ___ Python tracker ___ _

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Thanks for pointing that out! I've fixed it in my local branch, though I'm fooling around with some new syntax so I'm not ready to publish it yet. It's not a general problem, exactly; it's a problem with extension types. After just a little tweaking and refa

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: I've started working on the alternative DSL PEP, but I hit a DSL-independent roadblock: The preprocessor passes copies of structs to the _impl functions, which could lead to subtle bugs. I pointed out a benign example on Rietveld, but in general I think this shoul

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: write_clinic_file tells Clinic to start writing to the clinic file, which I think is best to do at the very top of the file. include_clinic_file spits out the #include, which you probably want near the bottom of the file, just before the static module declara

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > * Antoine: I've implemented shunting the bulk of Argument Clinic's > output into a separate file. Please see Modules/zlibmodule.c > and Modules/zlibmodule_clinic.c for more. Interesting, thanks :-) Why do you need both clinic.write_clinic_file() and clin

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Behnel added the comment: I mentioned it in a couple of places during the discussion, you might just have missed them. The code that generates the unpacking C code starts here: https://github.com/cython/cython/blob/4ebc647c2fa54179d25335b2dcf5d845e7fc9a79/Cython/Compiler/Nodes.py#L3068

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: As a rule I'm unlikely to mention things I haven't heard about. I've never used Cython, and I don't recall anyone mentioning this technology previously. Once skrah posts his alternative DSL proposal, I'll amend the PEP to discuss both these alternatives. Ca

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Behnel added the comment: I still can't see a reference to Cython in the PEP. Larry, I don't really mind adding a newly designed DSL for this, but regarding the implementation of the actual argument parser, could you at least add a short paragraph to the PEP that mentions it as worth be

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: A quick note about the extension mechanism. Currently the only way to extend PyArg_Parse* is via O&. Therefore, any extended type you add will use O&, and will have a "converter". So internally all I did was say "if the parameter has a converter, ignore the

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Argument Clinic is now PEP 436. http://www.python.org/dev/peps/pep-0436/ -- ___ Python tracker ___

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Okay, I have finally addressed all the comments so far. Changes described below are my patch #2. They're also checked in to https://bitbucket.org/larry/python-clinic/ . * Antoine, Nick, et al: I've converted clinic.txt into a PEP. I've already sent it to

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Serhiy Storchaka wrote: > > { path: [string, bytes, int] => path_converter => path_t }, > > And register types somewhere: I must admit that I had a similar thought when I first heard about the project: If we're going through all this anyway, why not have a regist

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Serhiy Storchaka added the comment: > { path: [string, bytes, int] => path_converter => path_t }, > *, > { dir_fd: [int, None] = None => OS_STAT_DIR_FD_CONVERTER => int }, > { follow_symlinks: bool = True => "p" => int } Why not just: path: path_t * dir_fd: dir_fd_t = None => DEFAULT_

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: OK, I'll have a go at the PEP then. In addition to the proposed syntax in my previous mail, I'm going to suggest this alternative: /*[preprocessor] # Declaration os.stat [PyOs_Stat] ( { path: [string, bytes, int] => path_converter => path_t }, *, { dir_fd: [i

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Stefan Behnel : -- nosy: +scoder ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Nick Coghlan added the comment: Stefan, that proposal definitely looks like it is worth writing up as a PEP to me. One thing that I particularly like about it is that it should be possible to pluck out the first element of the {} entries fairly easily in order to get the ordinary Python signat

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: I would be willing to write an (alternative) PEP, but I'm not quite satisfied with my own proposal yet. Also, I'd need to understand the positional-only part better, but the _cursesmodule.c example does not seem to compile here. :) So, *disregarding* the positional

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > I agree with Stefan that the file is a lot more readable if the > docstring > is not repeated twice. Couldn't the generated C docstring end up in a separate file? It's only a constant after all. It seems to me that one reason we don't give much love to C doc

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Nick Coghlan added the comment: Stefan, would you be willing to write up your version as a PEP? (FWIW, I personally favour your suggested C'ish style for the preprocessor, since my brain would be in that mode anyway when hacking on C code) -- title: Integrate "Argument Clinic" specialize

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Jim Jewett wrote: > (1) The first several lines ( "/* pymacro.h */" until "/* could go into a > separate header file */" ) would not be written at all, and are just there to > help reviewers understand. Yes, they should ultimately go into Include/pymacro.h. >

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Jim Jewett added the comment: I'm not sure I correctly understand skrah's proposal. If I do, then (1) The first several lines ( "/* pymacro.h */" until "/* could go into a separate header file */" ) would not be written at all, and are just there to help reviewers understand. (2) The "#def

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Bradley Froehle added the comment: > As for the docstring: I would like it better if I could avoid typing > the cumbersome "\n\"s. I agree with Stefan that the file is a lot more readable if the docstring is not repeated twice. It's unfortunate that C doesn't have a notion of a raw string (as op

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > Here's a proposal for an alternative without parameter docstrings and > a > different DSL (see os_stat.c). I guess it's easiest to present my > thoughts > in list form. It's a bit difficult to give an opinion without a more formal definition. For example it se

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Here's a proposal for an alternative without parameter docstrings and a different DSL (see os_stat.c). I guess it's easiest to present my thoughts in list form. Changes and rationale: == Split docstring into function header and rest --

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Nick Coghlan added the comment: +1 for documenting the clinic DSL as a PEP. It's a bit too hard to find the rationale for toolchain changes like this when they're hidden away in tracker issues. It doesn't need to be an extended essay like most of the PEPs I write, it can be relatively short a

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Bradley Froehle added the comment: First off, thanks for all the work so far. This has proven incredibly useful to me in a personal project. However, I think there needs to be some additional discussion of how to handle situations where the arguments passed to PyArg_ParseTuple require additiona

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > That said, I think a lot of that has been done if you look at the > clinic.txt file within the patch. That already contains the meat of > what would go into a PEP. Certainly. A PEP doesn't have to be a 100% new text. -- _

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Gregory P. Smith added the comment: PEPs are perceived as a hurdle. Regardless, clinic.txt would turn into one pretty easily so just doing it may be easiest. :) On Fri, Dec 14, 2012 at 12:11 PM, Antoine Pitrou wrote: > > Antoine Pitrou added the comment: > > > I have no current plan to write a

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > I have no current plan to write a PEP. I don't know how to settle the > difference of opinion here; the easiest thing would be if you > convinced Guido we needed one. This is silly. Nobody is opposed to you writing a PEP and a couple people would like you to

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Gregory P. Smith added the comment: There's a bit of an impasse here on the PEP front. Guido said it wasn't needed. Antoine brings up the point that CPython developers will have to live with the result of this work so maybe something more easy to read and see in one place like a PEP would be

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: I have no current plan to write a PEP. I don't know how to settle the difference of opinion here; the easiest thing would be if you convinced Guido we needed one. -- ___ Python tracker

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: I'll ask again, do you plan to write a PEP to iron out the functional details? I don't think a tracker entry is the right format for this. Le vendredi 14 décembre 2012 à 07:21 +, Larry Hastings a écrit : > Larry Hastings added the comment: > > I don't think

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Larry Hastings wrote: > Finally, since you have not addressed it directly, let me ask you: > are you interested in Clinic having a more boilerplate-friendly macro > processing mode, with templates or recycling old entries or something? > Or do you not care? If it

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: I don't think we can solve the problem of the output being too long for your liking. Personally I like the output; I find it eminently readable, and making it shorter would impair that. I think in the majority of uses Clinic will be a win for readability. I

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Gregory, I'm talking about the size of the *total output* of the tool, not about the size of the actual DSL parts: posixmodule.c: 1138211514+132 (2 functions) _cursesmodule.c:3434 3499 +65 (1 function) zlibmodule.c: 1295 13

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Gregory P. Smith added the comment: doesn't the size of code setting up ParseTupleWithKeywords and checking the values afterwards bother you? that's the same thing only much less consistent. -- ___ Python tracker

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Larry Hastings wrote: > I disagree that the Clinic DSL is "verbose". Certainly I find it > more succinct than what we do now. I was referring to the total size of the generated output, not to the DSL. > On the other hand, the syntax you proposed in the python-de

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Andrew Svetlov added the comment: How Argument Clinic solves problems like #16490? -- ___ Python tracker ___ ___ Python-bugs-list mail

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > The semicolon is optional, permitted explicitly so you can > copy-and-paste the original C variable declarations in and you don't > have to go hunting for semicolons. The spaces are permitted on > those lines for the same reason; they aren't on the "flag" line

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Ah. In that case, may I rewrite your critique as "[...] some lines may end with a semicolon, some lines may not, some assignment signs ("=") permit spaces around them, some don't." The semicolon is optional, permitted explicitly so you can copy-and-paste the

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > > What lines end with a colon? > > He probably means semicolon, for example: > > +int dir_fd = DEFAULT_DIR_FD; > +default=None Oops, yes, sorry. Semicolon indeed. -- ___ Python tracker

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Chris Jerdonek added the comment: > What lines end with a colon? He probably means semicolon, for example: +int dir_fd = DEFAULT_DIR_FD; +default=None -- ___ Python tracker ___

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: What lines end with a colon? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: Le dimanche 09 décembre 2012 à 19:20 +, Stefan Krah a écrit : > > If I'm the only one who is slightly bothered by the aesthetic and readability > aspects, then 5) obviously need not be discussed. You're not the only one. The vertical space argument also res

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: I disagree that the Clinic DSL is "verbose". Certainly I find it more succinct than what we do now. On the other hand, the syntax you proposed in the python-dev message you cite is insufficient to the task. Consider a function that takes a "char *". How might

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: Here are some things that I'm interested in right now: 1) From what kind of data structure are signature objects generated? 2) Is there an easy way to do this manually? 3) Can at least part of the verboseness go into header files (virtually all com

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Bradley Froehle : -- nosy: +bfroehle ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: (and, oh, I agree there isn't a bright line separating "this should get a PEP" from "this doesn't need a PEP") -- ___ Python tracker ___ ___

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Alex Gaynor added the comment: For what it's worth, I'm not as concerned with the process of the PEP, as having a single document we can review and discuss. -- ___ Python tracker _

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: > Antoine, Stefan: There doesn't appear to be a bright line separating > "this should get a PEP" from "this doesn't need a PEP". That said, > changes to the C API for CPython don't seem to merit PEPs very often, > the recent "Stable ABI" being the lone exception

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Antoine, Stefan: There doesn't appear to be a bright line separating "this should get a PEP" from "this doesn't need a PEP". That said, changes to the C API for CPython don't seem to merit PEPs very often, the recent "Stable ABI" being the lone exception to t

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Stefan Krah added the comment: I second Antoine's request for a PEP. I'm not only concerned about the DSL: This approach may work for modules like posixmodule.c, where each function is largely a self-contained unit wrapping some API function. On the other hand, I don't want to imagine how _deci

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Chris Jerdonek : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Chris Jerdonek added the comment: It seems the clinic.txt DSL document should be proofread for proper/consistent use of argument/parameter (e.g. as described in the recently added http://docs.python.org/dev/glossary.html#term-parameter ). To choose a couple random examples-- +Argument declar

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Antoine Pitrou added the comment: Can't you propose a PEP for this? I'd like to see the DSL officially discussed (or at least validated). -- nosy: +pitrou ___ Python tracker ___

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Larry Hastings added the comment: Sorry for the delay; this head cold is slowing me down. Here's the current state of Argument Clinic as a patch for review. I look forward to your comments! -- Added file: http://bugs.python.org/file28248/larry.clinic.patch.1.txt

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by David Villa Alises : -- nosy: +david.villa ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Mark Dickinson : -- nosy: +mark.dickinson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Daniel Urban : -- nosy: +daniel.urban ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Jeremy Kloth : -- nosy: +jkloth ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Eric V. Smith : -- nosy: +eric.smith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Andreas Stührk : -- nosy: +Trundle ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Alex Gaynor : -- nosy: +alex ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Dave Malcolm : -- nosy: +dmalcolm ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

Changes by Glenn Linderman : -- nosy: +v+python ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue16612] Integrate "Argument Clinic" specialized preprocessor into CPython trunk

New submission from Larry Hastings: This bug tracker entry is to track adding "Argument Clinic" to CPython trunk. Please see http://mail.python.org/pipermail/python-dev/2012-December/122920.html for more information. -- assignee: larry components: Interpreter Core messages: 176965 nos