Re: [Python-Dev] [python-committers] Proposed release schedule for Python 3.5.4

[Victor Stinner]

2017-06-22 17:56 GMT+02:00 Brett Cannon :
> On Thu, 22 Jun 2017 at 02:32 Larry Hastings  wrote:
>> Seriously, though, I was mostly hoping other people would handle the
>> security stuff and just keep me informed.  If I'm the only one permitted to
>> accept PRs into 3.4 (and soon 3.5), okay, I can work with that.  I'm still
>> probably gonna delegate the actual judgment of the validity of the PRs.  But
>> obviously it'll mean I'll have to be more hands-on, where so far I was
>> assuming I could just let other people handle it.
>
> Currently the security-only branches are set so that only release managers
> can merge PRs since they technically are on the hook if some compatibility
> breaks due to some patch (e.g. I expect Ned to use this for 3.7 once we hit
> rc to really control what goes in last minute). It's easy enough to turn
> this protection off, though, if people want.

Larry: would you be ok to turn this protection off on the 3.4 branch?
Or would you feel more confortable if only a few people would be
allowed to push to the 3.4 branch, so add me a whitelist group or
something like that?

As I wrote, my plan is not only to merge my security fixes, but also
to work on a CI. I would feel more confortable to see the Travis CI
test result on my security PRs.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Proposed release schedule for Python 3.5.4

[Larry Hastings]

On 06/23/2017 01:55 AM, Victor Stinner wrote:

Larry: would you be ok to turn this protection off on the 3.4 branch?
Or would you feel more confortable if only a few people would be
allowed to push to the 3.4 branch, so add me a whitelist group or
something like that?


Actually I kind of like the idea of the branch being restricted. Let's 
try it for now and see if it works.




As I wrote, my plan is not only to merge my security fixes, but also
to work on a CI. I would feel more confortable to see the Travis CI
test result on my security PRs.


Do you need write access to the branch in order to get Travis CI working?


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed release schedule for Python 3.5.4

[Larry Hastings]

On 06/21/2017 07:58 PM, Larry Hastings wrote:
If you have any feedback / concerns about this schedule, or if you 
think it's important that I release 3.4.7 with these minor changes, 
please reply here.  If I don't hear anything back in a day or two I'll 
go ahead and make this the official schedule.


I haven't heard any concerns, so I'm declaring that the official 
schedule for 3.5.4, and I'm not scheduling 3.4.7 at this time.



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Proposed release schedule for Python 3.5.4

[Victor Stinner]

2017-06-23 15:19 GMT+02:00 Larry Hastings :
> Do you need write access to the branch in order to get Travis CI working?

As soon as someone reviews my proposed 3.4 patches, no :-) I will work on a PR.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[markus.wissinger]

What I try to do is a new workflow:
- 'declare' types of method parameters by usage in a unit test. Therefore 
ensure the interface is obeyed inside.- publish the used types via hints to 
help callers of the method. -Do express all hints in test code. Do not insert 
any hints manually. Any type used in any test will result in a hint (with some 
clever exceptions). This requires some discipline while writing tests, but 
worked quite well already.
I think I just realized pep544 protocols will not break that scheme, just make 
it a little more difficult to handle. I can create intermediate signatures that 
still contain superfluous subtypes and remove those only in the end when 
protocol hints are sure to exist.
Thanks.Markus 


 Ursprüngliche Nachricht Von: Stefan Richthofer 
 Datum: 23.06.17  01:08  (GMT+01:00) An: Ivan 
Levkivskyi  Cc: Markus Wissinger 
, Python-Dev  Betreff: Re: 
[Python-Dev] PEP 544: Protocols - second round 

>> I am currently exploring a type hint generator that produces hints out of 
>> types used in unit tests.

 

Note that pytypes (https://github.com/Stewori/pytypes) already supports this. 
It can dump PEP 484 style stubfiles from runtime type observations (made via 
decorators or profiler hook).

 

>> isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted 
>> generic like List[int], so that again nothing new here. To check runtime 
>> subtyping with such types one can write a third party introspection tool 
>> based on typing_inspect package on PyPI

 

There are public API functions in pytypes for isinstance and issubclass with 
support for most PEP 484 types, see 
https://github.com/Stewori/pytypes#is_of_typeobj-cls and 
https://github.com/Stewori/pytypes#is_subtypesubclass-superclass respectively. 
We also use them internally for pytypes' runtime typechecking features.

 

Unfortunately there is no release finalized yet, but it's mostly cleanup work 
and pip integration left to do. Hope to get a beta release done soon.

 

Best

 

-Stefan

 

Gesendet: Donnerstag, 22. Juni 2017 um 23:53 Uhr

Von: "Ivan Levkivskyi" 

An: "Markus Wissinger" 

Cc: Python-Dev 

Betreff: Re: [Python-Dev] PEP 544: Protocols - second round




On 22 June 2017 at 10:44, Markus Wissinger  wrote:


I have to admit I am not happy with separating the concepts of 'runtime' and 
'static' types as implied by pep544.


 

This is not something new, already PEP 483 makes a clear distinction between 
types (a static concept) and classes (a runtime concept).

 



Failing isinstance/issubclass calls for protocols would hurt there. I 
understand that any type checker code that could provide isinstance 
functionality for pep544 protocols would rely on method signatures that my hint 
generator is just producing.



 

isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted 
generic like List[int], so that again nothing new here. To check runtime 
subtyping with such types one can write a third party introspection tool based 
on typing_inspect package on PyPI (which potentially might in future become an 
official wrapper for currently internal typing API).



--

Ivan



 



___ Python-Dev mailing list 
Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev 
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2017-06-16 - 2017-06-23)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6026 ( -8)
  closed 36470 (+65)
  total  42496 (+57)

Open issues with patches: 2361 


Issues opened (49)
==

#27425: Tests fail because of git's newline preferences on Windows
http://bugs.python.org/issue27425  reopened by haypo

#30383: [3.5] Backport regrtest features from master to Python 3.5
http://bugs.python.org/issue30383  reopened by haypo

#30597: Show expected input in custom "print" error message
http://bugs.python.org/issue30597  reopened by ncoghlan

#30685: Multiprocessing Send to Manager Fails for Large Payload
http://bugs.python.org/issue30685  opened by maxehr

#30686: make `make install` faster
http://bugs.python.org/issue30686  opened by inada.naoki

#30688: support named Unicode escapes (\N{name}) in re
http://bugs.python.org/issue30688  opened by jonathaneunice

#30691: WeakSet is not pickleable
http://bugs.python.org/issue30691  opened by serhiy.storchaka

#30692: Automatic upcast does not occur for custom classes
http://bugs.python.org/issue30692  opened by rharke

#30693: tarfile add uses random order
http://bugs.python.org/issue30693  opened by bmwiedemann

#30694: Update embedded copy of expat to 2.2.1
http://bugs.python.org/issue30694  opened by ned.deily

#30695: add a nomemory_allocator to the _testcapi module
http://bugs.python.org/issue30695  opened by xdegaye

#30696: infinite loop in PyRun_InteractiveLoopFlags()
http://bugs.python.org/issue30696  opened by xdegaye

#30697: segfault in PyErr_NormalizeException() after memory exhaustion
http://bugs.python.org/issue30697  opened by xdegaye

#30698: asyncio sslproto do not shutdown ssl layer cleanly
http://bugs.python.org/issue30698  opened by grzgrzgrz3

#30699: Misleading class names in datetime.tzinfo usage examples
http://bugs.python.org/issue30699  opened by Robert.Tasarz

#30700: fileinput inplace clobbers file without leaving backup on deco
http://bugs.python.org/issue30700  opened by switchnode

#30701: Exception parsing certain invalid email address headers
http://bugs.python.org/issue30701  opened by timb07

#30703: test_multiprocessing_forkserver hangs on the master branch
http://bugs.python.org/issue30703  opened by haypo

#30704: test_free_different_thread() of test_code leaks references on 
http://bugs.python.org/issue30704  opened by haypo

#30705: python2.7 -m test -R 3:3 hangs on Refleaks 2.7 buildbots
http://bugs.python.org/issue30705  opened by haypo

#30706: EmailMessage Object Creation
http://bugs.python.org/issue30706  opened by Michael Salsone

#30707: Incorrect description of "async with" in PEP492 and documentat
http://bugs.python.org/issue30707  opened by Damien George

#30708: Ensure that the result of PyUnicode_AsWideCharString() doesn't
http://bugs.python.org/issue30708  opened by serhiy.storchaka

#30710: getaddrinfo raises OverflowError
http://bugs.python.org/issue30710  opened by smejkar

#30711: getaddrinfo invalid port number
http://bugs.python.org/issue30711  opened by smejkar

#30713: Reject newline character (U+000A) in URLs in urllib.parse
http://bugs.python.org/issue30713  opened by haypo

#30714: test_ssl fails with openssl 1.1.0f
http://bugs.python.org/issue30714  opened by cstratak

#30715: Test_winreg, test_dynamic_key hangs on my Win 10
http://bugs.python.org/issue30715  opened by terry.reedy

#30716: Failing tests with installed 3.6.2rc1 on Win 10-64
http://bugs.python.org/issue30716  opened by terry.reedy

#30717: str.center() is not unicode aware
http://bugs.python.org/issue30717  opened by Guillaume Sanchez

#30718: open builtin function: specifying the size of buffer has no ef
http://bugs.python.org/issue30718  opened by direprobs

#30719: IDLE: Make PyShell visible upon error.
http://bugs.python.org/issue30719  opened by terry.reedy

#30721: Show expected input for right shift operator usage in custom "
http://bugs.python.org/issue30721  opened by CuriousLearner

#30722: Tools/demo/redemo.py broken
http://bugs.python.org/issue30722  opened by Christoph Sarnowski2

#30723: IDLE parenmatch - left and right highlighting, independent fla
http://bugs.python.org/issue30723  opened by wohlganger

#30724: ZipFile.open treats directory path as empty file
http://bugs.python.org/issue30724  opened by Kit Yan Choi

#30725: Windows installer for 2.7.13 doesn't install pip when installi
http://bugs.python.org/issue30725  opened by KevKeating

#30726: [Windows] Warnings in elementtree due to new expat
http://bugs.python.org/issue30726  opened by Segev Finer

#30728: IDLE: Modernize configdialog code.
http://bugs.python.org/issue30728  opened by terry.reedy

#30730: Injecting environment variable in subprocess on Windows
http://bugs.python.org/issue30730  opened by serhiy.storchaka

#30731: Use correct executable manifest for windows
http://bugs.pytho

[Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

[Terry Reedy]

Example: appveyor passes for
https://github.com/python/cpython/pull/2335#discussion_r123724857

Appveyor build fails for 3.6 backport
https://github.com/python/cpython/pull/2359
https://ci.appveyor.com/project/python/cpython/build/3.6.1+.3673
The compile and build errors have nothing to do with the patch.

This failure is routine; turning off appveyor for 3.6, until fixed, 
would reduce noise.


I presume this is a workflow issue, but I don't know if Windows experts 
would see it.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Core-mentorship] C version of weakrefset.py.

[Brett Cannon]

On Thu, 22 Jun 2017 at 16:26 Bhavishya  wrote:

> Also I saw your conversation with "Brett Cannon" on lazy-loading some
> modules at  startup, and also doing so using ModuleProxy(also several
> implementations that exist like importlib's Lazyloader
> ,
> PEAK  ).
> So your suggestion's on this too?
>

As stated earlier, if you wish to discuss seeing if lazy loading will help
in Python's startup performance then please start a thread over at
sp...@python.org and we can discuss it there.

-Brett


>
> Thank You
>
> regards,
> Bhavishya
>
> On Thu, Jun 22, 2017 at 7:24 PM, Bhavishya 
> wrote:
>
>> Hello,
>> 1)I might be totally wrong here, but even if we go with stripping
>> annotation(in .pyc)...still the "lag" that comes from ABCs needs to be
>> addressed.
>>
>> 2) I been reading for past few days about your fat-optimizer project and
>> the corresponding TO-DO list, if you think that PEP-0511 should be
>> improved, I can work on that.
>>
>> 3)Also I was seeing to existing repos which implement some-kind of
>> optimizaton, like
>> numpy,snake-oil
>>
>> Finally I wanted to decide upon a roadmap, so that I could put more
>> specific efforts.(fat optimizer?)
>>
>> Thank You
>> Regards,
>> Bhavishya
>>
>> On Wed, Jun 21, 2017 at 8:05 PM, Victor Stinner > > wrote:
>>
>>> 2017-06-21 15:21 GMT+02:00 INADA Naoki :
>>> > ABC slowdown Python startup only 2ms.  But importing typing module
>>> take 11ms.
>>> > While typing is not imported from site.py, many new Python application
>>> > will import it.
>>> > It may take over 100ms for applications or libraries heavily depending
>>> on ABCs.
>>>
>>> When typing is not used in the application, only used for static
>>> checks, you can try to "strip" annotations to avoid any overhead on
>>> the application startup. It's not only a matter of "import typing",
>>> it's also the cost of instanciating types like "List[int]" (or even
>>> more complex ones).
>>>
>>> I discussed with Jukka Lehtosalo at Pycon US about stripping
>>> completely annotations. He told me that my PEP 511 may be a good
>>> solution to keep annotation in the .py code, but strip them for
>>> "production code", in the cached .pyc files:
>>>
>>> "PEP 511 -- API for code transformers"
>>> https://www.python.org/dev/peps/pep-0511/
>>>
>>> This PEP is somehow controversal. Some people fear that it would allow
>>> people to hack the Python language to write their own incompatible
>>> variant of Python. I don't think that my PEP adds anything new, it's
>>> already possible to do that, importlib made it even more easy. I used
>>> my FAT Python optimizer project to sell this PEP. Since FAT Python is
>>> also controversal (it hasn't been proved to be actually faster), the
>>> PEP didn't go far at my last attempt.
>>>
>>> Note: Is core-menthorship the best place for such performance
>>> discussion? :-) Maybe we should open a thread on python-dev@ or speed@
>>> mailing list.
>>>
>>> Victor
>>>
>>
>>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

[Zachary Ware]

On Fri, Jun 23, 2017 at 11:36 AM, Terry Reedy  wrote:
> Example: appveyor passes for
> https://github.com/python/cpython/pull/2335#discussion_r123724857
>
> Appveyor build fails for 3.6 backport
> https://github.com/python/cpython/pull/2359
> https://ci.appveyor.com/project/python/cpython/build/3.6.1+.3673
> The compile and build errors have nothing to do with the patch.
>
> This failure is routine; turning off appveyor for 3.6, until fixed, would
> reduce noise.
>
> I presume this is a workflow issue, but I don't know if Windows experts
> would see it.

I had not seen that failure before, but it is troubling.  For
AppVeyor, it looks like we might be able to get around it by using the
`cache` feature (which we should probably do anyway), but if that
failure happens routinely otherwise, we may have to completely rethink
the approach I've taken in https://bugs.python.org/issue30450

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

[Terry Reedy]

On 6/23/2017 2:24 PM, Zachary Ware wrote:

On Fri, Jun 23, 2017 at 11:36 AM, Terry Reedy  wrote:

Example: appveyor passes for
https://github.com/python/cpython/pull/2335#discussion_r123724857

Appveyor build fails for 3.6 backport
https://github.com/python/cpython/pull/2359
https://ci.appveyor.com/project/python/cpython/build/3.6.1+.3673
The compile and build errors have nothing to do with the patch.

This failure is routine; turning off appveyor for 3.6, until fixed, would
reduce noise.

I presume this is a workflow issue, but I don't know if Windows experts
would see it.


I had not seen that failure before, but it is troubling.  For
AppVeyor, it looks like we might be able to get around it by using the
`cache` feature (which we should probably do anyway), but if that
failure happens routinely otherwise, we may have to completely rethink
the approach I've taken in https://bugs.python.org/issue30450


If by 'cache', you mean keep a current build of the main x.y branch, and
only recompile what is needed, and skip compile for .py and .rst only 
patches, thereby speeding up CI testing, I would be all in favor.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

[Steve Dower]

We could improve life for everyone even more if we built Tcl/Tk once per update 
and made it a binary dependency. Nobody is updating our version of it regularly 
anyway, and this would significantly improve build time in CI.

The externals script probably just needs some retry logic. Doesn’t look like a 
“throw it all out” problem to me.

Cheers,
Steve

Top-posted from my Windows phone

From: Terry Reedy
Sent: Friday, June 23, 2017 11:44
To: python-dev@python.org
Subject: Re: [Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

On 6/23/2017 2:24 PM, Zachary Ware wrote:
> On Fri, Jun 23, 2017 at 11:36 AM, Terry Reedy  wrote:
>> Example: appveyor passes for
>> https://github.com/python/cpython/pull/2335#discussion_r123724857
>>
>> Appveyor build fails for 3.6 backport
>> https://github.com/python/cpython/pull/2359
>> https://ci.appveyor.com/project/python/cpython/build/3.6.1+.3673
>> The compile and build errors have nothing to do with the patch.
>>
>> This failure is routine; turning off appveyor for 3.6, until fixed, would
>> reduce noise.
>>
>> I presume this is a workflow issue, but I don't know if Windows experts
>> would see it.
> 
> I had not seen that failure before, but it is troubling.  For
> AppVeyor, it looks like we might be able to get around it by using the
> `cache` feature (which we should probably do anyway), but if that
> failure happens routinely otherwise, we may have to completely rethink
> the approach I've taken in https://bugs.python.org/issue30450

If by 'cache', you mean keep a current build of the main x.y branch, and
only recompile what is needed, and skip compile for .py and .rst only 
patches, thereby speeding up CI testing, I would be all in favor.

-- 
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

[Terry Reedy]

On 6/23/2017 4:14 PM, Steve Dower wrote:
We could improve life for everyone even more if we built Tcl/Tk once per 
update and made it a binary dependency. Nobody is updating our version 
of it regularly anyway, and this would significantly improve build time 
in CI.


On Windows, I believe compiling tcl/tk takes longer to compile than 
python.  It is certainly close.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] New workflow change: Welcome to blurb

[Larry Hastings]

One minor but ongoing problem we've had in CPython core development has 
been the mess of updating Misc/NEWS.  Day-to-day developers may have a 
conflict if they lose a push race, which means a little editing.  You'll 
have a similar, if slightly worse, problem when cherry-picking a fix 
between versions.  Worst of all used to be the manual merges necessary 
after cutting a release--this was the bane of a CPython release 
manager's existence.  (Though the new git-based workflow may have 
obviated the worst of this.)


The real problem is that we have one central file that everybody 
continually edits in a haphazard way.  We aren't actually editing the 
same information, we aren't actually changing the same lines. But our 
revision control systems and diff algorithms don't understand the 
structure of Misc/NEWS and so they get confused. And for what? It's not 
like there's a tremendous benefit to having this central file everyone's 
fighting over.


We've been talking about addressing this for years.  Fixing this was one 
of the goals of the new workflow.  And finally, as of right now, the 
future is here.  Ladies and gentlemen, I present: blurb.


   https://github.com/python/core-workflow/tree/master/blurb


blurb is an interactive command-line tool that helps you write Misc/NEWS 
entries.  You simply run blurb from anywhere inside a CPython repo.  
blurb runs an editor for you with a template open. You fill in these 
three pieces of information:


 * the bugs.python.org or "bpo" issue number,
 * what "section" of Misc/NEWS this entry should go in (by uncommenting
   the correct line), and
 * the text of the Misc/NEWS entry, in ReST format.

You save and exit and you're done.  blurb even stages the Misc/NEWS 
entry in git for you!



Behind the scenes, blurb writes your information here:

   Misc/NEWS.d/next//

The "" is the name of the section in Misc/NEWS where your 
entry should go.   contains the current date and time, the bpo 
number, and a nonce to prevent collisions.


These "next" files get merged together into a single aggregate .rst file 
by the release manager when cutting a release (using "blurb release").  
One nice feature of this approach: when you cherry-pick a change, its 
Misc/NEWS entry in "next" gets cherry-picked along with it.



One important change: Misc/NEWS will no longer be checked in. It'll 
still be present in CPython tarballs; it will be generated by the 
release manager as part of cutting a release.  But as a repository of 
information, it's been superseded by the various blurb data files.  And 
by regenerating it from data files, we ensure that we'll never ever have 
a Misc/NEWS conflict ever again!


The plan is to leave Misc/NEWS in the CPython repo for maybe another 
week, to let the current crop of PRs get merged.  But new work should 
switch to using blurb immediately.



You can install blurb from pip:

   % pip3.6 install blurb

In fact--please do!


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New workflow change: Welcome to blurb

[Craig Rodrigues]

On Fri, Jun 23, 2017 at 8:24 PM, Larry Hastings  wrote:

>
>
> We've been talking about addressing this for years.  Fixing this was one
> of the goals of the new workflow.  And finally, as of right now, the future
> is here.  Ladies and gentlemen, I present: blurb.
>
> https://github.com/python/core-workflow/tree/master/blurb
>
>
Yes, when you have a single NEWS file that needs to get updated,
the process begins  to fall apart when you have a pull request type of
workflow
which results in many merge conflicts to the single NEWS file.

Twisted and Buildbot use towncrier ( https://github.com/hawkowl/towncrier )
which tries
to solve the same problem as blurb.  The developer needs to
commit individual newsfragment files to the tree along with their pull
request.
At release time, the release engineer runs towncrier which aggregates the
individual
newsfragments into a single NEWS file.  The NEWS file is not checked in,
but the release engineer makes it available as part of official release
tarballs.

--
Craig
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New workflow change: Welcome to blurb

[Steve Dower]

One quick heads up – the NEWS file is included in the docs build (if not in the 
html docs, certainly in the CHM for Windows releases). You may have to do some 
extra work to keep that from breaking when you remove it. We might also include 
it as plain text in the installers, I forget right now.

Is blurb going to be embedded in the main repository? Not necessarily a problem 
if not, but I'd rather not have the build process depend on pip. Though I guess 
Sphinx is dependency already, so perhaps I should just integrate it better into 
the build…

Top-posted from my Windows phone

From: Larry Hastings
Sent: Friday, June 23, 2017 20:26
To: Python Dev; python-committers
Subject: [Python-Dev] New workflow change: Welcome to blurb



One minor but ongoing problem we've had in CPython core development has been 
the mess of updating Misc/NEWS.  Day-to-day developers may have a conflict if 
they lose a push race, which means a little editing.  You'll have a similar, if 
slightly worse, problem when cherry-picking a fix between versions.  Worst of 
all used to be the manual merges necessary after cutting a release--this was 
the bane of a CPython release manager's existence.  (Though the new git-based 
workflow may have obviated the worst of this.)

The real problem is that we have one central file that everybody continually 
edits in a haphazard way.  We aren't actually editing the same information, we 
aren't actually changing the same lines.  But our revision control systems and 
diff algorithms don't understand the structure of Misc/NEWS and so they get 
confused.  And for what? It's not like there's a tremendous benefit to having 
this central file everyone's fighting over.

We've been talking about addressing this for years.  Fixing this was one of the 
goals of the new workflow.  And finally, as of right now, the future is here.  
Ladies and gentlemen, I present: blurb.
https://github.com/python/core-workflow/tree/master/blurb

blurb is an interactive command-line tool that helps you write Misc/NEWS 
entries.  You simply run blurb from anywhere inside a CPython repo.  blurb runs 
an editor for you with a template open.  You fill in these three pieces of 
information:
• the bugs.python.org or "bpo" issue number,
• what "section" of Misc/NEWS this entry should go in (by uncommenting the 
correct line), and
• the text of the Misc/NEWS entry, in ReST format.
You save and exit and you're done.  blurb even stages the Misc/NEWS entry in 
git for you!


Behind the scenes, blurb writes your information here:
Misc/NEWS.d/next//
The "" is the name of the section in Misc/NEWS where your entry 
should go.   contains the current date and time, the bpo number, and 
a nonce to prevent collisions.

These "next" files get merged together into a single aggregate .rst file by the 
release manager when cutting a release (using "blurb release").  One nice 
feature of this approach: when you cherry-pick a change, its Misc/NEWS entry in 
"next" gets cherry-picked along with it.


One important change: Misc/NEWS will no longer be checked in.  It'll still be 
present in CPython tarballs; it will be generated by the release manager as 
part of cutting a release.  But as a repository of information, it's been 
superseded by the various blurb data files.  And by regenerating it from data 
files, we ensure that we'll never ever have a Misc/NEWS conflict ever again!

The plan is to leave Misc/NEWS in the CPython repo for maybe another week, to 
let the current crop of PRs get merged.  But new work should switch to using 
blurb immediately.


You can install blurb from pip:
% pip3.6 install blurb
In fact--please do!


/arry

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] New workflow change: Welcome to blurb

[Nick Coghlan]

On 24 June 2017 at 13:24, Larry Hastings  wrote:
>
>
> One minor but ongoing problem we've had in CPython core development has been
> the mess of updating Misc/NEWS.  Day-to-day developers may have a conflict
> if they lose a push race, which means a little editing.  You'll have a
> similar, if slightly worse, problem when cherry-picking a fix between
> versions.  Worst of all used to be the manual merges necessary after cutting
> a release--this was the bane of a CPython release manager's existence.
> (Though the new git-based workflow may have obviated the worst of this.)
>
> The real problem is that we have one central file that everybody continually
> edits in a haphazard way.  We aren't actually editing the same information,
> we aren't actually changing the same lines.  But our revision control
> systems and diff algorithms don't understand the structure of Misc/NEWS and
> so they get confused.  And for what? It's not like there's a tremendous
> benefit to having this central file everyone's fighting over.
>
> We've been talking about addressing this for years.  Fixing this was one of
> the goals of the new workflow.  And finally, as of right now, the future is
> here.  Ladies and gentlemen, I present: blurb.
>
> https://github.com/python/core-workflow/tree/master/blurb

Thanks Larry, great to see this go live!

> Behind the scenes, blurb writes your information here:
>
> Misc/NEWS.d/next//
>
> The "" is the name of the section in Misc/NEWS where your
> entry should go.   contains the current date and time, the bpo
> number, and a nonce to prevent collisions.

Folks are also free to handcraft these files if they really want to do
so. The Developer Guide has the necessary details:
https://docs.python.org/devguide/committing.html#what-s-new-and-news-entries

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New workflow change: Welcome to blurb

[Nick Coghlan]

On 24 June 2017 at 14:01, Craig Rodrigues  wrote:
> On Fri, Jun 23, 2017 at 8:24 PM, Larry Hastings  wrote:
>>
>>
>>
>> We've been talking about addressing this for years.  Fixing this was one
>> of the goals of the new workflow.  And finally, as of right now, the future
>> is here.  Ladies and gentlemen, I present: blurb.
>>
>> https://github.com/python/core-workflow/tree/master/blurb
>
>
> Yes, when you have a single NEWS file that needs to get updated,
> the process begins  to fall apart when you have a pull request type of
> workflow
> which results in many merge conflicts to the single NEWS file.
>
> Twisted and Buildbot use towncrier ( https://github.com/hawkowl/towncrier )
> which tries
> to solve the same problem as blurb.

Aye, towncrier and OpenStack's reno were the two main alternatives we
looked at in addition to Larry's offer of creating a tool specifically
for CPython: https://github.com/python/core-workflow/issues/6

We ultimately settled on blurb mainly because we wanted to be able to
customise various details differently from the way towncrier works,
and we figure they're close enough in spirit that folks familiar with
one won't have any problems adapting to the other.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Appveyor builds fail on Windows for 3.6 backports.

[Nick Coghlan]

On 24 June 2017 at 06:14, Steve Dower  wrote:
> We could improve life for everyone even more if we built Tcl/Tk once per
> update and made it a binary dependency. Nobody is updating our version of it
> regularly anyway, and this would significantly improve build time in CI.

I'm pretty sure the *nix builds already default to using the system
Tcl/Tk, so figuring out a way to do something similar on Windows (and
then bundling the DLLs in the installer) would make sense to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] New workflow change: Welcome to blurb

[Serhiy Storchaka]

2017-06-24 6:24 GMT+03:00 Larry Hastings :
> One minor but ongoing problem we've had in CPython core development has been
> the mess of updating Misc/NEWS.  Day-to-day developers may have a conflict
> if they lose a push race, which means a little editing.  You'll have a
> similar, if slightly worse, problem when cherry-picking a fix between
> versions.  Worst of all used to be the manual merges necessary after cutting
> a release--this was the bane of a CPython release manager's existence.
> (Though the new git-based workflow may have obviated the worst of this.)

Thanks Larry! With the old hg-based workflow this was only slightly
annoying, but the new git-based workflow turned this into a hell. If
you have several PRs that updates the same Misc/NEWS section you
needed to spent many time for just resolving conflicts one by one and
waiting CI tests. And be lucky if other core developer trying to do
the same withis PRs at the same time.

> You can install blurb from pip:
>
> % pip3.6 install blurb
>
> In fact--please do!

I have installed it, but how to use it?

$ python3 -m pip install --user blurb
Collecting blurb
  Using cached blurb-1.0-py3-none-any.whl
Installing collected packages: blurb
Successfully installed blurb-1.0
$ python3 -m blurb
/usr/bin/python3: No module named blurb
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com