[Python-Dev] Re: Steering Council update for February

[Titus Brown]

> On Mar 9, 2021, at 4:27 PM, Terry Reedy  wrote:
> 
> On 3/9/2021 3:27 PM, Pablo Galindo Salgado wrote:
> 
>> The Steering Council just published the community update for February:
> 
> Thank you for posting this.
> 
>>The Steering Council discussed renaming the master branch to main
>>and the consensus was that we should do that.
> 
> I did not that this was an issue.  This will break 4 of my .bat files (update 
> repository and workspaces, build python.exes, run sphinx on docs,  run 
> coverage.) and, I presume, many other workflow scripts and bots.  What is the 
> upside?  ('next (version)' might be more accurately descriptive, but 'main' 
> works as the 'main branch we are working on'.) Does 'master' confuse people?

Hi Terry,

this is happening in many places - for example, see

https://sfconservancy.org/news/2020/jun/23/gitbranchname/
https://github.com/github/renaming

tl;dr ‘master’ has unpleasant connotations for many, so it’s being changed in 
many places/many projects.

best,
—titus

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GDAUZKYB6GP3A3ZGBSQ4KQ7R6QFIZHZC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Release of a responsive python-docs-theme 2021.5

[Titus Brown]

Thanks, Carol (and Olga)! This sounds great!

I’m sure you can all google :), but here’s the link:

https://github.com/python/python-docs-theme

I wasn’t able to find examples - is this what is up and running on 
docs.python.org, or is that a future plan?

thanks much,
—titus

p.s. Not sure what address I’m subscribed to on python-dev, so this might get 
blocked for posting there. Is ok.

> On May 7, 2021, at 4:32 PM, Carol Willing  
> wrote:
> 
> It’s with great pleasure that I announce that python-docs-theme has been 
> released to PyPI.
> 
> Thanks to the hard work and patience of Olga Bulat, @obulat, Python’s doc 
> theme is now responsive. Many thanks to everyone who has contributed to this 
> release by filing issues, writing PRs, reviewing PRs, and testing the theme. 
> It was a great team effort.
> 
> Here are the highlights from the CHANGELOG.rst:
> 
> - Make the theme responsive (#46) Contributed by Olga Bulat.
> - Use Python 3.8 for the Github Actions (#71) Contributed by Stéphane Wirtel.
> - Use default pygments theme (#68) Contributed by Aaron Carlisle.
> - Test Github action to validate the theme against docsbuild scripts. (#69) 
> Contributed by Julien Palard.
> - Add the copy button to pycon3 highlighted code blocks. (#64) Contributed by 
> Julien Palard.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/75SE5KPBFEWXOGOAFEKM6FBHJQ3AORXK/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2TN4YG7WXI4X6VUMO2F3KCNSWBG33AOE/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-Dev] Encouraging developers

[Titus Brown]

On Wed, Mar 07, 2007 at 11:10:22AM +0100, "Martin v. L?wis" wrote:
-> Giovanni Bajo schrieb:
-> > On 06/03/2007 10.52, Martin v. L?wis wrote:
-> > 
-> >> I can't see that the barrier at contributing is high. 
-> > 
-> > I think this says it all. It now appears obvious to me that people 
-> > inside the "mafia" don't even realize there is one. Thus, it looks like 
-> > we are all wasting time in this thread, since they think there is 
-> > nothing to be changed.



Hi, I just wanted to interject -- when I used the word "mafia", I meant
it in this sense:

"""
Informal. A tightly knit group of trusted associates, as of a political
leader: [He] is one of the personal mafia that [the chancellor]
brought with him to Bonn.
"""

(Martin, I certainly didn't intend to offend anyone by implying that
they were part of a criminal organization. ;)

I have a longer explanation of my view in the blog entry, and anyway I
don't want to belabor my own experiences too much, but I would like to
point out three things:

 * there's definitely a group of "trusted associates" -- committers,
   perhaps? -- and it's not at all clear to outsiders like me how new
   features, old patches, and old bugs are dealt with.

 * python-dev is an all-volunteer community.  In true open-source
   fashion, that means that it's incumbent upon people who HAVE
   problems/issues with a process (like me, and Giovanni) to propose
   solutions that either someone wants to implement, or that we can
   implement.  I certainly don't expect any of the committers to put
   tons of effort into a new initiative.

 * Much of the discussion on this issue of encouraging developers comes
   down to communicating better with non-python-dev people.
   
Unless someone is already doing it, I will try to write a summary of the
last few days of discussion and post it for review. The idea would be to
provide a simple one stop wiki page for people who want to contribute.

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Tue, Mar 13, 2007 at 02:16:21PM -0700, Guido van Rossum wrote:
-> On 3/13/07, Georg Brandl <[EMAIL PROTECTED]> wrote:
-> > I'd like to deprecate commands.getstatus() in 2.6.
-> >
-> > Reason: there is getoutput() and getstatusoutput(). In the latter, "status"
-> > means the exit code. getstatus(), however, returns the output of "ls -ld
-> > " which is completely nonobvious and confusing.
-> 
-> +1.
-> 
-> > Perhaps the whole commands module can be deprecated in favor of subprocess.
-> 
-> -1.
-> 
-> Reason (for both voteS): I have learned that the getoutput() and
-> getstatusoutput() functions in the commands module are exceedingly
-> popular amongst Googlers who write simple Python scripts that
-> occasionally invoke an external command. It's much simpler than using
-> os.popen() or the subprocess module (which is also very popular BTW).

What about reimplementing commands.* using subprocess?  Or providing a
commands.*-compatible interface in the subprocess module?

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Tue, Mar 13, 2007 at 04:55:07PM -0700, Guido van Rossum wrote:
-> On 3/13/07, Titus Brown <[EMAIL PROTECTED]> wrote:
-> > What about reimplementing commands.* using subprocess?  Or providing a
-> > commands.*-compatible interface in the subprocess module?
-> 
-> What does that buy us?

The simplicity of the commands interface on top of the more functional
subprocess interface, no?  subprocess is very powerful but there isn't
a simple way to get the output.

See http://docs.python.org/lib/node530.html, and see the docs comparing
os.system() with subprocess,

http://docs.python.org/lib/node537.html

So, if you added 'getstatusoutput' and 'getoutput'-style commands to the
subprocess module, you would

(a) be able to deprecate a module in the stdlib, simplifying it a bit,

and

(b) provide simple commands implementing a common use case for subprocess,
"run this command and give me the output".  (You can already
do 'getstatus' with a 'Popen(cmd).wait()'.)

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

-> > >> What about reimplementing commands.* using subprocess?  Or providing a
-> > >> commands.*-compatible interface in the subprocess module?

OK, so as I understand it, the next step would be for me to provide a
patch implementing this, right? Or is this PEP-required (please no...)?

What do people think of this basic interface?

(status, output) = subprocess.get_status_output(cmd)

output = subprocess.get_output(cmd)

Here 'status' is the 'returncode' from subprocess.Popen, and 'output'
would be the combined stdout/stderr.  'commands.getstatus' would be
removed entirely [0].

This mimics 'commands' fairly closely, while adhering to PEP 8
guidelines; it's a simple API; and it should be dead easy to implement.
It will also have the various advantages people have mentioned:

 * better error trapping;
 * better post-fork behavior;
 * multi-platform support;

If this sort of thing goes in, I guess commands.* would then be
deprecated with a note saying "go look at these similar commands in
subprocess", right?

An additional function that I would personally like is:

(status, output, errout) = subprocess.get_status_output_err(cmd)

although the name is hideous.  I'd like to change 'get_status_output'
to return a triple, but I'm not sure how big a change that would be
relative to the expected behavior from the ancestral commands function.

cheers,
--titus

[0] As per GvR,

http://mail.python.org/pipermail/python-dev/2007-March/071926.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Fri, Mar 16, 2007 at 09:50:29AM -0700, Guido van Rossum wrote:
-> > What do people think of this basic interface?
-> >
-> > (status, output) = subprocess.get_status_output(cmd)
-> >
-> > output = subprocess.get_output(cmd)
-> >
-> > Here 'status' is the 'returncode' from subprocess.Popen, and 'output'
-> > would be the combined stdout/stderr.  'commands.getstatus' would be
-> > removed entirely [0].
-> >
-> > This mimics 'commands' fairly closely, while adhering to PEP 8
-> > guidelines; it's a simple API; and it should be dead easy to implement.
-> 
-> Right. Does it also match the style of the API provided by the
-> subprocess module?

Yes; subprocess has only two "convenience" functions apart from the main
class, in fact, and their names are quite short and simple.

-> > It will also have the various advantages people have mentioned:
-> >
-> >  * better error trapping;
-> >  * better post-fork behavior;
-> >  * multi-platform support;
-> >
-> > If this sort of thing goes in, I guess commands.* would then be
-> > deprecated with a note saying "go look at these similar commands in
-> > subprocess", right?
-> 
-> Yes. Another intermediate step might be to rewrite the commands module
-> to call the new APIs in the subprocess module.

My preference would be to push people towards subprocess, and the more
capable & complex API there, by deprecating commands completely.  I'm
happy to do it either way.

-> > An additional function that I would personally like is:
-> >
-> > (status, output, errout) = subprocess.get_status_output_err(cmd)
-> >
-> > although the name is hideous.  I'd like to change 'get_status_output'
-> > to return a triple, but I'm not sure how big a change that would be
-> > relative to the expected behavior from the ancestral commands function.
-> 
-> Call it get_status_output_errors and you have my blessings. If at all
-> possible, get_status_output should *not* just concatenate the output
-> and errors from this API, but attempt to really *merge* the stdout and
-> stderr stream so that if they are interleaved they come out in the
-> right order. That's what the old getstatusoutput() did by putting
-> '2>&1' around the command.

I believe that this is what

subprocess.Popen(..., stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

does, but I will check to be sure.

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

Hi all,

I posted about adding 'get_output', 'get_status_output', and
'get_status_output_errors' to subprocess here,

http://ivory.idyll.org/blog/mar-07/replacing-commands-with-subprocess

and got some interesting responses.

Briefly, my original proposal was to add these three functions:

  output = get_output(cmd, input=None, cwd=None, env=None)

  (status, output) = get_status_output(cmd, input=None, cwd=None, env=None)
  (status, output, errout) = get_status_output_errors(cmd, input=None,
  cwd=None, env=None)

Commenters convinced me to propose a few additions.  In order of
estimated plausibility,

 * first, all sensical keyword args to subprocess.Popen
   (everything but universal_newlines, stdout, stderr, and bufsize,
   which don't make much sense in the context of the proposed functions)
   should be accepted by the 'get_' functions.

   This complicates the function signatures but does make them
   potentially much more useful.

 * second, I'd like to add a 'require_success' bool keyword, that is
   by default False (and does nothing in that case).  However, when
   True, the functions would emulate check_call, i.e. they would raise
   CalledProcessError when the returncode was not zero.

 * third, the 'popen2' module should be deprecated for 2.6.  I don't see
   that it has anything in it that subprocess doesn't have.

Thoughts?

--titus

p.s. This has been a fun learning process... ;)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Thu, Mar 22, 2007 at 09:34:46PM +, Michael Foord wrote:
-> Guido van Rossum wrote:
-> > Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(),
-> > that abomination invented by Microsoft? I also hear no opposition
-> > against killign os.system() and os.popen()
-> 
-> Except that 'os.system' is really easy to use and I use it rarely enough 
-> that I *always* have to RTFM for subprocess which makes you jump through 
-> a few more (albeit simple) hoops.

Hopefully the patch I'm making will change that, no?

I could add in a 'system'-alike call easily enough; that was suggested.
But I think

returncode = subprocess.call("program")

is pretty simple, isn't it?

-> Additionally, AFAIK subprocess is still broken for py2exe'd applications 
-> which is a problem.

Explain?

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Thu, Mar 22, 2007 at 02:47:58PM -0700, Guido van Rossum wrote:
-> On 3/22/07, Michael Foord <[EMAIL PROTECTED]> wrote:
-> > Guido van Rossum wrote:
-> > > Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(),
-> > > that abomination invented by Microsoft? I also hear no opposition
-> > > against killign os.system() and os.popen()
-> >
-> > Except that 'os.system' is really easy to use and I use it rarely enough
-> > that I *always* have to RTFM for subprocess which makes you jump through
-> > a few more (albeit simple) hoops.
-> 
-> So let's add subprocess.system() which takes care of the hoops (but
-> still allows you more flexibility through optional keyword
-> parameters).

How would this differ from subprocess.call()?

http://docs.python.org/lib/node530.html

(And should I add this in my patch, or is this for Py3k?)

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Thu, Mar 22, 2007 at 11:12:26PM +0100, Andr? Malo wrote:
-> * Titus Brown wrote:
-> 
-> > On Thu, Mar 22, 2007 at 02:47:58PM -0700, Guido van Rossum wrote:
-> > -> On 3/22/07, Michael Foord <[EMAIL PROTECTED]> wrote:
-> > -> > Guido van Rossum wrote:
-> > -> > > Sure. os.fork() and the os.exec*() family can stay. But
-> > os.spawn*(), -> > > that abomination invented by Microsoft? I also hear
-> > no opposition -> > > against killign os.system() and os.popen()
-> > -> >
-> > -> > Except that 'os.system' is really easy to use and I use it rarely
-> > enough -> > that I *always* have to RTFM for subprocess which makes you
-> > jump through -> > a few more (albeit simple) hoops.
-> > ->
-> > -> So let's add subprocess.system() which takes care of the hoops (but
-> > -> still allows you more flexibility through optional keyword
-> > -> parameters).
-> >
-> > How would this differ from subprocess.call()?
-> >
-> >http://docs.python.org/lib/node530.html
-> 
-> It doesn't implement the system() spec:
-> <http://opengroup.org/onlinepubs/007908799/xsh/system.html>
-> 
-> nd

OK, but I'm still confused.  This isn't about moving os.system into
subprocess, it's about reimplementing os.system *with* subprocess.Popen,
right?  And how would that be substantially different from call()?
Different defaults?  (like shell=True, close_fds=False?)

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] deprecate commands.getstatus()

[Titus Brown]

On Fri, Mar 23, 2007 at 10:30:37AM -0600, Steven Bethard wrote:
-> On 3/23/07, Hrvoje Nik??i?? <[EMAIL PROTECTED]> wrote:
-> > On Thu, 2007-03-22 at 13:38 -0700, Guido van Rossum wrote:
-> > > Sounds good to me. In 3.0 we should probably not have os.popen*(), nor
-> > > the popen2 module at all, and do everything via the subprocess module.
-> > > I wonder if we should even get rid of os.system(); then there should
-> > > be a subprocess.system() instead. And do we even need os.fork(),
-> > > os.exec*(), os.spawn*()?
-> >
-> > Please don't remove os.fork and os.exec*.  Some people need to fine-tune
-> > process creation and don't need portability to non-Unix OS'es.  For
-> > them, the functions that call the underlying system API and little or
-> > nothing else are a god-send.
-> 
-> Right, but if you're really using only Posix, you can simply use
-> ``posix.fork`` and ``posix.exec*`` and then you're even being explicit
-> about the fact.

Yes, but:

http://docs.python.org/lib/module-posix.html

"""Do not import this module directly."""

Unless people want me to try to extract something coherent from the
recent discussion, I'm going to avoid doing anything with os.*
functions.  That can be done separately from the contemplated subprocess
patch, anyway.

The whole thread is here:

http://www.gossamer-threads.com/lists/engine?post=553519;list=python

and I will finish up a patch to do this:

http://www.gossamer-threads.com/lists/python/dev/555743#555743

(add get_*output* functions to subprocess, modify docs appropriately,
add 'require_success', and put in a docs deprecation for
popen2/commands).

I'll post again when I have a patch ready so there's something concrete
to complain about ;).

thanks,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Titus Brown]

On Tue, May 22, 2007 at 07:32:42AM +0200, "Martin v. L?wis" wrote:
-> > Addressing only the issues of PCBuild8 and 64-bit architectures, I
-> > have tried to establish "free" buildbot support for the 64-bit
-> > architectures without any real success.
-> > 
-> > Should the PSF be considering paying for infrastructure that will 
-> > support 64-bit build reporting?
-> 
-> You can bring it up to the board, of course. However, given that
-> all other buildbot machines were contributed by volunteers, the
-> fact that nobody volunteers to contribute buildbot machines for
-> PCbuild8 indicates that there is not a lot of interest in that
-> build infrastructure.
-> 
-> Note that there *are* 64-bit buildbot slaves, e.g. for AMD64
-> (contributed by Neal Norwitz), Alpha (contributed by Neal
-> Norwitz), Itanium (contributed by Matthias Klose, offering
-> a machine from the Debian project), and PPC64 (likewise).
-> 
-> These machines all run Linux or Tru64, and (to my understanding)
-> serve other purposes as well, making the buildbot slave
-> just a minor detail.
-> 
-> It's an unfortunate fact of life that Microsoft Windows does not
-> support multi-tasking multi-user workloads that well, so the
-> Windows buildbot slave are (to my knowledge) typically dedicated
-> machines (except for Tim's machine, which is just disconnected
-> from the master when he doesn't feel like running buildbot
-> jobs).

I haven't really been listening to this conversation, so forgive me if
this isn't relevant, but:

I'd be happy to install Windows and the latest VisualStudio on a 64-bit
VMware image.  I just can't be responsible for day-to-day administration
of the buildslave; Windows requires too much attention for me to do
that.

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The docs, reloaded

[Titus Brown]

On Tue, May 22, 2007 at 11:45:04AM -0500, [EMAIL PROTECTED] wrote:
-> 
-> >> You at least take away a common excuse for lack of contributions.
-> >> True whiners will just come up with new ones (e.g., "the
-> >> documentation isn't available in Sanskrit yet" or "the dog ate my
-> >> changes before I could type them into the computer"). ;-)
-> 
-> Steve> But doesn't *everyone* now know that documentation contributions
-> Steve> don't have to be marked up? It's certainly been said
-> Steve> enough.
-> 
-> Sure, but that doesn't stop the true whiners. ;-)

Nothing stops the true whiners ;).

I think new and exciting ways of viewing, searching, annotating, linking
to/from, and indexing the docs are more important than new formats for
(not) writing the docs.

For example, this rocks! ::

http://pydoc.gbrandl.de/search.html?q=os.path&area=default

There have been (good) efforts to wikify the docs in the past. IMO what
would make them really work would be to have docs.python.org, the
"official" Python docs location, start hosting these efforts.  As long
as that location is static, I think the majority of users will ignore
other efforts.

cheers,
--titus

p.s. Are there good directions for installing the toolchain for current
docs building anywhere?  I've tried once or twice, but despite a lot of
LaTeX experience I could never get everything hooked up right.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Windows buildbot (Was: buildbot failure in x86 W2k trunk)

[Titus Brown]

On Wed, May 23, 2007 at 12:08:52PM -0700, Trent Mick wrote:
-> Thomas Heller wrote:
-> >> Are there others that can provide a Windows buildbot? It would probably 
-> >> be good to have two -- and a WinXP one would be good.
-> > 
-> > How much work is it to set one up, and to maintain it?  Maybe I can offer 
an XP VMWare image.
-> 
-> It has been a while since I set it up. Tim did so at about the same time 
-> and wrote down his steps to setup... but I can't find the reference to 
-> those instructions right now.
-> 
-> I've found maintenance to be low -- just have to have to start a cmd 
-> shell and run the buildbot slave command. However, that may be because 
-> the box on which it is running isn't one I use regularly, so I don't 
-> have to worry about accidentally killing the process, frequent reboots 
-> or anything like that.
-> 
-> I'll try to dig around and see what I can find for setup instructions.

It's mildly tricky to install, but very easy to set up a slave process;
I have several.

I can offer an image, but what kills me is the maintenance.  It's rarely
a big deal -- reboot after some updates, install some startup scripts,
etc. -- but when it *does* require an effort I hate it off because I dislike
Windows so intensely ;).  Not a good reason, I know, but...

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python developers are in demand

[Titus Brown]

On Thu, Oct 25, 2007 at 07:59:58AM -0700, Anna Ravenscroft wrote:
-> I noticed at the Grace Hopper Celebration of Women in Computing that
-> several major universities in the US are starting to offer intro (CS1)
-> courses based on Python, among them:
-> Georgia Tech
-> CMU
-> Bryn Mawr

It's been adopted at Michigan State U. this past year, and I'll be
teaching a Web dev followup course *next* year.

Python is also being used for bioinformatics at Caltech (not just me)
and at Michigan State (with no connection to the intro course).

The SciPy conferences have been eye opening as well: adoption here,
there, and everywhere.

Good to finally see this happening ;)

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_doctest failing

[Titus Brown]

On Fri, Nov 23, 2007 at 10:32:25PM -0600, [EMAIL PROTECTED] wrote:
-> 
-> Brett> Looks like Skip's r59137 fix for working with tracing has led to
-> Brett> test_doctest to be broken on 2.5 and the trunk (at least
-> Brett> according to the buildbots).  Can someone either revert the
-> Brett> commit or fix it?  I would normally do it myself but US
-> Brett> Thanksgiving has me tied up to not do too much beyond bitching
-> Brett> about the amount of buildbot emails I have been getting.  =)
-> 
-> Sorry.  I did verify that it solved the problem I was having with trace &
-> doctest, but I completely forgot to make test after applying that patch.
-> The checkins have been reverted.

Skip, this set_trace rewrite fixes the problem in both 25-maint and
trunk:

def set_trace(self, frame=None):
self.__debugger_used = True
if frame is None:
frame = sys._getframe().f_back
pdb.Pdb.set_trace(self, frame)

from the old:

def set_trace(self):
self.__debugger_used = True
pdb.Pdb.set_trace(self)

This was an actual bug: the patch didn't correctly allow for the fact
that Bdb.set_trace(self, None) went and got the calling frame -- which,
when the function was overriden in a subclass, was the subclass
function.

Let me know if you want a formal patch or somethin'.  And thanks for
tackling the doctest/trace situation - some users have been complaining
about it to me and it was on my list for tomorrow!

Happy post-Thanksgiving, all!

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_doctest failing

[Titus Brown]

On Sat, Nov 24, 2007 at 10:23:06AM -0800, Guido van Rossum wrote:
-> On Nov 24, 2007 6:35 AM,  <[EMAIL PROTECTED]> wrote:
-> > Thanks, Titus.  Both the doctest and trace tests pass with your change.
-> > Checked back in.  I didn't run the full test suite, as test_sqlite causes a
-> > bus error on my Mac which I've yet to investigate.
-> 
-> It's easy though to run all tests *except* a known failure:
-> .../runtest.py -x test_sqline

Just to make sure ... do you mean regrtest?

./python Lib/test/regrtest.py -x test_sqlite

?

(There's no 'runtest.py' in the trunk, and 'runtests.py' exists under
Lib/ctypes/.)

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fwd: Google Highly Open Participation Contest

[Titus Brown]

Dear Python-Dev-ers,

about half of the tasks for the GHOP contest (described below) are
currently on the stdlib, Python core, or Py3K:


http://code.google.com/p/google-highly-open-participation-psf/issues/list?q=label:stdlib


http://code.google.com/p/google-highly-open-participation-psf/issues/list?q=label:core


http://code.google.com/p/google-highly-open-participation-psf/issues/list?q=label:py3k

We have room for another twenty or so; if you can come up with testing
tasks, documentation updates/review needs, or anything else that you can
write down in a fairly detailed and specific way, we'd love to have
them.

Here are the new task guidelines:

http://code.google.com/p/google-highly-open-participation-psf/wiki/NewTaskGuidelines

and I'm happy to write up the tasks if people send me good ideas.

cheers,
--titus

- Forwarded message from Titus Brown <[EMAIL PROTECTED]> -----

From: Titus Brown <[EMAIL PROTECTED]>
Subject: Google Highly Open Participation Contest

Dear Python community,

I'm happy to announce that the Python Software Foundation is part of a
new Google Open Source program, the Highly Open Participation Contest.
This contest is an effort by Google to engage pre-college students in
open source programming: Google is offering prizes and awards for
completing a variety of tasks for several organizations.

You can read the official announcement of the PSF's involvement over on
the Python Software Foundation blog, at

http://pyfound.blogspot.com/

or read about the overall Google Highly Open Participation Contest on
the GHOP page itself,

http://code.google.com/opensource/ghop/2007-8

Python's project page is:

http://code.google.com/p/google-highly-open-participation-psf/

There are several ways that the community can help; please check out the
MentorPage,

  http://code.google.com/p/google-highly-open-participation-psf/wiki/MentorPage

for more information.  We could use more mentors, and there's room for
~40 more tasks -- why not write one up on your own project?

Also be sure to thank the people who made this possible,

 http://code.google.com/p/google-highly-open-participation-psf/wiki/Contributors

and, of course, Google!

best,
--titus

- End forwarded message -
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Memory benchmarking?

[Titus Brown]

Hi all,

is there a good, or standard memory benchmarking system for Python?
pybench doesn't return significantly different results when Python 2.6
is compiled with pymalloc and without pymalloc.  Thinking on it, I'm not
too surprised -- pybench probably benchmarks a lot of stuff -- but some
guidance on how/whether to benchmark different memory allocation schemes
would be welcome.

thanks,
--titus

refs:

http://code.google.com/p/google-highly-open-participation-psf/issues/detail?id=105&colspec=ID%20Status%20Summary

http://evanjones.ca/memoryallocator/

http://www.advogato.org/person/wingo/diary/225.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Split up the c-api documentation in smaller files?

[Titus Brown]

On Fri, Dec 07, 2007 at 07:24:04AM -0500, Steve Holden wrote:
-> Christian Heimes wrote:
-> > Good afternoon everybody!
-> > 
-> > The new C API documentation contains some large files:
-> > 
-> > 105K abstract.html
-> > 300K concrete.html
-> > 183K newtypes.html
-> > 
-> > The concrete.html takes noticeable time to render on my computer (P4 2.4
-> > with 1GB RAM, Firefox 2.0 and Ubuntu Linux). I estimate the load and
-> > rendering time is about 4-5 seconds! It also takes long to search for a
-> > string and scrolling isn't smooth, too.
-> > 
-> > Can we split the files in smaller chunks, please?
-> > 
-> This might make a good GHOP project, though as Titus has just uploaded 
-> about seventy (70!) new projects I am unsure as to how long it would 
-> have to wait ti get in the queue.

At the current rate, approximately 10 days.

;)

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Spurious Buildbot Reports

[Titus Brown]

On Wed, Dec 19, 2007 at 05:58:35PM -0800, Brett Cannon wrote:
-> On Dec 19, 2007 4:33 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
-> > The bots are kicking-off so many false alarms that it is becoming 
difficult to tell whether a check-in genuinely broke a build.
-> >
-> > At the root of the problem is a number of tests in the test suite that 
randomly blow-up.  I now tend to automatically dismiss failures in test_logging 
and test_threading for example.
-> 
-> Yeah, certain tests need some TLC to make them more predictable and
-> less prone to throw a failure because of some touch race condition or
-> something on the machine was not available to make the test work.
-> 
-> As I have stated on my blog, once I am done with importlib and the
-> stdlib reorg I plan on working on dev docs and then attack the whole
-> structure of the unit tests.
-> 
-> But who knows when that will happen.  =)

OK, but this isn't really a structural problem, right? This is
non-determinism in certain tests ;)

How broken are these tests?  Can we point a 17-yr-old at them and tell
them to fix 'em (yes think "GHOP")?  (If the problem is reproducible on a
1-in-10 basis then I think the answer is "yes".)

And are test_threading and test_logging the two that need the most
work?

Hmm, perhaps a good starting task would be to run the tests 10-100
times, in random order, on a single machine, to get a statistical
picture of of the problem.

cheers,
--titus
(always on the lookout for core => GHOP tasks)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Converting tests to unittest/doctest?

[Titus Brown]

Hi all,

a bit of grep'ping and personal examination discovered the following
tests in trunk/ that could be converted to unittest or doctests.  Any
thoughts, pro or con?

(I understand from Brett that the goal is to eradicate "old-style"
tests, by which I think he means tests that do not use unittest or
doctest.  There are some good reasons to switch to using unittests, not
least of which is that you can use a variety of frameworks (nose,
py.test) to do value-added wrapping and management of such tests.)

Suggestions for additional things to test or tests to modify, clean up,
or extend and expand are welcome.

thanks,
--titus

---

test_select
test_contains
test_crypt
test_dbm
test_dummy_threading
test_errno
test_getargs
test_gdbm
test_pep247
test_strftime

test_thread

test_queue

test_fcntl

test_format

test_curses

test_descr

test_funcattrs

test_gdbm

test_socketserver
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Converting tests to unittest/doctest?

[Titus Brown]

On Fri, Dec 21, 2007 at 04:02:21PM +0100, Quentin Gallet-Gilles wrote:
-> (oops, realized I didn't send it to the list, just to Titus)
-> 
-> I remember that it was one of the tasks at the Python Sprint at Google last
-> summer, so I guess this is a good idea (for GHOP, right ?)

Yep!

-> >From what remains of the spreadsheet used during the Sprint
-> (http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA&pli=1),
-> I believe you can add the following tests to your list:
-> 
-> test_tokenize
-> test_cProfile
-> test_extcall
-> test_logging
-> test_profile
-> test_thread
-> (and maybe test_pep277 that seems to use both unittest and test.test_support
-> )

These were already done as part of GHOP, just not all checked in yet --
we're waiting for some contributor agreements.  But thanks, that's
exactly the sort of info I'm looking for!

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Updating descrintro

[Titus Brown]

Hi all,

re

http://www.python.org/download/releases/2.2.3/descrintro/

on "Unifying types and classes in Python 2.2",

we have a GHOP task to "fill in" the Additional Topics section of this
document.  'novanasa', the student who took this task, has written up a
nice set of doctest additions in reST:

http://code.google.com/p/google-highly-open-participation-psf/issues/attachment?aid=-5215187968540901644&name=types-classes.rst

and has asked for feedback.  My feedback at the moment is, "looks good,
and useful!"  I thought I'd ask on python-dev to see if anyone has any
additional comments or ways to make this more directly useful to Python.

You can just add comments on to the task directly, if you like, or send
them to me.

One set of possible additions is to go through and make annotations on
stuff that is going to change in Py3K.  Anything else?

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributing to Python

[Titus Brown]

On Thu, Jan 03, 2008 at 11:15:04AM -0800, Guido van Rossum wrote:
-> We're getting a fair number of doc contributions, especially since the
-> docs were converted from Latex to ReST, and especially since the start
-> of the GHOP project.
-> 
-> My main gripe is with code contributions to Py3k and 2.6; Py3k is
-> mostly done by a handful of people, and almost nobody is working much
-> on 2.6.

What needs to be done with 2.6?  I'm happy to review patches, although
even were commit access on offer I'm too scatterbrained to do a good job
of it.

Incidentally, I'm planning to set up an SVK repos containing the GHOP
doc patches; that way they can stay sync'ed with 2.6 work.  I'd be happy
to do the same thing with reviewed-and-probably-OK patches, although I
don't know if repository proliferation is a generally good idea ;).

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributing to Python

[Titus Brown]

On Thu, Jan 03, 2008 at 05:48:25PM -0200, Facundo Batista wrote:
-> 2008/1/3, Titus Brown <[EMAIL PROTECTED]>:
-> 
-> > What needs to be done with 2.6?  I'm happy to review patches, although
-> > even were commit access on offer I'm too scatterbrained to do a good job
-> > of it.
-> 
-> We have 109 patches open for 2.5 [1], and 118 patches open for 2.6 [2].

The question is, is reviewing patches a good place to contribute?  Also,
if I (and others) could have a "core mentor" with commit access, that
might streamline things.  As it is, I am worried that patch reviews will
pass through the ether without leaving a visible trace; that's OK and
understandable, but it's demotivating.

(I've had a very good experience with Georg and documentation commits
for GHOP.)

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributing to Python

[Titus Brown]

On Thu, Jan 03, 2008 at 02:49:27PM -0500, Fred Drake wrote:
-> On Jan 3, 2008, at 2:15 PM, Guido van Rossum wrote:
-> > My main gripe is with code contributions to Py3k and 2.6; Py3k is
-> > mostly done by a handful of people, and almost nobody is working much
-> > on 2.6.
-> 
-> For those of us still using Python 2.4 and earlier, it's hard to be  
-> motivated to worry about Python 3.0, no matter how wonderful it  
-> looks.  (It doesn't help that my own available time appears to  
-> decrease daily with the kids and all.)
-> 
-> Python 2.6 seems to be entirely targeted at people who really want to  
-> be on Python 3, but have code that will need to be ported.  I  
-> certainly don't view it as interesting in it's own right.

3k and 26 are, however, the only place where we can propose new features
-- which makes it the place for cleanup and additional testing, as well
as backwards-incompatible bug fixes...

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributing to Python

[Titus Brown]

-> > Incidentally, I'm planning to set up an SVK repos containing the GHOP
-> > doc patches; that way they can stay sync'ed with 2.6 work.  I'd be happy
-> > to do the same thing with reviewed-and-probably-OK patches, although I
-> > don't know if repository proliferation is a generally good idea ;).
-> 
-> IMO patches should just be applied to the trunk ASAP.

(We're waiting on contributor forms for the GHOP students.  That's on my
TODO list, too.)

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributing to Python

[Titus Brown]

On Thu, Jan 03, 2008 at 09:55:44PM +0100, Christian Heimes wrote:
-> Jeroen Ruigrok van der Werven wrote:
-> > You don't put the bar high for newbies on the Python project eh? :)
-> > 
-> > I am assumign that most of those contributions code-wise need a fair 
amount of
-> > knowledge of Python's internals?
-> 
-> It's neither impossible nor too hard to get involved. I got from
-> "haven't done serious C coding in years" to "Python core developer with
-> full svn access" in less than 9 months. OK, I've more than 5 years of
-> Python experience but you don't need it to contribute.
-> 
-> You can start by updating or enhancing the existing documentation,
-> writing new docs and tutorials or updating the unit test suite. New
-> tests need to be written and existing test should be ported to the new
-> unit test module.

As far as I am aware, we/GHOP have patches for all of the tests in 2.6
=> unittest or doctest.  They have not all been applied yet, but that
will come soon.

Speaking as someone with a few years experience in Python and quite a
bit of C knowledge, it is not at all easy to come up with a way to dive
into the core code.  For GHOP, it was a bit of a struggle to come up
with tasks; I ended up with three main sources of tasks:

 - a blog post asking people about their favorite poorly documented
   module;

 - the 3.0 spreadsheet and some grepping, to figure out which tests
   still needed to be ported over to unittest/doctest;

 - a test coverage analysis to figure out which modules were largely
   untested;

A parent of a GHOP student asked us if we could somehow come up with
a suggested task list for new contributors (not just GHOP students).
While this could be a lot of work, I think the contributions from GHOP
to core in the areas of test coverage, test porting, and module
documentation indicate that this kind of outreach may be very effective.
If you have ideas on how to keep such a task list fresh and up-to-date
as well as populated with good, simple tasks, I'm interested in hearing
from you!

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributing to Python

[Titus Brown]

On Thu, Jan 03, 2008 at 03:24:16PM -0500, Joseph Armbruster wrote:
-> Having a "core mentor" would be great but do they really have time for
-> that?  I've been lucky at finding people in #python / #python-dev) that can
-> answer development inquiries (or at least verify something is or is not a
-> bug).

Again, IMO as someone on the lunatic fringe of core development (i.e.
I'm a happy spectator, but I'm too busy to actually get much done):

Mentoring coders may not be a traditional route for hard-core OSS
developers, but it sure can be effective, as I've found with GHOP.

For example, many core Python developers can save an outsider hours of
effort by simply and quickly outlining the issues involved in a
particular patch or coding effort.  Having actual committers involved is
especially good, because they can evaluate whether or not a patch is
likely to be accepted, potentially cutting out more hours of effort; and
they can directly commit patches, leading to the very important
gratification of an actual commit.

>From another angle, there are a lot of "easy" fixes/patches/updates to
be done to Python, but I'll be damned if I can figure out which ones are
easy meat, or complex, or likely to touch a nerve.  Having someone
experienced to quickly give an opinion is invaluable.  (I'm an
overconfident loudmouth, so I don't mind posting to this list, but I
think python-dev is pretty intimidating for people new to the hurly
burly of OSS development.)

As I've said in other responses in this thread, I'm not sure how to make
it happen, but I'm leaning towards asking the active GHOP mentors to try
to extend the GHOP mentoring effort into a general python-love effort.
We've got a good group of experienced people, and it's been a pretty
friendly list IMO.

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57

[Titus Brown]

On Thu, Jan 17, 2008 at 06:12:52AM +0900, Stephen J. Turnbull wrote:
-> Bill Janssen writes:
->  > > >> ~/Library/ is a Mac OS X thing.
->  > > 
->  > > Bill> Sure, but it's clearly where this should be on an OS X 
system, by
->  > > Bill> default.  
-> 
->  > > [etc.]
-> 
-> > [tocatta and fugue ad lib]
-> 
-> Doesn't Apple publish standards for this?  They do for everything
-> else, it seems.

Yep -- finally found it:

http://www.codinghorror.com/blog/archives/001032.html

http://developer.apple.com/documentation/MacOSX/Conceptual/BPFileSystem/Articles/WhereToPutFiles.html

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Patches: 1 for the price of 10.

[Titus Brown]

Hello all,

per previous discussion,

http://mail.python.org/pipermail/python-dev/2004-October/049495.html

I'd like to push a trivial little patch to sgmllib (#1087808) on you
gents, in exchange for my reviews & effort etc. on 10 other patches.

Without further ado:

No-brainers:

1055159 -- a docstring+docs update to CGIHTTPServer describing already-
existing behavior.  Recommend apply.

1037974 -- fix HTTP digest auth for broken servers, e.g. LiveJournal.
Trivial code fix, should break nothing.  Recommend apply +
backport.

1028908 -- JJ Lee's updates to urllib2.  Passes regr tests, by an
original author of much of the code (I think).  Recommend apply.

901480 -- patch to urllib2.parse_http_list (bug 735248).  Works.
Updated patch.  Recommend apply + backport.

827559 -- SimpleHTTPServer redirects to 'dir/' when asked for 'dir'.
This behavior mimics common behavior online and fixes a problem
with relative URLs when clicking on files within 'dir'.
Recommend apply.

810023 -- fixes off-by-one bug in urllib reporthook.  regr tests & all.
Good stuff.  Recommend apply + backport.

893642 -- adds allow_none option to SimpleXMLRPCServer & associated
classes.  Doesn't change default behavior.  Recommend apply.

755670 -- modify HTMLParser to accept clearly broken HTML.
Recommend reject.

Slightly more complicated:

1067760 -- float-->long conversion on fileobj.seek calls, rather than
float-->int.  Permits larger floats (2.0**62) to match large
int (2**62) arguments.  rhettinger marked as "won't fix" in
the original bug report; this seems like a clean solution,
tho.  Recommend apply.

755660 -- should HTMLParser fail on all bad input, or do best effort?
I'd recommend more sweeping changes where must-fail situations
are distinguished from fails-by-default situations.
Alternatively take a stand and say "nein!" once and for all.
(See my comment for more information.)

--

For no particularly good reason, all of these were tested against
the current CVS HEAD rather than 2.4.  All of them should be trivial
to backport, although I think only a few are real problems worthy
of the effort.

--

I'm kind of curious to see how this goes, I must admit ;).  Please CC
me on replies so I can listen in...

One comment to Martin: it clearly isn't worth the effort of reviewing 10
patches to push a patch the size of my sgmllib patch.  On the other
hand, it's nice to have a guarantee & it's an educational experience,
that's for sure.

A 5:1 ratio might be more reasonable, since that in practice will
mean 1 serious patch, 2 or 3 updates, and 1 drop-dead easy patch.

cheers,
--titus
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patches: 1 for the price of 10.

[Titus Brown]

-> Jeremy Hylton wrote:
-> >I got started on these this morning, will likely finish them tomorrow.
-> > It would be perverse to apply your patch last, wouldn't it?
-> 
-> It turns out that Titus' patch might be more involved than he thought
-> it would be.

*shrug* that's life ;).  I stole my patch from the other HTMLParser &
thought that would be sufficient; now I'll have to fix both!

-> In any case, the review itself is a highly appreciated contribution.

It was very educational; just wish I could remember to always submit
context diffs!  

The only patch that I think deserves some actual discussion - here or on
c.l.p, not sure which -- is patch 755660, which deals with
HTMLParser.HTMLParser.  The goal of the original submitter was to allow
subclasses of HTMLParser deal with bad HTML in a more robust way;
essentially this comes down to allowing returns from self.error() calls.

I have now come across the same problem in my work with PBP
(pbp.berlios.de): it turns out that many Web pages (such as the
SourceForge mailman admindb page...) contain errors that cause
HTMLParser to raise an exception.  It's simply not possible to reliably
change this behavior within either htmllib.HTMLParser or
HTMLParser.HTMLParser as they're currently written.  This is a big
problem for people basing packages on either HTMLParser class.

An additional problem is that both HTMLParser.HTMLParser and
htmllib.HTMLParser are based on other classes that call self.error(), so 
those base classes would have to altered to fit the new behavior.

What I proposed doing in my comment on patch 755660 was changing
HTMLParser.HTMLParser (and its base class markupbase, too) to call
_fail() when a hard failure was called for, and otherwise to call
error() and proceed parsing on an as-best-possible basis.  This wouldn't
change the *behavior* of the existing code, but would allow for it to be
overridden when necessary.

Right now the error() call is undocumented and so it's probably
ok to change what happens upon return.  As is it can leave the parser
in a borked state upon return, and that's the behavior I propose to
fix.

I'd of course be willing to do the work & submit the more involved
patch.

Your opinions are not only welcome but (as I understand it) necessary ;).

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patches: 1 for the price of 10.

[Titus Brown]

-> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than
-> >float-->int.  Permits larger floats (2.0**62) to match large
-> >int (2**62) arguments.  rhettinger marked as "won't fix" in
-> >the original bug report; this seems like a clean solution,
-> >tho.  Recommend apply.
-> 
-> Wouldn't this cause subtle errors when the float -> long conversion is
-> no longer precise? Or is this a non issue because it could only happen
-> when seeking on impossibly large files?

When would the float --> long conversion not be precise?  The docs
say PyLong_FromDouble takes the integer part, so it's like doing
a floor(), yes?

The current behavior is to convert directly from float to int, even
though seek() can take longs as an argument.  Thus 2.0**31 works,
2**31 works, 2**62 works, but 2.0**62 doesn't.  This seems mildly
inconsistent and prompted the guy to submit a bug report, and then
a patch.

The patch (with a bit more code context) is:

--
 if (!PyArg_ParseTuple(args, "O|i:seek", &offobj, &whence))
  return NULL;

 #if !defined(HAVE_LARGEFILE_SUPPORT)
offset = PyInt_AsLong(offobj);
  #else
+   if(PyFloat_Check(offobj)) {
+ offobj = PyLong_FromDouble(PyFloat_AsDouble(offobj));
+   }
offset = PyLong_Check(offobj) ?
PyLong_AsLongLong(offobj) : PyInt_AsLong(offobj);
  #endif
--

so the issue only comes into play with large file support.

Heh, and I just noticed that PyLong_FromDouble returns a new reference,
so this is a memory leak, I believe.  Whps.  I'll fix that.

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patches: 1 for the price of 10.

[Titus Brown]

-> > Apparently file.seek doesn't have this DeprecationWarning though..
-> > Strange, that.
-> >  >>> f.seek(3.6)
-> >  >>> f.tell()
-> > 3L
-> 
-> That's a bug. Who'll fix it?

Added:

+ if (PyFloat_Check(offobj))
+ PyErr_Warn(PyExc_DeprecationWarning,
+"integer argument expected, got float" );

see attached diff.  I also attached it to the patch at SourceForge,
with a comment, just to keep the record straight.

The DeprecationWarning wasn't given because a manual PyObject -->
int/long conversion was being done, rather than the usual conversion
implicit in ParseTuple.

Regression tests run w/o problem on RH 9.0/i386.  Now you get the
correct behavior:

--
>>> f = open('/dev/zero')
>>> f.seek(5)
>>> f.tell()
5L
>>> f.seek(5.2)
__main__:1: DeprecationWarning: integer argument expected, got float
>>> f.tell()
5L
--

This seems like a reasonable resolution to patch #1067760, to me...

cheers,
--titus
? Objects/.fileobject.c.swp
Index: Objects/fileobject.c
===
RCS file: /cvsroot/python/python/dist/src/Objects/fileobject.c,v
retrieving revision 2.193
diff -c -c -r2.193 fileobject.c
*** Objects/fileobject.c7 Nov 2004 14:15:28 -   2.193
--- Objects/fileobject.c23 Dec 2004 08:19:20 -
***
*** 462,467 
--- 462,472 
whence = 0;
if (!PyArg_ParseTuple(args, "O|i:seek", &offobj, &whence))
return NULL;
+ 
+ if (PyFloat_Check(offobj))
+ PyErr_Warn(PyExc_DeprecationWarning,
+"integer argument expected, got float" );
+ 
  #if !defined(HAVE_LARGEFILE_SUPPORT)
offset = PyInt_AsLong(offobj);
  #else
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Code coverage reporting.

[Titus Brown]

Folks,

I've just run a code coverage report for the python2.4 branch:

http://vallista.idyll.org/~t/temp/python2.4-svn/

This report uses my figleaf code,

http://darcs.idyll.org/~t/projects/figleaf-latest.tar.gz

I'm interested in feedback on a few things --

 * what more would you want to see in this report?

 * is there anything obviously wrong about the report?

In other words... comments solicited ;).

By the by, I'm also planning to integrate this into buildbot on some
projects.  I'll post the scripts when I get there, and I'd be happy
to help Python itself set it up, of course.

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Code coverage reporting.

[Titus Brown]

On Mon, Jun 19, 2006 at 02:21:04PM +1000, Nick Coghlan wrote:
-> Brett Cannon wrote:
-> >But it does seem accurate; random checking of some modules that got high 
-> >but not perfect covereage all seem to be instances where dependency 
-> >injection would be required to get the tests to work since they were 
-> >based on platform-specific things.
-> 
-> There's something odd going on with __future__.py, though. The module level 
-> code all shows up as not executed, but the bodies of the two _Feature 
-> methods both show up as being run.
-> 
-> I'm curious as to how a function body can be executed without executing the 
-> function definition first :)

Coverage recording probably wasn't on at the time the module was
imported; I only turn on recording in the 'runtest' function, and
not before.  I should probably start it earlier ;)

-> As far as making the comments/docstrings less obvious goes, grey is usually 
-> a good option for that.

I'll try it out and see...

thanks,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Code coverage reporting.

[Titus Brown]

On Mon, Jun 19, 2006 at 08:37:30AM -0400, Benji York wrote:
-> Brett Cannon wrote:
-> >But it does seem accurate; random checking of some modules that got high 
-> >but not perfect covereage all seem to be instances where dependency 
-> >injection would be required to get the tests to work since they were 
-> >based on platform-specific things.
-> 
-> >I don't know if we need it hooked into the buildbots (unless it is dirt 
-> >cheap to generate the report).
-> 
-> It would be interesting to combine the coverage over several platforms 
-> and report that.

Yes, I noticed that the platform specific stuff doesn't get covered, of
course.  It's very easy to do, *if* there's any way to get the coverage
database from a central location (or send it back to a central location).

It might be interesting to run coverage analysis -- either figleaf or
Ned Batchelder's module[0] -- once a week on select buildbot machines
(one linux, one windows, one mac, or some such) and make the coverage
databases available via something like a downloadable static file.  Then
anyone could download those files and do Interesting Things with them.

--titus

[0] I'm sorry, I don't know how Walter Dorwald generates his coverage;
if it's OSS, then it'd be better to use because it shows C code coverage
as well.

p.s. Here's the diff for regr_test:

Index: Lib/test/regrtest.py
===
--- Lib/test/regrtest.py(revision 46972)
+++ Lib/test/regrtest.py(working copy)
@@ -1,4 +1,5 @@
 #! /usr/bin/env python
+import figleaf

 """Regression test.

@@ -333,7 +334,11 @@
 tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
   globals=globals(), locals=vars())
 else:
+figleaf.start(False)
 ok = runtest(test, generate, verbose, quiet, testdir, huntrleaks)
+figleaf.stop()
+figleaf.write_coverage('.figleaf')
+
 if ok > 0:
 good.append(test)
 elif ok == 0:
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Code coverage reporting.

[Titus Brown]

On Sun, Jun 18, 2006 at 08:12:39PM -0700, Brett Cannon wrote:
-> On 6/15/06, Titus Brown <[EMAIL PROTECTED]> wrote:
-> >
-> >Folks,
-> >
-> >I've just run a code coverage report for the python2.4 branch:
-> >
-> >http://vallista.idyll.org/~t/temp/python2.4-svn/
-> >
-> >This report uses my figleaf code,
-> >
-> >http://darcs.idyll.org/~t/projects/figleaf-latest.tar.gz
-> 
-> 
-> Very nice, Titus!
-> 
-> I'm interested in feedback on a few things --
-> >
-> >* what more would you want to see in this report?
-> >
-> >* is there anything obviously wrong about the report?
-> >
-> >In other words... comments solicited ;).
-> 
-> Making the comments in the code stand out less (i.e., not black) might be
-> handy since my eye still gets drawn to the comments a lot.

I think I'd have to use the tokenizer to do this, no?  The comments
aren't kept in the AST, and I don't want to write a half-arsed regexp
because I'm sure I'll stumble on comments in strings etc ;)

-> It would also be nice to be able to sort on different things, such as
-> filename.

Easy enough; just the index needs to be generated in multiple ways.

-> But it does seem accurate; random checking of some modules that got high but
-> not perfect covereage all seem to be instances where dependency injection
-> would be required to get the tests to work since they were based on
-> platform-specific things.

Great!

-> By the by, I'm also planning to integrate this into buildbot on some
-> >projects.  I'll post the scripts when I get there, and I'd be happy
-> >to help Python itself set it up, of course.
-> 
-> 
-> I don't know if we need it hooked into the buildbots (unless it is dirt
-> cheap to generate the report).  But hooking it up to the script in
-> Misc/build.sh that Neal has running to report reference leaks and
-> fundamental test failures would be wonderful.

Hmm, ok, I'll take a look.

The general cost is a ~2x slowdown for running with trace enabled, and the
HTML generation itself takes less than 5 minutes (all of that in AST
parsing/traversing to figure out what lines *should* be looked at).

cheers,
--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: Annoying user on GitHub

[C. Titus Brown]

> On Jul 2, 2019, at 9:09 AM, Steve Dower  wrote:
> 
> On 02Jul2019 0840, Mariatta wrote:
>> I've used the "Report abuse" feature on GitHub for such situations. Most of 
>> the time I see the user suspended, and the associated comments deleted.
>> Our GitHub admins can delete comments too.
>> On Tue, Jul 2, 2019, 1:42 AM Victor Stinner > > wrote:
>>Hi,
>>Sometimes, I get an email notification about strange comments
>>(unrelated
>>or make no sense) on commits made 6 months ago if not longer.
>>Usually, I
>>go to the user profile page and I click on "Block or report user":
>>"Report abuse". I'm not sure what happens in this case. I never checks
>>if these strange comments are removed by GitHub.
> 
> Maybe there's also a way to automatically lock conversations on commits and 
> old issues?
> 
> Obviously we can lock them manually, but simply disallowing conversation in 
> places where we don't want to have to pay attention to will force people 
> towards the places where we are paying attention.

Hi Steve et al.,

I think this is eminently do-able via the GitHub API. Happy to put together a 
script if people are interested. Is there a repo being used for github magic?

(I did a lot of this on a recent project - 
http://ivory.idyll.org/blog/2019-github-project-reporting.html - the PyGithub 
package is excellent.)

best,
—titus

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KGJ4DXRDZ2F5FBXGNOUYE5BKA55UV7RX/

[Python-Dev] Re: [Possibly off-topic] Python-Announce floods and delays

[C. Titus Brown]

On Mon, Jul 08, 2019 at 03:27:50PM -0400, Jonathan Goble wrote:
> (I don't know the best list to post this to, so if this is not it,
> please forgive me and point me in the right direction. Thanks.)
> 
> So my inbox, and probably many of yours, was flooded this afternoon
> with a dozen-plus emails from the Python-Announce list. I understand
> that this list requires every email to be manually approved by a
> moderator. The problem is that this is done infrequently. Prior to
> today, the last round of approvals was June 26, almost two weeks ago.
> This is not an atypical delay; on the contrary, it seems that
> moderators only look at the queue once every one to two weeks.
> 
> There are several problems with these delays:
> 
> 1. They result in floods of emails, with a large number of emails in a
> short period of time. This makes inbox management difficult on the
> days that approvals are done. Before you argue that "it's fine if you
> have the right tools configured in the right way", consider that there
> are probably many people who are subscribed to Python-Announce who
> have no interest in and are not subscribed to any of the actual
> discussion lists where such tools are most beneficial. Complex tool
> configurations should not be a requirement for managing incoming
> emails from what is essentially (to those people) a notification-only
> mailing list. These people would be better served by frequent
> approvals several times a week, allowing them to get fewer emails at
> one time, but in a more timely manner.
> 
> 2. Speaking of a more timely manner, the lengthy delays result in
> redundant and outdated emails going through after the point where they
> are no longer relevant. One such issue exemplified by today's set of
> approvals (and seen on previous occasions before) is an announcement
> of a new release of a PyPI package not being approved until after
> there has already been a subsequent release to that same package. In
> this case I am referring to the pytest 5.0.0 announcement sent to the
> list on June 28 (according to the headers), followed by the pytest
> 5.0.1 announcement sent to the list on July 5. Neither was approved
> and delivered to subscribers until today.
> 
> 3. More importantly in terms of delays, on July 3 an announcement was
> sent to the list regarding the impending switch of EuroPython ticket
> rates to the late registration rate on July 6. This is an example of a
> time-sensitive announcement that needs to not be delayed. Instead, the
> email was not approved and delivered to subscribers until today, July
> 8, after the conference has already begun, and not in time for list
> subscribers to react and avoid the late registration rates.
> 
> Is there a solution to this that would enable moderators to approve
> more frequently? I understand that they are probably volunteers and
> need to find spare time to wade through the queue, but if approvals
> are done more frequently (even daily), then it will consume much less
> time on each occasion. It would go from a task requiring an entire
> hour (as it apparently did today based on the delivery timestamps) to
> something that can be done on a coffee break.

I already help moderate python-ideas and would be happy to help moderate
announce.

best,
--titus
-- 
C. Titus Brown, ctbr...@ucdavis.edu
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SJIXMHD734APMOANNTFHE7QDNHZZXO6U/

Re: [Python-Dev] Core projects for Summer of Code

[C. Titus Brown]

On Wed, Mar 18, 2009 at 09:24:25PM +, Antoine Pitrou wrote:
-> Rather than performance, I think some more interesting areas would be 
related to
-> some of the standard library modules. For instance, the unittest module could
-> welcome some new features (test discovery, support for skipped tests, 
probably
-> others that I'm forgetting about). Since this is pure Python stuff not 
needing
-> any deep experience with the interpreter's internals, it would be appropriate
-> for an outsider.

Hi Antoine,

interesting idea -- but I've seen too many arguments about what the
*right* functionality to add to unittest would be to want to give it to
a student.  I think a student would probably not be willing or able to
fight the battles necessary to get his/her changes into the core...

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [snakebite] Re: snakebite for GSoC?

[C. Titus Brown]

On Fri, Mar 20, 2009 at 07:37:40AM +, Trent Nelson wrote:
-> 
-> On Thu, Mar 19, 2009 at 10:32:03AM -0700, ajaksu wrote:
-> > Does anyone have good ideas for assigning students to snakebite? Is it
-> > too early?
-> 
-> Perhaps a little too early, python-dev@ won't know anything about
-> Snakebite yet as I haven't publicly announced it there ;-)  Watch
-> this space closer to PyCon.

I do have a snakebite-motivated project, listed here:

  http://ivory.idyll.org/blog/mar-09/gsoc-projects.html

(#7)

Right now an independent study student is building something, but he
can't work on it over the summer, so continuing it in various ways could
be a GSoC project.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Core Python development tools

[C. Titus Brown]

On Sun, Mar 22, 2009 at 07:30:01PM -0300, Daniel (ajax) Diniz wrote:
-> Even if neither is considered worthy, I'll keep them on my to-do list
-> and hope to slowly and hackishly work towards both proposals' goals.
-> Barring feedback saying that they're out of scope, stupid and
-> downright offensive, I'll post details on each to this thread very
-> soon.

Given the relative paucity of core Python GSoC ideas, I think you should
go for both of these, *especially* if you have a mentor up front.  So,
write 'em up, add 'em to the GSoC page, and let's see who we get...
If we get good applications for both, then I think we can "fund" both of
them.

I do think you should be prepared for pushback from python-dev on any
such ideas ;).  Don't get too ambitious about writing up *your* way of
fixing things, but rather make sure you and the students are prepared to
adapt to what people on python-dev think.  Mind you, ultimately the
people doing the work should make the decisions, but input from
python-dev is usually pretty useful even when it's contradictory!

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Core projects for Summer of Code

[C. Titus Brown]

On Fri, Mar 20, 2009 at 03:18:00PM -0700, average wrote:
-> > Summer of Code is ramping up. ?Every year the common complaint is that not
-> > enough Python core projects get proposed by students, and of course a big
-> > reason for that is often the only encouragement we offer prospective
-> > students is a link to the PEP index.
-> >
-> > The challenge is finding project ideas for them that could reasonably 
occupy
-> > them for the entire Summer and which the results of their work can be
-> > demonstrated. ?They're being paid for specific projects so "Spend the 
Summer
-> > fixing bugs on the tracker" is a no-go, and Google has outlined that Summer
-> > of Code is about code, not documentation.
-> 
-> Improve doctest by allowing it to be aware of nested test scopes such
-> that a variable defined at "class-level scope" (i.e. the variable b
-> defined at the class-level doctest """>>> b=Bag("abacab")""") can be
-> used in "method-level scopes" without re-defining it every time for
-> each method's doctest (each method would reset the given variable (if
-> used) to its original state rather than live mutated between
-> equal-level scopes).
-> 
-> Would be a great improvement for doctest in my opinion--both in
-> ease-of-use, and reduction of redundant, error-prone ("did you define
-> your test variable the same in each method?") code)--as well as other
-> benefits.
-> 
-> Appreciate any consideration...

Hi Marcos,

my primary concern here would be that the student would do all this work
and then python-dev would reject it for incorporation into core!  Plus
it's probably not a summer-long project.

If, however, you wanted to suggest a general "gather disparate doctest
features and integrate them, for consideration for the core" project, I
would definitely recommend posting that as a possible project on the
Python GSoC site.  I know that zope has done some good doctest stuff,
for example; the 'testing-in-python' list would be a good place to go
for finding out more.

Note, you don't have to offer to be the mentor to post it, but it would
help ;)

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC: Core Python development tools

[C. Titus Brown]

On Mon, Mar 23, 2009 at 10:26:54PM +1000, Nick Coghlan wrote:
-> C. Titus Brown wrote:
-> > On Sun, Mar 22, 2009 at 07:30:01PM -0300, Daniel (ajax) Diniz wrote:
-> > I do think you should be prepared for pushback from python-dev on any
-> > such ideas ;).  Don't get too ambitious about writing up *your* way of
-> > fixing things, but rather make sure you and the students are prepared to
-> > adapt to what people on python-dev think.  Mind you, ultimately the
-> > people doing the work should make the decisions, but input from
-> > python-dev is usually pretty useful even when it's contradictory!
-> 
-> Everything I've seen from Daniel so far seems to be about either making
-> things we already do more efficient, or else providing additional
-> features in ways that don't make the tools any more confusing for others
-> already used to a particular way of doing things. So he seems to be
-> navigating this particular minefield pretty well so far.
-> 
-> Then again, I may be a little biased - some of the recent bug tracker
-> changes are things I had wished for in the past, but never chipped in
-> any code to help make them happen :)

Oh, I heartily endorse his suggestions! I just want to make sure that we
make maximum use of students (and their code doesn't get tossed at the
end of the summer, which has serious morale consequences ;)

--t
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] More GSoC - make sure you list your projects!

[C. Titus Brown]

Hi all,

just a quick note -- anyone who has an idea for a core Python Google
Summer of Code project, has not had that project panned on this list ;),
and is willing to act as a CONTACT for students who want to apply (not
necessarily a mentor!) should make sure that those ideas are posted on
the wiki, at

http://wiki.python.org/moin/SummerOfCode/2009

Without a contact and a link on the wiki page, students won't have any
idea that the project exists...

I will endeavor to find mentors for as many of the core Python projects
as possible, once we have a clearer sense of how many applicants we have
and how many applications are good.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] core python tests (was: Re: PyDict_SetItem hook)

[C. Titus Brown]

On Fri, Apr 03, 2009 at 07:00:43PM +0100, Michael Foord wrote:
-> Collin Winter wrote:
-> >On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord
-> > wrote:
-> >  
-> >>Collin Winter wrote:
-> >>
-> >>>As part of the common standard library and test suite that we agreed
-> >>>on at the PyCon language summit last week, we're going to include a
-> >>>common benchmark suite that all Python implementations can share. This
-> >>>is still some months off, though, so there'll be plenty of time to
-> >>>bikeshed^Wrationally discuss which benchmarks should go in there.
-> >>>
-> >>>  
-> >>Where is the right place for us to discuss this common benchmark and test
-> >>suite?
-> >>
-> >>As the benchmark is developed I would like to ensure it can run on
-> >>IronPython.
-> >>
-> >>The test suite changes will need some discussion as well - Jython and
-> >>IronPython (and probably PyPy) have almost identical changes to tests that
-> >>currently rely on deterministic finalisation (reference counting) so it
-> >>makes sense to test changes on both platforms and commit a single 
-> >>solution.
-> >>
-> >
-> >I believe Brett Cannon is the best person to talk to about this kind
-> >of thing. I don't know that any common mailing list has been set up,
-> >though there may be and Brett just hasn't told anyone yet :)
-> >
-> >Collin
-> >  
-> Which begs the question of whether we *should* have a separate mailing list.
-> 
-> I don't think we discussed this specific point in the language summit - 
-> although it makes sense. Should we have a list specifically for the test 
-> / benchmarking or would a more general implementations-sig be appropriate?
-> 
-> And is it really Brett who sets up mailing lists? My understanding is 
-> that he is pulling out of stuff for a while anyway, so that he can do 
-> Java / Phd type things... ;-)

'tis a sad loss for both Python-dev and the academic community...

I vote for a separate mailing list -- 'python-tests'? -- but I don't
know exactly how splintered to make the conversation.  It probably
belongs at python.org but if you want me to host it, I can.

N.B. There are a bunch of GSoC projects to work on or with the CPython
test framework (increase test coverage, write plugins to make it
runnable in nose or py.test, etc.).  I don't know that the students
should be active participants in such a list, but the mentors should at
least try to stay in the loop so that we don't completely waste our time.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] GSoC (was Re: PyDict_SetItem hook)

[C. Titus Brown]

On Sat, Apr 04, 2009 at 06:28:01AM -0700, Aahz wrote:
-> On Fri, Apr 03, 2009, Collin Winter wrote:
-> >
-> > I don't believe that these are insurmountable problems, though. A
-> > great contribution to Python performance work would be an improved
-> > version of PyBench that corrects these problems and offers more
-> > precise measurements. Is that something you might be interested in
-> > contributing to? As performance moves more into the wider
-> > consciousness, having good tools will become increasingly important.
-> 
-> GSoC work?

Alas, it's too late to submit new proposals; the deadline was
yesterday.

The next "Google gives us money to wrangle students into doing
development" project will probably be GHOP for highschool students, in
the winter, although it has not been announced and may not happen.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] graphics maths types in python core?

[C. Titus Brown]

Hi all,

we're having a discussion over on the GSoC mailing list about basic
math types, and I was wondering if there is any history that we should
be aware of in python-dev.  Has this been brought up before and
rejected?  Should the interested projects work towards a consensus and
maybe write up a PEP?

(The proximal issue is whether or not this is of direct relevance to the
python core and hence should be given priority.)

tnx,
-titus

Rene Dudfield wrote:

-> there's seven graphics math type proposals which would be a good project
-> for the graphics python using projects -- especially if they can get
-> into python.
->
-> It would be great if one of these proposals was accepted to work towards
-> getting these simple types into python.
->
-> Otherwise we'll be doomed to have each project implement vec2, vec3,
-> vec4, matrix3/4, quaternion (which has already happened many times) -
-> and continue to have interoperability issues.
->
-> The reason why just these basic types, and not full blown numpy is that
-> numpy is never planned to get into python.  Numpy doesn't want to tie
-> it's development into pythons development cycle.  Whereas a small set of
-> types  can be implemented and stabalised for python more easily.
->
-> Also, it's not image, or 3d format types -- since those are also a way
-> larger project.

-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Google Summer of Code/core Python projects - RFC

[C. Titus Brown]

Hi all,

this year we have 10-12 GSoC applications that I've put in the "relevant
to core Python development" category.  These projects, if mentors etc
are found, are *guaranteed* a slot under the PSF GSoC umbrella.  As
backup GSoC admin and general busybody, I've taken on the work of
coordinating these as a special subgroup within the PSF GSoC, and I
thought it would be good to mention them to python-dev.

Note that all of them have been run by a few different committers,
including Martin, Tarek, Benjamin, and Brett, and they've been obliging
enough to triage a few of them.  Thanks, guys!

Here's what's left after that triage.  Note that except for the four at
the top, these have all received positive support from *someone* who is
a committer and I don't think we need to discuss them here -- patches
etc. can go through normal "python-dev" channels during the course of the
summer.

I am looking for feedback on the first four, though.  Can these
reasonably be considered "core" priorites for Python?  Remember, this
"costs" us something in the sense of preferring these over Python
subprojects like (random example) Cython, NumPy, PySoy, Tahoe, Gajim,
etc.

---

Questionable "core":

2x "port NumPy to py3k" -- NumPy is a major Python module and porting it
to py3k fits with Guido's request that "more stuff get ported".
To be clear, I don't think anyone expects all of NumPy to get
ported this summer, but these students will work through issues
associated with porting big chunks o' code to py3k.

One medium/strong proposal, one medium/weak proposal.

Comments/thoughts?

2x "improve testing tools for py3k" -- variously focus on improving test
coverage and testing wrappers.

One proposes to provide a nice wrapper to make nose and py.test
capable of running the regrtests, which (with no change to
regrtest) would let people run tests in parallel, distribute or
run tests across multiple machines (including Snakebite), tag
and run subsets of tests with personal and/or public tags, and
otherwise take advantage of many of the nice features of nose
and py.test.

The other proposes to measure & increase the code coverage of
the py3k tests in both Python and C, integrate across multiple
machines, and otherwise provide a nice set of integrated reports
that anyone can generate on their own machines.  This proposal,
in particular, could move smoothly towards the effort to produce
a "Python-wide" test suite for CPython/IronPython/PyPy/Jython.
(This wasn't integrated into the proposal because I only found
out about it after the proposals were due.)

I personally think that both testing proposals are good, and
they grew out of conversations I had with Brett, who thinks that
the general ideas are good.  So, err, I'm looking for pushback,
I guess ;).  I can expand on these ideas a bit if people are
interested.

Both proposals are medium at least, and I've personally been
positively impressed with the student interaction.

Comments/thoughts?

---

Unquestionably "core" by my criteria above:

3to2 tool -- 'nuff said.

subprocess improvement -- integrating, testing, and proposing some of
the various subprocess improvements that have passed across this
list & the bug tracker

IDLE/Tkinter patch integration & improvement -- deal with ~120 tracker
issues relating to IDLE and Tkinter.

roundup VCS integration / build tools to support core development --
a single student proposed both of these and has received some
support.  See http://slexy.org/view/s2pFgWxufI for details.

sphinx framework improvement -- support for per-paragraph comments and
user/developer interface for submitting/committing fixes 

2x "keyring package" -- see
http://tarekziade.wordpress.com/2009/03/27/pycon-hallway-session-1-a-keyring-library-for-python/.
The poorer one of these will probably be axed unless Tarek gives it
strong support.

--

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Google Summer of Code/core Python projects - RFC

[C. Titus Brown]

On Fri, Apr 10, 2009 at 05:53:23PM -0300, Guilherme Polo wrote:
-> >
-> > IDLE/Tkinter patch integration & improvement -- deal with ~120 tracker
-> > ? ? ? ?issues relating to IDLE and Tkinter.
-> >
-> 
-> Is it important, for the discussion, to mention that it also involves
-> testing this area (idle and tkinter), Titus ? I'm considering this
-> more important than "just" dealing with the tracker issues.

What, I tell you that your app is going to be accepted and we shouldn't
argue about it, and you want to argue about it? ;)

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Google Summer of Code/core Python projects - RFC

[C. Titus Brown]

On Fri, Apr 10, 2009 at 06:05:02PM -0500, Benjamin Peterson wrote:
-> 2009/4/10 C. Titus Brown :
-> > 2x "improve testing tools for py3k" -- variously focus on improving test
-> > ?? ?? ?? ??coverage and testing wrappers.
-> >
-> > ?? ?? ?? ??One proposes to provide a nice wrapper to make nose and py.test
-> > ?? ?? ?? ??capable of running the regrtests, which (with no change to
-> > ?? ?? ?? ??regrtest) would let people run tests in parallel, distribute or
-> > ?? ?? ?? ??run tests across multiple machines (including Snakebite), tag
-> > ?? ?? ?? ??and run subsets of tests with personal and/or public tags, and
-> > ?? ?? ?? ??otherwise take advantage of many of the nice features of nose
-> > ?? ?? ?? ??and py.test.
-> >
-> > ?? ?? ?? ??The other proposes to measure & increase the code coverage of
-> > ?? ?? ?? ??the py3k tests in both Python and C, integrate across multiple
-> > ?? ?? ?? ??machines, and otherwise provide a nice set of integrated reports
-> > ?? ?? ?? ??that anyone can generate on their own machines. ??This proposal,
-> > ?? ?? ?? ??in particular, could move smoothly towards the effort to produce
-> > ?? ?? ?? ??a "Python-wide" test suite for CPython/IronPython/PyPy/Jython.
-> > ?? ?? ?? ??(This wasn't integrated into the proposal because I only found
-> > ?? ?? ?? ??out about it after the proposals were due.)
-> >
-> > ?? ?? ?? ??I personally think that both testing proposals are good, and
-> > ?? ?? ?? ??they grew out of conversations I had with Brett, who thinks that
-> > ?? ?? ?? ??the general ideas are good. ??So, err, I'm looking for pushback,
-> > ?? ?? ?? ??I guess ;). ??I can expand on these ideas a bit if people are
-> > ?? ?? ?? ??interested.
-> >
-> > ?? ?? ?? ??Both proposals are medium at least, and I've personally been
-> > ?? ?? ?? ??positively impressed with the student interaction.
-> 
-> To me, both of those proposals seem to say "measure and improve test
-> coverage" or "nose integration" with a severe lack specific details.
-> Especially the nose plugin one seems like very little work. (Running
-> default nose in the test directory in fact works fairly well.)

...fairly, yes ;).  But not perfectly.  And certainly not with
equivalent guarantees to regrtest, which is really what Python
developers need.  Tracking down the corner cases, writing up examples,
setting up tags, getting multiprocess to work properly, and making sure
that coverage recording works properly, and then getting people to try
it out on THEIR machines, is likely to be a lot of work.

The plugin ecosystem for nose is growing daily and supporting that for
core would be fantastic; extending it to py.test (whose plugin interface
is now mostly compatible with nose) would be even better.

The lack of detail on the code coverage is intentional, IMO.  It's
non-trivial to get a full handle on C code coverage integrated with
Python code coverage -- or at least it has been for me -- so I supported
the student focusing on first writing robust coverage analysis tools,
and only then deciding what to "hit" with more tests.  I will encourage
the student to talk to this list (or the "tests" list in the stdlib sig)
in order to target areas that are more relevant to people.

I have had a hard time getting a good sense of what core code is well
tested and what is not well tested, across various platforms.  While
Walter's C/Python integrated code coverage site is nice, it would be
even nicer to have a way to generate all that information within any
particular checkout on a real-time basis.  Doing so in the context of
Snakebite would be icing... and I think it's worth supporting in core,
especially if it can be done without any changes *to* core.

-> Another small nit is that they should address Python 2.x, too.

I asked that they focus on EITHER 2.x or 3.x, since "too broad" is an
equally valid criticism.  Certainly 3.x is the future so I though
focusing on increasing code coverage, and especially C code coverage,
could best be applied to 3.x.

cheers,
--titus
--
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Google Summer of Code/core Python projects - RFC

[C. Titus Brown]

On Sat, Apr 11, 2009 at 12:21:18PM +0200, Mario wrote:
-> > He says vague things about patches too, but I'm not sure what.  If he
-> > wanted to make that into a 'patchbot' that just applied every patch in
-> > isolation and ran 'make && make test' and posted results in the
-> > tracker I'd be a happy camper.
-> >
-> Jack, how about you write that idea down on the wiki page mentioned in the
-> proposal, along with the use case? Following that, I'll see if I can do
-> anything about it to make it a reality.

We had a GSoC student two years back who worked on something like this;
his name is Michal Kwiatkowski.  He probably has the code working
somewhere.

It's a nontrivial problem if you want to do it properly with VMs etc.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Google Summer of Code/core Python projects - RFC

[C. Titus Brown]

On Sat, Apr 11, 2009 at 08:13:35AM +0200, "Martin v. L?wis" wrote:
-> > 2x "keyring package" -- see
-> > 
http://tarekziade.wordpress.com/2009/03/27/pycon-hallway-session-1-a-keyring-library-for-python/.
-> > The poorer one of these will probably be axed unless Tarek gives it
-> > strong support.
-> 
-> I don't think these are good "core" projects. Even if the students come
-> up with a complete solution, it shouldn't be integrated with the
-> standard library right away. Instead, it should have a life outside the
-> standard library, and be considered for inclusion only if the user
-> community wants it.

Tarek has said he can put it into distutils on a trial basis, although
I'm sure that'll depend on what the student comes up with.

I'm using "core projects" as a shorthand for projects that directly
address the core development environment, the stdlib, and priorities of
committers on python-dev.  Tarek is a committer, and it sounded like
you, Jim, and Georg were all interested in this project, too -- that
pushes it well into "core" territory IMO.

-> I'm also skeptical that this is a good SoC project in the first place.
-> Coming up with a wrapper for, say, Apple Keychain, could be a good
-> project. Coming up with a unifying API for all keychains is out of
-> scope, IMO; various past attempts at unifying APIs have demonstrated
-> that creating them is difficult, and might require writing a PEP
-> (whose acceptance then might not happen within a summer).

Well, that's a more unassailable argument and one I agree with ;).

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr)

[C. Titus Brown]

On Thu, Jun 04, 2009 at 11:03:22AM -0400, Alexander Belopolsky wrote:
-> On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore  wrote:
-> ...
-> > I could still argue that there are downsides (need to take action to
-> > set myself as nosy on an issue, possibly setting up a new mail filter,
-> > housekeeping cruft, the fact that people don't quote in the same way
-> > on tracker items) that make tracker discussions less attractive to me,
-> > but it's very personal things.
-> 
-> How hard would it be to set up a bot that will subscribe to python-dev
-> and archive messages that have [issue] in the subject under
-> appropriate roundup ticket?  This way if discussion started on
-> roundup, it would only take adding python-dev in CC to bring it to the
-> larger python-dev audience and adding issue number to the python-dev
-> thread subject will be all it takes to archive new messages on
-> roundup.  This would not solve a problem of linking threads that start
-> on python-dev before a ticket is opened, but with some proper
-> conventions this problem can be solved as well.

In my experience this kind automated stuff is too fragile to work
reliably & specifically over time, which is what we would want -- "fire
and forget".

Is there a good python-dev archive search mechanism (other than to
google "python-dev " ;) out there?  Wouldn't that help?

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr)

[C. Titus Brown]

On Thu, Jun 04, 2009 at 05:54:23PM +0200, "Martin v. L?wis" wrote:
-> > Is there a good python-dev archive search mechanism (other than to
-> > google "python-dev " ;) out there?  Wouldn't that help?
-> 
-> I would add site:mail.python.org into the google query, but apart from
-> that: what's wrong with google search?

It generally points you towards the right messages in the python-dev
archives, but those are kept by month and the threading doesn't always
work well.  Sometimes conversations span many months, and threading can
be an immense help for understanding the discussion.

Something like MarkMail (as Dirkjan mentioned) may have a better
interface.  I'll give it a try.

thanks,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Snakebite (was: Re: py3k buildbots)

[C. Titus Brown]

On Fri, Jun 05, 2009 at 11:26:34AM +0200, Thomas Heller wrote:
-> Antoine Pitrou schrieb:
-> > Hello
-> > 
-> > Only one of the py3k buildbots seems up:
-> > http://www.python.org/dev/buildbot/3.x.stable/
-> 
-> Maybe they are waiting for the snakebite network ;-) (what's up with it, 
anyway?).

We're still getting the machines set up.  It turns out delivering power
and A/C to a wide variety of architectures is more complicated than it
may sound ;)

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FWD: Front Runner Program

[C. Titus Brown]

On Thu, Sep 10, 2009 at 08:11:55AM -0700, Aahz wrote:
-> I'm still no-mail on python-dev, forwarding as FYI
->
-> - Forwarded message from Eric Albrecht  
-
-> 
-> From: Eric Albrecht 
-> To: "webmas...@python.org" 
-> Date: Thu, 10 Sep 2009 10:48:11 -0400
-> Subject: Front Runner Program 
-> 
-> Regarding: Windows 7 Compatibility for Python Application.

Thanks, Aahz!

I don't see a Windows 7 buildbot up here:

http://www.python.org/dev/buildbot/all/

but I confess that I'm bad at reading these pages.  Has anyone tried
compiling either trunk or py3k on Win 7?  Would this be useful?

My recently acquired* MSDN account has led me to getting XP up and
running in a VM, and I would be happy to try other Windows OSes of
interest.

--titus

* acquired courtesy of Snakebite/Trent Nelson.
--
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] thinking about 2.7 / buildbots / testing

[C. Titus Brown]

On Thu, Sep 24, 2009 at 12:23:31AM -0400, David Lyon wrote:
> On Wed, 23 Sep 2009 15:13:55 -, exar...@twistedmatrix.com wrote:
> 
> > I would also personally recommend that this person first (well, after 
> > tracking down all the slave operators and convincing them to bring their 
> > slaves back online) acquire shell access to all of the slave machines so 
> > that the owners of the slave hosts themselves no longer need to be the 
> > gating factor for most issues.
> 
> Depends on where the machines are. There are good tools do check all
> automatically. Nagios is one.
> 
> Anyway, this would suite my work schedule for the next 12 months.
> 
> Do we already have the machines? or do they need to be acquired?

Hi David,

as Brett leaked, I am developing software towards making this buildbot
dilemma easier to deal with.  It's not really ready for primetime yet;
I'll send you a link off-list.

I am also (physically) hosting Snakebite, which Trent Nelson will be
tackling over the next few months.

And, finally, I now have Windows VMs (XP, Vista, 7) and a Mac OS X box
that I can dedicate to Python purposes.  I even have a student that will
be monitoring those boxes.  I am happy to give you access to these
machines if you want to set up buildbot on them; buildbot is compatible
with my own endeavors, so that would actually help me out ;)

I will be buying more/better/bigger Linux hardware over the next few
months, too, and I am happy to slice that off to provide more VMs --
basically whatever x86-based platforms people think are needed.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] BDFL pronouncement?

[C. Titus Brown]

On Fri, Oct 09, 2009 at 01:56:42PM -0700, Guido van Rossum wrote:
> On Fri, Oct 9, 2009 at 1:50 PM, Georg Brandl  wrote:
> > Chris Withers schrieb:
> >> Guido van Rossum wrote:
> >>> I'm saying that I don't expect setuptools 0.7 to appear before Tarek's
> >>> Distribute is mature and in widespread use. IOW I support Tarek's fork
> >>> and suggest nobody hold their breath waiting for setuptools 0.7.
> >>
> >> Well, if this was the BDFL pronouncement that a lot of people have taken
> >> it to be, does that allow Martin von Lewis to give the keys to
> >> http://pypi.python.org/pypi/setuptools to the "distribute" developers,
> >> so we can get on and use the original "setuptools" name without all the
> >> confusion and hackery needed to make "distribute" work?
> >
> > That's absurd. ?There's a certain area where Guido can make pronouncements,
> > but third-party packages is not it. ?Even if they're hosted on python.org
> > infrastructure.
> 
> Right.

Is that a pronouncement?

:)

"GvR, the self-limiting BDFL."

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible language summit topic: buildbots

[C. Titus Brown]

On Sun, Oct 25, 2009 at 08:54:46AM +, Mark Dickinson wrote:
> Would it be worth spending some time discussing the buildbot situation
> at the PyCon 2010 language summit?  In the past, I've found the
> buildbots to be an incredibly valuable resource;  especially when
> working with aspects of Python or C that tend to vary significantly
> from platform to platform (for me, this usually means floating-point,
> and platform math libraries, but there are surely many other things it
> applies to).  But more recently there seem to have been some
> difficulties keeping a reasonable number of buildbots up and running.
> A secondary problem is that it can be awkward to debug some of the
> more obscure test failures on buildbots without having direct access
> to the machine.  From conversations on IRC, I don't think I'm alone in
> wanting to find ways to make the buildbots more useful.
> 
> So the question is: how best to invest time and possibly money to
> improve the buildbot situation (and as a result, I hope, improve the
> quality of Python)?  What could be done to make maintenance of build
> slaves easier?  Or to encourage interested third parties to donate
> hardware and time?  Are there good alternatives to Buildbot that might
> make a difference? What do other projects do?
> 
> These are probably the wrong questions;  I'm hoping that a discussion
> would help produce the right questions, and possibly some answers.

[ x-posting to testing-in-python; please redirect followups to one list or
the other! ]

Hi Mark,

a few bits of information...

---

I have a set of VM machines running some "core" build archs -- Linux, Mac OS X,
Win XP, Win Vista, and Win 7 -- that I am going to dedicate to this purpose.
I am happy to give out remote admin access to a few people.  This
infrastructure is also going to increase slowly as I build up my lab's internal
network.

I'm giving Tarek an account on my Linux box later today to serve as a build
slave for Distribute.

--

More machines, and more esoteric machines, will be coming online as Snakebite
unfolds.  Trent Nelson (Snakepit master) has been finishing up some consulting
work and is going to dedicate his time to it starting in November.  This means
more 64 bit, bigmem, and "weird" archs, with full login access.

---

I've also been working on a buildbot alternative that I'm calling pony-build.
pony-build is based on a client-push architecture in which client machines
do builds and push results to the master, which then acts as a record-keeper
rather than a slave driver.  The result is a less coordinated but (AFAICT) much
less fragile continuous integration system.  I'm hoping to give a talk at PyCon
on the differences, and there will be a sprint on pony-build + pyhton-dev at
PyCon, regardless.

The current status of pony-build is "functional but ugly inside".  In
particular, the data model is horrible, and the internal API needs much
more fleshing out.  Nonetheless, my server has been running for two months
or so, and you can look at the results here,

http://lyorn.idyll.org/ctb/pb-dev/

The most fully-fleshed out set of build clients is for pygr,

http://lyorn.idyll.org/ctb/pb-dev/pygr/

but you can view daily build results for Python 2.7 at

http://lyorn.idyll.org/ctb/pb-dev/python/

with an exhaustive list here

http://lyorn.idyll.org/ctb/pb-dev/python/show_all

(and why the heck am I sorting in reverse date order, anyway?!)

The most interesting (?) part of pony-build right now is the client
config, which I'm struggling to make simple and potentially universal
enough to serve under buildbot as well:

http://github.com/ctb/pony-build/blob/master/client/build-cpython

(see 'commands' list).

The most *exciting* part of pony-build, apart from the always-riveting
spectacle of "titus rediscovering problems that buildbot solved 5 years ago",
is the loose coupling of recording server to the build slaves and build
reporters.  My plan is to enable a simple and lightweight XML-RPC and/or
REST-ish interface for querying the recording server from scripts or other Web
sites.  This has Brett aquiver with anticipation, I gather -- no more visual
inspection of buildbot waterfall pages ;)

--

If you're interested in bashing on, contributing to, or helping figure out
what color the pony-build main screen should be, contact me off-list; I'm
reluctant to spam up python-dev or testing-in-python.

That having been said, the results of taking it and trying it out -- you can
post results to my own recording server at 

http://lyorn.idyll.org/ctb/pb-dev/xmlrpc

-- would be most welcome.

Once I fix the data model, code collaboration will be much more feasible,
too.

---

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___

Re: [Python-Dev] Possible language summit topic: buildbots

[C. Titus Brown]

On Sun, Oct 25, 2009 at 07:32:52PM +0100, "Martin v. L?wis" wrote:
> > I've been trying to get some feedback about firing up buildbots on Cloud
> > Servers for a while now and haven't had much luck.  I'd love to find a
> > way of having buildbots come to life, report to the mother ship, do the
> > build, then go away 'till next time they're required.
> 
> I'm not quite sure whom you have been trying to get feedback from, and
> can't quite picture your proposed setup from above description.
> 
> In any case, ISTM that your approach isn't compatible with how buildbot
> works today (not sure whether you are aware of that): a build slave
> needs to stay connected all the time, so that the build master can
> trigger a build when necessary.

Hi Martin,

it shouldn't be difficult to cobble together a build script that spins up a
buildslave on EC2 and runs the tests there; I wrote something similar a
few years ago for an infrequently connected home machine.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible language summit topic: buildbots

[C. Titus Brown]

On Fri, Oct 30, 2009 at 04:21:06PM +, Antoine Pitrou wrote:
> 
> Hello,
> 
> Sorry for the little redundancy, I would like to underline Jean-Paul's
> suggestion here:
> 
> Le Sun, 25 Oct 2009 14:05:12 +, exarkun a ??crit??:
> > I think that money can help in two ways in this case.
> > 
> > First, there are now a multitude of cloud hosting providers which will 
> > operate a slave machine for you.  BuildBot has even begun to support 
> > this deployment use-case by allowing you to start up and shut down vms 
> > on demand to save on costs.  Amazon's EC2 service is supported out of 
> > the box in the latest release.
> 
> I'm not a PSF member, but it seems to me that the PSF could ask Amazon 
> (or any other virtual machine business anyway) to donate a small number
> of permanent EC2 instances in order to run buildslaves on.

[ ... ]

I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
ia64; Linux x86; and Mac OS X.  Others have made similar offers.  The
architectures supported by the cloud services don't really add anything
(and generally don't have Mac OS X support, AFAIK).

What we really need (IMO) is someone to dig into the tests to figure out which
tests fail randomly and why, and to fix them on specific architectures that
most of us don't personally use.  This is hard work that is neither glamorous
nor popular.

I think the idea of paying a dedicated developer to make the CPython+buildbot
tests reliable is better, although I would still be -0 on it (I don't think
the PSF should be paying for this kind of thing at all).

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible language summit topic: buildbots

[C. Titus Brown]

On Fri, Oct 30, 2009 at 05:41:39PM +0100, Antoine Pitrou wrote:
> Le vendredi 30 octobre 2009 ?? 09:31 -0700, C. Titus Brown a ??crit :
> > [ ... ]
> > 
> > I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
> > ia64; Linux x86; and Mac OS X.  Others have made similar offers.  The
> > architectures supported by the cloud services don't really add anything
> > (and generally don't have Mac OS X support, AFAIK).
> 
> Well these VMs would have to run buildslaves on them, then. Are you
> ready to host some and connect them to the current buildbot
> infrastructure?
> (VMs without buildslaves are less interesting IMO)

No, I'm not willing to spend the time to install and maintain buildbot. But
I'm happy to give the necessary access to those who are interested and
willing.

(...and let me tell you, getting these !#%!#$! Windows VMs up and running
already took an immense amount of effort ;)

> > What we really need (IMO) is someone to dig into the tests to figure out 
> > which
> > tests fail randomly and why, and to fix them on specific architectures that
> > most of us don't personally use.  This is hard work that is neither 
> > glamorous
> > nor popular.
> 
> I'm sure some of us are ready to do so (*). The situation has already
> improved quite a lot in the recent times. But fixing platform- or,
> worse, setup-specific issues often requires shell access to the target
> system, otherwise you spend too much time trying fixes on the SVN and
> waiting for the buildbot to react.
> 
> (*) After all, if we weren't, we wouldn't even care about buildbots,
> we'd be content with running the test suite on our own machines

I look forward to it!

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [TIP] Possible language summit topic: buildbots

[C. Titus Brown]

On Fri, Oct 30, 2009 at 11:42:30AM -0500, Olemis Lang wrote:
> On Sun, Oct 25, 2009 at 9:13 AM,   wrote:
> > On 12:48 pm, c...@msu.edu wrote:
> >>
> >> [snip]
> >>
> >> The most *exciting* part of pony-build, apart from the always-riveting
> >> spectacle of "titus rediscovering problems that buildbot solved 5 years
> >> ago",
> >> is the loose coupling of recording server to the build slaves and build
> >> reporters. ?My plan is to enable a simple and lightweight XML-RPC and/or
> >> REST-ish interface for querying the recording server from scripts or other
> >> Web
> >> sites. ?This has Brett aquiver with anticipation, I gather -- no more
> >> visual
> >> inspection of buildbot waterfall pages ;)
> >
> > BuildBot has an XML-RPC interface. ?So Brett can probably do what he wants
> > with BuildBot right now.
> 
> ... but pony-build follows a different model

I'd rather not get into discussions of why my vaporware is going to be
way, way better than anything anyone else could possibly do -- that's
flamebait and not very friendly, in the end.  Let's just say that I'm wasting
my own time on it to scratch my own itch and leave it at that!

thanks,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible language summit topic: buildbots

[C. Titus Brown]

On Fri, Oct 30, 2009 at 04:49:51PM +, Paul Moore wrote:
> 2009/10/30 C. Titus Brown :
> > On Fri, Oct 30, 2009 at 04:21:06PM +, Antoine Pitrou wrote:
> >>
> >> Hello,
> >>
> >> Sorry for the little redundancy, I would like to underline Jean-Paul's
> >> suggestion here:
> >>
> >> Le Sun, 25 Oct 2009 14:05:12 +, exarkun a ??crit??:
> >> > I think that money can help in two ways in this case.
> >> >
> >> > First, there are now a multitude of cloud hosting providers which will
> >> > operate a slave machine for you. ?BuildBot has even begun to support
> >> > this deployment use-case by allowing you to start up and shut down vms
> >> > on demand to save on costs. ?Amazon's EC2 service is supported out of
> >> > the box in the latest release.
> >>
> >> I'm not a PSF member, but it seems to me that the PSF could ask Amazon
> >> (or any other virtual machine business anyway) to donate a small number
> >> of permanent EC2 instances in order to run buildslaves on.
> >
> > [ ... ]
> >
> > I'm happy to provide VMs or shell access for Windows (XP, Vista, 7); Linux
> > ia64; Linux x86; and Mac OS X. ?Others have made similar offers. ?The
> > architectures supported by the cloud services don't really add anything
> > (and generally don't have Mac OS X support, AFAIK).
> 
> As Antione pointed out, it's not clear (at least, it isn't to me) what
> that leaves to be done.

Great!  We've solved the problem ;)

> As a counter-offer: Given remote access to however many Windows VMs
> you want to provide, I'll get them up and running with buildslaves on
> them. If that requires software such as Visual Studio, I have copies
> via the MSDN licenses that I am happy to provide.

I, too, have MSDN licenses, and I have functioning build environments
on all of the VMs (I think -- I've only tested Win XP currently:

http://lyorn.idyll.org/ctb/pb-dev/python/detail?result_key=8276

)

I also have an OS X 10.5 machine that I can let you into through a firewall;
it's building Python 2.7 quite nicely:

http://lyorn.idyll.org/ctb/pb-dev/python/detail?result_key=8229

> Once things are up and running, I'll be prepared to do basic care and
> feeding of the buildslave, but as my time is limited, it would be nice
> if others would pitch in to help.

I would be somewhat unhappy about giving more than three or four people
admin access, but am prepared to lie back and think of England.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Cloud build slaves (was Re: Possible language summit topic: buildbots)

[C. Titus Brown]

On Fri, Oct 30, 2009 at 04:58:29PM -, exar...@twistedmatrix.com wrote:
> On 04:31 pm, c...@msu.edu wrote:
>> On Fri, Oct 30, 2009 at 04:21:06PM +, Antoine Pitrou wrote:
>>>
>>> Hello,
>>>
>>> Sorry for the little redundancy, I would like to underline Jean-Paul's
>>> suggestion here:
>>>
>>> Le Sun, 25 Oct 2009 14:05:12 +, exarkun a ??crit??:
>>> > I think that money can help in two ways in this case.
>>> >
>>> > First, there are now a multitude of cloud hosting providers which  
>>> will
>>> > operate a slave machine for you.  BuildBot has even begun to support
>>> > this deployment use-case by allowing you to start up and shut down  
>>> vms
>>> > on demand to save on costs.  Amazon's EC2 service is supported out  
>>> of
>>> > the box in the latest release.
>>>
>>> I'm not a PSF member, but it seems to me that the PSF could ask Amazon
>>> (or any other virtual machine business anyway) to donate a small  
>>> number
>>> of permanent EC2 instances in order to run buildslaves on.
>>
>> [ ... ]
>>
>> I'm happy to provide VMs or shell access for Windows (XP, Vista, 7);  
>> Linux
>> ia64; Linux x86; and Mac OS X.
>
> Okay, let's move on this.  Martin has, I believe, said that potential  
> slave operators only need to contact him to get credentials for new  
> slaves.  Can you make sure to follow up with him to get slaves running  
> on these machines?  Or would you rather give out access to someone else  
> and have them do the build slave setup?

I think we crossed threads here; I can provide the VMs, and access to them,
but I won't (empirically, don't have the regular time available to ;) maintain
buildbot buildslaves.

You or Antoine or others are welcome to contact me off-list.  Just give me an
account name and ssh key, and I'll give you login access via tunneled Remote
Desktop to the Windows machines.

>> Others have made similar offers.
>
> I'll similarly encourage them to take action, then.  Do you happen to  
> remember who?

Every few months this thread seems to pop up and then fizzles when people
realize the level of work and attention involved (<- he says cynically) in
exploiting the offer of resources; I hope that anyone interested in
offering resources will pop their head up again to look around.

> I hope everyone is on board with the idea of fixing bugs in CPython,  
> either in the actual implementation of features or in the tests for  
> those features.  That being the case, the discussion of whether or not  
> the PSF should try to fund such a task is perhaps best discussed on the  
> PSF members list.

Sure.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Building a Windows MSI for Python /trunk

[C. Titus Brown]

Hi all,

I got an MSI build working on my WinXP VM just now, and I wanted to
touch base with whomever it is that is maintaining this (wonderful!)
set of scripts...

I ran into three problems, and I managed to figure out two of them; the third
wasn't fatal.  Note, the diff of my fixed checkout is attached.

First, the script that finds & builds the external dependencies has two
minor problems.

 * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
   tcl8.* and tk8.* to grab the license text.  I changed the glob strings
   appropriately and that seemed to work.

 * Tix isn't downloaded/installed/built automatically like everything else,
   and msi.py looks for its license file, too.  I just removed the
   Tix reference.  I can't figure out how to build Tix appropriately; any
   tips?

Second, the buildmsi.bat file refers to python26a3.hhp instead of
python27a0.hhp.

Third, I could not get _tkinter to build properly, although it wasn't fatal
to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
tcl85g.lib existed.

Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I
had to figure out with google -- things like nasm/nasmw, for example.  Are
these documented somewhere, or would it be helpful to document them?  I think I
had to install:

 - Microsoft HTML Help Compiler
 - cygwin with make and python2.5 to build the docs
 - nasm (and copy nasm.exe to nasmw.exe)
 - cabarc

Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;)

I'd love to get this build process working completely automatically and
100% correctly, too.

Hat tip to Trent Nelson, who helped me figure out where the scripts are
and what other things I needed...

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
Index: Tools/msi/msi.py
===
--- Tools/msi/msi.py	(revision 76532)
+++ Tools/msi/msi.py	(working copy)
@@ -875,9 +875,8 @@
 for name, pat, file in (("bzip2","bzip2-*", "LICENSE"),
   ("Berkeley DB", "db-*", "LICENSE"),
   ("openssl", "openssl-*", "LICENSE"),
-  ("Tcl", "tcl8*", "license.terms"),
-  ("Tk", "tk8*", "license.terms"),
-  ("Tix", "tix-*", "license.terms")):
+  ("Tcl", "tcl-8*", "license.terms"),
+  ("Tk", "tk-8*", "license.terms")):
 out.write("\nThis copy of Python includes a copy of %s, which is licensed under the following terms:\n\n" % name)
 dirs = glob.glob(srcdir+"/../"+pat)
 if not dirs:
Index: Tools/buildbot/buildmsi.bat
===
--- Tools/buildbot/buildmsi.bat	(revision 76532)
+++ Tools/buildbot/buildmsi.bat	(working copy)
@@ -9,7 +9,7 @@
 
 @rem build the documentation
 bash.exe -c 'cd Doc;make PYTHON=python2.5 update htmlhelp'
-"%ProgramFiles%\HTML Help Workshop\hhc.exe" Doc\build\htmlhelp\python26a3.hhp
+"%ProgramFiles%\HTML Help Workshop\hhc.exe" Doc\build\htmlhelp\python27a0.hhp
 
 @rem buold the MSI file
 cd PC
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Building a Windows MSI for Python /trunk

[C. Titus Brown]

On Thu, Nov 26, 2009 at 09:54:35AM +0100, "Martin v. L?wis" wrote:
> > First, the script that finds & builds the external dependencies has two
> > minor problems.
> > 
> >  * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in
> >tcl8.* and tk8.* to grab the license text.  I changed the glob strings
> >appropriately and that seemed to work.
> 
> Not sure what "it" is that put it there; perhaps you are referring to
> the Tools/buildbot scripts?

Yes, sorry, Tools/buildbot/external*.bat, as in the diff.

> Unfortunately, the naming convention that these scripts establish
> doesn't quite work, as Tix would fail to build under these conventions.
> So in my own checkout, I manually renamed the trees, and that's what
> msi.py refers to.

OK, thanks!

> >  * Tix isn't downloaded/installed/built automatically like everything else,
> >and msi.py looks for its license file, too.  I just removed the
> >Tix reference.  I can't figure out how to build Tix appropriately; any
> >tips?
> 
> See above. I keep forgetting how to build Tix; one set of command lines
> is in PCbuild/build_tkinter.py.

Ok, thanks!

> > Second, the buildmsi.bat file refers to python26a3.hhp instead of
> > python27a0.hhp.
> 
> Yes, that script hasn't been updated for a long time, ever since we
> stopped having automated builds.
> 
> Ideally, the script would find the Python version on its own.

Right...

> > Third, I could not get _tkinter to build properly, although it wasn't fatal
> > to the endeavor.  It couldn't find ..\..\tcltk\lib\tcl85.lib, although
> > tcl85g.lib existed.
> 
> You'll need to build release versions of Tcl/Tk/Tix.

Yes, I saw a reference to that online, but I wasn't sure if that was the
problem -- is the naming convention really that 'tcl85g.lib' is the debug
lib!?

> > Oh, and there were a bunch of missing commands that (as a non-Windows 
> > xpert) I
> > had to figure out with google -- things like nasm/nasmw, for example.  Are
> > these documented somewhere, or would it be helpful to document them?
> 
> See PCbuild/readme.txt.

OK, thanks.

> > I'd love to get this build process working completely automatically and
> > 100% correctly, too.
> > 
> > Hat tip to Trent Nelson, who helped me figure out where the scripts are
> > and what other things I needed...
> 
> Feel free to submit patches. There may be a misunderstanding, though, in
> that buildmsi.bat isn't actually used for anything, and isn't "meant" to
> be run by users, but instead by build slaves.

Well, yes, although wouldn't we want the scripts to run without any user
editing, for the build slaves?

> One consequence is that these scripts build in debug mode (perhaps
> except for buildmsi), whereas end users would typically want to build in
> release mode.

Right.

Is there a reason the buildbot builds were stopped? I've added binary
file uploads to pony-build, so that platform-specific files can be
uploaded to build results -- see, e.g.

http://lyorn.idyll.org/ctb/pb-dev/p/python/10768/files/

or

http://lyorn.idyll.org/ctb/pb-dev/p/pygr/10805/files/

and I thought daily -latest builds for Python latest on Windows and Mac would
be a good demo...

Incidentally, the pony-build results & file upload scripts can be used from
within buildbot, if people are interested.

> Another consequence is that different committers actually use different
> procedures. Trent created much of the Tools/buildbot setup, so he is
> obviously in favor of that strategy. Christian Heimes created
> PCbuild/build_tkinter, so he would probably prefer to use that instead.
> When I took over Windows builds from Tim Peters, PCbuild/readme.txt
> was the official reference, and I try to stick to that.

OK, I see.  Thanks!  So if I can bring the scripts into concordance with
that it might be valuable?

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] /trunk test_distutils failing on Mac OS X 10.5

[C. Titus Brown]

Hi all,

/trunk test_distutils is failing with the following error on Mac OS X 10.5:

Here's the error:

test_distutils
test test_distutils failed -- Traceback (most recent call last):
File
"/private/tmp/tmp8UfLPT/python27/Lib/distutils/tests/test_sdist.py",
line 342, in test_make_distribution_owner_group 
self.assertEquals(member.gid, os.getgid())
AssertionError: 0 != 20

It has been a problem for over a week, perhaps longer.  I've filed it as:

http://bugs.python.org/issue7408

... So why am I posting this to python-dev?

I went to double-check this on the buildbots and noticed that there
aren't any Mac OS X buildbots.  I would be happy to give one or two
people remote access to my Mac OS X 10.5 iMac if someone wanted to
set up a buildbot and/or debug this issue further.

Tarek, I can give you access immediately through your lyorn account, too.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5

[C. Titus Brown]

On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote:
> > I went to double-check this on the buildbots and noticed that there
> > aren't any Mac OS X buildbots. 
> 
> That's not true, see
> 
> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk

Ahh!  Sorry, was searching for 'mac' somewhere in the string.

Those tests are also broken but in different areas from mine.  Again,
if people would like shell access, just ask.

best,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5

[C. Titus Brown]

On Sun, Nov 29, 2009 at 06:43:50PM +, Mark Dickinson wrote:
> On Sun, Nov 29, 2009 at 6:36 PM, C. Titus Brown  wrote:
> > On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote:
> >> That's not true, see
> >>
> >> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk
> >
> > Ahh! ??Sorry, was searching for 'mac' somewhere in the string.
> >
> > Those tests are also broken but in different areas from mine. ??Again,
> > if people would like shell access, just ask.
> 
> I think that buildbot is running OS X 10.6 (despite its name).  It
> would definitely be useful to have an OS X 10.5 buildbot as well,
> since there are significant differences:  most obviously, the
> default build on 10.6 is an LP64 build.
> 
> I'm not seeing the test_distutils failure you report on my own
> 10.5 machine, for some reason.

Yes, neither is Tarek... I'll see if I can track down why.  I don't
think I installed anything other than the minimum necessary to build
Python, but that's a bit worrisome.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] GSoC 2010 is on -- projects?

[C. Titus Brown]

Hi all,

once again, the PSF has been accepted as a mentoring foundation for the Google
Summer of Code!  This year, we're going to emphasize python 3 porting, so
please think of projects you'd like to see tackled.

Please submit ideas for projects as soon as possible, as students will be able
to start applying soon.  You can add them to this page:

   http://wiki.python.org/moin/SummerOfCode/2010

You can subscribe to the mentors list here,

   http://mail.python.org/mailman/listinfo/soc2010-mentors

and the general discussion list (students included) here:

   http://mail.python.org/mailman/listinfo/soc2010-general

thanks,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC 2010 is on -- projects?

[C. Titus Brown]

On Thu, Mar 18, 2010 at 10:13:42PM -0500, Benjamin Peterson wrote:
> 2010/3/18 C. Titus Brown :
> > Hi all,
> >
> > once again, the PSF has been accepted as a mentoring foundation for the 
> > Google
> > Summer of Code! ??This year, we're going to emphasize python 3 porting, so
> > please think of projects you'd like to see tackled.
> 
> This is not completely to the exclusion of other, projects, though,
> correct? For example, I think building a 64 bit PyPy backend would be
> a very worthy task.

Right -- emphasize only ;).  Good projects + good students + good mentors are
always welcome (and, actually, we'd like to focus on good students -- better
that something get done than that something particularly relevant get chosen
but NOT done by a poor student).

thanks,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC 2010 is on -- projects?

[C. Titus Brown]

On Sat, Mar 20, 2010 at 12:00:48AM +0100, "Martin v. L?wis" wrote:
> > Whether this is worth weeks of work or not will depend on a given
> > student's knowledge about Python 3, and I'd suspect that the GSoC would
> > be an opportunity for a number of applicant to actually learn the
> > intricacies of Python 3.
> > Developing Python 3-specific features could be used to increase the
> > amount of work on the project, but I am uncertain of whether this is
> > worth a full GSoC.
> 
> I'd say this would definitely make a GSoC project; any non-trivial
> porting will be.

Sounds good to me.  Line up a good student and bob's your uncle.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC 2010 is on -- projects?

[C. Titus Brown]

On Mon, Mar 29, 2010 at 10:40:06AM +0300, anatoly techtonik wrote:
> I would vote for allowing student work on community infrastructure
> tasks. Tracker, Wiki, Web site management tools are all outdated and
> everybody who cares agrees that they've seen a better tools.

As long as it's programming, it's allowed by Google.  So let's find a good
student or two, and outline a few good projects!

I do worry that that kind of work is difficult to evaluate, and requires really
great communication on both sides...

--titus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GSoC 2010 is on -- projects?

[C. Titus Brown]

On Wed, Mar 31, 2010 at 05:41:47PM +0900, Stephen J. Turnbull wrote:
> anatoly techtonik writes:
> 
>  > I would vote for allowing student work on community infrastructure
>  > tasks. Tracker, Wiki, Web site management tools are all outdated and
>  > everybody who cares agrees that they've seen a better tools.

[ munch ]

> I would recommend changing the tools only if *current* maintainers are
> either planning to step down (and so we face the problem of how to
> support maintenance in the future in any case), or are willing to
> supervise (ie, the people who will come back and fix problems in the
> future find the proposed "improvements" to be real improvements).  Eg,
> MvL should be intimately involved in any move to use different
> software for the tracker.  If the GSoC student(s) is (are) willing to
> work within the constraints of the existing software (ie, incremental
> improvements), then the constraints on mentors could be substantially
> relaxed to any of the senior folks who have recently contributed in
> those areas.

Agreed.

It'd be great to have students work on prototyping improvements, with an
eye to making changes that can then be evaluated in terms of maintainability,
extensibility, etc.  Having them actually change the infrastructure itself
seems to me like a bad idea :)

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Code coverage metrics

[C. Titus Brown]

On Tue, Apr 06, 2010 at 10:36:14PM +0300, anatoly techtonik wrote:
> On Tue, Apr 6, 2010 at 7:31 PM, Georg Brandl  wrote:
> >
> >>>> Where can I find public reports with Python tests code coverage?
> >>>
> >>> Here:
> >>>
> >>> http://coverage.livinglogic.de/
> >>
> >> Thank you. What is the status of getting these stats on python.org?
> >
> > Wouldn't "status" imply that there is a plan to do so?
> 
> It is not that hard to create that plan given that there is a fair
> amount of developers who care about code coverage.

Anatoly,

nonetheless, I don't know of any plan :).

I keep on shaving yaks when trying to get test coverage posted somewhere;
it's actually not a trivial problem to set it up automatically and
reliably, in my experience.  If you are interested in volunteering to
help, I'm sure it would be appreciated -- I suspect the right place to
start would be get test coverage running reproducibly and reliably on
your own machines and posted somewhere; then you could ask for permissions
to post it somewhere on *.python.org.  (I don't see why we care where it's
posted, but that's what you asked about.)

I keep on running into technical barriers in getting cross-platform code
coverage analysis working, which would be quite valuable; it's easy to
get it working once, but to keep it working is a maintenance task that
involves regular effort, again, in my experience.

best,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)

[C. Titus Brown]

On Wed, Apr 14, 2010 at 06:33:03AM -0400, Tres Seaver wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Martin v. L?wis wrote:
> > Tres Seaver wrote:
> >> Steve Holden wrote:
> >>> Why is it unavoidable that the Mac build will languish behind others?
> >>> Are we supporting MacOs or aren't we? If we are, why isn't the creation
> >>> of the build a part of the release process?
> >>> Clearly it's not a priority given that nobody has seen fit to (or had
> >>> time to) reply to this mail in three weeks.
> >> Maybe the PSF should make it a priority by funding acquisition of the
> >> appropriate proprietary hardware (Mac / Windows) for the release
> >> manager.  Otherwise the avaialbility of binaries is going to lag source
> >> releases forever.
> > 
> > Tres,
> > 
> > can you be more explicit? How would such hardware help (whom specifically)?
> 
> I assumed that creation of installer binaries for a release depends on
> having the release manager or a lieutenant have access to the given
> platform (Windows, OS/X) and tools,  For instance, the RM or lieutenant
> might only have access to such a platform part-time (e.g., only while at
> work, or only at home).  In such a case, providing additional hardware
> could expedite creation of the binaries.

As with Windows, I personally find that building Python with all the associated
libraries is blocked on getting the right libraries installed, not on
getting the compilers (which are available to us for free) ... but I'm sure
that easy access to hardware is an issue, too.

I can provide command-line access to a Mac OS X machine but I'm not sure that's
enough.  Let me know if anyone wants that.

Separately, I'd be happy to put forward a proposal to the PSF to fund RMs and
their lieutenants with a Mac or a PC, whichever they needed to keep things
moving.  It's the least "we" can do, IMO, and hardware is just not that
expensive compared to the dedication of the volunteers.  If Georg, Benjamin,
Martin, or Ronald are interested, please just tell me (or Steve, or the PSF
board, or ...) what you want and I'll work on getting it funded.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)

[C. Titus Brown]

On Wed, Apr 14, 2010 at 07:36:25AM +0200, "Martin v. L?wis" wrote:
> >> In a wider sense of "to support", MacOS is certainly supported by
> >> Python. There is everything in the source code that you need to make
> >> Python run on a Mac. Just download the sources and compile them yourself.
> >>
> > And yet we don't regard the Windows release as complete until you have
> > built the binaries (for which service you deserve many thanks, by the way).
> 
> This phenomenon exists for a lot of other systems, as well. For example,
> we also support Solaris, but stopped providing Solaris binaries since
> Python 1.5 (when I last built binaries for Das Python-Buch). People
> still can get Solaris binaries from ActiveState or Sunfreeware; Sun also
> ships Python as part of the system.

I personally think the Mac is pretty important, as one of the big three
consumer operating systems...

> > Is the Mac platform one on which users will be happy to compile from
> > source? I know its users are savvier than Windows users, and have a
> > better tool set available to them, but they still seem to expect
> > downloadable installers.
> 
> The major difference in the "do it yourself" attitude is that Mac user
> get a compiler for free, as part of the operating system release,
> whereas for Windows, they have to pay for it (leaving alone VS Express
> for the moment).

Actually, I think the more pernicious factor is that a version of Python comes
pre-installed on Mac OS X, which means the up-front demand is lower
for a pre-compiled version.  This is problematic, though, because that
version of Python only gets upgraded with full releases of Mac OS X
(which are not very well correlated with releases of Python, of course).
So we have lots of Python installs out there that, in the absence of
a precompiled binary version, can't be upgraded without installing
the developer tools.

> However, the real difference is motivation for contribution to open
> source projects. You normally contribute to scratch an itch.
> Unfortunately, these binaries don't come out such a motiviation. So the
> release manager roles are either altruistic, or rely on extrinsic
> motivations (money, reputation).

I don't know what to do about motivation but if there are barriers that
we can lower, please let me know.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Automatic installer builds (was Re: Fwd: Broken link to download (Mac OS X))

[C. Titus Brown]

On Thu, Apr 15, 2010 at 12:47:50AM +1000, Nick Coghlan wrote:
> Martin v. L?wis wrote:
> >> Wasn't that problem fixed weeks ago?  The installer image has been 
> >> available there since several days after the release.  And the link 
> >> seems fine now.
> > 
> > The inherent problem remains. There is no binary for 2.7b1, for example.
> > The last binaries produced in the 2.7 testing process were for 2.7a2.
> 
> I seem to recall there was some issue (aside from the current lack of a
> reliable OS X buildbot) that prevented us from regularly grabbing the
> head of the tree and using it to automatically build the Windows and Mac
> installers (to check that the installers could actually be created,
> preventing a repeat of the Mac situation with 2.7b1).
> 
> Am I misremembering the discussion from last time this topic came up?

Making the Windows binary build process automatic and robust is challenging
if you hate Windows (like I do).  Martin also made the point that it's
been broken forever, so people don't seem to care :).  ISTR Martin just
makes them manually.

It's on my TODO list to robustify this process.

OTOH, my Mac automated builds/tests are working just fine.  I could produce
nightly binaries from those easily enough:

http://lyorn.idyll.org/ctb/pb-dev/p/python/

Just need to figure out the magic doohickey commands... will add to list.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)

[C. Titus Brown]

On Wed, Apr 14, 2010 at 09:37:34AM -0700, Bill Janssen wrote:
> Michael Foord  wrote:
> 
> > > Isn't that just a matter of having the recipe written down somewhere?
> > >
> > 
> > Yes, that would be nice. :-) Preferably a recipe that doesn't involve
> > Macports or Fink which some of us are allergic to.
> 
> Yes, ditto the MacPorts/Fink allergy.
> 
> All we need is a script, right?  The released branches should be built
> automatically every night anyway, just for regression testing.

Please pass it along to me, when you get it working... I build python2.7
nightly on Mac OS X, but just at the command line.

see:

http://lyorn.idyll.org/ctb/pb-dev/p/python/

http://lyorn.idyll.org/ctb/pb-dev/p/python/show_all

(the Windows build is flaky for me, so the 'show_all' shows mostly
Windows builds).

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Fwd: Broken link to download (Mac OS X)

[C. Titus Brown]

On Wed, Apr 14, 2010 at 06:52:46PM +0200, Michael Foord wrote:
> On 14/04/2010 18:49, Steve Holden wrote:
>> Bill Janssen wrote:
>>
>>> Michael Foord  wrote:
>>>
>>>  
>>>>> Isn't that just a matter of having the recipe written down somewhere?
>>>>>
>>>>>  
>>>> Yes, that would be nice. :-) Preferably a recipe that doesn't involve
>>>> Macports or Fink which some of us are allergic to.
>>>>
>>> Yes, ditto the MacPorts/Fink allergy.
>>>
>>> All we need is a script, right?  The released branches should be built
>>> automatically every night anyway, just for regression testing.
>>>
>>> Perhaps we should take this up on Mac-Python?
>>>
>>>  
>> Please do, and let me know if resources of some kind would help.
>>
>
> A Mac buildbot would *definitely* be useful and I know some of the  
> Python-dev crew would like access to a Mac OS X machine for trying out  
> fixes when none of the core Mac maintainers are around. A couple of  
> co-located, or otherwise hosted, Mac boxes would be very useful.
>
> The obvious question is who would hold the keys (maintain the buildbot)?  
> I don't know if Ronald has the spare capacity to do this (?). If someone  
> will help me with the initial setup I can otherwise volunteer.

We have some space in our machine room, and some sysadmins that like open
source.  I will ask them if they are willing to do physical maintenance (profs
wisely aren't allowed access to the machine room).  That would really be
ideal... I will report back to interested people.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Anyone can do patch reviews

[C. Titus Brown]

On Tue, Apr 27, 2010 at 03:23:19PM -0400, Tres Seaver wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Antoine Pitrou wrote:
> > Tres Seaver  palladion.com> writes:
> >> This is an excellent set of guidelines.  The only drawback I see here is
> >> that the current VCS situation makes doing the review more tedious than
> >> it should be, especially for non-committers.  Or maybe the Hg mirrors
> >> are truly up-to-date and working?  Last I looked, they were lagging or
> >> unavailable.
> > 
> > If you only a review a patch (rather than say maintain and evolve it), 
> > there's
> > no point in using hg rather than SVN.
> 
> Hmm, it feels exactly the other way around to me:  working with the DVCS
> tools while reviewiing a patch allows me to be more productive (e.g.,
> using 'bzr shelve' or the equivalent hg subcommand).
> 
> Making a local branch / clone for each issue also feels more natural
> than working in a read-only SVN checkout.

+1.  I find it to be an excellent way to muck around with patches and
make my own changes / diffs / etc. for a review process.  (Not that I
do any Python reviews, note.  But it's a great technique in general.)

It's also fantastically simple and esay to interact with patches that are
branches on someone's bitbucket or github repo; much better than uploading and
downloading patch files while in the middle of a discussion.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FHS compliance of Python installation

[C. Titus Brown]

On Sat, Jun 26, 2010 at 10:25:28PM +0200, Matthias Klose wrote:
> On 25.06.2010 02:54, Ben Finney wrote:
>> James Y Knight  writes:
>>
>>> Really, python should store the .py files in /usr/share/python/, the
>>> .so files in /usr/lib/x86_64- linux-gnu/python2.5-debug/, and the .pyc
>>> files in /var/lib/python2.5- debug. But python doesn't work like that.
>>
>> +1
>>
>> So who's going to draft the ???Filesystem Hierarchy Standard compliance???
>> PEP? :-)
>
> This has nothing to do with the FHS.  The FHS talks about data, not code.

Really?  It has some guidelines here for object files, etc., at least as
of 2004.

http://www.pathname.com/fhs/pub/fhs-2.3.html

A quick scan suggests /usr/lib is the right place to look:

http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration)

[C. Titus Brown]

On Thu, Jul 01, 2010 at 09:31:13AM -0500, Daniel Stutzbach wrote:
> On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik wrote:
> 
> > 4. Even if I make patch in my Mercurial clone - you still can't pull
> > it and I have to attach it to tracker. No gain.
> >
> 
> Was there ever any discussion about hosting the central repository on a site
> such as bitbucket or github?  I tried searching the python-dev archives but
> was unable to find much.

We discussed this briefly at the Python language summit at PyCon '10, the
discussion intersected and/or merged with the whole infrastructure discussion,
got borg-ified by that, and I think discussions continue(d) on pydotorg-www in
general.

Personally I think it makes a great deal of sense to pay someone to manage
this for us, but there were concerns about customizability and control that
warrant further investigation.  Since it's DVCS, it's much easier to change
the "official" repo, but it still incurs costs in terms of documentation,
workflow changes, etc.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] query: docstring formatting in python distutils code

[C. Titus Brown]

Hi all,

over on the fellowship o' the packaging mailing list, one of our GSoC students
(merwok) asked about how much formatting info should go into Python stdlib
docstrings.  Right now the stdlib docstrings are primarily text, AFAIK; but
with the switch to Sphinx for the official Python docs, should we permit
ReST-general and/or Sphinx-specific markup in docstrings?

Hmm, I don't actually see that the stdlib docstrings are imported into the
Python documentation anywhere, so maybe the use of Sphinx isn't that
relevant.  But how about ReST in general?

See

http://sphinx.pocoo.org/markup/index.html

for sphinx-specific markup constructs.

thanks,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] query: docstring formatting in python distutils code

[C. Titus Brown]

On Wed, Jul 07, 2010 at 09:36:10PM +0530, Shashwat Anand wrote:
> On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown  wrote:
> 
> > Hi all,
> >
> > over on the fellowship o' the packaging mailing list, one of our GSoC
> > students
> > (merwok) asked about how much formatting info should go into Python stdlib
> > docstrings.  Right now the stdlib docstrings are primarily text, AFAIK; but
> > with the switch to Sphinx for the official Python docs, should we permit
> > ReST-general and/or Sphinx-specific markup in docstrings?
> >
> > Hmm, I don't actually see that the stdlib docstrings are imported into the
> > Python documentation anywhere, so maybe the use of Sphinx isn't that
> > relevant.  But how about ReST in general?
> 
> So will we be able to use .__docs__ within python interpretor, which is
> quite handy feature.
> >>> print(os.getcwd.__doc__)
> getcwd() -> path
> 
> Return a string representing the current working directory.
> Also some python interpretors like bpython uses it ; a snapshot here -  h
> ttp://cl.ly/c5bb3be4a01d9d44732f
> So will it be ok to break them ?

I don't understand...

Frist, you can already use

help(os.getcwd)

to get the same result.

Second, what would we be breaking?  We'd be making the straight text
representation a bit more cluttered in return for adding certain kinds
of meta-information into the markup.  I think it's a judgement call...

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] query: docstring formatting in python distutils code

[C. Titus Brown]

On Wed, Jul 07, 2010 at 05:09:40PM +0100, Michael Foord wrote:
> On 07/07/2010 17:06, Shashwat Anand wrote:
>> On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown > <mailto:c...@msu.edu>> wrote:
>>
>> Hi all,
>>
>> over on the fellowship o' the packaging mailing list, one of our
>> GSoC students
>> (merwok) asked about how much formatting info should go into
>> Python stdlib
>> docstrings.  Right now the stdlib docstrings are primarily text,
>> AFAIK; but
>> with the switch to Sphinx for the official Python docs, should we
>> permit
>> ReST-general and/or Sphinx-specific markup in docstrings?
>>
>> Hmm, I don't actually see that the stdlib docstrings are imported
>> into the
>> Python documentation anywhere, so maybe the use of Sphinx isn't that
>> relevant.  But how about ReST in general?
>>
>>
>> So will we be able to use .__docs__ within python interpretor, which  
>> is quite handy feature.
>> >>> print(os.getcwd.__doc__)
>> getcwd() -> path
>>
>> Return a string representing the current working directory.
>> Also some python interpretors like bpython uses it ; a snapshot here -  
>>  http://cl.ly/c5bb3be4a01d9d44732f
>> So will it be ok to break them ?
>
> Using ReST won't *break* these tools, but may make the output less  
> readable.
>
> I would say that the major use of docstrings is for interactive help -  
> so interactive readability should be *the most important* (but perhaps  
> not only) factor when considering how to format standard library 
> docstrings.

OK.

I guess docutils isn't in the stdlib (should it be?) or else we could modify
'help' to use it to prepare a straight text formatting.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Continuing 2.x

[C. Titus Brown]

On Fri, Oct 29, 2010 at 06:57:54PM +0200, "Martin v. L?wis" wrote:
> > "Infrastructure" sounds to me like code for "money".
> 
> No, it's rather "volunteer time". Of course, people keep proposing
> that this should be replaced by hired time that gets paid from
> donations, but all such proposals so far got stuck at implementation
> details (i.e. it's actual work that nobody has done).
> 
> > How much of the
> > PSF's money, for instance, comes from organizations whose primary
> > interest is still Python2?  How many of them are only or principally
> > only interested in Python3?  Then again, how much of the PSF's budget
> > goes toward infrastructure?
> 
> The first two questions are difficult to answer: the PSF doesn't
> maintain records of what Python versions are of primary interest
> to sponsor members. A significant portion of the donations comes
> from the conference surplus (being saved for the also-likely risk
> of a massive conference loss); in this case, it's even difficult to
> identify the donors (as you can't really attribute the surplus to
> being from, say, attendee fees, as opposed to conference sponsors).
> 
> As for the budget that goes into infrastructure: you'll find the details
> in the treasurer reports, but it is comparatively minor and goes
> primarily into hardware purchases. Connectivity and colocation is
> donated by companies who may not have an actual interest in Python
> at all (e.g. XS4ALL, which do this out of a general support for
> free software and in positive recollection of their former employee
> Thomas Wouters).

I'd just like to add my 2c that AFAICT the volunteer effort that goes into
Python, and in particular into python-dev and the infrastructure foo,
absolutely *dwarfs* all other aspects of "official" Python and PSF (including
$$ in all forms).

So, good job, -dev guys!

But they're already pretty overwhelmed.  Independent of talk, unless there's a
proposal to continue 2.x that actually involves someone *new* stepping up to
put in hugely substantial and ridiculously large amounts of seriously expert
time, I don't see the point of talking about it.

cheers,
--titus

p.s. I would be happy to enter into discussions on how to clone Martin and
others, though.  I just need some epithelial cells, I think.  And about $20 bn
dollars, and relocation to Israel (which I think has the best combination of
tech and human use guidelines for cloning).  Martin's permission is not
*strictly* necessary but should probably be obtained, too.

p.p.s.  The PSF isn't foolish enough to let me speak for them, in case
anyone is wondering.

-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Cleaning-up the new unittest API

[C. Titus Brown]

On Mon, Nov 01, 2010 at 02:37:33PM -0400, Terry Reedy wrote:
> On 10/31/2010 10:55 PM, Michael Foord wrote:
>
>> fact that sets / frozensets can't be sorted in the standard Python way
>> (their less than comparison adheres to the set definition). This is
>> something that will probably surprise many Python developers:
>
> Any programmer who sorts (or uses functions that depend on proper  
> sorting) should know and respect the difference between partial orders,  
> such as set inclusion, and total orders, such as lex order of sequences.  
> So I am surprised by the above claim ;-).

Huh.  Count me out.  I guess I don't live up to your standards.

--titus

p.s. Seriously?  I can accept that there's a rational minimalist argument
for this "feature", but arguing that it's somehow the responsibility of
a programmer to *expect* this seems kind of whack.
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Cleaning-up the new unittest API

[C. Titus Brown]

On Tue, Nov 02, 2010 at 05:28:43PM +0900, Stephen J. Turnbull wrote:
> C. Titus Brown writes:
> 
>  > p.s. Seriously?  I can accept that there's a rational minimalist argument
>  > for this "feature",
> 
> It is a feature, even if you aren't gonna need it.  I want it.
> 
> Many programmers do know that sets are partially ordered by inclusion,
> preordered by size, and (in Python) totally ordered by memory address.
> There's nothing wrong with not knowing that -- these are rather
> abstract mathematical concepts.  But it's very useful that sorted() or
> .sort() use <=, and it's very useful that Python so often obeys simple
> consistent rules, and it would be quite confusing to those who do
> understand that "in Python the set type is partially ordered by
> inclusion" if sorted() used some other relation to order collections
> of sets.
> 
> It's not so hard to change this:

[ ... ]

> If there were an obvious way to compare sets for use in sorting, that
> way would very likely be the most useful definition for <=, too.  But
> there isn't, really (it's pretty obvious that comparing memory
> addresses is implausible, but otherwise, there are lots of candidates
> that are at least sometimes useful).  Do you think otherwise?  If so,
> what do you propose for the OOWTDI of sorting a collection of sets?

I don't have one...

>  > but arguing that it's somehow the responsibility of a programmer to
>  > *expect* this seems kind of whack.
> 
> I don't quite agree that everyone should "expect exactly the
> implemented behavior", but I do think it's a Python *programmer's*
> responsibility to refrain from expecting something else in this case.

...but, as someone who has to figure out how to teach stuff to CSE undergrads
(and biology grads) I hate the statement "...any programmer should
expect this..." because (unless you're going to disqualify a huge swathe of
people from being programmers) it's *just not true*.  I don't expect Python
to cater to the lowest common denominator but we should be mindful of our
audience, too.

I think Python has a great advantage in not being too surprising much of the
time, which helps quite a bit with learning. I hope people keep that in mind
for future features.

cheers,
--t
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snakebite, buildbot and low hanging fruit -- feedback wanted! (Was Re: SSH access against buildbot boxes)

[C. Titus Brown]

On Sun, Nov 07, 2010 at 09:32:33PM -0500, Scott Dial wrote:
> On 11/7/2010 7:09 PM, Martin v. L?wis wrote:
> >> Luckily, the problems that we faced 2.5 years ago when I came up with
> >> the idea of Snakebite are still just as ever present today ;-)
> > 
> > Is this bashing of existing infrastructure really necessary?
> > People (like me) might start bashing about vaporware and how
> > a bird in the hand is worth two in the bush. Cooperate, don't
> > confront.
> 
> +1 Respect your (software) elders.
> 
> The Snaketbite rhetoric has always been less than generous with regard
> to Buildbot, but Buildbot has been providing an infinitely more useful
> service to the community for much longer than Snakebite has for those
> 2.5 years.

Yes, yes, I agree that some graciousness is a good idea.

Oh, wait... you're not helping.

Anyway, I think buildbot is a good local optimum for python-dev, largely
because it's maintained by someone who cares enough to do it well.  And, if
Trent had been talking about buildbot only, MvL's comment would be more than
fair.  But Trent, and I, and others, have talked about quite a bit more than
buildbot being "the" problem. Things like enabling *and maintaining* easy EC2
spin-up with buildbot, or providing SSH key access, or making a 'try' server
available and maintaining it, would be clearly beneficial.  And that's some of
what Trent has been talking about providing.  It turns out it's hard to do
without lots and lots of time and money.  If you truly think it's not useful,
I'd be interested in hearing your opinions, because we've spent an ungodly
amount of the above on it.

In the larger context, I worry very much that we're settling for a rather
suboptimal support setup (on svn, and on cont integration, and on some other
aspects of Python infrastructure) because the current maintainers are so
overloaded and few others are stepping up to bear burdens.  This is a big
concern of at least some people in the PSF.  But it's not an easy problem to
solve - quelle surprise.  And I'm not in a personal position to help, so I've
basically tried to shut up about it :).

As for buildbot, I've been pretty hard on buildbot myself, and I'm happy to
justify it to others -- I've done so in public fora so I'm sure you can find
the records, if you care to look.  But it's not really very relevant to this
conversation, especially since Trent has always been interested in building off
the buildbot setup rather than replacing it.

--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Buildbot for AIX

[C. Titus Brown]

On Mon, Nov 08, 2010 at 06:50:32PM +0100, Antoine Pitrou wrote:
> > However running 2 different slaves per host in order to distinguish xlc 
> > and gcc would be OK; though I would appreciate if they could run 
> > sequentially rather than in parallel as that would limit the host load.
> 
> If there are two separate slaves, I can't think of any simple way to run
> builds sequentially. Perhaps you can assign both of them to a single CPU
> (assuming AIX allows that).

You can specify a slave lock to do this, in buildbot:

http://buildbot.net/buildbot/docs/0.8.1/Interlocks.html

One the neat things that a master/slave system like buildbot provides...

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Addition of "pyprocessing" module to standard lib.

[C. Titus Brown]

On Tue, May 13, 2008 at 09:23:02PM -0400, Tom Pinckney wrote:
-> Why not use MPI? It's cross platform, cross language and very widely  
-> supported already. And there're Python bindings already.

MPI requires far more setup than the pyprocessing module does; it's by
no means plug & play.

--titus
-- 
C. Titus Brown, [EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement?

[C. Titus Brown]

On Wed, Jun 18, 2008 at 10:55:32PM -0700, Alex Martelli wrote:
-> On Wed, Jun 18, 2008 at 9:58 PM, Cesare Di Mauro <[EMAIL PROTECTED]> wrote:
-> > Very very, interesting. Thanks. :)
-> >
-> > Somebody thinks that Python is unsuitable to implement a DSL: IMO your 
example prove the contrary. :D
-> 
-> As long as you're willing to do the "DSL" within the strictures of
-> Python syntax, it's OK - not quite as flexible as LISP or Scheme or
-> even Ruby, but better than most;-).  I did design and implement "DSL"s
-> in Python (mostly specialized class trees with suitable metaclasses,
-> decorators &c) in many jobs in my recent past, I'll admit -- it never
-> feels as free-flowing as Scheme did back when I used THAT, but, close
-> enough to make my jobs successful!-)

It's pretty easy to put a simple syntax remapper around Python, too;
this is what twill does.  So, for example, this:

--
setlocal username 
setlocal password 

go http://www.slashdot.org/
formvalue 1 unickname $username
formvalue 1 upasswd $password
submit

code 200 # make sure form submission is correct!
--

translates very directly to

---
setlocal('username', '...')
setlocal('password', '...')

go('http://www.slashdot.org/')
formvalue('1', 'unickname', username)
formvalue('1', 'upasswd', password)
submit()

code('200')
---

which is DSL-ish enough for me...

More generally, I've never understood why some people insist that
certain features make Ruby better for DSLs -- are code blocks really
that important to DSLs?  Or is it just the lack of parens??

--titus
-- 
C. Titus Brown, [EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed unittest changes

[C. Titus Brown]

On Mon, Jul 14, 2008 at 09:37:30PM -0700, Raymond Hettinger wrote:
-> From: "Michael Foord" <[EMAIL PROTECTED]>
-> >Maybe Python needs a good mocking module in the standard library. There 
-> >are plenty, but we use a particularly nice one at Resolver Systems [1]. :-)
-> 
-> -1
-> 
-> This comes up occassionally and gets shot down.
-> http://bugs.python.org/issue708125
-> 
-> Mock objects mean different things to different people.
-> Some expect more simulated behavior and others want less.
-> It's rare to find agreement about general purpose mock objects and 
-> frameworks.
-> Mock libraries create their own complexities and burdens on a programmer's 
-> memory.
-> It's often easier to create a small special case mock object
-> than to remember how to configure a general purpose one.
-> And, afaict, there is no fan club for some particular python mock
-> object library -- it seems to only come up in general discussions
-> about possibilities for growing the unittest module, and almost
-> never comes up in the context of solving a real problem that
-> hasn't already be addressed in some other way.

Also see:

http://lists.idyll.org/pipermail/testing-in-python/2007-November/000406.html

& associated thread, for those interested in the variety of mock
libraries...

cheers,
--titus
-- 
C. Titus Brown, [EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP: Consolidating names and classes in the `unittest` module (updated 2008-07-15)

[C. Titus Brown]

On Tue, Jul 15, 2008 at 03:00:30PM +0100, Richard Thomas wrote:
-> On Tue, Jul 15, 2008 at 2:48 PM, Ben Finney <[EMAIL PROTECTED]> wrote:
-> > So far I have "precedent and tradition" and "positive admonition looks
-> > better" in support of preferring the 'assert*' names. Are there any
-> > others?
-> 
-> I've been told by a couple of non-programmers that "failUnless" is
-> more intuitive than "assert" if only for the reason that its unclear
-> what "assert" might do. This is similar to one of the arguments raised
-> in the PEP, but considered from the point of view of someone new to
-> test frameworks it could be all the more important.

Maybe this is an unnecessarily hard line, but if someone doesn't know
what "assert" does, then they should go look up the keyword -- it's
part of Python (and present in C, C++, Perl, and PHP as well).  So
unless the person is new to Python AND testing, they should know it.

-> On another note, while I understand that consistency is a good thing
-> is it really that important in test suites? Obviously the unittest
-> module itself should be internally consistent but why not provide
-> people using the all the synonyms they might want? I can see people
-> just wrapping TestCase to add the synonyms back in.

While I don't have a strong position on the assert* vs fail*, I think
consistency and simplicity in test suites is at least as important as in
code: if you have to work hard to understand and debug your test suites,
you've done something seriously wrong in building your tests.

Tests should be as simple as possible, and no simpler.

--titus
-- 
C. Titus Brown, [EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com