Re: [Python-Dev] Massive test_sqlite failure on Mac OSX ... sometimes

2006-10-22 Thread Ronald Oussoren


On Oct 21, 2006, at 8:03 PM, [EMAIL PROTECTED] wrote:


Followup #2...

Yesterday I whittled my problems with test_sqlite on my OSX g5 to
test_ctypes and test_sqlite:

./python.exe Lib/test/regrtest.py -l -f tests
test_ctypes
test_sqlite
test test_sqlite failed -- errors occurred; run in verbose mode  
for details

1 test OK.
1 test failed:
test_sqlite

Today I refined things further.  I renamed all the test_*.py files in
Lib/ctypes/test/ until all I was left with was test_find.py.  It  
fails if

that's the only ctypes test script run:

$ ls -l *.py
-rw---   1 buildbot  buildbot  6870 Oct 20 06:30 __init__.py
-rw---   1 buildbot  buildbot   624 Oct 20 06:30 runtests.py
-rw---   1 buildbot  buildbot  3463 Oct 21 12:52 test_find.py
montanaro:~/pybot/trunk.montanaro-g5/build/Lib/ctypes/test  
buildbot$ cd -

/Library/Buildbot/pybot/trunk.montanaro-g5/build
montanaro:~/pybot/trunk.montanaro-g5/build buildbot$ ./ 
python.exe Lib/test/regrtest.py -l -f tests

test_ctypes
test_sqlite
test test_sqlite failed -- errors occurred; run in verbose mode  
for details

1 test OK.
1 test failed:
test_sqlite

test_find.py contains checks for three OpenGL libraries on darwin:  
gl, glu
and glut.  If I comment out all those tests, test_sqlite succeeds.   
If any

of them are enabled, test_sqlite fails.


According to a comment in (IIRC) the pyOpenGL sources GLUT on OSX  
does a chdir() during initialization, that could be the problem here.


Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] readlink and unicode strings (SF:1580674) Patch http://www.python.org/sf/1580674 fixes readlink's behaviour w.r.t. Unicode strings: without this patch this function uses the system defaul

2006-10-22 Thread Ronald Oussoren
Patch http://www.python.org/sf/1580674 fixes readlink's behaviour  
w.r.t. Unicode strings: without this patch this function uses the  
system default encoding instead of the filesystem encoding to convert  
Unicode objects to plain strings. Like os.listdir, os.readlink will  
now return a Unicode object when the argument is a Unicode object.


What I'd like to know is if this can be backported to the 2.5 branch.  
The first part of this patch (use filesystem encoding instead of the  
system encoding) is IMHO a bugfix, the second part might break  
existing applications (that might not  expect a unicode result from  
os.readlink).


The reason I did this patch is that os.path.realpath currently breaks  
when the path is a unicode string with non-ascii characters and at  
least one element of the path is a symlink.


Ronald

smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] readlink and unicode strings (SF:1580674) Patch http://www.python.org/sf/1580674 fixes readlink's behaviour w.r.t. Unicode strings: without this patch this function uses the system de

2006-10-22 Thread M.-A. Lemburg
Ronald Oussoren wrote:
> Patch http://www.python.org/sf/1580674 fixes readlink's behaviour w.r.t.
> Unicode strings: without this patch this function uses the system
> default encoding instead of the filesystem encoding to convert Unicode
> objects to plain strings. Like os.listdir, os.readlink will now return a
> Unicode object when the argument is a Unicode object.
> 
> What I'd like to know is if this can be backported to the 2.5 branch.
> The first part of this patch (use filesystem encoding instead of the
> system encoding) is IMHO a bugfix, the second part might break existing
> applications (that might not  expect a unicode result from os.readlink).
> 
> The reason I did this patch is that os.path.realpath currently breaks
> when the path is a unicode string with non-ascii characters and at least
> one element of the path is a symlink.

I don't think that an application that passes a Unicode object to
os.readlink() would have problems dealing with a Unicode return
value.

+1 on backporting it to 2.5.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 22 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] readlink and unicode strings (SF:1580674)

2006-10-22 Thread Ronald Oussoren


On Oct 22, 2006, at 12:54 PM, Ronald Oussoren wrote a message with an  
annoyingly large subject...


Sorry about that, I guess it's time to book a course on basic  
computer usage :-(


Ronald



smime.p7s
Description: S/MIME cryptographic signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Massive test_sqlite failure on Mac OSX ... sometimes

2006-10-22 Thread skip

Ronald> According to a comment in (IIRC) the pyOpenGL sources GLUT on
Ronald> OSX does a chdir() during initialization, that could be the
Ronald> problem here.

How would that explain that it fails on my g5 but not on my powerbook?  They
are at the same revision of the operating system and compiler.  The
checksums on the libraries are different though the file sizes are the same.
The dates on the files are different as well.  I suspect the checksum
difference is caused by the different upgrade dates of the two machines and
the resulting different times the two systems were "optimized".

Is there anyone else with a g5 who can do a vanilla Unix (not framework)
build on an up-to-date g5 from an up-to-date Subversion repository?  It
would be nice if someone else could at least confirm or not confirm this
problem.

Skip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PSF Infrastructure has chosen Roundup as the issue tracker for Python development

2006-10-22 Thread Brett Cannon
Forgot to send this to python-dev.  =)-- Forwarded message --From: Brett Cannon <[EMAIL PROTECTED]
>Date: Oct 20, 2006 1:35 PMSubject: PSF Infrastructure has chosen Roundup as the issue tracker for Python developmentTo: python-list@python.orgAt the beginning of the month the PSF Infrastructure committee announced that we had reached the decision that JIRA was our recommendation for the next issue tracker for Python development.  Realizing, though, that it was a tough call between JIRA and 
Roundup we said that we would be willing to switch our recommendation to Roundup if enough volunteers stepped forward to help administer the tracker, thus negating Atlassian's offer of free managed hosting.
Well, the community stepped up to the challenge and we got plenty of volunteers!  In fact, the call for volunteers has led to an offer for professional hosting for Roundup
 from Upfront Systems.  The committee is currently evaluating that offer and will hopefully have a decision made soon.  Once a decision has been made we will contact the volunteers as to whom we have selected to help administer the installation (regardless of who hosts the tracker).  The administrators and python-dev can then begin working towards deciding what we want from the tracker and its configuration.
Once again, thanks to the volunteers for stepping forward to make this happen!-Brett CannonPSF Infrastructure committee chairman


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Massive test_sqlite failure on Mac OSX ... sometimes

2006-10-22 Thread Jean-Paul Calderone
On Sun, 22 Oct 2006 07:51:27 -0500, [EMAIL PROTECTED] wrote:
>
>Ronald> According to a comment in (IIRC) the pyOpenGL sources GLUT on
>Ronald> OSX does a chdir() during initialization, that could be the
>Ronald> problem here.
>
>How would that explain that it fails on my g5 but not on my powerbook?  They
>are at the same revision of the operating system and compiler.  The
>checksums on the libraries are different though the file sizes are the same.
>The dates on the files are different as well.  I suspect the checksum
>difference is caused by the different upgrade dates of the two machines and
>the resulting different times the two systems were "optimized".
>
>Is there anyone else with a g5 who can do a vanilla Unix (not framework)
>build on an up-to-date g5 from an up-to-date Subversion repository?  It
>would be nice if someone else could at least confirm or not confirm this
>problem.

Robert Gravina has seen a problem which bears some resemblance to this one
while using PySQLite in a real application on OS X.  I've pointed him to
this thread; hopefully it's the same issue and a second way of producing
the issue will shed some more light on the matter.

The top of that thread is available here:

http://divmod.org/users/mailman.twistd/pipermail/divmod-dev/2006-October/000707.html

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PSF Infrastructure has chosen Roundup as the issue tracker for Python development

2006-10-22 Thread Anthony Baxter
Thanks to the folks involved in this prcocess - I'm looking forward to getting 
the hell away from SF's bug tracker. :-)

Anthony
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PSF Infrastructure has chosen Roundup as the issue tracker for Python development

2006-10-22 Thread Talin
Anthony Baxter wrote:
> Thanks to the folks involved in this prcocess - I'm looking forward to 
> getting 
> the hell away from SF's bug tracker. :-)

Yes, let us know when the new tracker is up, I want to start using it :)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Massive test_sqlite failure on Mac OSX ... sometimes

2006-10-22 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Oct 22, 2006, at 8:51 AM, [EMAIL PROTECTED] wrote:

> Is there anyone else with a g5 who can do a vanilla Unix (not  
> framework)
> build on an up-to-date g5 from an up-to-date Subversion  
> repository?  It
> would be nice if someone else could at least confirm or not confirm  
> this
> problem.

By "vanilla unix" you mean a pretty simple ./configure; make; make test?

Works for me with Python 2.5 on both my G5s and Intel Macs, all  
running 10.4.8.  Note though that I usually build with CPPFLAGS and  
LDFLAGS pointing to /opt/local in order to pick up DarwinPorts  
readline, and you do the same and have a version of sqlite from there  
you can have problems.

For example, we were seeing some very odd infloops in our sqlite  
layer.  We have our own version of sqlite that we expected to be  
dynamically linked against, but when I used otool -L to check it, I  
realized we were dynamically linked against a version of sqlite in  
DarwinPorts.  Getting rid of the unnecessary DarwinPorts version and  
making sure that we were dynamically linking against our version  
eliminated the infloops.

What do you get when you check _sqlite3?

% otool -L build/lib.macosx-10.3-ppc-2.5/_sqlite3.so
build/lib.macosx-10.3-ppc-2.5/_sqlite3.so:
 /usr/lib/libsqlite3.0.dylib (compatibility version 9.0.0,  
current version 9.6.0)
 /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current  
version 92.0.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,  
current version 88.1.7)

Any possibility something like that's going on?
- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRTwSqHEjvBPtnXfVAQLvwQP/VuTQwwXwsauiuQt8E3k05scWsykarLaZ
YMJyVwq++DH/X8C5RODG9seYhSMQLF8PKMStmhKWLmlQ9mfFPIobMgsFqXBuI+bD
njUOh74O6vcJw1RNKXaERdQ6ABb2t79S6w+Psu5hGOP1NDy/e9GQazw05HpJWWvG
7Py+bDt24oE=
=9TjL
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Massive test_sqlite failure on Mac OSX ... sometimes

2006-10-22 Thread skip

Barry> What do you get when you check _sqlite3?

$ otool -L ./build/lib.mac-10.3-ppc-2.6/_sqlite3.so
./build/lib.macosx-10.3-ppc-2.6/_sqlite3.so:
/usr/local/lib/libsqlite3.0.dylib (compatibility version 9.0.0, 
current version 9.6.0)
/usr/lib/libmx.A.dylib (compatibility version 1.0.0, current 
version 93.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 88.1.7)

Which I apparently installed on Oct 15 but seem to have forgotten...
According to the source in my directory, it's sqlite 3.3.8.  On my powerbook
it's linked against /usr/lib/libsqlite3.0.dylib...

Make clean, run the failing test pair, now it's fine.  Otool shows linkage
against /usr/lib/libsqlite3.0.dylib...:

$ otool -L ./build/lib.macosx-10.3-ppc-2.6/_sqlite3.so
./build/lib.macosx-10.3-ppc-2.6/_sqlite3.so:
/usr/lib/libsqlite3.0.dylib (compatibility version 9.0.0, current 
version 9.6.0)
/usr/lib/libmx.A.dylib (compatibility version 1.0.0, current 
version 93.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 88.1.7)

According to /usr/include/sqlite3.h, what's installed by Apple is 3.1.3.
Aside from the possibility that I somehow compiled against
/usr/include/sqlite3.h and linked against /usr/local/lib/libsqlite3.0.dylib,
what difference should 3.3.8 vs. 3.1.3 have made?

Skip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Massive test_sqlite failure on Mac OSX ... sometimes

2006-10-22 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Oct 22, 2006, at 11:24 PM, [EMAIL PROTECTED] wrote:

> According to /usr/include/sqlite3.h, what's installed by Apple is  
> 3.1.3.
> Aside from the possibility that I somehow compiled against
> /usr/include/sqlite3.h and linked against /usr/local/lib/ 
> libsqlite3.0.dylib,
> what difference should 3.3.8 vs. 3.1.3 have made?

Dunno, but as much as I love SQLite, I've also found it to be pretty  
finicky.  For example, I once tried to upgrade us from 3.2.1 to 3.2.8  
but that caused us a world of hurt, so I reverted back to the last  
known good version.  At some point I'll try to get us on the latest  
release, but I'm a little gunshy about it.

- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRTw8e3EjvBPtnXfVAQJbKgP+MjAz/NfUOaDd+ZEg9haJVr7v5JsKTHEl
i9n7pLLFToIE81RX3iGHMZwIZyIGHqT9d3gqan8INrvcAtL7hxVvkqAAFRJTmX2Z
XVLAjWLYCp9nY6Q3K+yXls798RDoHhZIWvHnNXZJ7Ya2wwSVQoADFdV1GN0pIB07
PnNHa/S83+Q=
=4fX8
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The "lazy strings" patch

2006-10-22 Thread Larry Hastings

Martin v. Löwis wrote:
> It's not clear to me what you want to achieve with these patches,
> in particular, whether you want to see them integrated into Python or
> not.
>   
I would be thrilled if they were, but it seems less likely with every 
passing day.  If you have some advice on how I might increase the 
patch's chances I would be all ears.

It was/is my understanding that the early days of a new major revision 
was the most judicious time to introduce big changes.  If I had offered 
these patches six months ago for 2.5, they would have had zero chance of 
acceptance.  But 2.6 is in its infancy, and so I assumed now was the 
time to discuss sea-change patches like this.

Anyway, it was my intent to post the patch and see what happened.  Being 
a first-timer at this, and not having even read the core development 
mailing lists for very long, I had no idea what to expect.  Though I 
genuinely didn't expect it to be this brusque.

> I think this specific approach will find strong resistance.
I'd say the "lazy strings" patch is really two approaches, "lazy 
concatenation" and "lazy slices".  You are right, though, *both* have 
"found strong resistance".

> Most recently, it was discussed under the name "string view" on the Py3k 
> list, see
>   http://mail.python.org/pipermail/python-3000/2006-August/003282.html
> Traditionally, the biggest objection is that even small strings may
> consume insane amounts of memory.
>   
Let's be specific: when there is at least one long-lived small lazy 
slice of a large string, and the large string itself would otherwise 
have been dereferenced and freed, and this small slice is never examined 
by code outside of stringobject.c, this approach means the large string 
becomes long-lived too and thus Python consumes more memory overall.  In 
pathological scenarios this memory usage could be characterized as "insane".

True dat.  Then again, I could suggest some scenarios where this would 
save memory (multiple long-lived large slices of a large string), and 
others where memory use would be a wash (long-lived slices containing 
the all or almost all of a large string, or any scenario where slices 
are short-lived).  While I think it's clear lazy slices are *faster* on 
average, its overall effect on memory use in real-world Python is not 
yet known.  Read on.

>> I bet this generally reduces overall memory usage for slices too.
>> 
> Channeling Guido: what real-world applications did you study with
> this patch to make such a claim?
>   
I didn't; I don't have any.  I must admit to being only a small-scale 
Python user.  Memory use remains about the same in pybench, the biggest 
Python app I have handy.  But, then, it was pretty clearly speculation, 
not a claim.  Yes, I *think* it'd use less memory overall.  But I 
wouldn't *claim* anything yet.

The "stringview" discussion you cite was largely speculation, and as I 
recall there were users in both camps ("it'll use more memory overall" 
vs "no it won't").  And, while I saw a test case with microbenchmarks, 
and a "proof-of-concept" where a stringview was a separate object from a 
string, I didn't see any real-word applications tested with this approach.

Rather than start in on speculation about it, I have followed that old 
maxim of "show me the code".  I've produced actual code that works with 
real strings in Python.  I see this as an opportunity for Pythonistas to 
determine the facts for themselves.  Now folks can try the patch with 
these real-world applications you cite and find out how it really 
behaves.  (Although I realize the Python community is under no 
obligation to do so.)

If experimentation is the best thing here, I'd be happy to revise the 
patch to facilitate it.  For instance, I could add command-line 
arguments letting you tweak the run-time behavior of the patch, like 
changing the minimum size of a lazy slice.  Perhaps add code so there's 
a tweakable minimum size of a lazy concatenation too.  Or a tweakable 
minimum *ratio* necessary for a lazy slice.  I'm open to suggestions.

Cheers,


/larry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The "lazy strings" patch

2006-10-22 Thread Talin
Larry Hastings wrote:
> Martin v. Löwis wrote:

> Let's be specific: when there is at least one long-lived small lazy 
> slice of a large string, and the large string itself would otherwise 
> have been dereferenced and freed, and this small slice is never examined 
> by code outside of stringobject.c, this approach means the large string 
> becomes long-lived too and thus Python consumes more memory overall.  In 
> pathological scenarios this memory usage could be characterized as "insane".
> 
> True dat.  Then again, I could suggest some scenarios where this would 
> save memory (multiple long-lived large slices of a large string), and 
> others where memory use would be a wash (long-lived slices containing 
> the all or almost all of a large string, or any scenario where slices 
> are short-lived).  While I think it's clear lazy slices are *faster* on 
> average, its overall effect on memory use in real-world Python is not 
> yet known.  Read on.

I wonder - how expensive would it be for the string slice to have a weak 
reference, and 'normalize' the slice when the big string is collected? 
Would the overhead of the weak reference swamp the savings?

-- Talin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The "lazy strings" patch

2006-10-22 Thread Martin v. Löwis
Larry Hastings schrieb:
> Anyway, it was my intent to post the patch and see what happened.  Being 
> a first-timer at this, and not having even read the core development 
> mailing lists for very long, I had no idea what to expect.  Though I 
> genuinely didn't expect it to be this brusque.

I could have told you :-) The "problem" really is that you are
suggesting a major, significant change to the implementation of
Python, and one that doesn't fix an obvious bug. The new code
is an order of magnitude more complex than the old one, and the
impact that it will have is unknown - but in the worst case,
it could have serious negative impact, e.g. when the code is
full of errors, and causes Python application to crash in masses.

This is, of course, FUD: it is the fear that this might happen,
the uncertainty about the quality of the code and the doubt
about the viability of the approach.

There are many aspects to such a change, but my experience is
that it primarily takes time. Fredrik Lundh suggested you give
up about Python 2.6, and target Python 3.0 right away; it may
indeed be the case that Python 2.6 is too close for that kind
of change to find enough supporters.

If your primary goal was to contribute to open source, you
might want to look for other areas of Python: there are plenty
of open bugs ("real bugs" :-), unreviewed patches, etc. For
some time, it is more satisfying to work on these, since
the likelihood of success is higher.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The "lazy strings" patch

2006-10-22 Thread Josiah Carlson

Larry Hastings <[EMAIL PROTECTED]> wrote:
> It was/is my understanding that the early days of a new major revision 
> was the most judicious time to introduce big changes.  If I had offered 
> these patches six months ago for 2.5, they would have had zero chance of 
> acceptance.  But 2.6 is in its infancy, and so I assumed now was the 
> time to discuss sea-change patches like this.

It would be a radical change for Python 2.6, and really the 2.x series,
likely requiring nontrivial changes to extension modules that deal with
strings, and the assumptions about strings that have held for over a
decade.  I think 2.6 as an option is a non-starter.  Think Py3k, and
really, think bytes and unicode.


> The "stringview" discussion you cite was largely speculation, and as I 
> recall there were users in both camps ("it'll use more memory overall" 
> vs "no it won't").  And, while I saw a test case with microbenchmarks, 
> and a "proof-of-concept" where a stringview was a separate object from a 
> string, I didn't see any real-word applications tested with this approach.
> 
> Rather than start in on speculation about it, I have followed that old 
> maxim of "show me the code".  I've produced actual code that works with 
> real strings in Python.  I see this as an opportunity for Pythonistas to 
> determine the facts for themselves.  Now folks can try the patch with 
> these real-world applications you cite and find out how it really 
> behaves.  (Although I realize the Python community is under no 
> obligation to do so.)

One of the big concerns brought up in the stringview discussion was that
of users expecting one thing and getting another.  Slicing a larger
string producing a 'view', which then keeps the larger string alive,
would be a surprise.  By making it a separate object that just *knows*
about strings (or really, anything that offers a buffer interface), I
was able to make an object that was 1) flexible, 2) usable in any Python,
3) doesn't change the core assumptions about Python, 4) is expandable to
beyond just *strings*.  Reason #4 was my primary reason for writing it,
because str disappears in Py3k, which is closer to happening than most
of us realize.


> If experimentation is the best thing here, I'd be happy to revise the 
> patch to facilitate it.  For instance, I could add command-line 
> arguments letting you tweak the run-time behavior of the patch, like 
> changing the minimum size of a lazy slice.  Perhaps add code so there's 
> a tweakable minimum size of a lazy concatenation too.  Or a tweakable 
> minimum *ratio* necessary for a lazy slice.  I'm open to suggestions.

I believe that would be a waste of time.  The odds of it making it into
Python 2.x without significant core developer support are pretty close
to None, which in Python 2.x is less than 0.  I've been down that road,
nothing good lies that way.

Want my advice?  Aim for Py3k text as your primary target, but as a
wrapper, not as the core type (I put the odds at somewhere around 0 for
such a core type change).  If you are good, and want to make guys like
me happy, you could even make it support the buffer interface for
non-text (bytes, array, mmap, etc.), unifying (via wrapper) the behavior
of bytes and text.


 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] The "lazy strings" patch

2006-10-22 Thread Fredrik Lundh
Josiah Carlson wrote:

> It would be a radical change for Python 2.6, and really the 2.x series,
> likely requiring nontrivial changes to extension modules that deal with
> strings, and the assumptions about strings that have held for over a
> decade.

the assumptions hidden in everyone's use of the C-level string API is 
the main concern here, at least for me; radically changing the internal 
format is not a new idea, but it's always been held off because we have 
no idea how people are using the C API.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com