Re: [Python-Dev] String conversion issues implementing new curses module method addchstr, etc.

[Ulrich Eckhardt]

On Wednesday 25 February 2009, Heracles wrote:
> addchstr((chtype*)PyString_AsString(pS))

You are effectively disabling the well-deserved warnings with the cast here. 
Don't do that.

> Now the thing is that when I make calls from python like so:
>
>curses.addchstr(5,10, "@ < Row 5, Col 10")
[...]
> What prints out on the screen is gobledygook not the strings you would
> expect below.
[...]
> Any thoughts on how to correctly convert the PyStringObject arguments to
> chtype* pointers so that the ncurses library will be able to understand
> them?

In this case, I suggest man ncurses:

| cchar_t
|corresponds to chtype.  However it is a structure, because more
|data is stored than can fit into an integer.  The characters are
|large enough to require a full  integer  value  -  and there may
|be more than one character per cell. The video attributes and
|color are stored in  separate fields of the structure.  

I'm pretty sure that you can find functions to generate such a string both 
from a char-string (with whatever encoding) or a wchar_t-string (using 
Unicode probably) in the curses library.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] sequence slice that wraps, bug or intention?

[Ulrich Eckhardt]

Hi!

I just stumbled across something in Python 2.6 where I'm not sure if it is by 
design or a fault:

x = 'abdc'
x[-3:-3] -> ''
x[-3:-2] -> 'b'
x[-3:-1] -> 'bc'
x[-3: 0] -> ''

The one that actually bothers me here is the last one, I would have expected 
it to yield 'bcd' instead, because otherwise I don't see a way to specify a 
slice that starts with a negative index but still includes the last element.

Similarly, I would expect x[-1,1] to yield 'ca' or at least raise an error, 
but not to return an empty string.

Bug?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python on PowerPC?

[Ulrich Eckhardt]

On Wednesday 20 May 2009, Chris Plasun wrote:
> I'm to develop console apps on a Linux embedded PowerPC board (Freescale
> MPC8313).
>
> Is there a Python release for the PowerPC platform?

This has pretty little to do with the development of the Python language 
itself, so it is rather off topic here.

That said, Linux systems are barely thinkable without Python, even when 
running on PPC, so yes, Python runs on PPC, too, and is included in probably 
every Linux distro, e.g. Debian.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Clean up Python/thread_*.h ?

[Ulrich Eckhardt]

On Saturday 24 October 2009, Christian Heimes wrote:
> Antoine Pitrou wrote:
> > * The unused file: thread_wince.h
> >
> > Windows CE has actually been using thread_nt.h since January 2009 (see
> > http://bugs.python.org/issue4893 ). thread_wince.h is still there, but
> > it's unused and grep brings no instance of it being mentioned anywhere in
> > the source tree.
>
> What about older versions of Windows CE? Maybe they still require the
> wince thread header?

Windows CE support in recent 2.x (and probably 3.x) is broken due to heavy 
bitrot. I can't imagine any CE platform this code would compile against. 
Further, I don't see many people using CE < 4, with CE 6 being current, in 
fact they are _very_ rare judging from Usenet activities.

For current CE versions, porting the NT-threads code is the easier way out.

+1 for removal of thread_wince.{h,c}

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] standard libraries don't behave like standard 'libraries'

[Ulrich Eckhardt]

On Thursday 12 November 2009, Sriram Srinivasan wrote:
> I don't know if you have used Dev-C++. It has a 'package management'
> mechanism for the standard libraries.

I disagree. It has a package management system for libraries, not for the 
standard libraries. The point is that the Python standard library is supplied 
as part of Python itself, as is e.g. the C++ standard library as part of a 
C++ compiler.

> In python we have the PyPI which is equivalent to the
> http://devpacks.org but in PyPI the packages are all user made
> applications.

...as in devpacks.org. Please scroll down on that page and read the warning 
that mentions than any person can upload a package possibly containing 
malware or broken code or whatever.

> What I want is similar to PyPI but for the python standard libraries, so
> that they (libraries) are as add-on as possible.

So, you want to make libs that are currently part of the standard library 
external libs available via PyPI? If that is what you want, it would be 
helpful if you provided a reason and also which libraries you're talking 
about.

Uli

-- 
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] patch to make list.pop(0) work in O(1) time

[Ulrich Eckhardt]

On Tuesday 26 January 2010, Steve Howell wrote:
> Here are the benefits of an O(1) implementation.
[...]
> Did I leave anything out?

Good summary, Steve, thanks!

Anyway, you left two out:

 * Inserting at the front gets the same complexity as inserting at the back.

 * Inserting and erasing anywhere in the middle can memmove() the smaller 
tail, so it changes from O(N) to O(N/2).


Finally, if you drop the invariant first<=last, you can simply wrap around the 
pointers. This allows creating a producer/consumer queue which never needs to 
move its content.


Cheers!

Uli

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Drive suffix

[Ulrich Eckhardt]

On Wednesday 04 August 2010, Rob Cliffe wrote:
> As it happens, what sparked the question was trying to determine in a
> platform-independent way whether a path consisted of a bare drive
> specification (e.g. "C:").  I guess
> os.path.splitdrive(MyPath)[1] == ""
> takes care of that.

"platform-independent"? What is a "bare drive specification" on a FreeBSD 
system? Do you perhaps mean "mount point" instead? I don't think this 
discussion belongs Here(tm) though.

BTW: Embedded MS Windows CE uses a filesystem tree like *nix, i.e. without 
drive letters. Not that I have any kind of Python crawling on one of those 
yet, just wanted to mention...

Uli

-- 
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] new buffer in python2.7

[Ulrich Eckhardt]

On Wednesday 27 October 2010, Kristján Valur Jónsson wrote:
> Although 2.7 has the new buffer interface and memoryview objects, these are
> widely not accepted in the built in modules. Examples are the structmodule,
> some of the socketmodule apis, structmodule, etc.
>
> IMHO this is unfortunate.  For example when doign network io, you would
> want code like this: Buffer = bytearray(10)
> Socket.recv_into(Buffer)
> Header = struct.unpack("i", memoryview(Buffer)[:4])[0]

Actually I would like code like
  s = socket()
  ...
  header = struct.unpack("i", s)

In other words, struct should interact with files/streams directly, instead of 
requiring me to first read a chunk who's size I manually have to determine 
etc.

Otherwise, I'm +1 on your suggestion, avoiding copying is a good thing.

Uli
(who's going to shut up, because he doesn't have a patch either)

-- 
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Removal of Win32 ANSI API

[Ulrich Eckhardt]

On Thursday 11 November 2010, Hirokazu Yamamoto wrote:
> Is it possible to remove Win32 ANSI API (ie: GetFileAttributesA)
> and only use Win32 WIDE API (ie: GetFileAttributesW)?
> Mainly in posixmodule.c.
> I think we can simplify the code hugely.

+1

MS Windows variants that only support the ANSI API (win9x) are officially 
unsupported since 2.5 or 2.6. Further, this also eases porting to MS Windows 
CE, which I'd still like to to see one day.

> (This means droping bytes support for os.stat etc on windows)

I disagree that not using the ANSI win32 API means dropping byte support for 
os.stat. I'd rather say that it means converting bytes at the earliest 
possible time and only using unicode internally. But I'm guessing a bit here, 
I haven't looked at the code for a while.

> # I recently did it for winsound.PlaySound with MvL's approval

Interesting, is there a ticket associate with this? Also, was that on Python 3 
or 2? Which commits?

Uli

-- 
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] python 2 for building python 3

[Ulrich Eckhardt]

On Saturday 04 December 2010, Antoine Pitrou wrote:
> Perhaps SVN doesn't get timestamps right.

SVN doesn't touch timestamps explicitly, it just writes the files as they come 
and the system gives them the timestamps. This also makes sense, because the 
build system depends on them - you don't want the build to skip compiling 
files after rewinding your working copy to an older version.

That said, you can tell SVN to do something with timestamps, I'd have to 
search a bit in order to find out what exactly that is.

Uli

**
Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bugs in thread_nt.h

[Ulrich Eckhardt]

On Thursday 10 March 2011, Sturla Molden wrote:
> As for InterlockedCompareExchange et al., MSDN says this: "The
> parameters for this function must be aligned on a 32-bit boundary;

bit != byte

Uli



**
Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] time.sleep(-1) behaviour

[Ulrich Eckhardt]

Hi!

This is a request for clarification for the thread "how to call a function for 
evry 10 seconds" from the user mailinglist/newsgroup.


The gist is this:
1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError.
2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that 
converting this to a 32-bit integer for Sleep() causes an underflow.


Questions:
1. Is the IOError somehow explainable?
2. Is the IOError acceptable or a bug?
3. Is the behaviour under MS Windows acceptable or a bug?
4. Since time.sleep() is documented to possibly sleep a bit longer than 
specified, someone suggested that the overall sleep time could also be 
truncated to a minimum of zero, i.e. treating negative values as zero.


Greetings!

Uli
**
Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.sleep(-1) behaviour

[Ulrich Eckhardt]

On Thursday 30 June 2011, R. David Murray wrote:
> Please file a bug report at bugs.python.org.

Filed as bug #12459.

Uli

**
Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: no one passes NULL here (or should anyway)

[Ulrich Eckhardt]

On Monday 04 July 2011, Benjamin Peterson wrote:
> If someone's static analysis tool starts complaining about it, I'd be
> happy to consider adding an assert...

Here is one! To me an assertion is what actually documents that you don't 
expect NULL in this context and that the missing check is not an oversight.

+1 for assert()

Uli
**
Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Status of MS Windows CE port

[Ulrich Eckhardt]

Greetings!

I'm currently trying to assess the effort required for a CE port. I'm already 
aware of the project at http://pythonce.sourceforge.net, but I find it a 
waste of time to have two separate projects which then require syncing 
changes back and forth.

Questions:
- What are the feelings towards a port in the mainline? I'm aware that you are 
trying to get 2.6 and 3.0 out of the door and are probably not willing to 
risk such changes to the release code right now, but in general?
- Is anyone already working on this? I know that Brad Clements made an attempt 
some years ago but didn't get it integrated, are there any others? Maybe even 
any ongoing work?
- I see that you are using Subversion and Bazaar, would it be possible to get 
a feature branch for the CE port? I'm thinking about the effort going on at 
pythonce.sf.net and some further porting that I need for my specific 
platform. IMHO both would be better off in the same repository as your 
upstream sources, since merging changes back and forth is much easier that 
way.

Thanks!

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of MS Windows CE port

[Ulrich Eckhardt]

On Wednesday 24 September 2008, Neal Norwitz wrote:
> In general, we try to ensure that there is an active maintainer,
> preferably more than one.  As long as it doesn't make the code that
> much harder to maintain, it is desirable to support more platforms.

Actually, looking at the sources, there are several places where even existing 
win32 ports would benefit. There are four main differences between CE and the 
desktop variants:
1. TCHAR is always WCHAR
2. some APIs are missing
3. the vendor-supplied C or C++ stdlibs are rather minimal
4. you always cross-compile

The first point is interesting to the desktop win32 variants because 
everything since NT4 (or even 3.x) also used UCS2 or UTF-16 internally. Only 
win9x and ME(?) use 8-bit chars, and since support for them is being dropped, 
this would save a few conversions while allowing better Unicode support. I 
think getting Python to compile with _UNICODE #defined will be my first step.

The second point is mainly about features that don't exist like a current 
directory for a process or environment variables. Some other APIs don't 
support all parameters or similar differences. This should be rather easy to 
do.

The third means that functions like e.g. system() are simply not available in 
the supplied C API. If the functionality itself exists, it can be built on 
top of the win32 API. There might be a few cases where the desktop variants 
would benefit from one less adaption layer, too.

The last one only affects the build system and possibly the integration of 
unit testing. pythonce.sf.net already have one based on SCons, which is nice 
and easily extended. Being able to build 64 bit variants on 32 bit systems 
and vice versa might be another benefit.

> It's possible.  The best way to get the ball rolling is to produce
> some patches against the mainline.

Hmmm, which mainline? 2.5, 2.6 or 3.0? Up to now I have only looked at 2.5, 
because that one is stable. However, quite a few of above changes are not 
suitable for a stable version though, so I'll either have to restrict them to 
only CE or rather apply them to version 3. We can discuss that when I have 
actual patches.

> Perhaps the best way to start is using Bazaar or some distributed
> version control system for your patches.  Once you've demonstrated
> your ability and intent, we can make a branch.

I'll be digging into Bazaar, looks like an interesting topic by itself anyway. 
You'll hear from me!

cheers

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of MS Windows CE port

[Ulrich Eckhardt]

On Thursday 25 September 2008, Martin v. Löwis wrote:
> > 1. TCHAR is always WCHAR
>
> Python shouldn't be using TCHAR at all; that getpathp uses it is
> misguided (IMO).

It already does, see below...

> > The first point is interesting to the desktop win32 variants because
> > everything since NT4 (or even 3.x) also used UCS2 or UTF-16 internally.
> > Only win9x and ME(?) use 8-bit chars, and since support for them is being
> > dropped, this would save a few conversions while allowing better Unicode
> > support. I think getting Python to compile with _UNICODE #defined will be
> > my first step.
>
> Please don't. Whether or not _UNICODE is defined should have no effect.

Well, currently it does make a difference. Simple example: CreateFile(). One 
of the parameters is actually documented at LPCTSTR, which is a TCHAR string. 
That means that under CE, CreateFile(), which is actually a macro, resolves 
to CreateFileW and the existing code that passes a char-string doesn't work.

> > The third means that functions like e.g. system() are simply not
> > available in the supplied C API.
>
> In this case, not defining HAVE_SYSTEM would be already sufficient, no?

I'll take a look, I actually don't know.


> > The last one only affects the build system and possibly the integration
> > of unit testing. pythonce.sf.net already have one based on SCons, which
> > is nice and easily extended. Being able to build 64 bit variants on 32
> > bit systems and vice versa might be another benefit.
>
> Wouldn't it be easier to extend the VS projects? VS targets CE fairly
> well, no?

VS2005 works really good with CE5 and later. My problem here is that I can't 
provide a projectfile that suits everyone, because different devices have 
different SDKs and each configuration/SDK/CPU is another entry in that file. 
Further, the SDKs or devices differ in what they support, the platform CE as 
such doesn't exist.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of MS Windows CE port

[Ulrich Eckhardt]

On Friday 26 September 2008, Martin v. Löwis wrote:
> >> Please don't. Whether or not _UNICODE is defined should have no effect.
> >
> > Well, currently it does make a difference. Simple example: CreateFile().
>
> It's not so simple: Python doesn't actually call CreateFile, AFAICT
> (well, it does, in _msi.c, but I hope you aren't worried about that
> module on CE).

Martin, CreateFile() was just used as an example. You can substitute it with 
LoadString() or CreateProcess() if you like, the problem remains the same.


[about using SCons for building]
> And you *can* provide an SCons file that supports all the SDKs?

No, but I can provide one that allows parametrisation. ;)

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Filename as byte string in python 2.6 or 3.0?

[Ulrich Eckhardt]

On Sunday 28 September 2008, Gregory P. Smith wrote:
> "broken" systems will always exist.  Code to deal with them must be
> possible to write in python 3.0.
>
> since any given path (not just fs) can have its own encoding it makes
> the most sense to me to let the OS deal with the errors and not try to
> enforce bytes vs string encoding type at the python lib. level.

Actually I'm afraid that that isn't really useful. I, too, would like to kick 
peoples' back in order to get the to fix their systems or use the proper 
codepage while mounting etc, etc, but that is not going to happen soon. Just 
ignoring those broken systems is tempting, but alienating a large group of 
users isn't IMHO worth it.

Instead, I'd like to present a different approach:

1. For POSIX platforms (using a byte string for the path):
Here, the first approach is to convert the path to Unicode, according to the 
locale's CTYPE category. Hopefully, it will be UTF-8, but also codepages 
should work. If there is a segment (a byte sequence between two path 
separators) where it doesn't work, it uses an ASCII mapping where possible 
and codepoints from the "Private Use Area" (PUA) of Unicode for the 
non-decodable bytes.
In order to pass this path to fopen(), each segment would be converted to a 
byte string again, using the locale's CTYPE category except for segments 
which use the PUA where it simply encodes the original bytes.

2. For win32 platforms, the path is already Unicode (UTF-16) and the whole 
problem is solved or not solved by the OS.

In the end, both approaches yield a path represented by a Unicode string for 
intermediate use, which provides maximum flexibility. Further, it 
preserves "broken" encodings by simply mapping their byte-values to the PUA 
of Unicode. Maybe not using a string to represent a path would be a good 
idea, too. At least it would make it very clear that the string is not 
completely free-form.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Filename as byte string in python 2.6 or 3.0?

[Ulrich Eckhardt]

On Monday 29 September 2008, M.-A. Lemburg wrote:
> On 2008-09-29 12:50, Ulrich Eckhardt wrote:
> > 1. For POSIX platforms (using a byte string for the path):
> > Here, the first approach is to convert the path to Unicode, according to
> > the locale's CTYPE category. Hopefully, it will be UTF-8, but also
> > codepages should work. If there is a segment (a byte sequence between two
> > path separators) where it doesn't work, it uses an ASCII mapping where
> > possible and codepoints from the "Private Use Area" (PUA) of Unicode for
> > the non-decodable bytes.
> > In order to pass this path to fopen(), each segment would be converted to
> > a byte string again, using the locale's CTYPE category except for
> > segments which use the PUA where it simply encodes the original bytes.
>
> I'm not sure how this would work. How would you map the private use
> code points back to bytes ? Using a special codec that knows about
> these code points ? How would the fopen() know to use that special
> codec instead of e.g. the UTF-8 codec ?

Sorry, I wasn't clear enough. I'll try to explain further...

Let's assume we have a filename like this:

  0xc2 0xa9 0x2f 0x7f

The first two bytes are the copyright sign encoded in UTF-8, followed by a 
slash (0x2f, path separator) and a character encoded in an unknown codepage 
(0x7f is not ASCII!). The first thing when receiving that path from the 
system would be to split it into segments, here we would get two of them, one 
with 0xc2 0xa9 and the other with 0x7f. This uses the fact that the separator 
(slash/0x2f) is rather universal (Note: I'm not sure about encodings like 
BIG5, i.e. ones that are neither UTF-8 nor derived from ASCII).

For each segment, we would apply the locale's CTYPE facet and get the Unicode 
codepoint 0xa9 for the first segment, while the second one fails to convert. 
So, for the second one, we simply check for each byte if it is valid and 
printable ASCII (0x7f isn't). If it is, we emit the byte as Unicode 
codepoint. Otherwise, we map to the PUA.

The PUA reserves 0xe000 to 0xf8ff for private uses. I would simply encode the 
byte 0x7f as 0xe07f, i.e. map it to the beginning of that range. Eventually, 
we would end up with the following Unicode codepoints:

  0xa9, 0x2f, 0xe07f

When converting to a byte string for use with fopen(), we simply inspect the 
supplied string again. If a segment contains elements of the PUA, we simply 
reverse the mapping for those and leave the others in that segment as-is. For 
all other segments, we apply the CTYPE conversion.


Notes:
* This effectively converts the current path representation (a string) into a 
sequence of segments where each segment can either be a fully Unicode-capable 
string or a raw byte string without any known interpretation. However, 
instead of using an array for that, it uses a string, which is what most 
people's code expects anyway.
* You could also work on a byte-base instead of splitting the path in segments 
first. I just assumed that a single segment will not contain valid UTF-8 
sequences mixed with invalid ones. A path however can contain both correctly 
and incorrectly encoded segments.


> BTW: Private use areas in Unicode are meant for e.g. company specific
> code points. Using them for escaping purposes is likely to cause problems
> due to assignment clashes.

I'm not sure if the use I proposed is correct according to the intended use of 
the PUA. I know that ideally no such string would escape from Python, i.e. it 
should only be visible internally. I would guess that that is something the 
PUA was intended for.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Filename as byte string in python 2.6 or 3.0?

[Ulrich Eckhardt]

On Monday 29 September 2008, [EMAIL PROTECTED] wrote:
> Also, what about MacOS X?

AFAIK, OS X guarantees UTF-8 for filesystem encodings. So the OS also provides 
Unicode filenames and how it deals with broken or legacy media is left up to 
the OS.

Uli


-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Broken link to NASM download location

[Ulrich Eckhardt]

Hi!

In trunk/PCbuild/readme.txt it says NASM is available from kernel.org. The 
project has moved to Sourceforge though, please replace the link with

  http://nasm.sf.net

Thanks!

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of MS Windows CE port

[Ulrich Eckhardt]

On Tuesday 30 September 2008, Martin v. Löwis wrote:
> Ulrich Eckhardt wrote:
> >>> Well, currently it does make a difference. Simple example:
> >>> CreateFile().
> >>
> >> It's not so simple: Python doesn't actually call CreateFile
> >
> > Martin, CreateFile() was just used as an example. You can substitute it
> > with LoadString() or CreateProcess() if you like, the problem remains the
> > same.
>
> However, the solution should be different from the one you propose. I
> don't know what call of CreateProcess you are referring to specifically,
> but I think they should all be changed to call CreateProcessW.
>
> Again, whether or not _UNICODE is defined should have no effect. If it
> does, it's a bug, and the solution is not to sprinkle TCHAR all over the
> place.

I think we're misunderstanding each other, because that is exactly the 
solution I'm targetting. I'm aware that TCHAR is just a hack to ease 
transition between obsolete MS Windows versions and NT and later. However, 
current state is that Python uses CreateProcessA() and changing that is not 
always trivial.

Therefore, my first step in porting is also to remove the dependency on TCHAR, 
i.e. replace things like CreateProcess() with preferably CreateProcessW() or 
alternatively CreateProcessA(). Just #defining _UNICODE in the build already 
turns up around 50 places in pythoncore that need work. I'll send patches 
soon.

> > [about using SCons for building]
> >
> >> And you *can* provide an SCons file that supports all the SDKs?
> >
> > No, but I can provide one that allows parametrisation. ;)
>
> And, with proper parametrization, then supports all SDKs?

Hopefully, yes, but I'm not going to make any claims which I'm not sure about. 
SCons is just convenient because the PythonCE project already uses it but I'm 
not adamant on that matter. The approach of Jack Jansen also sounds good, 
generating the projectfiles (which are XML, btw) from a Python script also 
sounds nice.

cheers

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] when is path==NULL?

[Ulrich Eckhardt]

Hi!

I'm looking at trunk/Python/sysmodule.c, function PySys_SetArgv(). In that 
function, there is code like this:

  PyObject* path = PySys_GetObject("path");
  ...
  if (path != NULL) {
...
  }

My intuition says that if path==NULL, something is very wrong. At least I 
would expect to get 'None', but never NULL, except when out of memory. So, 
for the case that path==NULL', I would simply invoke Py_FatalError("no mem 
for sys.path"), similarly to the other call there.

Sounds reasonable?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue

[Ulrich Eckhardt]

On Tuesday 30 September 2008, M.-A. Lemburg wrote:
> On 2008-09-30 08:00, Martin v. Löwis wrote:
> >> Change the default file system encoding to store bytes in Unicode is
> >> like introducing a new Python type: .
> >
> > Exactly. Seems like the best solution to me, despite your polemics.
>
> Not a bad idea... have os.listdir() return Unicode subclasses that work
> like file handles, ie. they have an extra buffer that holds the original
> bytes value received from the underlying C API.

Why does it have to be a Unicode subclass? In my eyes, a Unicode object 
promises a few things, in particular that it contains a Unicode string. If it 
now suddenly contains bytes without any further meaning, that would be bad.


What I wonder is what the requirements on path handling are. I'll try to list 
the ones I can see:

1. A path received from the system should be preserved, so it can be given to 
the system later on. IOW, the internal representation should not loose any 
information compared to the one used by the OS.

2. Typical operations like joining two path segments or moving to the parent 
dir should be defined.

3. There must be a way to display the path to the user. IOW, there should be a 
way to turn the path into a string that the user can recognise, according to 
some encoding. Note that this is not always possible, so this can fail.

4. There must be a way to receive a path from the user. That means that there 
must be a way from a user-entered string to a path. Note that this, too, 
isn't always possible and can fail.

5. The conversion between a string and a path should be configurable, defaults 
retrieved from the system. This is so that most operations will just work and 
do the thing that the user expects.

6. There should be a way to modify the path data itself. This of course 
requires knowledge about the internals but gives full power to the 
programmer.


For requirement 3, I would say a lossy conversion to a string would be enough, 
i.e. try to convert the path to a Unicode string and use a question mark or 
some escaping to mark parts that can't be decoded. It will allow users to 
recognise the decodeable parts of the path with hopefully just a few 
characters left without decoding.

For requirement 4, a failure to encode a string to a path must result in a 
loud failure, i.e. an exception. This is because the user entered a path that 
we can't use, any guessing what the user might have wanted is futile.


Are there any points to add?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Subversion access down?

[Ulrich Eckhardt]

Hi!

Is it only me or does it fail for other people, too? I'm getting

| Server sent unexpected return value (503 Service
| Unavailable) in response to OPTIONS request 
| for 'http://svn.python.org/projects/python/trunk'  


Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] My patches

[Ulrich Eckhardt]

On Thursday 30 October 2008, Victor Stinner wrote:
> > One of the reasons why I'm very keen on us moving to a distributed
> > version control system is to help break the logjam on core developers.
>
> Yeah, exactly :-) Does anyone already maintain a distributed tree?
> Mercurial, GIT, anything else?

Bazaar. Take a look at the developers' pages on python.org, they mention that 
a BZR checkout is available. I know that it works (though the initial 
checkout is glacially slow) but I don't know what "official" support it has 
or what is planned with it.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Friday 05 December 2008, [EMAIL PROTECTED] wrote:
> Filenames and environment variables would all need to be encoded or
> decoded according to this magic encoding.

Those, and commandline arguments, too.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Friday 05 December 2008, Adam Olsen wrote:
> Many of the windows APIs use UTF-16 without validating it.  They'll
> pass through invalid strings until they hit something that does
> validate, at which point it'll blow up.
>
> I suspect that it doesn't happen very often in practice, as having
> only one encoding makes it quite clear that it's a broken file name,
> not a mixed encoding environment.

Actually, I wouldn't say that's a problem at all. The point is that stuff that 
is blissfully unaware of encodings typically uses some ASCII-de(p)rived text. 
Those char-strings are translated according to the current locale, which then 
does the filtering and validation. The result may be gibberish (GIGO 
principle) but at least it's UTF-16 gibberish. ;)

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Friday 05 December 2008, Guido van Rossum wrote:
> At the risk of bringing up something that was already rejected, let me
> propose something that follows the path taken in 3.0 for filenames,
> rather than doubling back:
>
> For os.environ, os.getenv() and os.putenv(), I think a similar
> approach as used for os.listdir() and os.getcwd() makes sense: let
> os.environ skip variables whose name or value is undecodable, and have
> a separate os.environb() which contains bytes; let os.getenv() and
> os.putenv() do the right thing when the arguments passed in are bytes.
>
> For sys.argv, because it's positional, you can't skip undecodable
> values, so I propose to use error=replace for the decoding; again, we
> can add sys.argvb that contains the raw bytes values. The various
> os.exec*() and os.spawn*() calls (as well as os.system(), os.popen()
> and the subprocess module) should all accept bytes as well as strings.
>
> On Windows, the bytes APIs should probably not exist.
>
> I predict that most developers can get away with not using the bytes
> APIs at all. The small minority that needs to be robust if not all
> filenames use the system encoding can use the bytes APIs.

I know some of those developers, you can contact them via 
[EMAIL PROTECTED] Seriously, what would you suggest to someone that 
wants to handle paths in a portable way? Using the Unicode variants of 
functions is fubar, because encoding/decoding is not universally possible. 
Using the byte variant is equally fubar, because e.g. on MS Windows it is not 
supported, except through a very lossy roundtrip through the locale's 
codepage, limiting your functionality.

I actually think it is about time to give up on trying to think about a path 
as a string. Dito for data received from os.environ or sys.argv. There are 
only very few things that are universal to them and a reliable encoding is 
none of them. Then, once you have let that idea go, meditate a bit over the 
Zen.

What I propose is that paths must be treated as OS-specific, with the only 
common reliable operations being joining them, concatenating them and 
splitting them into segments divided by the (again, OS-specific) separator. 
Other operations, like e.g. appending a string or converting it to a string 
in order to display it can fail. And if they fail, they should fail noisily. 
In 99% of all cases, using the default encoding will work and do what people 
expect, which is why I would make this conversion automatic. In all other 
cases, it will at least not fail silently (which would lead to garbage and 
data loss) and allow more sophisticated applications to handle it.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Friday 05 December 2008, James Y Knight wrote:
> On Dec 5, 2008, at 5:27 AM, Ulrich Eckhardt wrote:
> > Using the byte variant is equally fubar, because e.g. on MS Windows
> > it is not supported, except through a very lossy roundtrip through
> > the locale's codepage, limiting your functionality.
>
> Yeah, IMO whole mess could have been avoided by keeping the filename/
> args/environ simply *bytes*, like it really is, on unix. Then, make
> the Windows version of python use (always! not dependent upon locale!)
> utf-8 to decode the utf-8 bytestring to the UTF-16 that the Windows
> platform APIs expect (and vice versa).

If possible, I would try to avoid this useless roundtrip from UTF-16 to UTF-8 
and back.

> And never use the ASCII variant of the windows APIs.

That's okay, but I'm afraid it's not possible. The problem is not so much 
doing it, but finding all those places where it is currently done. Those 
could be outside of Python itself. So, even to Python code, there could still 
be APIs that would need the MBCS-encoded strings.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Sunday 07 December 2008, Guido van Rossum wrote:
> My problem with raising exceptions *by default* when an undecodable
> name exists is that it may render an app completely useless in a
> situation where the developer is no longer around. This happened all
> the time with the 2.x Unicode API, where the developer hadn't
> anticipated a particular input potentially containing non-ASCII bytes,
> and the user fed the application non-ASCII text. Making os.listdir
> raise an exception when a directory contains a single undecodable file
> means that the entire directory can't be read, and most likely the
> entire app crashes at that point. Most likely the developer never
> anticipated this situation (since in most places it is either
> impossible or very unlikely) -- after all, if they had anticipated it
> they would have used the bytes API in the first place.

There is another way to handle this that noisily signals errors but doesn't 
cause programs to suddenly fail. Using os.listdir as example, the problem 
there is that the OS actually returns a list of strings that can not be 
reliably decoded, so I would propose to simply not decode them.

Now, the idea is what if this function simply returned neither a byte string 
nor a Unicode string, but e.g. an environment string type (called env_str)? 
os.listdir would only fail if it really failed to read the dir. If a user 
wants to display an element from the returned list, they would get something 
akin to what repr() returns, i.e. a recognisable string that can be written 
to a logfile. However, this thing will also include additional markup that 
makes it clear that it is not just a piece of text and not suitable to 
display to the end user.

This type distinction is important, because it means that any developer will 
immediately see that something unexpected is going on here. They will 
invoke "type(lst[0])" and see the unexpected type env_str, which will (via 
documentation) redirect them to the issue with different encodings and that 
all they have to do is 'map( unicode, lst)' in order to get at a list of real 
text strings, but they will also read that this operation might fail, forcing 
an informed decision.

If they don't care about a textual representation at all but only want to 
invoke os.popen with arguments received from the commandline, then everything 
is fine, too, because that function will take the strings as they are and 
just give them back to the OS. This allows roundtripping from OS over Python 
and back to the OS without any conversions and thus without any conversions 
that could fail. In the case of e.g. a backup program, this is exactly what 
is needed.

Now, if you have any hard-coded strings in your program but a function like 
os.popen needs an env_str object, this string is converted via a default 
encoding, i.e. the same that is used when converting an env_str object to 
Unicode. In this case, I would go so far to say that os.popen should accept 
normal str strings, too, and perform that conversion itself. An alternative 
way would be to reject the string because it is the wrong type, but since 
this internal string's encoding is known, there is no reason to force users 
to convert explicitly, it is just that the conversion might fail.

Similarly, when modifying such an env_str object, like e.g. "bak = 
sys.argv[1]+'.backup'". In this case, the string '.backup' is converted 
according to the default encoding and then appended to the commandline 
argument, the result would again be an env_str object.


Note: There is an option in this design, and that is to make the default 
behaviour in case of nonconvertable env_str objects configurable. A 
filemanager would then replace the undecodable bytes by an approximation, a 
backup program would use strict mode and a music player would perhaps simply 
skip and ignore such strings. The problem there is that changing this option 
would possibly affect other library code that one doesn't even know about 
because it is only used indirectly and its implementation is unknown. For 
that reason, I would rather not make this policy a configurable element. If 
you want that, you can easily code it yourself.

BTW: there was a PEP that proposed a new path class, which was rejected. This 
class was actually pretty similar, except that it also included several other 
features (globbing, path handling, opening files and the kitchen sink) which 
eventually made it too bloated. Otherwise, the idea of creating a separate 
type for these strings is the same.


Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthal

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Monday 08 December 2008, Adam Olsen wrote:
> At this point someone suggests we have a type that can store an
> arbitrary mix of unicode and bytes, so the undecodable portions stay
> in their original form. :P

Well, not an arbitrary mix, but a type that just stores whatever comes from 
the system without further specifying it as either bytes or Unicode:

* If you want a string for displaying it, you first have to extract a string 
from that thing and there you optionally specify the encoding and error 
behaviour.
* If you want to append a string to it, it is automatically encoded in the 
default encoding, which obviously can fail.
* Similarly, e.g. globbing is done on the underlying representation's level, 
so "*.py" will first have to be converted according to the default encoding.
* If you just print it, you will get something that you can make out the 
decodable parts from, but it will probably be like "{Unicode:u'abcde'}" 
or "{bytes:b'ab\xf0\x0fcd'}".
* If you don't want to display it, but just want to pass it to the system, 
just use it as is.

Yes, this puts an inconvenience on application programmers that up to now 
always assumed that they received a list of strings from os.readdir(), but 
that's the way with false assumptions. In any case, they will be aware (from 
reading the docs) of what the problem is and why there is no way to return a 
text. Further, they will get tools to convert these paths or environment vars 
to texts, so it will be simply replacing "os.readdir()" 
with "map(to_unicode,os.readdir())".


Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Tuesday 09 December 2008, Adam Olsen wrote:
> On Tue, Dec 9, 2008 at 11:31 AM, Ulrich Eckhardt
>
> <[EMAIL PROTECTED]> wrote:
> > On Monday 08 December 2008, Adam Olsen wrote:
> >> At this point someone suggests we have a type that can store an
> >> arbitrary mix of unicode and bytes, so the undecodable portions stay
> >> in their original form. :P
> >
> > Well, not an arbitrary mix, but a type that just stores whatever comes
> > from the system without further specifying it as either bytes or Unicode:
> >
> > * If you want a string for displaying it, you first have to extract a
> > string from that thing and there you optionally specify the encoding and
> > error behaviour.
> > * If you want to append a string to it, it is automatically encoded in
> > the default encoding, which obviously can fail.
>
> So the 2.x str, but with a more interesting default encoding than
> ASCII.  It'll work fine on the developer's system, but one day a user
> will present it with strange input, and boom.

If the system's representation of filenames can not represent a Unicode 
codepoint that the user entered, trying to open such a file must fail. If it 
can be represented, for convenience I would allow an implicit conversion.

  for i in readdir():
  copy( i, i+".backup")
  ...

> You have to be pessimistic here.  The default operations should either
> always work or never work.  Using unicode internally and skipping
> garbage input means the operations always work.  Using a bytes API
> means mixing with unicode never works, unless the programmer
> explicitly converts, in which case the onus is on them to use proper
> error handling.

So, if I understand you correctly, you would prefer an explicit conversion to 
the system's representation:

  for i in readdir():
  copy( i, i+path(".backup"))
  ...

> The only thing separating this from a bikeshed discussion is that a
> bikeshed has many equally good solutions, while we have no good
> solutions.  Instead we're trying to find the least-bad one.  The
> unicode/bytes separation is pretty close to that.  Adding a warning
> gets even closer.  Adding magic makes it worse.

Well, I see two cases:
1. Converting from an uncertain representation to a known one.
2. Converting from a known representation to a known one.

The uncertain one is the one used by the filesystem or environment. The known 
representations are the expected(!) encoding for filesystem and environment 
and the internal text in Unicode. For case 1, I would require an explicit 
conversion to make the programmer really aware of the fact that it can fail. 
For the second case, I would allow an implicit conversion even though it can 
fail. Anyhow, that is a matter of taste, and I can actually live with your 
point of view.

However, one question still remains: What about the approach in general, i.e. 
that these texts with an uncertain representation are handled as a separate 
type? I find this much more appealing that duplicating APIs like readdir() 
using either overloading on the arguments or a separate readdirb().

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Wednesday 10 December 2008, Adam Olsen wrote:
> On Wed, Dec 10, 2008 at 3:39 AM, Ulrich Eckhardt
>
> <[EMAIL PROTECTED]> wrote:
> > On Tuesday 09 December 2008, Adam Olsen wrote:
> >> The only thing separating this from a bikeshed discussion is that a
> >> bikeshed has many equally good solutions, while we have no good
> >> solutions.  Instead we're trying to find the least-bad one.  The
> >> unicode/bytes separation is pretty close to that.  Adding a warning
> >> gets even closer.  Adding magic makes it worse.
> >
> > Well, I see two cases:
> > 1. Converting from an uncertain representation to a known one.
> > 2. Converting from a known representation to a known one.
>
> Not quite:
> 1. Using a garbage file name locally (within a single process, not
> talking to any libs)
> 2. Using a unicode filename everywhere (libs, saved to config files,
> displayed to the user, etc.)

I think there is some misunderstanding. I was referring to conversions and 
whether it is good to perform them implicitly. For that, I saw the above two 
cases.

> On linux the bytes/unicode separation is perfect for this.  You decide
> which approach you're using and use it consistently.  If you mess up
> (mixing bytes and unicode) you'll consistently get an error.
>
> We currently don't follow this model on windows, so a garbage file
> name gets passed around as if it was unicode, but fails when passed to
> a lib, saved to a config file, is displayed to a user, etc.

I'm not sure I agree with this. Facts I know are:
1. On POSIX systems, there is no reliable encoding for filenames while the 
system APIs use char/byte strings.
2. On MS Windows, the encoding for filenames is Unicode/UTF-16.

Returning Unicode strings from readdir() is wrong because it can't handle the 
case 1 above. Returning byte strings is wrong because it can't handle case 2 
above because it gives you useless roundtrips from UTF-16 to either UTF-8 or, 
worst case, to the locale-dependent MBCS. Returning something different 
depending on the system us also broken because that would make Python code 
that uses this function and assumes a certain type unportable.

Note that this doesn't get much better if you provide a separate readdirb() 
API or one that simply returns a byte string or Unicode string depending on 
its argument. It just shifts the brokenness from readdir() to the code that 
uses it, unless this code makes a distinction between the target systems. 
Since way too many programmers are not aware of the problem, they will not 
handle these systems differently, so code will become non-portable.

What I'd just like some feedback on is the approach to return a distinct type 
(neither a byte string nor a Unicode string) from readdir(). In order to use 
this, a programmer will have to convert it explicitly, otherwise e.g. 
printing it will just produce . This will 
immediately bump each programmer with their heads on the issue of unknown 
encodings and they will have to make the application-specific choice whether 
an approximation of the filename, an exception or ignoring the file is the 
right choice. Also, it presents the options for doing this conversion in a 
single class, which I personally find much better than providing overloads 
for hundreds of functions.


Sorry for ranting, but I'm a bit confused and desperate, because either I'm 
unable to explain what I mean or I'm really not understanding something that 
everybody else here seems to agree upon. I just know that using a distinct 
path type has helped me in C++ in the past, and I don't see why it shouldn't 
in Python.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Thursday 11 December 2008, Steve Holden wrote:
> Ulrich Eckhardt wrote:
> > What I'd just like some feedback on is the approach to return a distinct
> > type (neither a byte string nor a Unicode string) from readdir(). In
> > order to use this, a programmer will have to convert it explicitly,
> > otherwise e.g. printing it will just produce .
> > This will immediately bump each programmer with their heads on the issue
> > of unknown encodings and they will have to make the application-specific
> > choice whether an approximation of the filename, an exception or ignoring
> > the file is the right choice. Also, it presents the options for doing
> > this conversion in a single class, which I personally find much better
> > than providing overloads for hundreds of functions.
[...]
>
> Seems to me this just threatens to add to the confusion.
>
> If you know what your filesystem produces, you can take the appropriate
> action to convert it into a type that makes sense to the user. If you
> don't, then at least if you have the string in its bytes form you can
   ^^^

There are operating systems that don't use bytes to represent a file path, 
namely all the MS Windows variants. Even worse, when you use a byte string 
there, it typically means that you want to use the obsolete encoding that is 
based on codepages.

Why can we not preserve the representation of a path as it is? Why do we 
_have_ to convert it to anything at all, without even knowing if this 
conversion is needed? I just want to do something to a file's content, why 
does its path have to be converted to something and then be converted back in 
order for the system to digest it?

> re-present it to the filesystem to manipulate the file. What are we
> supposed to do with the "special type"?

You receive from readdir() and pass it to stat(), simple as that. No 
conversions from the native representation needed. If you need a textual 
representation, then you have to convert it and you have to do so explicitly 
according to whatever logic your application requires.

If readdir() returned Unicode text, people would start taking that for 
granted. If it returned bytes, just the same. Returning a completely 
unrelated type will give them enough hint that for this thing they have to 
rethink their assumptions. This runs along the lines of "In the face of 
ambiguity, refuse the temptation to guess.", as it makes guessing rather 
impossible.

I just don't see a case where using a separate path class would break things. 
Further, the special handling that is required would be made even clearer by 
using such a class.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Thursday 11 December 2008, Steve Holden wrote:
> Ulrich Eckhardt wrote:
> > If readdir() returned Unicode text, people would start taking that for
> > granted. If it returned bytes, just the same. Returning a completely
> > unrelated type will give them enough hint that for this thing they have
> > to rethink their assumptions. This runs along the lines of "In the face
> > of ambiguity, refuse the temptation to guess.", as it makes guessing
> > rather impossible.
>
> So you are suggesting this "special object" be used only to represent
> files to users? Now I understand.

Not only files, the same problem crops up when handling sys.argv and 
os.environ.

> > I just don't see a case where using a separate path class would break
> > things. Further, the special handling that is required would be made even
> > clearer by using such a class.
>
> But it does have to be implemented ...

Well, it isn't really terribly difficult to do so, after all its just a 
container for either a byte string or Unicode string plus some helper code to 
convert it to/from Unicode.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Thursday 11 December 2008, Adam Olsen wrote:
> The simplest solution there is to have windows bytes APIs that return
> raw UTF-16 bytes (note that windows does NOT guaranteed to be valid
> unicode, despite being much more likely than on linux).

Actually, I'm not aware of this case. I only know that the OS refuses to mount 
media it can't decode, but that is on the OS-level. Can you give me a hint?

> The only real issue I see is that UTF-16 isn't an ASCII superset, so it
> won't print nicely.

True, but I personally couldn't care less. Actually, I would even prefer if 
printing a byte string always produced \x escaped byte values, that way it 
would at least be consistent. 

> In other words, bytes can be your special type.

That would actually be a lot of work to do, but I do agree that it would be a 
way. 

The problem though is that I have seen quite a few places in Python where such 
a byte string is passed as 'char*' and treated with the assumption that 
strlen() would yield a meaningful value there, so this calls at least for a 
distinct 'Py_Byte' type. Also, this still doesn't even remotely handle the 
problem that you do have two valid encodings on win32, even though the MBCS 
one could be called deprecated. People will try to interface to other 
libraries that use win32 CHAR strings and that will be much harder or even 
impossible. Further, and that is IMHO the worst part of it, things will fail 
too silently and programmers aren't encouraged to write portable code, but 
maybe I'm just too pessimistic.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Friday 12 December 2008, Stephen J. Turnbull wrote:
> I gather that the BFDL's line on this thread of discussion is that
> forcing programmers to think about encodings every time they call out
> to the OS is unacceptable

Exactly that is not necessary.

  for n in os.readdir('.'):
  f = open(n)
  if grep('foo', f):
  print('found "foo"!')

Now, if you actually wanted to output the filename, you could never do so 
reliably anyway, because even though it is supposed to be text, the encoding 
isn't known. So, an archiving program will probably do something like this:

   try:
   for n in os.readdir():
   b = n.encode('UTF-8')
   f = open(n)
   archive.write_file_header(b)
   archive.write_file(f)
   catch ...
   print "oops, couldn't decode file '%s'" % n.unicode(error='replace')

If you're writing a filemanager, you would store the path alongside an 
approximated Unicode representation.


> when most programs will work acceptably 
> almost all of the time with a rather naive approach.  This means that
> almost all Python programs will be technically broken for the
> forseeable future, sorry, Ulrich.

Actually, they are already broken, only that few people notice it. :|

> And for the same pragmatic reasons, these functions are going to
> return strings (ie, Unicode), not bytes, I expect.  Sorry, Steve.
>
> What needs to be determined here is the best way to provide
> reliability to those who will go to the effort of asking for it if
> it's available.  I don't think "just return bytes" fits the bill for
> the reason above.
>
> What I would like to see is a type that is derived from string (so if
> you present it to an API expecting string, it is silently treated as
> string), but from which the original bytes can always be extracted on
> request.

I like that idea, this type would behave pretty much like the env_string I 
proposed. The main difference is that it does several implicit conversions 
where I personally would rather see explicit conversions. Other than that, 
I'm all for it.

> If the original bytes cannot be sensibly decoded to a 
> string, then the string field in the object would either contain
> something that should normally cause an error in a string API, or some
> made-up string (presumably it would attempt to be a more or less
> faithful representation of the bytes) at the caller's option.
> Probably they'd also contain some metadata useful in guessing
> encodings (the read time locale in particular).

Well, I wouldn't provide an approximation. Considering the archiving software 
above, you would end up with a file name "" in an 
archive. For that kind of software, it would be fatal. But, and that is much 
more important than my preference, at least your approach would allow writing 
reliable software that properly handles such environment strings. Further, 
and that is where it differs from just returning bytes, it even makes it easy 
by the using a distinct type.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-3.0, unicode, and os.environ

[Ulrich Eckhardt]

On Friday 12 December 2008, Adam Olsen wrote:
> Only pages like this, which indicate the underlying API is an array of
> WCHAR:
>
> http://blogs.msdn.com/michkap/archive/2005/05/11/416552.aspx

Hmm, true. So even there, the encoding isn't known...

> char * is just fine.  You need only pass a length along with it.  All
> internal APIs *must* already do this, as they support nul bytes.  Also
> note that the underlying POSIX APIs prohibit nul bytes in filenames,
> so it's irrelevant for them.

Hmmm, I see things like Py_GetPath() in the 2.7 sourcecode, which returns a 
plain char*. I really need to check if 3.0 is better.

thanks for the info

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.

**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] WinCE port (issues #4075 #4051)

[Ulrich Eckhardt]

Hi!

I'm currently working again on the CE port, and since 2.6 and 3.0 are now out 
of the door, could you apply the patches in #4075 & #4051? Both patches are 
fairly isolated and easy to review and I'm pretty sure they won't cause any 
inconveniences.

Note: this is far from everything that is necessary for Python to rock on CE, 
but these are prerequisites, as explained in both bugs' histories.

thanks

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] #ifdef __cplusplus?

[Ulrich Eckhardt]

Hi!

There are lots of files that are framed with an extern "C" stanza when 
compiled under C++. Now, I appreciate that header files are made suitable for 
use with C++ with that, but WTF are those doing in .c files???

puzzled greetings

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #ifdef __cplusplus?

[Ulrich Eckhardt]

On Friday 02 January 2009 06:17:24 Alexander Belopolsky wrote:
> Since that issue is closed, I have created
> http://bugs.python.org/issue4805 with a patch that restores C++
> compilability of the core and a few standard modules.

Looking at the patch, I see three main changes there:
1. Remove the extern "C" stanza from the .c files.
2. Add explicit casts where necessary for C++.
3. Include headers instead of locally declaring functions and adding some 
declarations to headers.

In particular the third part of above, I personally would definitely vote a 
+1. Since the first one is only necessary because things are declared 
sloppily, I'm also +1 on that one, i.e. #3 makes #1 possible.

As far as the second part is concerned, I personally wouldn't bother getting 
Python to compile with a C++ compiler. I'd also like to point out that a 
missing declaration of e.g. malloc(), and the ensuing implicit declaration 
as "int malloc(int)" will be hidden when there's an explicit cast, which is 
why e.g. the comp.lang.c FAQ is against it.

> Note that this discussion has deviated from OP's original question.
> While I argue that C++ compilability of source code is good thing, I
> agree with OP that wrapping non-header file code in extern "C" {} is
> bad practice.  For example, the only reason Objects/fileobject.c does
> not compile without extern "C" {} is because fclose is declared inside
> PyFile_FromString as follows:
>
>  PyObject *
>  PyFile_FromString(char *name, char *mode)
>  {
>   extern int fclose(FILE *);
>  ..
>
> I would rather #include  at the top of the file instead.

This is change #3 above, and I'm all for it. I'm actually surprised that this 
code has actually escaped the peer review here.

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ParseTuple question

[Ulrich Eckhardt]

Hi!

I'm looking at NullImporter_init in import.c and especially at the call to 
PyArg_ParseTuple there. What I'm wondering is what that call will do when I 
call the function with a Unicode object. Will it convert the Unicode to a 
char string first, will it return the Unicode object in a certain (default) 
encoding, will it fail?

I'm working on the MS Windows CE port, and I don't have stat() there. Also, I 
don't have GetFileAttributesA(char const*) there, so I need a wchar_t 
(UTF-16) string anyway. What would be the best way to get one?

Thanks!

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PyOS_GetLastModificationTime

[Ulrich Eckhardt]

Hi!

The function PyOS_GetLastModificationTime() is documented in sys.rst as taking 
a "char*". However, in reality, it takes a "char*" and a "FILE*". Actually, 
it should take a "char const*", as it doesn't and shouldn't modify the path. 
Further, the normal version doesn't use the path at all, the RISCOS version 
in 2.7 does however, and for a CE port it would be convenient to have that 
info, too.

There is another issue, and that is that the function isn't declared anywhere, 
and I'm not sure where it should be declared either. Actually, I'm not sure 
if is suitable/intended for public consumption, so I wonder if putting it 
into 'Include' would be right.

Any suggestions how to deal with that issue?

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyOS_GetLastModificationTime

[Ulrich Eckhardt]

On Friday 02 January 2009 22:30:39 Ulrich Eckhardt wrote:
> The function PyOS_GetLastModificationTime() is documented in sys.rst as
> taking a "char*". However, in reality, it takes a "char*" and a "FILE*".
> Actually, it should take a "char const*", as it doesn't and shouldn't
> modify the path. Further, the normal version doesn't use the path at all,
> the RISCOS version in 2.7 does however, and for a CE port it would be
> convenient to have that info, too.
>
> There is another issue, and that is that the function isn't declared
> anywhere, and I'm not sure where it should be declared either. Actually,
> I'm not sure if is suitable/intended for public consumption, so I wonder if
> putting it into 'Include' would be right.

Actually, the whole thing might be a non-issue. The point is that the function 
is not used anywhere.

> Any suggestions how to deal with that issue?

...of course that question remains. Remove all traces? I'd volunteer to make a 
patch, as it's one less function to port. ;)

cheers

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyOS_GetLastModificationTime

[Ulrich Eckhardt]

On Friday 02 January 2009 23:18:04 Martin v. Löwis wrote:
> Correct me if I'm wrong: it seems that the function isn't called
> anymore. So I propose to just remove it (and the file it lives
> in).

Filed as issue #4817, including patch.

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I would like an svn account

[Ulrich Eckhardt]

On Saturday 03 January 2009 16:52:56 Victor Stinner wrote:
> > A little offtopic: it seems to me it is a flaw of svn, that it
> > encourages the model of two classes of developers, those with a commit
> > access (first class) and those without it (second class).
>
> Yes, that's the problem. Is it not possible to have finer permission
> (instead of boolean permission: commit or not commit)? Eg. give commit
> access but only for a file or a directory?

Yes it is possible.

As far as your goal is concerned, couldn't you live with a branch where you 
develop the feature? That way, people could see your code and e.g. switch 
their working copies there for testing or even merge it into trunk some day. 
SVN actually supports that rather well, it would be guaranteed to not affect 
the quality of the releases negatively and saying "please merge r1234 from 
foo into trunk" is much easier than downloading and applying a patch, which 
doesn't even cover all possible changes that SVN does.

Actually, I'd like such a branch, too, where I could move much quicker and in 
particular with the backing of a VCS to port Python to MS Windows CE. 
Currently, I'm tempted to pull the code into a private repository, which 
causes problems when I want to push it back upstream.

Uli



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I would like an svn account

[Ulrich Eckhardt]

On Saturday 03 January 2009 17:21:16 Antoine Pitrou wrote:
> Ulrich Eckhardt  knuut.de> writes:
> > saying "please merge r1234 from
> > foo into trunk" is much easier than downloading and applying a patch,
> > which doesn't even cover all possible changes that SVN does.
>
> I don't know about others, but downloading and applying a patch doesn't
> bother me (it's actually much quicker than doing a whole new SVN checkout).

1. I think that a patch can not e.g. capture a moved, renamed or deleted file. 
Further, it can not handle e.g. things like the executable bit or similar 
things that SVN otherwise does manage. That is what makes a patch only 
partially suitable.
2. You don't checkout anew. You simply switch ("svn switch") your existing 
working copy to the branch which just pulls the differences and merges them 
into your existing working copy. Or, you could merge the changes on a branch 
("svn merge") into your working copy.

> What takes time and effort is to actually check and review the patch (or
> branch, or whatever).

Yes, full ACK.

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] I would like an svn account

[Ulrich Eckhardt]

On Saturday 03 January 2009 17:36:04 Martin v. Löwis wrote:
> > As far as your goal is concerned, couldn't you live with a branch where
> > you develop the feature?
>
> That still doesn't help the change getting merged into the trunk.
> Whether you store it in a patch file, in a DVCS, or in the very same
> VCS-but-different-branch - these are all minor details, which may affect
> the efficiency of producing and technically integrating the patch. It
> doesn't help to the least in speeding up reviews of the patch, or
> reduces the amount of work necessary to do a review.

This is true...

> > Actually, I'd like such a branch, too, where I could move much quicker
> > and in particular with the backing of a VCS to port Python to MS Windows
> > CE. Currently, I'm tempted to pull the code into a private repository,
> > which causes problems when I want to push it back upstream.
>
> [I guess you aren't happy with the DVCS systems, such as bazaar, which
> supposedly work perfect in exactly this case. I won't blame you for
> that, but still, consider trying out one of them for this project]

I tried bazaar, but it's just too much to tackle at once: porting to CE, 
learning BZR and maintaining a feature branch on trunk (though the latter 
should not be too difficult, according to BZR's reputation).

> We can setup such a branch, unless you reconsider and try bazaar first.
> There wouldn't be any pushing it back upstream, though - you would still
> need to go through the tracker for all changes. The only advantage I
> can see is that it simplifies repeated merging of the trunk into your
> branch.

Actually, now that I think about it, I must admit that it doesn't really 
matter to me whether I use a local mirror or work on a remote branch. The 
problem is still splitting up the whole port into pieces that can be digested 
(read: reviewed) at a time. Since I'm confident with the use of SVN, I'll for 
now stay with it and a local mirror, but any single change that can be 
submitted will be submitted, hopefully to have something working here soon.

Cheers!

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ParseTuple question

[Ulrich Eckhardt]

[sorry, dropped one pair of mails off the list, hence also the overquoting]

On Sunday 04 January 2009 01:07:08 Mark Hammond wrote:
> > > On 'normal' windows you generally would need to use
> > > WideCharToMultiByte() to get a 'char *' version of your wchar
> > > string but I expect you already know that, so I doubt I understand
> > > the question...
> >
> > Actually, I want the exact opposite.
>
> I'm not sure you can get it though :)  Unless you can change Python itself,
> you will be forced to convert filenames etc back and forward from wide to
> narrow strings.  This should roundtrip OK for all character in the current
> code set.
>
> > 'char' strings under CE are mostly useless, in particular for
> > filenames. Actually, they are useless under MS Windows in general,
> > at least for the versions supported by Python 2.6/3.0, they all use
> > UTF-16, only the desktop variants still have backward compatibility
> > code to use 'char' strings, though with restricted functionality.
>
> Exactly - but for python 2.6, we are still somewhat forced to use that
> 'char *' API in many cases.  If the API didn't offer the automatic unicode
> conversion, we would most likely have needed to implement it ourselves.
>
> > So, point is that I need a UTF-16 string in a function that might
> > get either a string or a Unicode string. Preferably, I'd like to take
> > the way that poses least resistance, so which one would you suggest?
>
> I'm not with you still: if a Python implemented function accepts either
> string or unicode, then your utf16 string is perfect - its already unicode.
> On the other hand, I thought you were faced with a Python function which,
> as currently implemented, only accepts whatever the 's' format string
> accepts. Such a function only accepts real PyString objects, so attempts at
> passing it unicode will be futile.  Obviously you could modify it to accept
> a unicode, but clearly that would also mean adjusting everything that uses
> the existing 'char *', which may end up fanning out to much more than you
> expect.
>
> If I'm still misunderstanding, can you be more specific about the exact
> problem (ie, the exact function you are referring to, and how you intend
> calling it)?

trunk/_fileio.c/fileio_init()

Let's leave aside that you can also pass a filedescriptor, that function 
either takes a string or a Unicode string as first parameter. Now, under CE, 
I always need a 'wchar_t*' in order to open a file, how would I get at that 
easiest?

My approach now is to simply use "O" as format specifier for the filename and 
then take a look at the object's type. If it is a char-string, I have to 
convert (under CE) or call the char-API (desktop MS Windows), if it is a 
Unicode string, I can use it as it is.

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] another Python Bug Day?

[Ulrich Eckhardt]

On Monday 05 January 2009 23:48:13 Malte Helmert wrote:
> For people who are not core developers but would still like to
> contribute, the Bug Days are quite exciting events. It would be great if
> they could keep going.

As a not core developer, I would like to know what exactly that means.

;)

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] a few strdup() questions...

[Ulrich Eckhardt]

Greetings!

MS Windows CE doesn't provide strdup(), so where should I put it? I guess I 
should just compile in Python/strdup.c, right?

However, where should I declare it? My approach would be to declare it in 
PC/pyconfig.h. I see that RISCOS also seems to lack that function, which is 
why it is declared locally in _localemodule.c, but I guess this isn't really 
the best of all possible ways.

Also, there is HAVE_STRDUP. I would actually expect that #undef HAVE_STRDUP 
would do the trick to at least declare this, but it doesn't. I guess that 
most modern OS have this so this will probably just be bitrot ... right? But 
where should I put the declaration?

BTW: there is another implementation (called my_strdup) in 
Modules/_ctypes/_ctypes_test.c, why not use the one in Python/strdup.c there?

Lastly: I would have written the thing a bit differently:

char* strdup(char const* s) {
char* res;
size_t len;

assert(s);

len = strlen(s);
res = malloc(len+1);
if(res)
memcpy(res, s, len+1);
return res;
}

First difference is that I wouldn't accept NULL as valid input, e.g. the glibc 
implementation doesn't either and GCC even warns you if you call 
strdup(NULL). Secondly, I would have used memcpy(), since the length is 
already known and then potentially quicker. Should I write a patch?

thanks

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] a few strdup() questions...

[Ulrich Eckhardt]

On Wednesday 07 January 2009 16:30:23 Daniel Stutzbach wrote:
> On Wed, Jan 7, 2009 at 5:30 AM, Ulrich Eckhardt  wrote:
> > MS Windows CE doesn't provide strdup(), so where should I put it? I guess
> > I should just compile in Python/strdup.c, right?
>
> I'm not an expert on Windows CE, but I believe it calls the function
> "_strdup()":
>
> http://msdn.microsoft.com/en-us/library/ms861162.aspx

Search with "Look in VC Include Directories" yields nothing. You are right 
though, the CE6 SDK I have does declare _strdup in stdlib.h and also provides 
an implementation to link with.

Summary: redefine strdup and work around the broken search feature.

thanks

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_END_ALLOW_THREADS and GetLastError()

[Ulrich Eckhardt]

On Saturday 10 January 2009 12:29:47 Kristján Valur Jónsson wrote:
> Currently on Windows, Py_END_ALLOW_THREADS can have the side effect of
> resetting the windows error code returned by GetLastError(). There is a
> number of cases, particularly in posixmodule, with a pattern like:
> Py_BEGIN_ALLOW_THREADS
> result = FindNextFile(hFindFile, &FileData);
> Py_END_ALLOW_THREADS
> /* FindNextFile sets error to ERROR_NO_MORE_FILES if
>it got to the end of the directory. */
> if (!result && GetLastError() != ERROR_NO_MORE_FILES) {
>
> That doesn´t work.  (This particular site is where I noticed the problem,
> running the testsuite in a debug build). Now, the thread swith macro does
> take care to preserve "errno", but not the windows system error.  This is
> easy to add, but it requires that windows.h be included by ceval.c and
> pystate.c The alternative fix is to find all these cases and manually
> preserve the error state, or query it right after the function call if
> needed. Any preferences?

Well, that's what you get for using globals and exactly that is the reason why 
their use is generally discouraged. My preference would be to fix all cases 
where there is an intervening call after the call that set errno to first 
preserve that state. As a short term fix, I would add a workaround to 
Py_END_ALLOW_THREADS though, both for errno and win32's GetLastError().

Generally, I would discourage non-local errno use. My motivation is that MS 
Windows CE simply doesn't have errno and MS Windows in general often uses 
different ways to signal errors, so not using it would restrict the 
conditionally compiled code further. Especially in the math module, I have no 
idea yet how to port that in a way that is still at least remotely clean and 
maintainable because errno is used everywhere there. Translating the errno 
value to a Python error directly after the failed call would help immensely.

just my two embedded cents

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_END_ALLOW_THREADS and GetLastError()

[Ulrich Eckhardt]

On Saturday 10 January 2009 15:11:16 Martin v. Löwis wrote:
> > Well, that's what you get for using globals
>
> Please do take a look at the issue at hand before pointing fingers.

I'm really sorry if this sounded like I was accusing someone,
that was not my intention.

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] How should I handle unsupported features?

[Ulrich Eckhardt]

Hi!

Porting to MS Windows CE, I find that e.g. signals or environment vars are not 
supported. How should I handle that? In particular, I'm talking about 
PyOS_getsig() and PyOS_setsig(). Should I just #ifdef them out completely or 
should I implement them by setting an error? Which error should I set?

cheers

Uli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Update on MS Windows CE port

[Ulrich Eckhardt]

Hi!

Just a short update on my porting issues. I tried various things in the 
holidays/vacation, filed several bug reports, got a few patches applied and I 
think the thing is taking shape now. However, I couldn't work on it for the 
past two weeks since I'm just too swamped at work here. I haven't given up 
though!


What still needs work?

The main component that requires porting is the pythoncore project, but that 
port isn't finished yet. I'm using the VS8.0 project files as a base and 
adapted it in the following steps (note that this is preliminary):
0. Check out trunk (i.e. 2.x, not 3.x).
1. Create new project configuration for your CE target and use the win32 one 
as a base.
2. In the preprocessor settings, add these entries:
   UNICODE,_UNICODE,UNDER_CE,_WIN32_WCE=$(CEVER)
3. Try compiling. ;)


What issues are left?

There are two classes of errors you will encounter, those that are related to 
CE itself (like missing errno and other parts of standard C) and those that 
are more general, like assuming TCHAR=char. Those that assume that TCHAR=char 
can also be found with the plain win32 variant by simply adding UNICODE and 
_UNICODE to the preprocessor defines.


What are the future plans?

I'm trying to fix errors in pythoncore one by one and provide separate bug 
reports with patches. Note that there are already several patches in the BTS 
which would merit reviews if you have time, even if it's only a "patch 
applies and doesn't cause regressions".

When I get pythoncore to compile (and be it by disabling a few builtin 
modules), I will try to get the commandline interface to work. Note that IIRC 
there are two commandlines availabel under CE, one is a common cmd.exe, the 
other is called AYGShell or some such. Both will/might require different 
approaches.

Another problem is the fact that there is not just one target platform but a 
potentially infinite number of them, since every CE device can have a 
separate SDK which further might provide different features. This probably 
requires autogenerated build files or some other autoconf-like procedure that 
allows enabling or disabling certain features for builds.


so much for now

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules.

[Ulrich Eckhardt]

On Sunday 25 January 2009, Luke Kenneth Casson Leighton wrote:
> matthieu, thank you for responding.  from
> http://en.wikipedia.org/wiki/Dynamic-link_library:
>
> "Third, dynamic linking is inherently the wrong model for paged memory
> managed systems. Such systems work best with the idea that code is
> invariant from the time of assembly/compilation on.
> ... Data references do not need to be so vectored because
>  DLLs do not share data."
> 
>
> does anyone happen to know what this means?

I can only guess: The difference between code and data is that code can be 
loaded into a process by simply mapping it into the virtual memory. For data 
that is constant, the same applies. For non-const data, you absolutely must 
not do that though, because it would make processes interfere with each 
other, and that is what the above text probably means.

So, the important difference is rather that read-only stuff can be 
memory-mapped while read-write stuff can't. Since code is read-only (barring 
self-modifying code and trampolines etc), it is automatically always 
sharable.

> curt, thank you for responding.  i'd seen this: i understood it -
> and... yet... mingw happily segfaults when asked to access _any_ data
> in _any_ object file of the python2N dll.

Dump the address of said data and its size from inside that DLL and from 
outside just to see if they differ, both from the same process. I'd also dump 
the size, in case different compiler settings messed up padding or something 
like that.

> from looking so far. e.g. i expected MSVCRT.DLL errno to be an
> int - it's not: it's a function).

'errno' can't be an int, because it needs to be thread-local. Also, note the 
important difference between "errno is an int" and "errno yields an lvalue of 
type int". The latter is how the standard defines it.

> *sigh*.  if this turns out to be yet another gcc / mingw bug i'm going
> to be slightly annoyed.  only slightly, because this _is_ free
> software, after all :)

Can you reproduce this with a separate example?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] FormatError() in callproc.c under win32

[Ulrich Eckhardt]

Hi!

In callproc.c from trunk is a function called SetException(), which calls 
FormatError() only to discard the contents. Can anyone enlighten me to the 
reasons thereof? Is it just to check if the errorcode is registered in the 
stringtables?

The reason I ask is the CE port. The FormatMessage API exists there (or, 
rather, the FormatMessageW API), but the tables containing the error messages 
are optional for the OS and for space reasons many vendors chose not to 
include them. That means that the function there regularly fails to retrieve 
the requested string.

My first approach was to fall back to simply providing a sting with a numeric 
representation of the errorcode, but that would change the meaning of above 
function, because then it could never fails.

My second approach was to enhance PyErr_SetFromWindowsErr() to handle the 
additional error codes that are checked in SetException(). However, those 
require more context than just the error code, they use the EXCEPTION_RECORD 
passed to SetException() for that.

My third approach would be to filter out the special error codes first and 
delegate all others to PyErr_SetFromWindowsErr(). The latter already handles 
the lack of a string for the code by formatting it numerically. This would 
also improve consistency, since the two functions use different ways to 
format unrecognised errors numerically. This approach would change where and 
how a completely unrecognised error code is formatted, but would otherwise be 
pretty equivalent.


Suggestions?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FormatError() in callproc.c under win32

[Ulrich Eckhardt]

On Monday 26 January 2009, Thomas Heller wrote:
> Ulrich Eckhardt schrieb:
> > In callproc.c from trunk is a function called SetException(), which calls
[...]
> > My third approach would be to filter out the special error codes first
> > and delegate all others to PyErr_SetFromWindowsErr(). The latter already
> > handles the lack of a string for the code by formatting it numerically.
> > This would also improve consistency, since the two functions use
> > different ways to format unrecognised errors numerically. This approach
> > would change where and how a completely unrecognised error code is
> > formatted, but would otherwise be pretty equivalent.
>
> The third approach is fine with me.  Sidenote: The only error codes that I
> remember having seen in practice are 'access violation reading...' and
> 'access violation writing...', although it may be that on WinCE 'datatype
> misalignment' may also be possible.

Submitted as patch for issue #5078.

Note: under CE, you can actually encounter datatype misalignments, since it 
runs on CPUs that don't emulate them. I wonder if the same doesn't also apply 
to win64

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at <http://www.satorlaser.de/>
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FormatError() in callproc.c under win32

[Ulrich Eckhardt]

On Monday 26 January 2009, Martin v. Löwis wrote:
> > In callproc.c from trunk is a function called SetException(), which calls
> > FormatError() only to discard the contents. Can anyone enlighten me to
> > the reasons thereof?
>
> Interestingly enough, the code used to say
>
>PyErr_SetString(PyExc_WindowsError, lpMsgBuf);
>
> Then it was changed to its current form, with a log message of
>
>Changes for windows CE, contributed by Luke Dunstan.  Thanks a lot!
>
> See
>
> 
http://ctypes.cvs.sourceforge.net/viewvc/ctypes/ctypes/source/callproc.c?hideattic=0&r1=1.127.2.15&r2=1.127.2.16
>
> I suggest you ask Thomas Heller and Luke Dunstan (if available) what the
> rationale for this partial change was.

I can only guess:
1. Those changes seem to generate TCHAR strings. This is necessary to compile 
it on both win9x (TCHAR=char) and CE (TCHAR=wchar_t). Since win9x was dropped 
from the supported platforms, that isn't necessary any more and all the code 
could use WCHAR directly.
2. Those changes also seem to change a few byte-strings to Unicode-strings, 
see format_error(). This is a questionable step, since those are changes that 
are visible to Python code. Worse, even on the same platform it could return 
different string types when the lookup of the errorcode fails. I wonder if 
that is intentional.

In any case, CCing Luke on the issue, maybe he can clarify things.

cheers

Uli

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python as a Metro-style App

[Ulrich Eckhardt]

Am 07.01.2012 18:57, schrieb "Martin v. Löwis":

I just tried porting Python as a Metro (Windows 8) App, and failed.

Metro Apps use a variant of the Windows API called WinRT that still
allows to write native applications in C++, but restricts various APIs
to a subset of the full Win32 functionality. For example, everything
related to subprocess creation would not work; none of the
byte-oriented file API seems to be present, and a number of file
operation functions are absent as well (such as MoveFile).


Just wondering, do Metro apps define UNDER_CE or _WIN32_WCE? The point 
is that the old ANSI functions (CreateFileA etc) have been removed from 
the embedded MS Windows CE long ago, too, and MS Windows Mobile used to 
be a custom CE variant or at least strongly related. In any case, it 
could help using the existing (incomplete) CE port as base for Metro.


Uli
**
Domino Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Domino Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Improve error message "UnboundLocalError: local variable referenced before assignment"

[Ulrich Eckhardt]

Am 31.10.2012 23:15, schrieb Steven D'Aprano:

On 01/11/12 06:57, anatoly techtonik wrote:
[...]

UnboundLocalError: local variable 'FONT_NAMES' referenced before
assignment

[...]

I wonder if this message can be improved with a
pointer to the concept on when global variables become local?


If you have a suggestion for an improved message, please tell us.


I'll take a shot, since I was also bitten by this when trying to learn 
Python. The important point is that some code earlier or later in that 
function does an assignment, so this location should be referenced in 
the error.


How about:

  "UnboundLocalError: Local variable 'FONT_NAMES' (created on
   line 11) referenced before assignment."

What I don't really like is the term "created". Maybe "implicitly 
created on line 11"? Or "implied by line 11"? Or how about "Local 
variable FONT_NAMES (implied by line 11) doesn't refer to an object", to 
avoid the multiple interpretations of the term "assignment"?


=just my 2cc=

Uli

**
Domino Laser GmbH, Fangdieckstra�e 75a, 22547 Hamburg, Deutschland
Gesch�ftsf�hrer: Hans Robert Dapprich, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschlie�lich s�mtlicher Anh�nge ist nur f�r den 
Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte 
benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte 
Empf�nger sein sollten. Die E-Mail ist in diesem Fall zu l�schen und darf 
weder gelesen, weitergeleitet, ver�ffentlicht oder anderweitig benutzt werden.
E-Mails k�nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte 
�nderungen enthalten. Domino Laser GmbH ist f�r diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] performance of {} versus dict()

[Ulrich Eckhardt]

Am 14.11.2012 10:12, schrieb Chris Withers:

Can someone with Python 3 handy compare there too?


C:\Python27\python -m timeit -n 100 -r 5 -v
   "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 0.918 0.924 0.922 0.928 0.926
100 loops, best of 5: 0.918 usec per loop

C:\Python27\python -m timeit -n 100 -r 5 -v
   "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.48 0.49 0.482 0.496 0.497
100 loops, best of 5: 0.48 usec per loop

C:\Python32\python -m timeit -n 100 -r 5 -v
   "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 0.898 0.891 0.892 0.899 0.891
100 loops, best of 5: 0.891 usec per loop

C:\Python32\python -m timeit -n 100 -r 5 -v
   "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.444 0.463 0.461 0.464 0.461
100 loops, best of 5: 0.444 usec per loop

C:\Python32_64\python -m timeit -n 100 -r 5 -v
   "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 0.908 0.923 0.927 0.912 0.923
100 loops, best of 5: 0.908 usec per loop

C:\Python32_64\python -m timeit -n 100 -r 5 -v
   "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.484 0.446 0.501 0.45 0.442
100 loops, best of 5: 0.442 usec per loop

C:\Python33_64\python -m timeit -n 100 -r 5 -v
   "dict(a=1, b=2, c=3, d=4, e=5, f=6, g=7)"
raw times: 1.02 1.07 1.03 1.11 1.07
100 loops, best of 5: 1.02 usec per loop

C:\Python33_64\python -m timeit -n 100 -r 5 -v
   "{'a':1, 'b':2, 'c':3, 'd':4, 'e':5, 'f':6, 'g':7}"
raw times: 0.444 0.449 0.455 0.452 0.46
100 loops, best of 5: 0.444 usec per loop


Tested on Win7/64 bit. Python 2.7 is the 32-bit version, 3.2 is 
installed as 32-bit and 64-bit versions and 3.3 only as 64-bit version. 
In any case, the difference is even a bit stronger than what you 
experience and it seems that all versions perform roughly similar.


Uli



**
Domino Laser GmbH, Fangdieckstra�e 75a, 22547 Hamburg, Deutschland
Gesch�ftsf�hrer: Hans Robert Dapprich, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschlie�lich s�mtlicher Anh�nge ist nur f�r den 
Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte 
benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte 
Empf�nger sein sollten. Die E-Mail ist in diesem Fall zu l�schen und darf 
weder gelesen, weitergeleitet, ver�ffentlicht oder anderweitig benutzt werden.
E-Mails k�nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte 
�nderungen enthalten. Domino Laser GmbH ist f�r diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposing "Argument Clinic", a new way of specifying arguments to builtins for CPython

[Ulrich Eckhardt]

Am 03.12.2012 23:29, schrieb Larry Hastings:
[...autogen some code from special comment strings...]

   /*[clinic]
   dbm.open -> mapping
   basename=dbmopen

   const char *filename;
   The filename to open.

   const char *flags="r";
   How to open the file.  "r" for reading, "w" for writing, etc.

   int mode=0666;
   default=0o666
   If creating a new file, the mode bits for the new file
   (e.g. os.O_RDWR).

   Returns a database object.

   [clinic]*/


Firstly, I like the idea. Even though this "autogenerate in-place" seems 
a bit strange at first, I don't think it really hurts in practice. Also, 
thanks for introducing me to the 'cog' tool, I think I'll use this now 
and then!


This also brings me to a single question I have for your proposal: Why 
did you create another DSL instead of using Python, i.e. instead of 
using cog directly? Looking at the above, I could imagine this being 
written like this instead:


/*[[[cog
import pycognize
with pycognize.function('dbmopen') as f:
f.add_param('self')
f.add_kwparam('filename',
  doc='The filename to open',
  c_type='char*')
f.add_kwparam('flags',
  doc='How to open the file.'
  c_type='char*',
  default='r')
f.set_result('mapping')
]]]*/
//[[[end]]]

Cheers!

Uli


**
Domino Laser GmbH, Fangdieckstra�e 75a, 22547 Hamburg, Deutschland
Gesch�ftsf�hrer: Hans Robert Dapprich, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschlie�lich s�mtlicher Anh�nge ist nur f�r den 
Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte 
benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte 
Empf�nger sein sollten. Die E-Mail ist in diesem Fall zu l�schen und darf 
weder gelesen, weitergeleitet, ver�ffentlicht oder anderweitig benutzt werden.
E-Mails k�nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte 
�nderungen enthalten. Domino Laser GmbH ist f�r diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com