Re: [ZODB] Problem with pickle in zodb

[Etienne Robillard]

Yup, that is correct! Thanks. :-)

Etienne


Le 2018-06-03 à 07:39, Jason Madden a écrit :

You appear to have mismatched versions of ZEO on the client and server, 5.2 on 
one side and 5.1.0 on the other. Both need to be 5.2.


On Jun 3, 2018, at 06:35, Etienne Robillard  wrote:

Hi,

Any ideas why this is happening ?

erob@marina:/home/www/open-neurosecurity.org/trunk$ schevo editor 
zodb://127.0.0.1:4545
07:32:16 environ  No en_CA translation found for domain kiwi
07:32:16 environ  No en_CA translation found for domain gazpacho
libschevo 4.0.1 :: Database Navigator ()
()

/home/erob/src/libschevo/lib/schevo/gtk2/window.py(295)database_open()

-> self._db = schevo.database.open(db_alias)
(Pdb) n
zodb://127.0.0.1:4545
127.0.0.1:4545
Found epoll library, good!
/home/erob/src/libschevo/lib/schevo/backends/zodb.py:7: RuntimeWarning: 
Patching more than once will result in the union of all True parameters being 
patched
   patch_all(thread=True)
07:32:23 ZEO.asyncio.marshal  can't decode message: 
'\x80\x03(K\x01K\x00U\x06.replyU\x08\x03\xc7\xf6]\xe0\xd...'
07:32:23 ZEO.asyncio.base data_received 4 0 True
Traceback (most recent call last):
   File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/base.py", line 128, in 
data_received
 self.message_received(collected)
   File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 206, in 
message_received
 msgid, async, name, args = self.decode(data)
   File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 91, in 
pickle_decode
 return unpickler.load() # msgid, flags, name, args
ValueError: unsupported pickle protocol: 3
07:32:52 ZEO.asyncio.client   Registration or cache validation failed, Closed
Traceback (most recent call last):
   File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/client.py", line 188, in 
finish_connect
 *credentials)
ClientDisconnected: Closed
ClientDisconnected: ClientDi...ection',)






--
Etienne Robillard
tkad...@yandex.com
https://www.isotopesoftware.ca/

--
https://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 10442: character maps to

[Peter J. Holzer]

On 2018-06-03 16:36:12 -0700, bellcanada...@gmail.com wrote:
> On Tuesday, 22 May 2018 17:23:55 UTC-4, Peter J. Holzer  wrote:
> > On 2018-05-20 15:43:54 +0200, Karsten Hilbert wrote:
> > > On Sun, May 20, 2018 at 04:59:12AM -0700, bellcanada...@gmail.com wrote:
> > > > thank you for the reply, but how exactly am i supposed to find
> > > > oout what is the correct encodeing??
> > > 
> > > One CAN NOT.
> > > 
> > > The best you can do is to go ask the canonical source of the
> > > file what encoding the file is _supposed_ to be in.
> > 
> > I disagree on both counts.
> > 
> > 1) For any given file it is almost always possible to find the correct
> >encoding (or *a* correct encoding, as there may be more than one).
> > 
> >This may require domain-specific knowledge (e.g. it may be necessary
> >to recognize the human language and know at least some distinctive
> >words, or to know some special symbols likely to be used in a data
> >file), and it almost always takes a bit of detective work and trial
> >and error. But I don't think I ever encountered a file where I
> >couldn't figure out the encoding.
[...]
> 
> hello peter ...how exactly would i solve this issue .

There is no "exactly" here. Determining the encoding of a file depends
on experience and trial and error. However, I can give you some general
guide lines:

Preparation:

 Make sure you have a way to reliably display files:

 1) As a stream of bytes. On Linux hd works well for this purpose,
although a hex editor might be even better

 2) As a unicode text. On Linux terminal emulators usually use UTF-8
encoding, so viewing a file with less should be sufficient.
Beware of programs which try to guess the encoding. They can
fool you. If you don't have anything which works reliably you
might want to have a look at my utf8dump script
(https://www.hjp.at/programs/utf8dump/ (Perl code, sorry ;-)).

First guess:

As has already been mentioned, chardet usually does a good job.
So first let chardet guess the encoding. Then use iconv to convert
from this encoding to UTF-8 (or any other UTF you can reliably read)
and open it in your text reader (preparation step 2 above) to check
whether the result makes sense. If it does, you are done.

Checking other encodings:

This is where it gets tedious. You could systematically try all
encodings supported by iconv, but there are a lot of them (over
1000!). So you should try to narrow it down: What language is the
file in? On what OS was the file (probably) created? If most of the
non-ascii characters are already correct, but a few are wrong, what
other encodings are there in the same family? But however you
determined the list of candidate encodings, the actual check is the
same as above: Use iconv to convert from the candidate encoding and
check the result for plausibility.

Use the encoding in your program:

When you are done, open the file in your with open(...,
encoding='...') with the encoding you determined above.


> i have a script that works in python 2 but not pytho3..i did 2 to 3.py
> ...but i still get the errro...character undefieed..unicode decode
> error cant decode byte 1x09 in line 7414 from cp 1252..like would you
> have a sraright solution answer??..i cant get a straight answer..it
> was ported from ansi to python...so its utf-8 as far asi can see

If it is utf-8, just open the file with open(filename, encoding="utf-8") 
(or open(filename, encoding="utf-8-sig"), if it starts with a BOM). 

And follow Steven's advice and read all the stuff he mentioned. It is
important to have a firm understanding of what "character", "byte",
"encoding" etc. mean. If you understand that, the rest is easy (sometimes
tedious, but not difficult). If you don't understand that, you can only
resort to try and error and will be continuously baffled by unexpected
results.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Barry Scott]

> On 1 Jun 2018, at 14:23, Paul Moore  wrote:
> 
> On 1 June 2018 at 13:15, Barry Scott  wrote:
>> I think the reason for the \0 check is that if the string is passed to the
>> operating system with the \0 you can get surprising results.
>> 
>> If \0 was not checked for you would be able to get True from:
>> 
>>os.file.exists('/home\0ignore me')
>> 
>> This is because a posix system only sees '/home'.

Turns out that this is a limitation on Windows as well.
The \0 is not allowed for Windows, macOS and Posix.

> 
> So because the OS API can't handle filenames with \0 in (because that
> API uses null-terminated strings) Python has to special case its
> handling of the check. That's fine.
> 
>> Surely ValueError is reasonable?
> 
> Well, if the OS API can't handle filenames with embedded \0, we can be
> sure that such a file doesn't exist - so returning False is
> reasonable.

I think most of the file APIs check for \0 and raise ValueError on python3
and  TypeError on python2.

os.path.exists() is not special and I don't think should be be changed.

> 
>> Once you know that all of the string you provided is given to the operating
>> system it can then do whatever checks it sees fit to and return a suitable
>> result.
> 
> As the programmer, I don't care. The Python interpreter should take
> care of that for me, and if I say "does file 'a\0b' exist?" I want an
> answer. And I don't see how anything other than "no it doesn't" is
> correct. Python allows strings with embedded \0 characters, so it's
> possible to express that question in Python - os.path.exists('a\0b').
> What can be expressed in terms of the low-level (C-based) operating
> system API shouldn't be relevant.
> 
> Disclaimer - the Python "os" module *does* expose low-level
> OS-dependent functionality, so it's not necessarily reasonable to
> extend this argument to other functions in os. But it seems like a
> pretty solid argument in this particular case.
> 
>> As an aside Windows has lots of special filenames that you have to know about
>> if you are writting robust file handling. AUX, COM1, \this\is\also\COM1 etc.
> 
> I don't think that's relevant in this context.

I think it is. This started because the OP was surprised that they needed to 
check for \0.
There are related surprised waiting. I'm point out that its more then \0 a 
robust
piece of code will need to consider.

Barry

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Steven D'Aprano]

On Mon, 04 Jun 2018 11:16:21 +0100, Barry Scott wrote:

[...]
> Turns out that this is a limitation on Windows as well. The \0 is not
> allowed for Windows, macOS and Posix.

We -- all of us, including myself -- have been terribly careless all 
through this discussion. The fact is, this should not be an OS limitation 
at all. It is a *file system* limitation.

If I can mount a HFS or HFS-Plus disk on Linux, it can include file names 
with embedded NULs or slashes. (Only the : character is illegal in HFS 
file names.) It shouldn't matter what the OS is, if I have drivers for 
HFS and can mount a HFS disk, I ought to be able to sensibly ask for file 
names including NUL.



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Marko Rauhamaa]

Barry Scott :
> os.path.exists() is not special and I don't think should be be changed.

You are right that os.path.exists() might be logically tied to other
os.* facilities. The question is, should the application be cognizant of
the seam between the standard library and the operating system kernel?

When a Linux system call contains an illegal value, it responds with
errno=EINVAL. In Python, that's represented by the OSError exception
with e.errno=EINVAL. However, when Python encounters an illegal value
itself, it usually raises a ValueError. Is it useful for the application
to have to be prepared for OSError/EINVAL and ValueError separately? Or
should the difference be paved over by Python?

As it stands, os.path.exists() really means: the operating system
doesn't have a reason to fail os.stat() on the pathname. Python
intercedes with an exception if it can't even ask the operating system
for its opinion. That dichotomy is not suggested by the os.path.exists()
documentation. In fact, the whole point of os.path.* is to provide for
an abstraction to isolate the application from the intricacies of the
operating system specifics.

BTW, I challenge you to find a test case that tests the proper behavior
of an application if it encounters a pathname with a NUL in it. Or code
that gracefully catches a ValueError from os.path.exists().


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Paul Moore]

On 4 June 2018 at 13:01, Steven D'Aprano
 wrote:

>> Turns out that this is a limitation on Windows as well. The \0 is not
>> allowed for Windows, macOS and Posix.
>
> We -- all of us, including myself -- have been terribly careless all
> through this discussion. The fact is, this should not be an OS limitation
> at all. It is a *file system* limitation.
>
> If I can mount a HFS or HFS-Plus disk on Linux, it can include file names
> with embedded NULs or slashes. (Only the : character is illegal in HFS
> file names.) It shouldn't matter what the OS is, if I have drivers for
> HFS and can mount a HFS disk, I ought to be able to sensibly ask for file
> names including NUL.

Agreed, being completely precise in this situation is both pretty
complicated, and essential.

The question of what are legal characters in a filename is, as you
say, a filesystem related issue. People traditionally forget this
point, but in these days of cross-platform filesystem mounting,
networked filesystems[1], etc, it's more and more relevant, and
thankfully people are getting more aware of the point.

But there's also the question of what capability the kernel API has to
express the queries. The fact that the Unix API (and the Windows one,
in most cases - although as Eryk Sun pointed out there are exceptions
in the Windows kernel API) uses NUL-terminated strings means that
querying the filesystem about filenames with embedded \0 characters
isn't possible *at the OS level*. (As another example, the fact that
the Unix kernel treats filenames as byte strings means that there are
translation issues querying an NTFS filesystem that uses Unicode
(UTF-16) natively - and vice versa when Windows queries a Unix-native
filesystem).

So "it's complicated" is about the best we can say :-)

Paul

[1] And of course if you mount (say) an NTFS filesystem over NFS, you
have *two* filesystems involved, each adding its own layer of
restrictions and capabilities.
-- 
https://mail.python.org/mailman/listinfo/python-list

EuroPython 2018: Looking for a photographer

[M.-A. Lemburg]

At last year’s event, we had a professional conference photographer,
Alessia Peviani, from our community, help us cover the event in
pictures.

  https://ep2017.europython.eu/en/europython/photos/


This year she unfortunately cannot attend, so we’re looking for help
from other photographers in the community.

Here’s what we can offer:
-

* recognition and publicity by listing you as the official EuroPython
  conference photographer

* free tickets for the conference and trainings

* refund for travel and accommodation up to EUR 500

* gratitude by our attendees who really appreciate having this kind of
  documentation available

What we are asking for:
---

* cover all aspects of the conference in photos

* photos licensed under the CC BY-NC license, with a special exception
  for the EPS, so that we can use the photos for promoting the
  conference

* self-management and help with administering the Flickr group,
  uploads by other community photographers and discussions

If you are interested in helping us, please write to me...@europython.eu.



Help spread the word


Please help us spread this message by sharing it on your social
networks as widely as possible. Thank you !

Link to the blog post:

https://blog.europython.eu/post/174564294432/europython-2018-looking-for-a-photographer

Tweet:

https://twitter.com/europython/status/1003627195460390914

Enjoy,
--
EuroPython 2018 Team
https://ep2018.europython.eu/
https://www.europython-society.org/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Steven D'Aprano]

On Mon, 04 Jun 2018 13:33:28 +0100, Paul Moore wrote:

> But there's also the question of what capability the kernel API has to
> express the queries. The fact that the Unix API (and the Windows one, in
> most cases - although as Eryk Sun pointed out there are exceptions in
> the Windows kernel API) uses NUL-terminated strings means that querying
> the filesystem about filenames with embedded \0 characters isn't
> possible *at the OS level*.

I don't know whether or not the Linux OS is capable of accessing files 
with embedded NULs in the file name. But Mac OS is capable of doing so, 
so it should be possible. Wikipedia says:

"HFS Plus mandates support for an escape sequence to allow arbitrary 
Unicode. Users of older software might see the escape sequences instead 
of the desired characters."

Apple File System is an even more modern FS (it replaced HFS Plus in 2017 
as Apple's preferred OS) which supports all Unicode code points, 
including NUL.



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Indented multi-line strings

[Dan Strohl via Python-list]

> > 
> No-one is saying a method is *worse* than a standalone function - they are
> just saying it's *not sufficiently better* to justify creating a string 
> method that
> replicates an existing stdlib function.
> 

What about performance?  I would expect a string method to perform better than 
a stdlib function.  (no?)  Especially if it's something that could be used 
commonly it seems like having that function be of higher performance would be a 
benefit?

At least for me, when I do use this, it's often in places like logging where it 
could well be called a lot.

Dan

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: add me

[Bob Gailer]

On Jun 3, 2018 10:12 AM,  wrote:
>
> i am saran. i want to learn python programming.

Welcome. In future please use the python tutor email list - it is there to
provide guidance. Go to the python. Org website - look for tutorials.
That's the best place to start.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Grant Edwards]

On 2018-06-03, Gregory Ewing  wrote:
> Steven D'Aprano wrote:
>> Do you really mean to say that a computer that won't boot is POSIX 
>> compliant?
>
> No, I was pointing out the absurdity of saying that the Windows
> kernel layer is POSIX compliant, which is what the post I was
> replying to seemed to be saying.

The normal Win32 API that all Windows apps use is not Posix compliant.

However, there is an API layer Microsoft provides (or provided) that
is/was Posix compliant.  At one point, I think it was an add-on that
had to be purchased seperately.  I've never heard of anybody actually
_using_ it, but it allowed some US government purchasing droid to
check the "Posix Compliant" box on an acquisition checklist back in
the 90's.

-- 
Grant Edwards   grant.b.edwardsYow! But they went to MARS
  at   around 1953!!
  gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Indented multi-line strings

[Chris Angelico]

On Mon, Jun 4, 2018 at 11:54 PM, Dan Strohl  wrote:
>> >
>> No-one is saying a method is *worse* than a standalone function - they are
>> just saying it's *not sufficiently better* to justify creating a string 
>> method that
>> replicates an existing stdlib function.
>>
>
> What about performance?  I would expect a string method to perform better 
> than a stdlib function.  (no?)  Especially if it's something that could be 
> used commonly it seems like having that function be of higher performance 
> would be a benefit?
>
> At least for me, when I do use this, it's often in places like logging where 
> it could well be called a lot.
>

Considering that a method on a string literal is a plausible candidate
for constant folding, it most definitely could perform better; it
could perform as well as the string literal itself does.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Attachments (was: How can an int be '+' with a tuple?)

[Peter Pearson]

On Sun, 3 Jun 2018 20:20:32 +0200, Peter J. Holzer  wrote:
[snip]
> On 2018-06-03 13:57:26 +1000, Ben Finney wrote:
>> (For good reasons, attachments are dropped when messages are distributed
>> on the forum.)
>
> By "the forum" you mean Gmane? (I got the attachment over the mailing
> list)

Comp.lang.python is a usenet newsgroup, also accessible through a mailing
list (and, I think, a web interface).  NNTP, the Network News Transfer
Protocol, does not provide for attachments.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[Peter J. Holzer]

On 2018-06-04 13:23:59 +, Steven D'Aprano wrote:
> On Mon, 04 Jun 2018 13:33:28 +0100, Paul Moore wrote:
> > But there's also the question of what capability the kernel API has to
> > express the queries. The fact that the Unix API (and the Windows one, in
> > most cases - although as Eryk Sun pointed out there are exceptions in
> > the Windows kernel API) uses NUL-terminated strings means that querying
> > the filesystem about filenames with embedded \0 characters isn't
> > possible *at the OS level*.
> 
> I don't know whether or not the Linux OS is capable of accessing files 
> with embedded NULs in the file name. But Mac OS is capable of doing so, 
> so it should be possible. Wikipedia says:
> 
> "HFS Plus mandates support for an escape sequence to allow arbitrary 
> Unicode. Users of older software might see the escape sequences instead 
> of the desired characters."

I don't know about MacOS. In Linux there is no way to pass a filename
with an embedded '\0' (or a '/' which is not path separator) between the
kernel and user space. So if a filesystem contained such a filename, the
kernel would have to map it (via an escape sequence or some other
mechanism) to a different file name. Which of course means that - from
the perspective of any user space process - the filename doesn't contain
a '\0' or '/'.

Theoretically that mapping could be reversed in the standard library of
a language which allows '\0' in strings (like Python), but since that
would mean that programs written in that language see different
filenames than programs written in other languages (especially C, which
covers the majority of the GNU command line tools), this would be a very
bad idea. Much better to have strange but consistent filenames if you
mount a "foreign" file system. (This is btw also what Samba does,
although it does a spectacularly bad job).

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Attachments

[Peter J. Holzer]

On 2018-06-04 16:34:08 +, Peter Pearson wrote:
> On Sun, 3 Jun 2018 20:20:32 +0200, Peter J. Holzer  wrote:
> > On 2018-06-03 13:57:26 +1000, Ben Finney wrote:
> >> (For good reasons, attachments are dropped when messages are distributed
> >> on the forum.)
> >
> > By "the forum" you mean Gmane? (I got the attachment over the mailing
> > list)
> 
> Comp.lang.python is a usenet newsgroup, also accessible through a mailing
> list (and, I think, a web interface).  NNTP, the Network News Transfer
> Protocol, does not provide for attachments.

This is not true. NNTP doesn't care about the content of the message at
all (as long as it contains the few mandatory headers). As a case in
point, Jach Fong's message got here fine (including the attachment) via
this path:
news.luga.at!news.albasani.net!eternal-september.org!feeder.eternal-september.org!news.szaf.org!fu-berlin.de!uni-berlin.de!not-for-mail
So it was 6 or 7 times transmitted via NNTP with the attachment intact.

There is a rather strong *convention* to avoid attachments in many news
hierarchies (comp.* being one of them), and some newsservers may filter
them. But clearly none of the 7 newsservers mentioned above does this,
at least not for *text* attachments (they might do it for binary
attachments).

So we have determined that "the forum" is not the mailing list and not
the newsgroup (Jach's message appeared on both with the attachment). Ben
uses Gmane, which makes it likely that he is talking about a
(mis)feature of Gmane,

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Attachments

[Peter Otten]

Peter J. Holzer wrote:

> On 2018-06-04 16:34:08 +, Peter Pearson wrote:
>> On Sun, 3 Jun 2018 20:20:32 +0200, Peter J. Holzer 
>> wrote:
>> > On 2018-06-03 13:57:26 +1000, Ben Finney wrote:
>> >> (For good reasons, attachments are dropped when messages are
>> >> distributed on the forum.)
>> >
>> > By "the forum" you mean Gmane? (I got the attachment over the mailing
>> > list)
>> 
>> Comp.lang.python is a usenet newsgroup, also accessible through a mailing
>> list (and, I think, a web interface).  NNTP, the Network News Transfer
>> Protocol, does not provide for attachments.
> 
> This is not true. NNTP doesn't care about the content of the message at
> all (as long as it contains the few mandatory headers). As a case in
> point, Jach Fong's message got here fine (including the attachment) via
> this path:
> news.luga.at!news.albasani.net!eternal-september.org!feeder.eternal-
september.org!news.szaf.org!fu-berlin.de!uni-berlin.de!not-for-mail
> So it was 6 or 7 times transmitted via NNTP with the attachment intact.
> 
> There is a rather strong *convention* to avoid attachments in many news
> hierarchies (comp.* being one of them), and some newsservers may filter
> them. But clearly none of the 7 newsservers mentioned above does this,
> at least not for *text* attachments (they might do it for binary
> attachments).
> 
> So we have determined that "the forum" is not the mailing list and not
> the newsgroup (Jach's message appeared on both with the attachment). Ben
> uses Gmane, which makes it likely that he is talking about a
> (mis)feature of Gmane,

However, I did see that particular particular attachment, test.py in Jach's 
original post, and I'm reading c.l.py via gmane.

> 
> hp
> 


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Attachments

[Peter J. Holzer]

On 2018-06-04 22:59:52 +0200, Peter Otten wrote:
> Peter J. Holzer wrote:
> > On 2018-06-04 16:34:08 +, Peter Pearson wrote:
> >> On Sun, 3 Jun 2018 20:20:32 +0200, Peter J. Holzer 
> >> wrote:
> >> > On 2018-06-03 13:57:26 +1000, Ben Finney wrote:
> >> >> (For good reasons, attachments are dropped when messages are
> >> >> distributed on the forum.)
> >> >
> >> > By "the forum" you mean Gmane? (I got the attachment over the mailing
> >> > list)
> >> 
> >> Comp.lang.python is a usenet newsgroup, also accessible through a mailing
> >> list (and, I think, a web interface).  NNTP, the Network News Transfer
> >> Protocol, does not provide for attachments.
> > 
> > This is not true. NNTP doesn't care about the content of the message at
> > all (as long as it contains the few mandatory headers). As a case in
> > point, Jach Fong's message got here fine (including the attachment) via
[...]
> > So we have determined that "the forum" is not the mailing list and not
> > the newsgroup (Jach's message appeared on both with the attachment). Ben
> > uses Gmane, which makes it likely that he is talking about a
> > (mis)feature of Gmane,
> 
> However, I did see that particular particular attachment, test.py in Jach's 
> original post, and I'm reading c.l.py via gmane.

comp.lang.python or gmane.comp.python.general? (I see the latter but not
the former on gmane, but I'm accessing it anonymously which might make a
difference.) But yes, I also just checked gmane and the attachment is
there. So we don't know why Ben didn't see the attachment (or thought
that others might not see it).

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Help with 'site' package.

[Erik Martinson via Python-list]

I am trying to dynamically add a site-package to a script that is run as a cron 
job. The method adduseristepackages does not seem to do anything.

import sys
import site

print('-')print(site.getusersitepackages())
print('add', 
site.addusersitepackages('/home/erik/.local/lib/python3.6/site-packages'))
print(site.getusersitepackages())
print(site.getsitepackages())
print(site.check_enableusersite())


output:-
/root/.local/lib/python3.6/site-packages
add /home/erik/.local/lib/python3.6/site-packages
/root/.local/lib/python3.6/site-packages
['/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages', 
'/usr/lib/python3.6/dist-packages']
True


Thanks,
Erik
"To error is human ... and to blame it on a computer is even more so."

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why exception from os.path.exists()?

[eryk sun]

On Sat, Jun 2, 2018 at 11:28 AM, Chris Angelico  wrote:
>
> I also can't find anything about path names there. What does POSIX say
> about the concept of relative paths? Does Windows comply with that?

Certainly Windows file-system paths are not POSIX compatible. Seven
path types are supported:

* Extended Local Device (\\?\)
* Local Device (\\.\)
* UNC
* Drive Absolute
* Drive Relative
* Rooted
* Relative

Extended local-device paths only allow backslash as a path separator.
The others allow either backslash or slash. I doubt POSIX would allow
magically reserved DOS device names in every directory or stripping of
trailing dots and spaces from filenames.

But this isn't relevant to NT's POSIX compatibility. A POSIX process
links with psxdll.dll, which connects to the POSIX environment
subsystem (psxss.exe). It gets run from Windows via posix.exe
(console) or psxrun.exe. In the 00s, Microsoft acquired Interix, which
extended the original POSIX subsystem, and integrated it as the
Subsystem for UNIX Applications (SUA). Notably SUA adds a kernel
driver, psxdrv.sys, which facilitates implementing system calls and
signals. There used to be a community website with overviews [1], a
FAQ [2], a forum [3], tool downloads [4], and various documentation
[5]. However, NT's environment subsystems never really had mass
appeal, probably because existing programs had to be ported and
recompiled. SUA is no longer supported as of Windows 8.1 and Server
2012 R2. The community website was closed, and the domain is now held
by a squatter.

Regarding file-system paths, SUA has a single root directory and uses
"/dev/fs/C" for drive "C:" and "/net/server/share" for
"\\server\share".

[1]: http://www.suacommunity.com/SUA_Tools_Env_Start.htm
 https://archive.li/45JG
[2]: http://www.suacommunity.com/FAQs.htm
 https://archive.li/5LFw
[3]: http://www.suacommunity.com/forum2
 https://archive.li/LzZxS
[4]: http://www.suacommunity.com/tool_warehouse.aspx
 https://archive.li/0luI9
[5]: http://www.suacommunity.com/dictionary/fork-entry.php
 https://archive.li/5k8vW

Windows 10 has a Linux subsystem (WSL), but this is not an NT
environment subsystem. WSL processes do not load ntdll.dll. They're
lightweight pico processes with an associated pico provider in the
kernel (lxss.sys, lxcore.sys), and they directly execute native Linux
binaries (no porting and recompiling from source). WSL only supports
the console, but at least the console was upgraded to support
virtual-terminal mode.

> We know that Ctrl-C maps to the internal Windows interrupt
> handler, and "kill process" maps to the internal Windows "terminate",
> but can you send a different process all the different signals and
> handle them differently?

IIRC, the original POSIX subsystem supported only single-threaded
processes, and SIGKILL called NtTerminateThread. Of course the
subsystem has its own client bookkeeping to handle here as well. (For
the Windows subsystem, csrss.exe also maintains shadow process and
thread structures for clients. This is how an environment subsystem
supplements base NT behavior.)

Regarding Ctrl+C, a console session is started by posix.exe, which is
a Windows console application. It translates console control events to
signals, e.g. CTRL_C_EVENT to SIGINT, CTRL_BREAK_EVENT to SIGQUIT, and
otherwise SIGKILL (e.g. closing the console, logoff, shutdown). It
sends the signal number and session ID to the subsystem, which signals
the processes in the given session.

One way for the subsystem to implement signal delivery is via NT's
runtime library function RtlRemoteCall (i.e. suspend the target
thread, get its CPU context and copy it to the stack, modify the
context and stack, and resume). Make a remote call to a known client
function (i.e. in psxdll.dll), which delivers the signal and then
continues the thread's original context via NtContinue. This approach
isn't really efficient, but it's basically how the original POSIX
subsystem worked. SUA probably uses NT asynchronous procedure calls
(APCs).

---

Appendix: NT APCs

NT doesn't have anything exactly like POSIX signals. It has
exceptions, which are handled using either Vectored Exception Handling
or Structured Exception Handling (i.e. MSVC __try, __except,
__finally), and asynchronous procedure calls (APCs). Some POSIX
signals correspond to NT exceptions (e.g. SIGSEGV corresponds to a
STATUS_ACCESS_VIOLATION). But APCs are what a POSIX subsystem would
likely use to implement signals.

There are two types of APC: kernel and user. A thread has an APC queue
for each type. User APCs can be queued from user mode via
NtQueueApcThread, or via WinAPI QueueUserAPC. Some APIs such as
ReadFileEx take an optional completion or notification APC routine,
for which a kernel component queues the user APC.

All APCs have a "kernel routine" and most also have a "normal"
routine. The kernel routine is called first, with the CPU in kernel
mode and its interrupt request level (IRQL) at APC_LEVEL (1). The
kern

Re: Attachments

[Ben Finney]

"Peter J. Holzer"  writes:

> So we have determined that "the forum" is not the mailing list and not
> the newsgroup (Jach's message appeared on both with the attachment).

By “the forum” I don't mean any specific server to the exclusion of
others. I mean the aggregate forum in which we are having this
discussion: comp.lang.python and python-list are a single discussion
forum, distributed across many servers.

-- 
 \“If you go parachuting, and your parachute doesn't open, and |
  `\you friends are all watching you fall, I think a funny gag |
_o__) would be to pretend you were swimming.” —Jack Handey |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: tkinter (ttk) combobox dropdown text is white on white

[Jim Lee]

Oops, I hit "reply" instead of "reply-list" last time.  Trying again...


On 06/03/2018 02:01 PM, Christian Gollwitzer wrote:

Am 03.06.18 um 21:54 schrieb Jim Lee:> import tkinter as tk

from tkinter import ttk

root = tk.Tk()
cb = ttk.Combobox(root)
cb.grid(row=0, column=0, sticky='NSEW')
cb['values'] = ['one', 'two', 'three', 'four']
root.mainloop()

The text of the values in the combobox dropdown list is white on 
white.   The *selected* item in the list is white on light grey, but 
all the unselected items are invisible (white on white). 


Which platform are you on, i.e. which operating system? I guess it is 
Linux. In that case the default colors are read by Tk from the X11 
options database, which is some cruft from the past to set options for 
X11 applications. Try to run


xrdb -query

That should give you a list of default values, and maybe you can see 
something. The dropdown list is actually a popdown menu, so look for 
Menu colors. For instance, if you use a dark theme in your desktop 
environment, it could be that the foreground is correctly set to 
white, but the background is hardcoded white for some reason e.g.



Christian


Thanks.  Yes, I am on Linux (Fedora 28, MATE desktop).  Yes, I am using 
a dark theme - however, xrdb does not show #ff for *any* background 
color, so I have no idea where ttk is picking it up from.  Even if I 
change to a light desktop theme, the ttk widget still displays white on 
white.  The only solution I have found so far is rather hackish, but it 
works:


class MyCombobox(ttk.Combobox):
    def __init__(self, *args, **kwargs):
    super().__init__(*args, **kwargs)
    self.bind('', self._change_popdown_color)

    def _change_popdown_color(self, *args):
    popdown = self.tk.eval('ttk::combobox::PopdownWindow 
{}'.format(self))
    self.tk.call('{}.f.l'.format(popdown), 'configure', 
'-foreground', 'black')


However, I am still looking for a more elegant solution that preferably 
doesn't hardcode colors...


-Jim

--
https://mail.python.org/mailman/listinfo/python-list

Re: Attachments (was: How can an int be '+' with a tuple?)

[Richard Damon]

On 6/4/18 12:34 PM, Peter Pearson wrote:
> On Sun, 3 Jun 2018 20:20:32 +0200, Peter J. Holzer  wrote:
> [snip]
>> On 2018-06-03 13:57:26 +1000, Ben Finney wrote:
>>> (For good reasons, attachments are dropped when messages are distributed
>>> on the forum.)
>> By "the forum" you mean Gmane? (I got the attachment over the mailing
>> list)
> Comp.lang.python is a usenet newsgroup, also accessible through a mailing
> list (and, I think, a web interface).  NNTP, the Network News Transfer
> Protocol, does not provide for attachments.
>
NNTP most certainly can provide for attachments, and I would guess that
well over 90% of the traffic (might even be 99%) on usenet is carried in
attachments. Just wander over to the swamp of alt.binaries.*

Now, most news servers are configured to discard most attachments on
messages (or the whole message if it has an attachment) if posted to a
group not specifically marked as a 'binary' newsgroup, just like a lot
of mailing list are similarly setup.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

user defined modules

[Sharan Basappa]

Is there a specific location where user defined modules need to be kept?
If not, do we need to specify search location so that Python interpreter can 
find it?

Also, when does Python interpreter compile the module code?
When it is imported?
-- 
https://mail.python.org/mailman/listinfo/python-list