Re: Python gotcha of the day

[Thomas Jollans]

On 2018-03-14 05:08, Steven D'Aprano wrote:
> Explain the difference between these two triple-quoted strings:
> 
> Here is a triple-quoted string containing spaces and a triple-quote:
> 
> py> """ \""" """
> ' """ '
> 
> 
> But remove the spaces, and two of the quotation marks disappear:
> 
> py> """\""
> '"'
> 
> 
> If nobody gets the answer, I shall reveal all later.
> 
> (Hint: it is not a bug.)
> 

Ah, subtle!

Initially I thought the first one was being interpreted as

''' """ '''

and the second one as

"" '"' "" ""

which left me rather puzzled as to why the first wasn't being interpreted as

"" ' "' " " ""

but of course that's not what's going on at all. The second one is

'''"''' ""

As to WHY - in both your examples, the literal can be interpreted as a
triple-quoted string, so it is (rather than some combination of
single-quoted strings). And, in both cases, the SHORTEST possible
reading as a triple-quoted string is used.

There, now I can go back to work.

- Thomas

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Enumerating all 3-tuples

[Denis Kasak]

On 2018-03-10 02:13, Steven D'Aprano wrote:


But I've stared at this for an hour and I can't see how to extend the
result to three coordinates. I can lay out a grid in the order I want:

1,1,1   1,1,2   1,1,3   1,1,4   ...
2,1,1   2,1,2   2,1,3   2,1,4   ...
1,2,1   1,2,2   1,2,3   1,2,4   ...
3,1,1   3,1,2   3,1,3   3,1,4   ...
2,2,1   2,2,2   2,2,3   2,2,4   ...
...

and applying Cantor's diagonal order will give me what I want, but 
damned

if I can see how to do it in code.


If you want this exact order, it can be reproduced in the following way.

The triples can be viewed as a pair of a pair and a natural number:

(1,1),1 (1,1),2 (1,1),3 ...
(2,1),1 (2,1),2 (2,1),3 ...
(1,2),1 (1,2),2 (1,2),3 ...
   .   .   .
   .   .   .
   .   .   .

Decomposing this by the diagonals yields

1: (1,1),1
2: (2,1),1  (1,1),2
3: (1,2),1  (2,1),2  (1,1),3
.
.
.

Notice that the first elements of each pair (i.e. the inner pairs) are 
given by the (inverse of the) original Cantor pairing function, in 
decreasing order. The second elements are just the natural numbers. 
Naming the inverse c, we can rewrite the above like this:


1: c(1),1
2: c(2),1  c(1),2
3: c(3),1  c(2),2  c(1),3
.
.
.

Rewriting this to spell out the mapping between the natural numbers and 
the pairs, we get


1 -> c(1),1
2 -> c(2),1
3 -> c(1),2
4 -> c(3),1
5 -> c(2),2
6 -> c(1),3
.
.
.

Squinting a bit, this might seem familiar. This is exactly the same as 
the original pairing function, except for an additional application of c 
to the first element of the pair!


1 -> 1,1
2 -> 2,1
3 -> 1,2
4 -> 3,1
5 -> 2,2
6 -> 1,3

This leads fairly naturally to the implementation.

from itertools import accumulate, count

def c(i):
"""
Inverse of the Cantor pairing function, mapping N → N×N.
"""
assert i >= 1

# partial sums of the series 1 + 2 + 3 + ...
sums = accumulate(count(1))
n = 0

while True:
m = next(sums)
if m < i:
n += 1
else:
r = m - i
break

return r + 1, n - r + 1


def c3(i):
"""
Inverse of the Cantor pairing function generalization to 
triples,

mapping N → N×N×N.
"""
n, m = c(i)
return c(n) + (m,)

Applying c3 to the natural numbers gives the sequence you wanted:


s = map(c3, count(1))
pprint([next(s) for _ in range(10)])

[(1, 1, 1),
 (2, 1, 1),
 (1, 1, 2),
 (1, 2, 1),
 (2, 1, 2),
 (1, 1, 3),
 (3, 1, 1),
 (1, 2, 2),
 (2, 1, 3),
 (1, 1, 4)]

--
Denis Kasak
--
https://mail.python.org/mailman/listinfo/python-list

Re: 2.7 EOL = 2020 January 1

[Mark Lawrence]

On 13/03/18 18:30, Tim Chase wrote:

On 2018-03-13 10:58, Terry Reedy wrote:

Two days later, Benjamin Peterson, the 2.7 release manager, replied
"Sounds good to me. I've updated the PEP to say 2.7 is completely
dead on Jan 1 2020." adding "The final release may not literally be
on January 1st".


Am I the only one saddened by this announcement?  I mean, it could
have been something like

"""
"VOOM"?!?  Mate, 2.7 wouldn't "voom" if you put four million volts
through it!  It's bleedin' demised!  It's not pinin'!  It's passed on!
This 2.x series is no more!  It has ceased to be!  It's expired and
gone to meet its maker!  It's a stiff!  Bereft of life, it rests in
peace!  If you hadn't nailed it to your dependencies it'd be pushing
up the daisies!  Its metabolic processes are now 'istory!  It's off
the twig!  It's kicked the bucket, it's shuffled off its mortal coil,
run down the curtain and joined the bleedin' choir invisible!!
THIS IS AN EX-VERSION!!
"""

Pythonically-yers,

-tkc




I'd still like to know what the core developers have ever done for us. 
Apart from .


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

TLSServer: certificate one request behind...

[Fabiano Sidler]

Hi folks!

I have written a TLSServer for testing purposes that generates
self-signed certificates upon request. This works pretty well
except that the certificates are always supplied one request too
late:

# gets no cert, but a handshake failure instead
$ openssl s_client -connect localhost:1234 -servername server1

# gets the cert for server1
$ openssl s_client -connect localhost:1234 -servername server2

# gets the cert for server2
$ openssl s_client -connect localhost:1234 -servername server3

What's the reason for this? Please find attached my TLSServer.

Best wishes,
Fabiano
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python gotcha of the day

[Brian Oney via Python-list]

explicit is better than implicit.

That gives me an idea for a module with the following debugging command line 
functionality.

import sass

>>> "" ":p"
Traceback: 
Are you telling me that ' ' is supposed to an operator? (Rock thrown)




On March 14, 2018 10:40:38 AM GMT+01:00, Thomas Jollans  wrote:
>On 2018-03-14 05:08, Steven D'Aprano wrote:
>> Explain the difference between these two triple-quoted strings:
>> 
>> Here is a triple-quoted string containing spaces and a triple-quote:
>> 
>> py> """ \""" """
>> ' """ '
>> 
>> 
>> But remove the spaces, and two of the quotation marks disappear:
>> 
>> py> """\""
>> '"'
>> 
>> 
>> If nobody gets the answer, I shall reveal all later.
>> 
>> (Hint: it is not a bug.)
>> 
>
>Ah, subtle!
>
>Initially I thought the first one was being interpreted as
>
>''' """ '''
>
>and the second one as
>
>"" '"' "" ""
>
>which left me rather puzzled as to why the first wasn't being
>interpreted as
>
>"" ' "' " " ""
>
>but of course that's not what's going on at all. The second one is
>
>'''"''' ""
>
>As to WHY - in both your examples, the literal can be interpreted as a
>triple-quoted string, so it is (rather than some combination of
>single-quoted strings). And, in both cases, the SHORTEST possible
>reading as a triple-quoted string is used.
>
>There, now I can go back to work.
>
>- Thomas
>
>-- 
>https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

matplotlib icon

[tedo . vrbanec]

I am getting this logging.INFO notice:
Could not load matplotlib icon: bad option "foobar": must be aspect, 
attributes, client, colormapwindows, command, deiconify, focusmodel, forget, 
frame, geometry, grid, group, iconbitmap, iconify, iconmask, iconname, 
iconphoto, iconposition, iconwindow, manage, maxsize, minsize, 
overrideredirect, positionfrom, protocol, resizable, sizefrom, stackorder, 
state, title, transient, or withdraw
I even set icon, but this notice remains.
There's nothing on web or in matplotlib documentation.
Python 2.7, Debian
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: matplotlib icon

[Terry Reedy]

On 3/14/2018 2:30 PM, tedo.vrba...@gmail.com wrote:

I am getting this logging.INFO notice:
Could not load matplotlib icon: bad option "foobar": must be aspect, 
attributes, client, colormapwindows, command, deiconify, focusmodel, forget, frame, 
geometry, grid, group, iconbitmap, iconify, iconmask, iconname, iconphoto, iconposition, 
iconwindow, manage, maxsize, minsize, overrideredirect, positionfrom, protocol, 
resizable, sizefrom, stackorder, state, title, transient, or withdraw
I even set icon, but this notice remains.
There's nothing on web or in matplotlib documentation.
Python 2.7, Debian


We need more context, such as the code that emits the warning, to be 
sure of anything.  However, some code apparently tries to modify 
non-existent option 'foobar' on, I suspect, a GUI widget.  I have seen 
similar messages from tkinter/tk.


The option mistake prevents the icon load.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: TLSServer: certificate one request behind...

[Fabiano Sidler]

Thus wrote Fabiano Sidler:
> What's the reason for this? Please find attached my TLSServer.

Oh, sorry...! Apparently, the attachment has been stripped. Here inline:

=== tlsserver.py ===
from socketserver import ThreadingTCPServer,StreamRequestHandler
import ssl

class TLSServer(ThreadingTCPServer):
def __init__(self, *args, **kwargs):
super(TLSServer, self).__init__(*args, **kwargs)
ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ctx.set_servername_callback(self.servername_callback)
ctx.check_hostname = False
self._ctx = ctx
def get_request(self):
s,a = super(TLSServer, self).get_request()
s = self._ctx.wrap_socket(s, server_side=True)
return s,a
def servername_callback(self, sock, req_hostname, cb_context):
return ssl.ALERT_DESCRIPTION_INTERNAL_ERROR


from OpenSSL import crypto as x509
from tempfile import NamedTemporaryFile

class SelfSigningServer(TLSServer):
def servername_callback(self, sock, req_hostname, cb_context):
key = x509.PKey()
key.generate_key(x509.TYPE_RSA, 2048)
cert = x509.X509()
subj = cert.get_subject()
subj.C  = 'CH'
subj.ST = 'ZH'
subj.L  = 'Zurich'
subj.O  = 'ACME Inc.'
subj.OU = 'IT dept.'
subj.CN = req_hostname
cert.set_version(0x02)
cert.set_serial_number(1000)
cert.gmtime_adj_notBefore(0)
cert.gmtime_adj_notAfter(10*365*24*60*60)
cert.set_issuer(subj)
cert.set_pubkey(key)
cert.sign(key, 'sha256')
certfile = NamedTemporaryFile()
keyfile = NamedTemporaryFile()
certfile.write(x509.dump_certificate(x509.FILETYPE_PEM, cert))
keyfile.write(x509.dump_privatekey(x509.FILETYPE_PEM, key))
certfile.seek(0)
keyfile.seek(0)
cb_context.load_cert_chain(certfile=certfile.name, 
keyfile=keyfile.name)
cb_context.set_servername_callback(self.servername_callback)
sock.context = cb_context
certfile.close()
keyfile.close()

class SelfSigningHandler(StreamRequestHandler):
def handle(self):
self.wfile.write(b'Hello World!\r\n')

server = SelfSigningServer(('localhost',1234), SelfSigningHandler)
server.serve_forever()
=== tlsserver.py ===

Thanks again!
-- 
https://mail.python.org/mailman/listinfo/python-list

macOS specific - reading calendar information

[Jan Erik Moström]

I've been trying to find some example of how to read calendar info on 
macOS but I haven't found anything ... I'm probably just bad at 
searching !!


What I want to do is to read calendar info for a date range. Does anyone 
know of an example of how to do this?


= jem
--
https://mail.python.org/mailman/listinfo/python-list

Re: macOS specific - reading calendar information

[Larry Martell]

On Wed, Mar 14, 2018 at 4:31 PM, Jan Erik Moström  wrote:
> I've been trying to find some example of how to read calendar info on macOS
> but I haven't found anything ... I'm probably just bad at searching !!
>
> What I want to do is to read calendar info for a date range. Does anyone
> know of an example of how to do this?

What does 'read calendar info' mean? What exactly are you trying to read from?
-- 
https://mail.python.org/mailman/listinfo/python-list

urllib.request.urlopen fails with https (was: Re: Stock quote APi)

[Irv Kalb]

Thanks to Chris A and Roger C for their information.  Very helpful and I am 
working on both of their suggestions.  But now I've run into a new related 
problem.

I still am trying to get stock information using Standard Library calls for now 
(although I understand that the requests module makes this kind of thing very 
easy).

My general approach (Python 3) is to build up a URL by concatenating an 
parameters, and make a call to url.request.urlopen.  I then do a 'read' on what 
was returned. For example, in my class, I use the following code to get weather 
data from weatherman.org:

URL = 'http://api.openweathermap.org/data/2.5/weather?q=' + city + 
'&mode=xml'+ '&APPID=' + API_KEY
response = urllib.request.urlopen(URL)
responseString = str(response.read())

I then have some code to parse out the pieces of the returned information.  
This works great and an excellent example of the power of using API's with 
Python.


In looking for a stock data API, Chris suggested  alpha vantage.co (and Roger 
made a nice suggestion as to how to parse it).  But when I use their API, I get 
an error:  

ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 
(_ssl.c:749)

I have tried a number of different API's and found that if the URL starts with 
'https', it always fails.  Even a simple example that I found here:  
https://pythonspot.com/urllib-tutorial-python-3/ 
  fails the same way.

Here's what I see when I try that example:

>>> import urllib.request
>>> html = urllib.request.urlopen('https://arstechnica.com').read()
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
 line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
 line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
 line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
 line 1026, in _send_output
self.send(msg)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
 line 964, in send
self.connect()
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py",
 line 1400, in connect
server_hostname=server_hostname)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 
401, in wrap_socket
_context=self, _session=session)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 
808, in __init__
self.do_handshake()
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 
1061, in do_handshake
self._sslobj.do_handshake()
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 
683, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 
(_ssl.c:749)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
html = urllib.request.urlopen('https://arstechnica.com').read()
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 223, in urlopen
return opener.open(url, data, timeout)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 526, in open
response = self._open(req, data)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 544, in _open
'_open', req)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 504, in _call_chain
result = func(*args)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py",
 line 1320, in do_open
raise URLError(err)
urllib.error.URLError: 
>>> 

All my tests return the same tracebacks.

Am I doing something wrong?  Is there another way (besides using the requests 
module which DOES work for me) to get data from an https URL?

Any help would be appreciated.

Thanks,

Irv
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urllib.request.urlopen fails with https (was: Re: Stock quote APi)

[Chris Angelico]

On Thu, Mar 15, 2018 at 9:04 AM, Irv Kalb  wrote:
> ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 
> (_ssl.c:749)
>
> Am I doing something wrong?  Is there another way (besides using the requests 
> module which DOES work for me) to get data from an https URL?

So my reading of this is that you have a major issue with root
certificates. Bleh. The normal response is "use pip to install
something that has your certs", but if you really absolutely cannot do
that, you may have to just disable certificate verification. This is
NOT a good thing to do, and you should have a big noisy comment
explaining why you have to make your code insecure. Put that RIGHT
next to the code that disables verification, so that if anyone copies
and pastes it, they'll have a good chance of seeing it. (Basically,
what you're doing is downgrading the protection of HTTPS to something
nearer plain HTTP. That's fine for what you're doing, but any code you
give to students is likely to be copied and pasted into their
production code.)

See if you can tie in with your OS's cert store first. If you can't,
look at "import ssl" and creating a custom SSL context that doesn't
verify.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Restore via pip to new OS

[Tim Johnson]

I'm currently running both python and python3 on ubuntu 14.04.
Plan is to do a complete re-install of ubuntu 16.04 on a fresh
hard drive.

I've accumulated a list of pip-install packages via 
pip list > piplist.txt.

Can I duplicate the same packages on the new OS by
pip -r piplist.txt?

thanks
-- 
Tim 
http://www.akwebsoft.com, http://www.tj49.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urllib.request.urlopen fails with https

[Gregory Ewing]

Chris Angelico wrote:

(Basically,
what you're doing is downgrading the protection of HTTPS to something
nearer plain HTTP. That's fine for what you're doing, but any code you
give to students is likely to be copied and pasted into their
production code.)

See if you can tie in with your OS's cert store first. If you can't,
look at "import ssl" and creating a custom SSL context that doesn't
verify.


That's likely to distract and confuse students as well.

I would suggest looking for a different example that
doesn't require interacting with an https site.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: urllib.request.urlopen fails with https

[Chris Angelico]

On Thu, Mar 15, 2018 at 10:27 AM, Gregory Ewing
 wrote:
> Chris Angelico wrote:
>>
>> (Basically,
>> what you're doing is downgrading the protection of HTTPS to something
>> nearer plain HTTP. That's fine for what you're doing, but any code you
>> give to students is likely to be copied and pasted into their
>> production code.)
>>
>> See if you can tie in with your OS's cert store first. If you can't,
>> look at "import ssl" and creating a custom SSL context that doesn't
>> verify.
>
>
> That's likely to distract and confuse students as well.
>
> I would suggest looking for a different example that
> doesn't require interacting with an https site.

That means going back to the original problem: "how do we get a usable
stock price API?".

My preferred solution is to just install 'requests', of course.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Restore via pip to new OS

[Paul Moore]

Use pip freeze rather than pip list. That will give you the
information in  "requirements file" format that pip install -r can
read.
Paul

On 14 March 2018 at 23:20, Tim Johnson  wrote:
> I'm currently running both python and python3 on ubuntu 14.04.
> Plan is to do a complete re-install of ubuntu 16.04 on a fresh
> hard drive.
>
> I've accumulated a list of pip-install packages via
> pip list > piplist.txt.
>
> Can I duplicate the same packages on the new OS by
> pip -r piplist.txt?
>
> thanks
> --
> Tim
> http://www.akwebsoft.com, http://www.tj49.com
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Restore via pip to new OS

[Tim Johnson]

* Paul Moore  [180314 15:42]:
> Use pip freeze rather than pip list. That will give you the
> information in  "requirements file" format that pip install -r can
> read.
> 
> On 14 March 2018 at 23:20, Tim Johnson  wrote:
<...>
> > Can I duplicate the same packages on the new OS by
> > pip -r piplist.txt?
  Thank you Paul.
  Cheers
-- 
Tim 
http://www.akwebsoft.com, http://www.tj49.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Enumerating all 3-tuples (Posting On Python-List Prohibited)

[Ben Bacarisse]

Lawrence D’Oliveiro  writes:

> On Wednesday, March 14, 2018 at 2:18:24 PM UTC+13, Ben Bacarisse wrote:
>> Lawrence D’Oliveiro writes:
>> 
>> The original problem -- triples of natural numbers -- is
>> not particularly hard, but the general problem -- enumerating n-tuples
>> of some sequence -- is more interesting because it is a bit harder.
>
> It’s not harder--it’s exactly the same difficulty. You iterate the
> naturals, then interpose a function mapping them onto the set that you
> really want to use.

Yes, I said exactly that a couple of messages ago.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urllib.request.urlopen fails with https

[Gregory Ewing]

Chris Angelico wrote:

That means going back to the original problem: "how do we get a usable
stock price API?".


Does it have to be stock prices in particular?
Or just some simple piece of data that demonstrates
the principles of fetching a url and parsing the
result?

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: urllib.request.urlopen fails with https

[Chris Angelico]

On Thu, Mar 15, 2018 at 3:54 PM, Gregory Ewing
 wrote:
> Chris Angelico wrote:
>>
>> That means going back to the original problem: "how do we get a usable
>> stock price API?".
>
>
> Does it have to be stock prices in particular?
> Or just some simple piece of data that demonstrates
> the principles of fetching a url and parsing the
> result?
>

You'd have to ask the OP, but if you make too big a change to the API
used, the rest of the course might have to change too. If the next
thing done with the downloaded data is to graph it, for instance, then
you'll need some other API that gives graphable data. Etcetera,
etcetera, etcetera.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python gotcha of the day

[Ben Finney]

Steven D'Aprano  writes:

> py> """\""
> '"'

That's an empty string delimited by ‘"’; followed by a double-quote
character, escaped, delimited by ‘"’; followed by two more empty
strings. They concatenate to a single one-character string.

Equivalent to

"" + "\"" + "" + ""

without the ‘+’ operations.

> If nobody gets the answer, I shall reveal all later.

My interpretation may not be how the language defines it; but it does
match the result!

-- 
 \ “I have yet to see any problem, however complicated, which, |
  `\  when you looked at it in the right way, did not become still |
_o__)more complicated.” —Paul Anderson |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list