Why does asyncio.wait_for() need a timeout?

[Frank Millman]

Hi all

Below is a simple asyncio loop that runs two background tasks.

They both perform a simple count. The first one counts forever, wrapped in a 
try/except. The second one counts up to 5, then cancels the first one and 
stops the loop.


There are two ways of ensuring that then cancellation is complete -

   asyncio.wait([a sequence of futures])

   asyncio.wait_for(a single future)

Both take an optional timeout.

If I use the first method without a timeout, the cancellation completes and 
the loop stops.


If I use the second method without a timeout, the future is cancelled, but 
the program hangs.


If I add a timeout to the second one, it behaves the same as the first one.

Is there a reason for this?

I am using version 3.6.0.

Thanks

Frank Millman

import asyncio
from itertools import count

async def counter1():
   cnt = count(1)
   try:
   while True:
   print('From 1:', next(cnt))
   await asyncio.sleep(1)
   except asyncio.CancelledError:
   print('counter1 cancelled')

async def counter2():
   cnt = count(1)
   for i in range(5):
   print('From 2:', next(cnt))
   await asyncio.sleep(1)
   cnt1.cancel()
#   await asyncio.wait([cnt1])  # works
#   await asyncio.wait_for(cnt1)  # hangs
   await asyncio.wait_for(cnt1, 1)  # works
   loop.stop()

loop = asyncio.get_event_loop()
cnt1 = asyncio.ensure_future(counter1())
cnt2 = asyncio.ensure_future(counter2())
loop.run_forever()


--
https://mail.python.org/mailman/listinfo/python-list

ANN: PyDDF Python Herbst Sprint 2017

[eGenix Team: M.-A. Lemburg]

[This announcement is in German since it targets a Python sprint in
 Düsseldorf, Germany]


ANKÜNDIGUNG


  PyDDF Python Herbst Sprint 2017 in
  Düsseldorf


 Samstag, 25.11.2017, 10:00-18:00 Uhr
 Sonntag, 26.11.2017, 10:00-18:00 Uhr

trivago GmbH,  Karl-Arnold-Platz 1A,  40474 Düsseldorf

  Python Meeting Düsseldorf
 https://www.pyddf.de/sprint2017-11/


INFORMATION

Das Python Meeting Düsseldorf (PyDDF) veranstaltet mit freundlicher
Unterstützung der *trivago GmbH* ein Python Sprint Wochenende im
September.

Der Sprint findet am Wochenende 25./26.11.2017 in der trivago
Niederlassung am Karl-Arnold-Platz 1A statt (nicht am Bennigsen-Platz 1).

Folgende Themengebiete haben wir als Anregung angedacht:

 * Willkommens-Chatbot für Düsseldorf mir RasaHQ
 * PyEditor oder django-reversion-compare
 * Django Autotask oder FirtzConnection
 * YouTube Video Manager CLI
 * Kivy
 * Formula Pi (Roboter Rennen)
 * Jython

Natürlich kann jeder Teilnehmer weitere Themen vorschlagen.

Alles weitere und die Anmeldung findet Ihr auf der Sprint Seite:

https://www.pyddf.de/sprint2017-11/

Teilnehmer sollten sich zudem auf der PyDDF Liste anmelden, da wir
uns dort koordinieren:

https://www.egenix.com/mailman/listinfo/pyddf


ÜBER UNS

Das Python Meeting Düsseldorf (PyDDF) ist eine regelmäßige Veranstaltung
in Düsseldorf, die sich an Python Begeisterte aus der Region wendet:

 * https://pyddf.de/

Einen guten Überblick über die Vorträge bietet unser YouTube-Kanal,
auf dem wir die Vorträge nach den Meetings veröffentlichen:

 * http://www.youtube.com/pyddf/

Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld,
in Zusammenarbeit mit Clark Consulting & Research, Düsseldorf:

 * http://www.egenix.com/
 * http://www.clark-consulting.eu/

Mit freundlichen Grüßen,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 23 2017)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Finding the module object in Python 3 C extensions (Posting On Python-List Prohibited)

[Jon Ribbens]

On 2017-11-23, Lawrence D’Oliveiro  wrote:
> On Thursday, November 23, 2017 at 4:03:18 AM UTC+13, Jon Ribbens wrote:
>> In Python 3, you are supposed to store the global state in an
>> allocated memory area instead.
>
> Says who? Considering that this
> 
> example uses a global variable.

Says Benjamin Peterson on docs.python.org:

https://docs.python.org/3/howto/cporting.html

"Python 3 has a revamped extension module initialization system.
(See PEP 3121.) Instead of storing module state in globals, they
should be stored in an interpreter specific structure. Creating
modules that act correctly in both Python 2 and Python 3 is
tricky."

He's not kidding, not least because it appears that creating modules
that act correctly in Python 3 at all is tricky!
-- 
https://mail.python.org/mailman/listinfo/python-list

Python loop and web server (bottle) in the same script

[zljubisic]

I would like to have a script that collects data every minute and at the same 
time serve newly collected data as web pages.

Timely collecting data is more important than serving web pages, so collecting 
data should have priority and should never be interrupted by serving web pages.

My first idea was to use (while) loop and a bottle web server, so I wrote a 
program in which for loop simulates collecting data and bottle simulates web 
server that server newly collected data as web pages:

from bottle import route, run, request, template, static_file
import time

x = 0

@route('/test')
def test():
return 'x = {}'.format(x)

run(host='localhost', port=8080, debug=True)

for x in range(100):
time.sleep(1)

I hoped that run() command will be executed and script will continue bellow 
this line, but that is not the case, run() is the last command that is 
executed, and /test always shows x = 0 as output.

If I put for loop before run(), than run() will be executed after for loop is 
completed.

Looks like I need some sort of parallelization.

So my goals are:

collecting data (loop) has higher priority
web server should serve newly collected data
What would be the logic for achieving my goals? One solution is two scripts, 
one for web server and another one for collecting data. I am looking for an 
option for integration of both functionalities in one script.

Regards.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Allow additional separator character in variables

[Mikhail V]

Chris A wrote:

On Fri, Nov 24, 2017 at 1:10 AM, Mikhail V  wrote:
>
> > Well, then there is some bitter irony in this, so it allows pretty
> > much everything,
> > but does not allow me to beautify code with hyphens.
> > I can fully understand the wish to use non-latin scripts in strings or 
> > comments.
> > As for identifiers, IMO, apart from latin letters and underscore, the
> > first unicode candidate
> > I would add is U+2010. And probably the LAST one I would add.
> >
>
> Fortunately for the world, you're not the one who decided which
> characters were permitted in Python identifiers. The ability to use
> non-English words for function/variable names is of huge value; the
> ability to use a hyphen is of some value, but not nearly as much.
>


Fortunately for the world we have Chris A. Who knows what is
fortunate and of huge values.
So is there any real world projects example of usage of non-latin scripts
in identifiers? Or is it still only a plan for the new world?


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python loop and web server (bottle) in the same script

[Chris Angelico]

On Fri, Nov 24, 2017 at 1:27 AM,   wrote:
> I would like to have a script that collects data every minute and at the same 
> time serve newly collected data as web pages.
>
> Timely collecting data is more important than serving web pages, so 
> collecting data should have priority and should never be interrupted by 
> serving web pages.
>
> Looks like I need some sort of parallelization.

Sounds like you want threads. Spin off a thread that collects data and
sleeps, collects data and sleeps; meanwhile, serve web pages. But
there's no way you can *guarantee* that the serving of pages won't
interrupt data collection. With separate processes, you could
potentially use OS-level prioritization to help your data collection
process, but even then, it's not a guarantee. What are the
consequences of mistimed data collection?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Allow additional separator character in variables

[Chris Angelico]

On Fri, Nov 24, 2017 at 1:39 AM, Mikhail V  wrote:
> Chris A wrote:
>
> On Fri, Nov 24, 2017 at 1:10 AM, Mikhail V  wrote:
>>
>> > Well, then there is some bitter irony in this, so it allows pretty
>> > much everything,
>> > but does not allow me to beautify code with hyphens.
>> > I can fully understand the wish to use non-latin scripts in strings or 
>> > comments.
>> > As for identifiers, IMO, apart from latin letters and underscore, the
>> > first unicode candidate
>> > I would add is U+2010. And probably the LAST one I would add.
>> >
>>
>> Fortunately for the world, you're not the one who decided which
>> characters were permitted in Python identifiers. The ability to use
>> non-English words for function/variable names is of huge value; the
>> ability to use a hyphen is of some value, but not nearly as much.
>>
>
>
> Fortunately for the world we have Chris A. Who knows what is
> fortunate and of huge values.
> So is there any real world projects example of usage of non-latin scripts
> in identifiers? Or is it still only a plan for the new world?

Yes, I've used them personally. And I know other people who have.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Mikhail V]

Chris A wrote:

>> On Fri, Nov 24, 2017 at 1:10 AM, Mikhail V wrote:
>>
>>> Chris A wrote:
>>>
>>> Fortunately for the world, you're not the one who decided which
>>> characters were permitted in Python identifiers. The ability to use
>>> non-English words for function/variable names is of huge value; the
>>> ability to use a hyphen is of some value, but not nearly as much.
>>
>> Fortunately for the world we have Chris A. Who knows what is
>> fortunate and of huge values.
>> So is there any real world projects example of usage of non-latin scripts
>> in identifiers? Or is it still only a plan for the new world?


> Yes, I've used them personally. And I know other people who have.


Oh, I though it would be more impressive showcase for 'huge value'.
If we drop the benefit of the bare fact that you can do it, or you just
don't know English, how would describe the practical benefit?
If you don't know english, then programming at all will be just too hard.
(or one must define a new whole language specially for some local script)

I mean for a real practical situation - for example for an average
Python programmer or someone who seeks a programmer job.
And who does not have a 500-key keyboard, and who has
a not enough high threshold of vision sensitivity to bear the look
of various scripts in one small text piece?

Ok, I personally could find some practical usage for that, but
merely for fun. I doubt though that someone with less
typographical experience and overall computer literacy could
really make benefits even for personal usage.

So - fun is one benefit. And fun is important. But is that the
idea behind it?


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 5:42 AM, Mikhail V  wrote:
> Chris A wrote:
>
>>> On Fri, Nov 24, 2017 at 1:10 AM, Mikhail V wrote:
>>>
 Chris A wrote:

 Fortunately for the world, you're not the one who decided which
 characters were permitted in Python identifiers. The ability to use
 non-English words for function/variable names is of huge value; the
 ability to use a hyphen is of some value, but not nearly as much.
>>>
>>> Fortunately for the world we have Chris A. Who knows what is
>>> fortunate and of huge values.
>>> So is there any real world projects example of usage of non-latin scripts
>>> in identifiers? Or is it still only a plan for the new world?
>
>
>> Yes, I've used them personally. And I know other people who have.
>
>
> Oh, I though it would be more impressive showcase for 'huge value'.
> If we drop the benefit of the bare fact that you can do it, or you just
> don't know English, how would describe the practical benefit?
> If you don't know english, then programming at all will be just too hard.
> (or one must define a new whole language specially for some local script)

Let's start with a simpler question. Which of these is better code?

# == Option 1
class ZipExhausted(Exception):
pass

def zip_longest(*args, **kwds):
# zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
fillvalue = kwds.get('fillvalue')
counter = len(args) - 1
def sentinel():
nonlocal counter
if not counter:
raise ZipExhausted
counter -= 1
yield fillvalue
fillers = repeat(fillvalue)
iterators = [chain(it, sentinel(), fillers) for it in args]
try:
while iterators:
yield tuple(map(next, iterators))
except ZipExhausted:
pass

# = Option 2

class e(Exception):
pass

def zl(*a, **k):
f = f.get('fillvalue')
c = len(a) - 1
def s():
nonlocal c
if not c:
raise e
c -= 1
yield f
ff = repeat(f)
i = [chain(i, s(), ff) for i in args]
try:
while i:
yield tuple(map(next, i))
except e:
pass

# 

One of them is cribbed straight from the itertools docs. The other is
the same functionality with shorter variable names. What makes one of
them better than the other? Answer me that, and I'll continue.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Thomas Jollans]

On 23/11/17 19:42, Mikhail V wrote:
> I mean for a real practical situation - for example for an average
> Python programmer or someone who seeks a programmer job.
> And who does not have a 500-key keyboard, 

I don't think it's too much to ask for a programmer to have the
technology and expertise necessary to type their own language in its
proper alphabet.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Karsten Hilbert]

On Thu, Nov 23, 2017 at 08:46:01PM +0100, Thomas Jollans wrote:

> > I mean for a real practical situation - for example for an average
> > Python programmer or someone who seeks a programmer job.
> > And who does not have a 500-key keyboard, 
> 
> I don't think it's too much to ask for a programmer to have the
> technology and expertise necessary to type their own language in its
> proper alphabet.

Surely, but it can make reusing code a nightmare.

Using function arguments written in Thai script ?

Understanding, let alone being able to read, code written in Arabic ?

No, thanks.

Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Mikhail V]

On Thu, Nov 23, 2017 at 8:46 PM, Thomas Jollans  wrote:
> On 23/11/17 19:42, Mikhail V wrote:
>> I mean for a real practical situation - for example for an average
>> Python programmer or someone who seeks a programmer job.
>> And who does not have a 500-key keyboard,
>
> I don't think it's too much to ask for a programmer to have the
> technology and expertise necessary to type their own language in its
> proper alphabet.

And I don't think it is too much of benefit of using two scripts in one
source to compensate the need to constantly switching. Do you
have a method to input e.g. Cyrillic and Latin without switching the
layout? If I just use few extra chars, then I'll bind a keyboard shortcut.
but even a two-language input is annoyance.
And I need to use Cyrillic and Latin constantly, so I know how it feels.


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Mikhail V]

On Thu, Nov 23, 2017 at 8:15 PM, Chris Angelico  wrote:

>
> Let's start with a simpler question. Which of these is better code?
>
> # == Option 1
> class ZipExhausted(Exception):
> pass
>
> def zip_longest(*args, **kwds):
> # zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
> fillvalue = kwds.get('fillvalue')
> counter = len(args) - 1
> def sentinel():
> nonlocal counter
> if not counter:
> raise ZipExhausted
> counter -= 1
> yield fillvalue
> fillers = repeat(fillvalue)
> iterators = [chain(it, sentinel(), fillers) for it in args]
> try:
> while iterators:
> yield tuple(map(next, iterators))
> except ZipExhausted:
> pass
>
> # = Option 2
>
> class e(Exception):
> pass
>
> def zl(*a, **k):
> f = f.get('fillvalue')
> c = len(a) - 1
> def s():
> nonlocal c
> if not c:
> raise e
> c -= 1
> yield f
> ff = repeat(f)
> i = [chain(i, s(), ff) for i in args]
> try:
> while i:
> yield tuple(map(next, i))
> except e:
> pass
>
> # 
>
> One of them is cribbed straight from the itertools docs. The other is
> the same functionality with shorter variable names. What makes one of
> them better than the other? Answer me that, and I'll continue.


I see you manually 'optimise' the look?
I personally would end with something like this:

def zip_longest(*A, **K):
value = K.get ('fillvalue')
count = len(a) - 1
def sentinel():
nonlocal count
if not count:
raise ZipExhausted
count -= 1
yield  value
fillers = repeat (value)
iterators = [chain (it, sentinel(), fillers) for it in A]
try:
while iterators:
yield tuple (map (next, iterators))
except ZipExhausted:
pass


So I would say, my option would be something inbetween.
Note that I tweaked it for proportional font, namely Times New Roman.

Particularly I find too narrow lines/words a bit eye-straining at times.
Also self-explanation is important in many cases. But that depends of
what context you cut the example.

But if you only ask which code of two looks better for me,
then, probably Second, but it has some issues for me, e.g. "c" and "e"
almost homoglyhs, too loose 'sieve'-like, short lines.


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 7:38 AM, Mikhail V  wrote:
> I see you manually 'optimise' the look?
> I personally would end with something like this:
>
> def zip_longest(*A, **K):
> value = K.get ('fillvalue')
> count = len(a) - 1
> def sentinel():
> nonlocal count
> if not count:
> raise ZipExhausted
> count -= 1
> yield  value
> fillers = repeat (value)
> iterators = [chain (it, sentinel(), fillers) for it in A]
> try:
> while iterators:
> yield tuple (map (next, iterators))
> except ZipExhausted:
> pass
>
>
> So I would say, my option would be something inbetween.
> Note that I tweaked it for proportional font, namely Times New Roman.

I don't see how the font applies here, but whatever. Which is better?
The one-letter names or the longer ones that tie in with what they're
doing?

Also, why do you have those loose spaces stuck in random places, eg
before some of the open parentheses but not others?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Mikhail V]

On Thu, Nov 23, 2017 at 9:39 PM, Chris Angelico  wrote:
> On Fri, Nov 24, 2017 at 7:38 AM, Mikhail V  wrote:
>> I see you manually 'optimise' the look?
>> I personally would end with something like this:
>>
>> def zip_longest(*A, **K):
>> value = K.get ('fillvalue')
>> count = len(a) - 1
>> def sentinel():
>> nonlocal count
>> if not count:
>> raise ZipExhausted
>> count -= 1
>> yield  value
>> fillers = repeat (value)
>> iterators = [chain (it, sentinel(), fillers) for it in A]
>> try:
>> while iterators:
>> yield tuple (map (next, iterators))
>> except ZipExhausted:
>> pass
>>
>>
>> So I would say, my option would be something inbetween.
>> Note that I tweaked it for proportional font, namely Times New Roman.

> I don't see how the font applies here, but whatever.

For a different font, say CourierNew (monospaced) the tweaking strategy might
be different.

> Which is better? The one-letter names or the longer ones that tie in with 
> what they're
> doing?

I think I have answered more or less in previous post, that you cutted off.
So you were not satisfied?
But now I am probably not get your 'better' meaning.
Better for understanding, or purely visually, i.e. less eye-straining?

> Also, why do you have those loose spaces stuck in random places, eg
> before some of the open parentheses but not others?

Is it not allowed? I like how it looks with Times font.


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 8:02 AM, Mikhail V  wrote:
> On Thu, Nov 23, 2017 at 9:39 PM, Chris Angelico  wrote:
>> On Fri, Nov 24, 2017 at 7:38 AM, Mikhail V  wrote:
>>> I see you manually 'optimise' the look?
>>> I personally would end with something like this:
>>>
>>> def zip_longest(*A, **K):
>>> value = K.get ('fillvalue')
>>> count = len(a) - 1
>>> def sentinel():
>>> nonlocal count
>>> if not count:
>>> raise ZipExhausted
>>> count -= 1
>>> yield  value
>>> fillers = repeat (value)
>>> iterators = [chain (it, sentinel(), fillers) for it in A]
>>> try:
>>> while iterators:
>>> yield tuple (map (next, iterators))
>>> except ZipExhausted:
>>> pass
>>>
>>>
>>> So I would say, my option would be something inbetween.
>>> Note that I tweaked it for proportional font, namely Times New Roman.
>
>> I don't see how the font applies here, but whatever.
>
> For a different font, say CourierNew (monospaced) the tweaking strategy might
> be different.

If you have ANY font-specific "tweaking", you're doing it wrong.
Thanks for making it look worse on everyone else's screen.

>> Which is better? The one-letter names or the longer ones that tie in with 
>> what they're
>> doing?
>
> I think I have answered more or less in previous post, that you cutted off.
> So you were not satisfied?
> But now I am probably not get your 'better' meaning.
> Better for understanding, or purely visually, i.e. less eye-straining?

Which one would you prefer to maintain? Which would you prefer in a code review?

Do you want to have one- and two-letter variable names, or longer and
more descriptive ones?

Seriously? Do I need to wrench this part out of you? This was supposed
to be the EASY question that everyone can agree on, from which I can
then draw my line of argument.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Richard Damon]

On 11/23/17 2:46 PM, Thomas Jollans wrote:

On 23/11/17 19:42, Mikhail V wrote:

I mean for a real practical situation - for example for an average
Python programmer or someone who seeks a programmer job.
And who does not have a 500-key keyboard,

I don't think it's too much to ask for a programmer to have the
technology and expertise necessary to type their own language in its
proper alphabet.


My personal feeling is that the language needs to be fully usable with 
just ASCII, so the - character (HYPHEN/MINUS) is the 
subtraction/negation operator, not an in-name hyphen. This also means 
the main library should use just the ASCII character set.


I do also realize that it could be very useful for programmers who are 
programming with other languages as their native, to be able to use 
words in their native language for their own symbols, and thus useful to 
use their own character sets. Yes, doing so may add difficulty to the 
programmers, as they may need to be switching keyboard layouts 
(especially if not using a LATIN based language), but that is THEIR 
decision to do so. It also may make it harder for outside programmers to 
hep, but again, that is the teams decision to make.


The Unicode Standard provides a fairly good classification of the 
characters, and it would make sense to define that an character that is 
defined as a 'Letter' or a 'Number', and some classes of Punctuation 
(connector and dash) be allowed in identifiers.


Fully implementing may be more complicated than it is worth. An interim 
simple solution would be just allow ALL (or maybe most, excluding a 
limited number of obvious exceptions) of the characters above the ASCII 
set, with a warning that only those classified as above are promised to 
remain valid, and that other characters, while currently not generating 
a syntax error, may do so in the future. It should also be stated that 
while currently no character normalization is being done, it may be 
added in the future, so identifiers that differ only by code point 
sequences that are defined as being equivalent, might in the future not 
be distinct.


Since my native language is English, this isn't that important to me, 
but I do see it as being useful to others with different native tongues. 
The simple implementation shouldn't be that hard, you can just allow 
character codes 0x80 and above as being acceptable in identifiers, with 
the documented warning that the current implementation allows some forms 
that may generate errors in the future. If enough interest is shown, 
adding better classification shouldn't be that hard.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 8:19 AM, Richard Damon  wrote:
> On 11/23/17 2:46 PM, Thomas Jollans wrote:
>>
>> On 23/11/17 19:42, Mikhail V wrote:
>>>
>>> I mean for a real practical situation - for example for an average
>>> Python programmer or someone who seeks a programmer job.
>>> And who does not have a 500-key keyboard,
>>
>> I don't think it's too much to ask for a programmer to have the
>> technology and expertise necessary to type their own language in its
>> proper alphabet.
>
>
> My personal feeling is that the language needs to be fully usable with just
> ASCII, so the - character (HYPHEN/MINUS) is the subtraction/negation
> operator, not an in-name hyphen. This also means the main library should use
> just the ASCII character set.
>
> I do also realize that it could be very useful for programmers who are
> programming with other languages as their native, to be able to use words in
> their native language for their own symbols, and thus useful to use their
> own character sets. Yes, doing so may add difficulty to the programmers, as
> they may need to be switching keyboard layouts (especially if not using a
> LATIN based language), but that is THEIR decision to do so. It also may make
> it harder for outside programmers to hep, but again, that is the teams
> decision to make.
>
> The Unicode Standard provides a fairly good classification of the
> characters, and it would make sense to define that an character that is
> defined as a 'Letter' or a 'Number', and some classes of Punctuation
> (connector and dash) be allowed in identifiers.

That's exactly how Python's identifiers are defined (modulo special
handling of some of the ASCII set, for reasons of backward
compatibility).

> Fully implementing may be more complicated than it is worth. An interim
> simple solution would be just allow ALL (or maybe most, excluding a limited
> number of obvious exceptions) of the characters above the ASCII set, with a
> warning that only those classified as above are promised to remain valid,
> and that other characters, while currently not generating a syntax error,
> may do so in the future. It should also be stated that while currently no
> character normalization is being done, it may be added in the future, so
> identifiers that differ only by code point sequences that are defined as
> being equivalent, might in the future not be distinct.

No, that would be a bad idea; some of those characters are more
logically operators or brackets, and some are most definitely
whitespace. Also, it's easier to *expand* the valid character set than
to *restrict* it, so it's better to start with only those characters
that you know for sure make sense, and then add more later. If the
xid_start and xid_continue classes didn't exist, it might be
reasonable to use "Letter, any" and "Number, any" as substitutes; but
those classes DO exist, so Python uses them.

But broadly speaking, yes; it's not hard to allow a bunch of
characters as part of Python source code. Actual language syntax (eg
keywords) is restricted to ASCII and to those symbols that can easily
be typed on most keyboards, but your identifiers are your business.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Richard Damon]

On 11/23/17 4:31 PM, Chris Angelico wrote:

On Fri, Nov 24, 2017 at 8:19 AM, Richard Damon  wrote:

On 11/23/17 2:46 PM, Thomas Jollans wrote:

On 23/11/17 19:42, Mikhail V wrote:

I mean for a real practical situation - for example for an average
Python programmer or someone who seeks a programmer job.
And who does not have a 500-key keyboard,

I don't think it's too much to ask for a programmer to have the
technology and expertise necessary to type their own language in its
proper alphabet.


My personal feeling is that the language needs to be fully usable with just
ASCII, so the - character (HYPHEN/MINUS) is the subtraction/negation
operator, not an in-name hyphen. This also means the main library should use
just the ASCII character set.

I do also realize that it could be very useful for programmers who are
programming with other languages as their native, to be able to use words in
their native language for their own symbols, and thus useful to use their
own character sets. Yes, doing so may add difficulty to the programmers, as
they may need to be switching keyboard layouts (especially if not using a
LATIN based language), but that is THEIR decision to do so. It also may make
it harder for outside programmers to hep, but again, that is the teams
decision to make.

The Unicode Standard provides a fairly good classification of the
characters, and it would make sense to define that an character that is
defined as a 'Letter' or a 'Number', and some classes of Punctuation
(connector and dash) be allowed in identifiers.

That's exactly how Python's identifiers are defined (modulo special
handling of some of the ASCII set, for reasons of backward
compatibility).


Fully implementing may be more complicated than it is worth. An interim
simple solution would be just allow ALL (or maybe most, excluding a limited
number of obvious exceptions) of the characters above the ASCII set, with a
warning that only those classified as above are promised to remain valid,
and that other characters, while currently not generating a syntax error,
may do so in the future. It should also be stated that while currently no
character normalization is being done, it may be added in the future, so
identifiers that differ only by code point sequences that are defined as
being equivalent, might in the future not be distinct.

No, that would be a bad idea; some of those characters are more
logically operators or brackets, and some are most definitely
whitespace. Also, it's easier to *expand* the valid character set than
to *restrict* it, so it's better to start with only those characters
that you know for sure make sense, and then add more later. If the
xid_start and xid_continue classes didn't exist, it might be
reasonable to use "Letter, any" and "Number, any" as substitutes; but
those classes DO exist, so Python uses them.

But broadly speaking, yes; it's not hard to allow a bunch of
characters as part of Python source code. Actual language syntax (eg
keywords) is restricted to ASCII and to those symbols that can easily
be typed on most keyboards, but your identifiers are your business.

ChrisA


My thought is you define a legal only those Unicode characters that via 
the defined classification would be normally legal, but perhaps the 
first implementation doesn't diagnose many of the illegal combinations. 
If that isn't Pythonic, then yes, implementing a fuller classification 
would be needed. That might also say normalization questions would need 
to be decided too.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Thomas Jollans]

On 23/11/17 23:15, Richard Damon wrote:
> 
> My thought is you define a legal only those Unicode characters that via
> the defined classification would be normally legal, but perhaps the
> first implementation doesn't diagnose many of the illegal combinations.
> If that isn't Pythonic, then yes, implementing a fuller classification
> would be needed. That might also say normalization questions would need
> to be decided too.
> 

You do realise that Python has a perfectly good definition of what's
allowed in an identifier that is thoroughly grounded in the Unicode
standard and works very well, right?


-- Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Richard Damon]

On 11/23/17 5:45 PM, Thomas Jollans wrote:

On 23/11/17 23:15, Richard Damon wrote:

My thought is you define a legal only those Unicode characters that via
the defined classification would be normally legal, but perhaps the
first implementation doesn't diagnose many of the illegal combinations.
If that isn't Pythonic, then yes, implementing a fuller classification
would be needed. That might also say normalization questions would need
to be decided too.


You do realise that Python has a perfectly good definition of what's
allowed in an identifier that is thoroughly grounded in the Unicode
standard and works very well, right?


-- Thomas


No, I wasn't aware that Python was already Unicode enabled in the source 
code set. Still fairly new with it, but the fact that people seemed to 
argue about doing it made me think it was allowed yet.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Thomas Jollans]

On 24/11/17 00:18, Richard Damon wrote:
> On 11/23/17 5:45 PM, Thomas Jollans wrote:
>> On 23/11/17 23:15, Richard Damon wrote:
>>> My thought is you define a legal only those Unicode characters that via
>>> the defined classification would be normally legal, but perhaps the
>>> first implementation doesn't diagnose many of the illegal combinations.
>>> If that isn't Pythonic, then yes, implementing a fuller classification
>>> would be needed. That might also say normalization questions would need
>>> to be decided too.
>>>
>> You do realise that Python has a perfectly good definition of what's
>> allowed in an identifier that is thoroughly grounded in the Unicode
>> standard and works very well, right?
>>
>>
>> -- Thomas
> 
> No, I wasn't aware that Python was already Unicode enabled in the source
> code set. Still fairly new with it, but the fact that people seemed to
> argue about doing it made me think it was allowed yet.
> 

It's an old favourite some people to shout about on slow news days...

Python allows identifiers to start with any letter or an underscore, and
continue with any letter or number, or an underscore. The details follow
the Unicode XID_* properties.

In comments and strings, anything goes.


-- Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Ian Kelly]

On Thu, Nov 23, 2017 at 1:04 PM, Karsten Hilbert
 wrote:
> On Thu, Nov 23, 2017 at 08:46:01PM +0100, Thomas Jollans wrote:
>
>> > I mean for a real practical situation - for example for an average
>> > Python programmer or someone who seeks a programmer job.
>> > And who does not have a 500-key keyboard,
>>
>> I don't think it's too much to ask for a programmer to have the
>> technology and expertise necessary to type their own language in its
>> proper alphabet.
>
> Surely, but it can make reusing code a nightmare.
>
> Using function arguments written in Thai script ?
>
> Understanding, let alone being able to read, code written in Arabic ?

People are going to write code in Arabic whether you like it or not,
because not everybody speaks English, and not everybody who does
*wants* to use it. Now, would you prefer to read code where the
variable names are written in Arabic script, or where the variable
names are still in Arabic but transliterated to Latin characters?
Either way, you're not going to be able to understand it, so I'm not
sure why it makes a difference to you.

If Arabic characters are allowed however, then it might be of use to
the people who are going to code in Arabic anyway. And if it isn't,
then they have the option not to use it either.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Ian Kelly]

On Thu, Nov 23, 2017 at 2:19 PM, Richard Damon  wrote:
> The Unicode Standard provides a fairly good classification of the
> characters, and it would make sense to define that an character that is
> defined as a 'Letter' or a 'Number', and some classes of Punctuation
> (connector and dash) be allowed in identifiers.
>
> Fully implementing may be more complicated than it is worth. An interim
> simple solution would be just allow ALL (or maybe most, excluding a limited
> number of obvious exceptions) of the characters above the ASCII set, with a
> warning that only those classified as above are promised to remain valid,
> and that other characters, while currently not generating a syntax error,
> may do so in the future. It should also be stated that while currently no
> character normalization is being done, it may be added in the future, so
> identifiers that differ only by code point sequences that are defined as
> being equivalent, might in the future not be distinct.

It's already implemented; nothing needs to be done. Unicode Standard
Annex #31 defines a recommended syntax of identifiers, which Python
basically follows, except that for backward compatibility Python also
allows identifiers to begin with an underscore. Compare the
recommended syntax at
http://unicode.org/reports/tr31/#Default_Identifier_Syntax with the
Python syntax at
https://docs.python.org/3/reference/lexical_analysis.html#identifiers.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Mikhail V]

On Thu, Nov 23, 2017 at 10:05 PM, Chris Angelico  wrote:
> On Fri, Nov 24, 2017 at 8:02 AM, Mikhail V  wrote:
>> On Thu, Nov 23, 2017 at 9:39 PM, Chris Angelico  wrote:
>>> On Fri, Nov 24, 2017 at 7:38 AM, Mikhail V  wrote:
 I see you manually 'optimise' the look?
 I personally would end with something like this:

 def zip_longest(*A, **K):
 value = K.get ('fillvalue')
 count = len(a) - 1
 def sentinel():
 nonlocal count
 if not count:
 raise ZipExhausted
 count -= 1
 yield  value
 fillers = repeat (value)
 iterators = [chain (it, sentinel(), fillers) for it in A]
 try:
 while iterators:
 yield tuple (map (next, iterators))
 except ZipExhausted:
 pass


 So I would say, my option would be something inbetween.
 Note that I tweaked it for proportional font, namely Times New Roman.
>>
>>> I don't see how the font applies here, but whatever.
>>
>> For a different font, say CourierNew (monospaced) the tweaking strategy might
>> be different.
>
> If you have ANY font-specific "tweaking", you're doing it wrong.
> Thanks for making it look worse on everyone else's screen.

Trolling attempt counted :)
No I don't have any particular font-specific strategy,
it is just my wording reflecting the fact that things look different
in different fonts, even among proportional fonts.


>
>>> Which is better? The one-letter names or the longer ones that tie in with 
>>> what they're
>>> doing?
>>
>> I think I have answered more or less in previous post, that you cutted off.
>> So you were not satisfied?
>> But now I am probably not get your 'better' meaning.
>> Better for understanding, or purely visually, i.e. less eye-straining?
>
> Which one would you prefer to maintain? Which would you prefer in a code 
> review?
>
> Do you want to have one- and two-letter variable names, or longer and
> more descriptive ones?
>
> Seriously? Do I need to wrench this part out of you? This was supposed
> to be the EASY question that everyone can agree on, from which I can
> then draw my line of argument.


>From my above example, you could probably see that I prefer somewhat
middle-sized identifiers, one-two syllables. And naturally, they tend to
reflect some process/meaining, it is not always achievable,
but yes there is such a natural tendency, although by me personally
not so strong, and quite often I use totally meaningless names,
mainly to avoid visual similarity to already created names.
So for very expanded names, it ends up with a lot of underscores :(


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 1:44 PM, Mikhail V  wrote:
> From my above example, you could probably see that I prefer somewhat
> middle-sized identifiers, one-two syllables. And naturally, they tend to
> reflect some process/meaining, it is not always achievable,
> but yes there is such a natural tendency, although by me personally
> not so strong, and quite often I use totally meaningless names,
> mainly to avoid visual similarity to already created names.
> So for very expanded names, it ends up with a lot of underscores :(

Okay. So if it makes sense for you to use English words instead of
individual letters, since you are fluent in English, does it stand to
reason that it would make sense for other programmers to use Russian,
Norwegian, Hebrew, Korean, or Japanese words the same way?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Mikhail V]

On Fri, Nov 24, 2017 at 4:13 AM, Chris Angelico  wrote:
> On Fri, Nov 24, 2017 at 1:44 PM, Mikhail V  wrote:
>> From my above example, you could probably see that I prefer somewhat
>> middle-sized identifiers, one-two syllables. And naturally, they tend to
>> reflect some process/meaining, it is not always achievable,
>> but yes there is such a natural tendency, although by me personally
>> not so strong, and quite often I use totally meaningless names,
>> mainly to avoid visual similarity to already created names.
>> So for very expanded names, it ends up with a lot of underscores :(
>
> Okay. So if it makes sense for you to use English words instead of
> individual letters, since you are fluent in English, does it stand to
> reason that it would make sense for other programmers to use Russian,
> Norwegian, Hebrew, Korean, or Japanese words the same way?

I don't know. Probably, especially if those *programmers* don't know latin
letters, then they would want to write code with their letters and their
language. This target group, as I said, will have really hard time
with programming,
and in Python in particular, because they will be not only forced to learn
some english, but also will have all 'pleasures' of  multi-script editing.
But wait, probably one can write python code in, say Arabic script *only*?
How about such feature proposal?

As for non-english speaker who know some English already,
could of course want to include identifiers in those scripts.
But how about libraries?
Ok, so we return back to my original question: apart from
ability to do so, how beneficial is it on a pragmatical basis?
I mean, e.g. Cyrillic will introduce homoglyph issues.
CJK and Arabic scripts are metrically and optically incompatible with
latin, so such mixing will end up with messy look. So just for
the experiment, yes, it's fun.


Mikhail
-- 
https://mail.python.org/mailman/listinfo/python-list

Increasing the diversity of people who write Python (was: Benefits of unicode identifiers)

[Ben Finney]

Ian Kelly  writes:

> On Thu, Nov 23, 2017 at 1:04 PM, Karsten Hilbert
>  wrote:
> > Using function arguments written in Thai script ?
> >
> > Understanding, let alone being able to read, code written in Arabic
> > ?
>
> People are going to write code in Arabic whether you like it or not,
> because not everybody speaks English, and not everybody who does
> *wants* to use it.

This is, I think, a good reason to allow Unicode identifiers (at least,
those Unicode subsets which encode writing systems of languages) as
programming-language identifiers.

> Now, would you prefer to read code where the variable names are
> written in Arabic script, or where the variable names are still in
> Arabic but transliterated to Latin characters?

(On the – evidently correct, in Karsten's case and mine – assumption
that the reader does not understand Arabic script.)

I've thought about this, and if the quesition is which would *I* prefer,
the answer is I'd prefer the identifiers transliterated to the Latin
(English-writing) characters.

Because if I already can't understand the words, it will be more useful
to me to be able to type them reliably at a keyboard, for replication,
search, discussion with others about the code, etc.

Set against that, though, I want the preferences of *others* to be taken
into consideration also. And there are many more people who do not
natively write English/Latin characters, that I want to feel welcome in
the Python community.

So it's a good thing that my own reading preference *does not* have
weight in this matter. I'm not the primary audience for code identifiers
written in Arabic script, so my preference should matter less than those
who understand it.

> Either way, you're not going to be able to understand it, so I'm not
> sure why it makes a difference to you.

I hope you can see that it can simultaneously make a difference – I
would definitely prefer to read Latin-writing identifiers – while also
being a lesser consideration that should not outweigh the benefits of
allowing non-Latin-script identifiers.

> If Arabic characters are allowed however, then it might be of use to
> the people who are going to code in Arabic anyway. And if it isn't,
> then they have the option not to use it either.

This is a necessary consequence of increasing the diversity of people
able to program in Python: people will express ideas originating in
their own language, in Python code.

For that diversity to increase, we English-fluent folk will necessarily
become a smaller proportion of the programming community than we are
today. That might be uncomfortable for us, but it is a necessary
adaptation the community needs to undergo.

-- 
 \ “In any great organization it is far, far safer to be wrong |
  `\  with the majority than to be right alone.” —John Kenneth |
_o__)Galbraith, 1989-07-28 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Increasing the diversity of people who write Python (was: Benefits of unicode identifiers)

[Andrew Z]

I have hard time seeing the benefits of this "necessity" , just
unreasonable  overcomplications for the name of "diversity".




On Nov 23, 2017 22:57, "Ben Finney"  wrote:

> Ian Kelly  writes:
>
> > On Thu, Nov 23, 2017 at 1:04 PM, Karsten Hilbert
> >  wrote:
> > > Using function arguments written in Thai script ?
> > >
> > > Understanding, let alone being able to read, code written in Arabic
> > > ?
> >
> > People are going to write code in Arabic whether you like it or not,
> > because not everybody speaks English, and not everybody who does
> > *wants* to use it.
>
> This is, I think, a good reason to allow Unicode identifiers (at least,
> those Unicode subsets which encode writing systems of languages) as
> programming-language identifiers.
>
> > Now, would you prefer to read code where the variable names are
> > written in Arabic script, or where the variable names are still in
> > Arabic but transliterated to Latin characters?
>
> (On the – evidently correct, in Karsten's case and mine – assumption
> that the reader does not understand Arabic script.)
>
> I've thought about this, and if the quesition is which would *I* prefer,
> the answer is I'd prefer the identifiers transliterated to the Latin
> (English-writing) characters.
>
> Because if I already can't understand the words, it will be more useful
> to me to be able to type them reliably at a keyboard, for replication,
> search, discussion with others about the code, etc.
>
> Set against that, though, I want the preferences of *others* to be taken
> into consideration also. And there are many more people who do not
> natively write English/Latin characters, that I want to feel welcome in
> the Python community.
>
> So it's a good thing that my own reading preference *does not* have
> weight in this matter. I'm not the primary audience for code identifiers
> written in Arabic script, so my preference should matter less than those
> who understand it.
>
> > Either way, you're not going to be able to understand it, so I'm not
> > sure why it makes a difference to you.
>
> I hope you can see that it can simultaneously make a difference – I
> would definitely prefer to read Latin-writing identifiers – while also
> being a lesser consideration that should not outweigh the benefits of
> allowing non-Latin-script identifiers.
>
> > If Arabic characters are allowed however, then it might be of use to
> > the people who are going to code in Arabic anyway. And if it isn't,
> > then they have the option not to use it either.
>
> This is a necessary consequence of increasing the diversity of people
> able to program in Python: people will express ideas originating in
> their own language, in Python code.
>
> For that diversity to increase, we English-fluent folk will necessarily
> become a smaller proportion of the programming community than we are
> today. That might be uncomfortable for us, but it is a necessary
> adaptation the community needs to undergo.
>
> --
>  \ “In any great organization it is far, far safer to be wrong |
>   `\  with the majority than to be right alone.” —John Kenneth |
> _o__)Galbraith, 1989-07-28 |
> Ben Finney
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 2:52 PM, Mikhail V  wrote:
> On Fri, Nov 24, 2017 at 4:13 AM, Chris Angelico  wrote:
>> On Fri, Nov 24, 2017 at 1:44 PM, Mikhail V  wrote:
>>> From my above example, you could probably see that I prefer somewhat
>>> middle-sized identifiers, one-two syllables. And naturally, they tend to
>>> reflect some process/meaining, it is not always achievable,
>>> but yes there is such a natural tendency, although by me personally
>>> not so strong, and quite often I use totally meaningless names,
>>> mainly to avoid visual similarity to already created names.
>>> So for very expanded names, it ends up with a lot of underscores :(
>>
>> Okay. So if it makes sense for you to use English words instead of
>> individual letters, since you are fluent in English, does it stand to
>> reason that it would make sense for other programmers to use Russian,
>> Norwegian, Hebrew, Korean, or Japanese words the same way?
>
> I don't know. Probably, especially if those *programmers* don't know latin
> letters, then they would want to write code with their letters and their
> language. This target group, as I said, will have really hard time
> with programming,
> and in Python in particular, because they will be not only forced to learn
> some english, but also will have all 'pleasures' of  multi-script editing.
> But wait, probably one can write python code in, say Arabic script *only*?
> How about such feature proposal?

If Python supports ASCII identifiers only, people have no choice but
to transliterate. As it is, people get to choose which is better for
them - to transliterate or not to transliterate, that is the
readability question.

> As for non-english speaker who know some English already,
> could of course want to include identifiers in those scripts.
> But how about libraries?

If you want to use numpy, you have to understand the language of
numpy. That's a lot of technical jargon, so even if you understand
English, you have to learn that. So there's ultimately no difference.

The most popular libraries, just like the standard library, are
usually going to choose to go ASCII-only as the
lowest-common-denominator. But that is, again, their own choice.

> Ok, so we return back to my original question: apart from
> ability to do so, how beneficial is it on a pragmatical basis?
> I mean, e.g. Cyrillic will introduce homoglyph issues.
> CJK and Arabic scripts are metrically and optically incompatible with
> latin, so such mixing will end up with messy look. So just for
> the experiment, yes, it's fun.

Does it really introduce homoglyph issues in real-world situations,
though? Are there really cases where people can't figure out from
context what's going on? I haven't seen that happening. Usually there
are *entire words* (and more) in a single language, making it pretty
easy to figure out.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Increasing the diversity of people who write Python (was: Benefits of unicode identifiers)

[Thomas Jollans]

On 24/11/17 05:45, Andrew Z wrote:
> I have hard time seeing the benefits of this "necessity" , just
> unreasonable  overcomplications for the name of "diversity".

What complications?
-- 
https://mail.python.org/mailman/listinfo/python-list