Re: ironpython not support py3.6

[Peter J. Holzer]

On 2018-06-22 17:20:29 -0700, denis.akhiya...@gmail.com wrote:
> Either wait for IronPython 3.6, use COM interop, pythonnet,
> subprocess, or things like gRPC. Based on PyPy experience, it is
> probably 1-2 years of sponsored development to get a working
> IronPython 3.6.

What is the current state of IronPython 3? The website is very
uninformative (there's a 4 year old blog post saying "we started a
repository", but that's it). The repository gets commits every now and
then (so it's obviously not dead yet), but neither the readme nor the
wiki give a hint beyond "not ready yet".

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/22/18 11:21 PM, Steven D'Aprano wrote:
> On Fri, 22 Jun 2018 20:06:35 +0100, Ben Bacarisse wrote:
>
>> Steven D'Aprano  writes:
>>
>>> On Fri, 22 Jun 2018 11:14:59 +0100, Ben Bacarisse wrote:
>>>
>> The code page remark is curious.  Will some "code pages" have digits
>> that are not ASCII digits?
> Good question.  I have no idea.
 It's much more of an open question than I thought.
>>> Nah, Python already solves that for you:
>> My understanding was that the OP does not (reliably) know the encoding,
>> though that was a guess based on a turn of phrase.
> I took it the other way: that Ethan *does* know the encoding, and his 
> problem is that knowing the encoding and/or locale is not enough to 
> recognise whether to use a period or comma as the decimal separator.
>
> Which it isn't.
If you know the Locale, then you do know what the decimal separator is,
as that is part of what a locale defines. The issue is that if you just
know the encoding, you don't necessarily know the locale. He also
commented that he didn't want to set the locale in the routine, as that
sets it globally for the full application (but perhaps that latter could
be fixed by first doing a locale.getlocale(), then setlocale for the
files locale, and then at the end of reading and processing restore back
the old locale.
>
> If he doesn't know the encoding, he has bigger problems than just 
> converting strings into floats. Without knowing the encoding, he cannot 
> even reliably detect non-ASCII digits at all.
>
>
>> Another guess is that the OP does not have Unicode data.  The term "code
>> page" hints at an 8-bit encoding or at least a pre-Unicode one.
> Assuming he is using Python 3, or using Python 2 sensibly, once he has 
> specified the encoding and read the data from the file, he has Unicode.
>
> Unicode is a superset of (ideally) all code pages. Once you have decoded 
> the data using the appropriate code page, you have a Unicode string, and 
> Python doesn't care where it came from.
>
> The point is, once Ethan can get the intended characters out of the file 
> into Python, it doesn't matter what code page they came from. They're now 
> full-fledged Unicode characters, and Python's float() and int() functions 
> can easily deal with non-ASCII digits. So long as he has digits in the 
> first place, float() and int() will deal with them correctly.
>
>

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Steven D'Aprano]

On Sat, 23 Jun 2018 06:26:22 -0400, Richard Damon wrote:

> If you know the Locale, then you do know what the decimal separator is,
> as that is part of what a locale defines.

A locale defines a set of common cultural conventions. It doesn't mandate 
the actual conventions in use in any specific document.

If I'm in Australia, using the en-AU locale, nevertheless I can generate 
a file using , as a decimal separator. Try and stop me :-)

But your point is taken -- I misread Ethan saying that he knew the locale 
and it wasn't helping, when in fact he was reluctant to change the locale 
as that's a process-wide global change.

> The issue is that if you just
> know the encoding, you don't necessarily know the locale. He also
> commented that he didn't want to set the locale in the routine, as that
> sets it globally for the full application (but perhaps that latter could
> be fixed by first doing a locale.getlocale(), then setlocale for the
> files locale, and then at the end of reading and processing restore back
> the old locale.

Indeed.


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Marko Rauhamaa]

Richard Damon :
> If you know the Locale, then you do know what the decimal separator
> is, as that is part of what a locale defines.

I don't know what that sentence means.

> The issue is that if you just know the encoding, you don't necessarily
> know the locale.

I always know my locale. The locale is tied to the human user.

> He also commented that he didn't want to set the locale in the
> routine, as that sets it globally for the full application (but
> perhaps that latter could be fixed by first doing a
> locale.getlocale(), then setlocale for the files locale, and then at
> the end of reading and processing restore back the old locale.

Setting a locale application-wise is

 * not in accordance with the idea of a locale (the locale should be
   constant within a user session)

 * not easily possible (the locale is seen by all threads
   simultaneously)


BTW, I think the locale is a terrible invention.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 7:46 AM, Steven D'Aprano wrote:
> On Sat, 23 Jun 2018 06:26:22 -0400, Richard Damon wrote:
>
>> If you know the Locale, then you do know what the decimal separator is,
>> as that is part of what a locale defines.
> A locale defines a set of common cultural conventions. It doesn't mandate 
> the actual conventions in use in any specific document.
>
> If I'm in Australia, using the en-AU locale, nevertheless I can generate 
> a file using , as a decimal separator. Try and stop me :-)
yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
One, just as you can misuse the language and write cat when you mean a
member of the Canine group, but then the misinterpretation is on the
creator of the document, not on the program that was told how the
document is to be read.
>
> But your point is taken -- I misread Ethan saying that he knew the locale 
> and it wasn't helping, when in fact he was reluctant to change the locale 
> as that's a process-wide global change.
>
>> The issue is that if you just
>> know the encoding, you don't necessarily know the locale. He also
>> commented that he didn't want to set the locale in the routine, as that
>> sets it globally for the full application (but perhaps that latter could
>> be fixed by first doing a locale.getlocale(), then setlocale for the
>> files locale, and then at the end of reading and processing restore back
>> the old locale.
> Indeed.
>
>

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Peter J. Holzer]

On 2018-06-23 08:12:52 -0400, Richard Damon wrote:
> On 6/23/18 7:46 AM, Steven D'Aprano wrote:
> > On Sat, 23 Jun 2018 06:26:22 -0400, Richard Damon wrote:
> >> If you know the Locale, then you do know what the decimal separator is,
> >> as that is part of what a locale defines.
> > A locale defines a set of common cultural conventions. It doesn't mandate 
> > the actual conventions in use in any specific document.
> >
> > If I'm in Australia, using the en-AU locale, nevertheless I can generate 
> > a file using , as a decimal separator. Try and stop me :-)
> yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
> One, just as you can misuse the language and write cat when you mean a
> member of the Canine group, but then the misinterpretation is on the
> creator of the document, not on the program that was told how the
> document is to be read.

How would he mis-use the en-AU locale to write 1 as "1,000"? I think
to do that he would simply NOT use the locale.

I think there are very good reasons to ignore the locale for specific
purposes. For example, a Python interpreter should not use the locale
when parsing Python, and a program producing Python should also ignore
the locale.

You two also seem to be writing about different things when you write
"THE locale". Steven seems to mean the global settings a user has
chosen, you seem to mean the specidic settings appropriate for parsing a
specific file.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 8:03 AM, Marko Rauhamaa wrote:
> Richard Damon :
>> If you know the Locale, then you do know what the decimal separator
>> is, as that is part of what a locale defines.
> I don't know what that sentence means.
When you set the locale
>
>> The issue is that if you just know the encoding, you don't necessarily
>> know the locale.
> I always know my locale. The locale is tied to the human user.
No, it should be tied to the data you are processing. If an English user
is feeding a program Chinese documents, while processing those documents
the program should be using the appropriate Chinese Locale. When
generating output to the user, it should switch (back) to the
appropriate English Locale (likely the system locale that the user set).
>
>> He also commented that he didn't want to set the locale in the
>> routine, as that sets it globally for the full application (but
>> perhaps that latter could be fixed by first doing a
>> locale.getlocale(), then setlocale for the files locale, and then at
>> the end of reading and processing restore back the old locale.
> Setting a locale application-wise is
>
>  * not in accordance with the idea of a locale (the locale should be
>constant within a user session)
Again, no, a locale is tied to the data, not the user (unless you want
to require the user to translate all data to his locale conventions
(without using a program that can use locale information) before
providing it to a program. Yes, the default for the interpretation
should be the users default/current locale, but you really want them to
be able to say I got this file from someone whose locale was different
than mine.

Data presented to the user should normally use his locale (unless he has
specified something different).
>
>  * not easily possible (the locale is seen by all threads
>simultaneously)
That is an implementation error. It should be possible to create a
thread specific locale, and it is really useful to create a local locale
that can be used by the various conversion operators to say for this
conversion use this specific locale as that is what this data indicated
how it is to be interpreted.
>
>
> BTW, I think the locale is a terrible invention.
>
>
> Marko

The locale is a lot better than the alternative, where every application
that needs to deal with internationalization need to recreate (and
debub) all of the mechanism. I agree it isn't perfect, and for small
simple programs it would be nice to be able to say "I don't want all
this stuff, make it go away".

Python took its locale (at least initially) from C, which was a single
global which does have more issues because of this. C++ objectified the
locale and allows the programmer to imbue a specific locale into
different parts of his program (in particular, each I/O Stream knows
what locale its data is to be processed with). Perhaps (maybe it has) it
could be good to adopt the object based locale concept of C++ (but that
does come at a significant cost for things like CPython) where streams
know their locale, and other operations can be optionally passed a
locale to use.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 8:28 AM, Peter J. Holzer wrote:
> On 2018-06-23 08:12:52 -0400, Richard Damon wrote:
>> On 6/23/18 7:46 AM, Steven D'Aprano wrote:
>>> On Sat, 23 Jun 2018 06:26:22 -0400, Richard Damon wrote:
 If you know the Locale, then you do know what the decimal separator is,
 as that is part of what a locale defines.
>>> A locale defines a set of common cultural conventions. It doesn't mandate 
>>> the actual conventions in use in any specific document.
>>>
>>> If I'm in Australia, using the en-AU locale, nevertheless I can generate 
>>> a file using , as a decimal separator. Try and stop me :-)
>> yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
>> One, just as you can misuse the language and write cat when you mean a
>> member of the Canine group, but then the misinterpretation is on the
>> creator of the document, not on the program that was told how the
>> document is to be read.
> How would he mis-use the en-AU locale to write 1 as "1,000"? I think
> to do that he would simply NOT use the locale.
Once you open the Locale can of worms, EVERYTHING has a locale, to say
you aren't using a locale is to say you are writing
something unintelligible, as you can thing of the locale as the set of
rules to interpret
>
> I think there are very good reasons to ignore the locale for specific
> purposes. For example, a Python interpreter should not use the locale
> when parsing Python, and a program producing Python should also ignore
> the locale.
Python, like many languages, define the formatting of things, so Python
programs should be interpreted according to the "Python" locale (which
may actually be named "C").
>
> You two also seem to be writing about different things when you write
> "THE locale". Steven seems to mean the global settings a user has
> chosen, you seem to mean the specidic settings appropriate for parsing a
> specific file.
>
> hp
>
You have THE locale for a given piece of data. My point is that Python
has adopted the C method of a single global locale for a program, so in
the program there is a 'THE Locale' which may actually need to be
different when processing different information, leading to some of the
issues.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Bart]

On 23/06/2018 04:51, Steven D'Aprano wrote:

On Wed, 20 Jun 2018 14:18:19 +1000, Chris Angelico wrote:


Ah. Yeah, that would be a plausible feature to add to Python. But in C,
a static variable is basically the same thing as a global variable,
except that its name is scoped to the function. There is only one of it.
What happens in Python? For instance:

def f():
 def g():
 static x = 0
 x += 1
 return x
 return g

Does the static variable exist once for each instance of g()? If so,
it'll behave like a closure variable; if not, it'll behave like a
global. Either way, I'm pretty much certain that people will expect the
other.


Yes, but given the normal execution model of Python, only one solution is
valid. Since the function g is created fresh each time f is called, each
one gets a fresh static x.

If you want all the g's to share the same x, you would write:

def f():
 static x = 0
 def g():
 x += 1
 return x
 return g


In this case, every invocation of f shares the same static x, and all the
g's refer to that same x, using the ordinary closure mechanism. In the
earlier case, each invocation of f creates a brand new g with its own x.

Simple and elegant.

This could at last get rid of that useful but ugly idiom:

 def function(real, arguments, len=len, int=int, str=str):
 ...

if we allowed the "static" declaration to access the values from the
surrounding scope:

 def function(real, arguments):
 static len=len, int=int, str=str

But I think nicer than that would be a decorator:

 @static(len=len, int=int, str=str)
 def function(real, arguments):
 ...

which adds local variables len, int, str to the function, with the given
values, and transforms all the bytecode LOAD_NAME len to LOAD_FAST len
(or whatever).

(We might need a new bytecode to SET_STATIC.)

That would be a nice bytecode hack to prove the usefulness of the concept!


This is an example of a simple concept getting so out of hand that it 
will either never be implemented, or the resulting implementation 
becomes impractical to use.


This is what we're trying to do:

def nextx():
static x = 0
x += 1
return x

And this is the simplest equivalent code in current Python that will 
cater for 99% of uses:


_nextx_x = 0

def nextx():
global _nextx_x
_nextx_x += 1
return _nextx_x

No nested functions. No generating new instances of functions complete 
with a new set of statics each time you happen to refer to the name. 
(Which sounds to me as useful as creating a new instance of an import 
when you copy its name, complete with a separate set of its globals. 
Isn't this stuff what classes are for?)


(At what point would that happen anyway; if you do this:

g = nextx  # hypothetical version would static

it will create a new instance of 'nextx'. But it won't create one here, 
just before () is applied:


nextx() # ?

Or how about here:

listoffunctions = (nextx1, nextx2, nextx3)

listoffunctions[i]() # ?

)

--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Ed Kellett]

On 2018-06-23 06:21, Chris Angelico wrote:
> Let's start finding all the edge cases that don't work, so I can work
> on fixing them :)

Very long functions (or, more specifically, functions with a very large
number of consts) will likely prove annoying.



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Marko Rauhamaa]

Richard Damon :

> On 6/23/18 8:03 AM, Marko Rauhamaa wrote:
>> I always know my locale. The locale is tied to the human user.
> No, it should be tied to the data you are processing.

   In computing, a locale is a set of parameters that defines the user's
   language, region and any special variant preferences that the user
   wants to see in their user interface.

   https://en.wikipedia.org/wiki/Locale_(computer_software)>

The data should not depend on the locale.

> If an English user is feeding a program Chinese documents, while
> processing those documents the program should be using the appropriate
> Chinese Locale.

Not true.

> Again, no, a locale is tied to the data, not the user (unless you want
> to require the user to translate all data to his locale conventions
> (without using a program that can use locale information) before
> providing it to a program. Yes, the default for the interpretation
> should be the users default/current locale, but you really want them
> to be able to say I got this file from someone whose locale was
> different than mine.

The locale is not directly related to data or data formats. Of course,
locales leak into data and create the sorry mess we are talking about.

> Data presented to the user should normally use his locale (unless he
> has specified something different).

Ok. Here's a value for you:

100€

I see '1', '0', '0', '€'. What do you see in your locale (LC_MONETARY)?

>> BTW, I think the locale is a terrible invention.
>
> The locale is a lot better than the alternative, where every
> application that needs to deal with internationalization need to
> recreate (and debub) all of the mechanism. I agree it isn't perfect,
> and for small simple programs it would be nice to be able to say "I
> don't want all this stuff, make it go away".

The locale doesn't solve a single problem in practice and often trips up
programs. For example, a customer-visible bug was once caused by:

   sort  Python took its locale (at least initially) from C, which was a single
> global which does have more issues because of this.

The single global is due to what the locale was introduced for. It came
about around the time when Unix applications were being made "8-bit
clean." Along with UCS-2 and XML, it's one of those things you wish
you'd never have to deal with.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 9:05 AM, Marko Rauhamaa wrote:
> Richard Damon :
>
>> On 6/23/18 8:03 AM, Marko Rauhamaa wrote:
>>> I always know my locale. The locale is tied to the human user.
>> No, it should be tied to the data you are processing.
>In computing, a locale is a set of parameters that defines the user's
>language, region and any special variant preferences that the user
>wants to see in their user interface.
>
>https://en.wikipedia.org/wiki/Locale_(computer_software)>
>
> The data should not depend on the locale.
So no one foreign ever gives you data? Note, that wikipedia article is
focused on the SYSTEM locale, which yes, that should reflect the what
the user wants in his interface.
>
>> If an English user is feeding a program Chinese documents, while
>> processing those documents the program should be using the appropriate
>> Chinese Locale.
> Not true.
How else is the program going to understand the Chinese data?
>
>> Again, no, a locale is tied to the data, not the user (unless you want
>> to require the user to translate all data to his locale conventions
>> (without using a program that can use locale information) before
>> providing it to a program. Yes, the default for the interpretation
>> should be the users default/current locale, but you really want them
>> to be able to say I got this file from someone whose locale was
>> different than mine.
> The locale is not directly related to data or data formats. Of course,
> locales leak into data and create the sorry mess we are talking about.
The fact that locale issues leak into data is the reason that the single
immutable global locale doesn't work. You really want to imbue into data
streams what locale their data represents (and use that in some of the
later processing of data from that stream).
>
>> Data presented to the user should normally use his locale (unless he
>> has specified something different).
> Ok. Here's a value for you:
>
> 100€
>
> I see '1', '0', '0', '€'. What do you see in your locale (LC_MONETARY)?
If I processed that on my system I would either get $100, or an error of
wrong currency symbol depending on the error checking.
>
>>> BTW, I think the locale is a terrible invention.
>> The locale is a lot better than the alternative, where every
>> application that needs to deal with internationalization need to
>> recreate (and debub) all of the mechanism. I agree it isn't perfect,
>> and for small simple programs it would be nice to be able to say "I
>> don't want all this stuff, make it go away".
> The locale doesn't solve a single problem in practice and often trips up
> programs. For example, a customer-visible bug was once caused by:
>
>sort 
> producing different results on different customers' machines.
>
> Mental note: *always* prefix GNU textutils commands with LANG=C.
Yes, one issue is that systems currently don't naturally tag data with
the locale to use (you can't even know for sure character set a file is
in, so your example above might be 100 some funny character(s). It is
starting be true that you can often assume UTF-8 (at least on Linux, on
Windows it is much less so), and validating that it is valid UTF-8 is a
pretty good sign that it is.
>
>> Python took its locale (at least initially) from C, which was a single
>> global which does have more issues because of this.
> The single global is due to what the locale was introduced for. It came
> about around the time when Unix applications were being made "8-bit
> clean." Along with UCS-2 and XML, it's one of those things you wish
> you'd never have to deal with.
>
>
> Marko

Locale predates UCS-2, it was the early attempt to provide
internationalization to C code so even programmers who didn't think
about it could add the line setlocale(LC_ALL, "") and make their code
work at least mostly right in more places. A single global was quick and
simple, and since threads didn't exist, not an issue.

In many ways it was the first attempt that should have been thrown away,
but got too intertwined. C++ made a significant improvement to it by
having streams remember their own locale.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Peter J. Holzer]

On 2018-06-23 08:41:38 -0400, Richard Damon wrote:
> On 6/23/18 8:28 AM, Peter J. Holzer wrote:
> > On 2018-06-23 08:12:52 -0400, Richard Damon wrote:
> >> On 6/23/18 7:46 AM, Steven D'Aprano wrote:
> >>> If I'm in Australia, using the en-AU locale, nevertheless I can generate 
> >>> a file using , as a decimal separator. Try and stop me :-)
> >> yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
> >> One, just as you can misuse the language and write cat when you mean a
> >> member of the Canine group, but then the misinterpretation is on the
> >> creator of the document, not on the program that was told how the
> >> document is to be read.
> > How would he mis-use the en-AU locale to write 1 as "1,000"? I think
> > to do that he would simply NOT use the locale.
> Once you open the Locale can of worms, EVERYTHING has a locale, to say
> you aren't using a locale is to say you are writing
> something unintelligible, as you can thing of the locale as the set of
> rules to interpret

I don't think that's a useful way to look at it. "Locale" in
(non-technical) English means "place" or "site". The idea behind the
locale concept is that some conventions (e.g. how to write numbers or
how to write strings) depend on the place where the program runs (or
maybe where the user is sitting or grew up or maybe where a file was
produced).

For stuff which doesn't depend on the place (e.g. how a Python program
should be parsed), the locale concept doesn't apply.


> > You two also seem to be writing about different things when you write
> > "THE locale". Steven seems to mean the global settings a user has
> > chosen, you seem to mean the specidic settings appropriate for parsing a
> > specific file.

While I was writing this paragraph I realized that I had also used "the
locale" in a specific meaning in the previous paragraph. I decided to
let it stand and see whether anyone would call me out it.

> You have THE locale for a given piece of data.

Well, you didn't. Even though I quite obviously used "the locale" in
Steven's meaning, you didn't react to that at all and just continue as
if your definition is the only possible one.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Marko Rauhamaa]

Richard Damon :

> On 6/23/18 9:05 AM, Marko Rauhamaa wrote:
>> Richard Damon :
>>
>>> On 6/23/18 8:03 AM, Marko Rauhamaa wrote:
 I always know my locale. The locale is tied to the human user.
>>> No, it should be tied to the data you are processing.
>>In computing, a locale is a set of parameters that defines the user's
>>language, region and any special variant preferences that the user
>>wants to see in their user interface.
>>
>>https://en.wikipedia.org/wiki/Locale_(computer_software)>
>>
>> The data should not depend on the locale.
> So no one foreign ever gives you data?

Never in my decades in computer programming have I found any use for
locales.

In particular, they have never helped me decode "foreign" data, whether
in ASCII, Latin-1, Latin-3, Latin-9, JIS or UTF-8.

> Note, that wikipedia article is focused on the SYSTEM locale, which
> yes, that should reflect the what the user wants in his interface.

I don't think locales have anything to do with anything else.


>>> If an English user is feeding a program Chinese documents, while
>>> processing those documents the program should be using the
>>> appropriate Chinese Locale.
>> Not true.
> How else is the program going to understand the Chinese data?

If someone gives me a file, they had better indicate the file format.

> The fact that locale issues leak into data is the reason that the
> single immutable global locale doesn't work.

Locales don't work. Period.

> You really want to imbue into data streams what locale their data
> represents (and use that in some of the later processing of data from
> that stream).

Can you refer to a standard for that kind of imbuement?

Of course, you have document types, schema definitions and other
implicit and explicit format indicators. You shouldn't call them
locales, though.

>>> Data presented to the user should normally use his locale (unless he
>>> has specified something different).
>> Ok. Here's a value for you:
>>
>> 100€
>>
>> I see '1', '0', '0', '€'. What do you see in your locale (LC_MONETARY)?
> If I processed that on my system I would either get $100, or an error of
> wrong currency symbol depending on the error checking.

Don't forget to convert the amount as well...

>> The single global is due to what the locale was introduced for. It
>> came about around the time when Unix applications were being made
>> "8-bit clean." Along with UCS-2 and XML, it's one of those things you
>> wish you'd never have to deal with.
>
> Locale predates UCS-2, it was the early attempt to provide
> internationalization to C code so even programmers who didn't think
> about it could add the line setlocale(LC_ALL, "") and make their code
> work at least mostly right in more places. A single global was quick
> and simple, and since threads didn't exist, not an issue.
>
> In many ways it was the first attempt that should have been thrown
> away, but got too intertwined. C++ made a significant improvement to
> it by having streams remember their own locale.

Noone should breathe any new life into locales.

And yes, add C++ to the list of things you wish you'd never have to deal
with...


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Peter J. Holzer]

On 2018-06-23 16:05:49 +0200, Peter J. Holzer wrote:
> I don't think that's a useful way to look at it. "Locale" in
> (non-technical) English means "place" or "site". The idea behind the
> locale concept is that some conventions (e.g. how to write numbers or
> how to write strings) depend on the place where the program runs

Sorry, I meant "how to *sort* strings.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: "Data blocks" syntax specification draft

[Abdur-Rahmaan Janhangeer]

the tab separated idea is used in :

e.g. see last section of files

Abdur-Rahmaan Janhangeer
https://github.com/Abdur-rahmaanJ


>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Steven D'Aprano]

On Sat, 23 Jun 2018 09:42:29 -0400, Richard Damon wrote:

> On 6/23/18 9:05 AM, Marko Rauhamaa wrote:

>> Ok. Here's a value for you:
>>
>> 100€
>>
>> I see '1', '0', '0', '€'. What do you see in your locale (LC_MONETARY)?
> 
> If I processed that on my system I would either get $100, or an error of
> wrong currency symbol depending on the error checking.

Then your system is so unbelievably broken that it should be nuked from 
orbit, just to be sure.

The data you were given was 100 Euros. If your system is incapable of 
reading that as 100 Euros, and errors out, then at least to know that it 
is brain-damaged and useless.

But if instead it silently changes the data to $100 (US dollars? 
Australian dollars? Zimbabwe dollars? the gods only know what a system 
that broken might do...) then it is not only broken but *dangerously* 
broken.



[...]
> Locale predates UCS-2, it was the early attempt to provide
> internationalization to C code so even programmers who didn't think
> about it could add the line setlocale(LC_ALL, "") and make their code
> work at least mostly right in more places. A single global was quick and
> simple, and since threads didn't exist, not an issue.

Threads were first used in 1967, five years before C even existed.

https://en.wikipedia.org/wiki/Thread_%28computing%29#History




-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Steven D'Aprano]

On Sat, 23 Jun 2018 08:12:52 -0400, Richard Damon wrote:

> On 6/23/18 7:46 AM, Steven D'Aprano wrote:
>> On Sat, 23 Jun 2018 06:26:22 -0400, Richard Damon wrote:
>>
>>> If you know the Locale, then you do know what the decimal separator
>>> is, as that is part of what a locale defines.
>> A locale defines a set of common cultural conventions. It doesn't
>> mandate the actual conventions in use in any specific document.
>>
>> If I'm in Australia, using the en-AU locale, nevertheless I can
>> generate a file using , as a decimal separator. Try and stop me :-)
>
> yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
> One, just as you can misuse the language and write cat when you mean a
> member of the Canine group, 

How about if I write "le chien" or "der Hund" or "собака"? Is that also a 
misuse of the locale because I choose to write in a foreign language, 
using foreign conventions for spelling, grammar and syntax?


> but then the misinterpretation is on the
> creator of the document, not on the program that was told how the
> document is to be read.

You're assuming that there will be a misinterpretation. That's an absurd 
assumption to make. There might be, of course, but the documentation for 
my document might be clear that comma is to be used for decimal 
separators. Or it might include numbers like

1.234.567,012345678

which is understandable to anyone who is aware of the possibility that 
comma may mean decimal separator and period the thousands separator.




-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 10:05 AM, Peter J. Holzer wrote:
> On 2018-06-23 08:41:38 -0400, Richard Damon wrote:
>> On 6/23/18 8:28 AM, Peter J. Holzer wrote:
>>> On 2018-06-23 08:12:52 -0400, Richard Damon wrote:
 On 6/23/18 7:46 AM, Steven D'Aprano wrote:
> If I'm in Australia, using the en-AU locale, nevertheless I can generate 
> a file using , as a decimal separator. Try and stop me :-)
 yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
 One, just as you can misuse the language and write cat when you mean a
 member of the Canine group, but then the misinterpretation is on the
 creator of the document, not on the program that was told how the
 document is to be read.
>>> How would he mis-use the en-AU locale to write 1 as "1,000"? I think
>>> to do that he would simply NOT use the locale.
>> Once you open the Locale can of worms, EVERYTHING has a locale, to say
>> you aren't using a locale is to say you are writing
>> something unintelligible, as you can thing of the locale as the set of
>> rules to interpret
> I don't think that's a useful way to look at it. "Locale" in
> (non-technical) English means "place" or "site". The idea behind the
> locale concept is that some conventions (e.g. how to write numbers or
> how to write strings) depend on the place where the program runs (or
> maybe where the user is sitting or grew up or maybe where a file was
> produced).
>
> For stuff which doesn't depend on the place (e.g. how a Python program
> should be parsed), the locale concept doesn't apply.
>
The Locale should NOT be the place the computer is running in (at least
not anymore), but where the data and the user are from (which can be
different). Do your really mean that when I travel to a place that uses
. as the thousands separator and , as the decimal separator (instead of
my normal environment when they are the other way around) all my
programs should immediately change how they read all my data files and
how I need to enter data? I hope not. I want my computer to use the
Locale of where "I" came from (not current am) to talk to me, and to be
able to set the Locale to interpret data to match the rules the person
who generated them used to generate them, so if they swap . and ,
compared to me, I can tell the program that. Your last parenthetical
comment in the first paragraph is my key point, the locale used to read
data should match the locale used to generate it, and that can easily be
different than the locale being used to interact with the user.

If a program doesn't care about the locale it is running in, like a
Python compiler, the either it needs to use routines that totally ignore
the locale or it needs to set the locale to one that matches the rules
it wants.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 11:44 AM, Steven D'Aprano wrote:
> On Sat, 23 Jun 2018 08:12:52 -0400, Richard Damon wrote:
>
>> On 6/23/18 7:46 AM, Steven D'Aprano wrote:
>>> On Sat, 23 Jun 2018 06:26:22 -0400, Richard Damon wrote:
>>>
 If you know the Locale, then you do know what the decimal separator
 is, as that is part of what a locale defines.
>>> A locale defines a set of common cultural conventions. It doesn't
>>> mandate the actual conventions in use in any specific document.
>>>
>>> If I'm in Australia, using the en-AU locale, nevertheless I can
>>> generate a file using , as a decimal separator. Try and stop me :-)
>> yes, you can MIS-use the en-AU locale and write 1,000 to mean the number
>> One, just as you can misuse the language and write cat when you mean a
>> member of the Canine group, 
> How about if I write "le chien" or "der Hund" or "собака"? Is that also a 
> misuse of the locale because I choose to write in a foreign language, 
> using foreign conventions for spelling, grammar and syntax?
>
>
>> but then the misinterpretation is on the
>> creator of the document, not on the program that was told how the
>> document is to be read.
> You're assuming that there will be a misinterpretation. That's an absurd 
> assumption to make. There might be, of course, but the documentation for 
> my document might be clear that comma is to be used for decimal 
> separators. Or it might include numbers like
>
> 1.234.567,012345678
>
> which is understandable to anyone who is aware of the possibility that 
> comma may mean decimal separator and period the thousands separator.
>
Then I shouldn't be using en-AU to decode the file. If I use a locale
based parser, I need to give it the right locale.

Now, if I have a parser that doesn't use the locale, but some other rule
base than I just need to provide it with the right rules, which is
basically just defining the right locale.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Bart]

On 23/06/2018 14:32, Stefan Ram wrote:

r...@zedat.fu-berlin.de (Stefan Ram) writes:

def f():
def g():
g.x += 1
return g.x
g.x = 0
return g


   Or, "for all g to share the same x":
   
   main.py


def f():
 def g():
 f.x += 1
 return f.x
 return g
f.x = 0


OK, problem solved: we just use attributes of function objects rather 
than locally static variables (I didn't even know that was possible). 
These apparently can be created, accessed and modified from anywhere in 
the program.


The only provisos are that functions with 'static' must be written as 
nested functions and the name of the function must be returned via the 
enclosing function in some setup code.


The initialising of the static is showed as happening in global space in 
your example, but may be possible to move that to the enclosing 
function. (For example, when the static data is a local table.)


However, here's a reminder of what the feature looks like implemented 
properly:


def g()
static x = 0
x += 1
return x

print (g())

No set up of g needed. 'static' can be added to any existing function 
without changing how its used. And it can be removed without having to 
dismantled all the extra machinery.


/And/ the access to x inside g() can be a fast local lookup not an 
attribute lookup (unless implemented on top of global variables).


--
bart
--
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[MRAB]

On 2018-06-23 05:16, Chris Angelico wrote:

On Sat, Jun 23, 2018 at 1:51 PM, Steven D'Aprano
 wrote:

On Wed, 20 Jun 2018 14:18:19 +1000, Chris Angelico wrote:


Ah. Yeah, that would be a plausible feature to add to Python. But in C,
a static variable is basically the same thing as a global variable,
except that its name is scoped to the function. There is only one of it.
What happens in Python? For instance:

def f():
def g():
static x = 0
x += 1
return x
return g

Does the static variable exist once for each instance of g()? If so,
it'll behave like a closure variable; if not, it'll behave like a
global. Either way, I'm pretty much certain that people will expect the
other.


Yes, but given the normal execution model of Python, only one solution is
valid. Since the function g is created fresh each time f is called, each
one gets a fresh static x.

If you want all the g's to share the same x, you would write:

def f():
static x = 0
def g():
x += 1
return x
return g


In this case, every invocation of f shares the same static x, and all the
g's refer to that same x, using the ordinary closure mechanism. In the
earlier case, each invocation of f creates a brand new g with its own x.

Simple and elegant.

This could at last get rid of that useful but ugly idiom:

def function(real, arguments, len=len, int=int, str=str):
...

if we allowed the "static" declaration to access the values from the
surrounding scope:

def function(real, arguments):
static len=len, int=int, str=str

But I think nicer than that would be a decorator:

@static(len=len, int=int, str=str)
def function(real, arguments):
...

which adds local variables len, int, str to the function, with the given
values, and transforms all the bytecode LOAD_NAME len to LOAD_FAST len
(or whatever).

(We might need a new bytecode to SET_STATIC.)

That would be a nice bytecode hack to prove the usefulness of the concept!



Okay, that makes sense. So in a way, static variables would be like
closure variables with an invisible outer function. These would be
roughly equivalent:


> def f():
>  static x = 0
>  x += 1
>  return x
>
You can already do something similar like this:

def f():
 f.x += 1
 return f.x
f.x = 0

[snip]
--
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Peter J. Holzer]

On 2018-06-23 12:11:34 -0400, Richard Damon wrote:
> On 6/23/18 10:05 AM, Peter J. Holzer wrote:
> > On 2018-06-23 08:41:38 -0400, Richard Damon wrote:
> >> Once you open the Locale can of worms, EVERYTHING has a locale, to say
> >> you aren't using a locale is to say you are writing
> >> something unintelligible, as you can thing of the locale as the set of
> >> rules to interpret
> > I don't think that's a useful way to look at it. "Locale" in
> > (non-technical) English means "place" or "site". The idea behind the
> > locale concept is that some conventions (e.g. how to write numbers or
> > how to write strings) depend on the place where the program runs (or
> > maybe where the user is sitting or grew up or maybe where a file was
> > produced).
> >
> > For stuff which doesn't depend on the place (e.g. how a Python program
> > should be parsed), the locale concept doesn't apply.
> >
> The Locale should NOT be the place the computer is running in (at least
> not anymore), but where the data and the user are from (which can be
> different).

Yes, it can be different, but for some *very* common cases (PCs,
smartphones most of the time) it isn't. More imporantly for the concept,
when the concept was developed (in the late 1980's) is was very common
(probably more common than 10 years earlier).

> Do your really mean that when I travel to a place that uses
> . as the thousands separator and , as the decimal separator (instead of
> my normal environment when they are the other way around) all my
> programs should immediately change how they read all my data files and
> how I need to enter data? I hope not.

Sometimes, yes. If you want to work with your colleagues at that place
they might thank you to use the local conventions.

> I want my computer to use the Locale of where "I" came from (not
> current am) to talk to me,

That's why I wrote "or grew up".

> and to be able to set the Locale to interpret data to match the rules
> the person who generated them used to generate them,

And that's why I wrote "where a file was produced".

So many words to repeat what I already wrote ...


> so if they swap . and , compared to me, I can tell the program that.
> Your last parenthetical comment in the first paragraph is my key
> point,

I think it is the weakest point. The locale is useful for interactive
use (input and output) and also for output intended for human users. For
parsing files it is woefully inadequate (also for generating files
intended to be parsed).

> the locale used to read data should match the locale used to generate
> it, and that can easily be different than the locale being used to
> interact with the user.

Which is basically why "locale" is a rather useless concept with files.
When I get a CSV file, I don't want to say "use locale en_US.cp437",
because the location "US" is almost completely irrelevant, the language
"English" is somewhat relevant but much too specific", and the list
separator isn't there at all. I want to tell it: Decode using CP437, a
decimal point, tabs as a list separator, CRLF as the record separator,
no quoting.

> If a program doesn't care about the locale it is running in, like a
> Python compiler, the either it needs to use routines that totally ignore
> the locale or it needs to set the locale to one that matches the rules
> it wants.

The former. Because locales are in general opaque, so you can never be
sure that a given locale will use the rules you want ("C" is the
exception, but not very useful).

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Peter J. Holzer]

On 2018-06-23 12:41:33 -0400, Richard Damon wrote:
> On 6/23/18 11:44 AM, Steven D'Aprano wrote:
> > You're assuming that there will be a misinterpretation. That's an absurd 
> > assumption to make. There might be, of course, but the documentation for 
> > my document might be clear that comma is to be used for decimal 
> > separators. Or it might include numbers like
> >
> > 1.234.567,012345678
> >
> > which is understandable to anyone who is aware of the possibility that 
> > comma may mean decimal separator and period the thousands separator.
> >
> Then I shouldn't be using en-AU to decode the file.

Quite right, You shouldn't.

> Now, if I have a parser that doesn't use the locale, but some other rule
> base than I just need to provide it with the right rules, which is
> basically just defining the right locale.

Nope. The right rules for almost any file format are much more than the
locale.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: syntax difference

[boB Stepp]

I've finally found time to examine this rather long, rambling thread.

On Wed, Jun 20, 2018 at 5:46 AM  wrote:
>
> Yeah, people keep bringing that up when they run out of arguments.
>
> So, every programmer must always use the most advanced, most esoteric 
> features possible at every opportunity? Coding should only be for the elite?
>
> There is a place for various levels of programming language. I'm saying that 
> Python which is always touted as a 'simple' language suitable for beginners, 
> is missing a surprising number of basics.

I still feel like a rank beginner, but on the Tutor list some
disagree.  Perhaps I feel this way because the more I try to study and
research programming/computer science topics, the more I realize how
little I know and how much remains to be learned, so I continue to
feel like a beginner.  But I am beginner enough that I can respond to
the above points.

A handful of years ago at the job I do, I started to get tired of
doing the same basic things over and over, so I investigated what
possibilities the software I used on the OS it used had in order to
automate these tasks.  Perl proved to be available, but Python wasn't
uniformly then, so I wrote some programs in Perl.  I got it figured
out and wrote some useful program in Perl, but the language never did
feel natural and easy to understand to me.

Later the OS and hardware got updated and I found Python was now fully
available in the 2.4 version.  I started writing new programs in
Python and found it quite easy to use and understand.  As I needed to,
I started rewriting earlier programs I had done in Perl in Python
instead, and was happier for it.  As an aside we just had another
round of software, OS and hardware upgrades.  Now I can use Python
2.7!

Because I read and study about new things as I take them up, I soon
learned that I had only so far scratched the surface of Python's
depths.  But despite knowing that Python had many more features to
explore, both in the core language and the standard library, this
never hindered me in writing my beginner-level programs.  I got things
done, and I got them done fairly easily, and never felt burdened by
all the "other stuff" that Python had to offer.

But I am continually grateful that this "other stuff" exists!  For
instance, recently I was working on a problem for home use that I was
doing in Python 3 (Not that 3 vs. 2 matters here.).  I was concerned
about loading potentially really large files into RAM and not having
enough memory for it.  Alan Gauld suggested I try a generator
approach.  I had not used these yet, though I was aware of this
feature's existence.  So I did some reading up on them, wrote some
code with my attempted implementation of them, submitted my efforts to
the Tutor list for critique, and while I am sure I did not do a
professional job of things, I was quite happy with the result as it
solved the problem I was concerned about.  So even dipping my big toe
into the "other stuff" proved an enjoyable and understandable
experience.  This has always been my experience with Python.  When the
time comes that I need something, I find it is there either in the
core language or the standard library, and it proves not burdensome to
learn the new feature(s).

This has been my beginner's journey with Python:  Easy to do useful
stuff, easy to read and easy to understand.  When I need something
_more_, I find it already exists and proves relatively easy to
understand *as long as* I am willing to put in a bit of study.  And I
don't think needing to put in a bit of study to learn how to use a new
feature is unreasonable for me or anyone else.

Anyway, so far Python has not lacked for anything I have needed so
far.  Of course, I realize that if I need to do something closer to
the machine level, Python is probably not going to be the preferred go
to tool, and even being a beginner I have enough sense to realize
this.  But then again, I would not be too surprised if Python or a
third party library did not already meet this future, hypothetical
need.

> That these are useful is demonstrated by the cumbersome add-ons that need to 
> be used to provide that functionality. Often in over the top ways and often 
> in a bewildering variety of ways in the case of records.

All I can say is I have yet to find much at all in Python cumbersome
or bewildering.

As to the original point of this thread concerning type-hints, I am
aware of them, once asked a bit on Tutor about them, but decided I am
not ready to go there yet.  But to (I hope.) help my learning of
Python syntax, I am forcing myself to *not* use linters, etc.  Once I
feel that I have Python in my head and in my fingers, then I will
start using such sensible tools and will probably reexamine
type-hints.

As an aside to Bart, if you strongly feel that Python is missing a
really useful feature, then why don't you do the usual thing, start a
very specific thread about just that feature (Not just a collection of
things you like 

Re: Static variables [was Re: syntax difference]

[Chris Angelico]

On Sat, Jun 23, 2018 at 10:41 PM, Bart  wrote:
> This is an example of a simple concept getting so out of hand that it will
> either never be implemented, or the resulting implementation becomes
> impractical to use.
>
> This is what we're trying to do:
>
> def nextx():
> static x = 0
> x += 1
> return x
>
> And this is the simplest equivalent code in current Python that will cater
> for 99% of uses:
>
> _nextx_x = 0
>
> def nextx():
> global _nextx_x
> _nextx_x += 1
> return _nextx_x
>
> No nested functions. No generating new instances of functions complete with
> a new set of statics each time you happen to refer to the name. (Which
> sounds to me as useful as creating a new instance of an import when you copy
> its name, complete with a separate set of its globals. Isn't this stuff what
> classes are for?)
>
> (At what point would that happen anyway; if you do this:
>
> g = nextx  # hypothetical version would static
>
> it will create a new instance of 'nextx'. But it won't create one here, just
> before () is applied:
>
> nextx() # ?
>
> Or how about here:
>
> listoffunctions = (nextx1, nextx2, nextx3)
>
> listoffunctions[i]() # ?
>
> )

Clearly you have completely misunderstood the entire concept of
Python's memory model, so I am not going to engage in debate with you
on this. This is my ONLY post explaining this to you.

NONE of your examples are taking copies of the function. They all are
making REFERENCES to the same function. That is all.

Creating new functions is the job of 'def' and 'lambda'. Not
assignment. So the concept of "multiple functions with the same name"
comes from executing the def statement more than once.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Bart]

On 23/06/2018 21:13, Chris Angelico wrote:

On Sat, Jun 23, 2018 at 10:41 PM, Bart  wrote:



(At what point would that happen anyway; if you do this:



NONE of your examples are taking copies of the function. They all are
making REFERENCES to the same function. That is all.


This is about your notion that invocations of the same function via 
different references, should maintain their own versions of the 
function's 'static' data.


Since these references are created via the return g statement here:

def f():
def g():

return g

(say to create function references i and j like this:

i = f()
j = f()
)

I'm assuming that something special must be happening. Otherwise, how 
does f() know which reference it's being called via?


What is different, what extra bit of information is provided when f() is 
invoked via i() or j()?


--
bart

--
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 11:27 AM, Steven D'Aprano wrote:
> On Sat, 23 Jun 2018 09:42:29 -0400, Richard Damon wrote:
>
>> On 6/23/18 9:05 AM, Marko Rauhamaa wrote:
>>> Ok. Here's a value for you:
>>>
>>> 100€
>>>
>>> I see '1', '0', '0', '€'. What do you see in your locale (LC_MONETARY)?
>> If I processed that on my system I would either get $100, or an error of
>> wrong currency symbol depending on the error checking.
> Then your system is so unbelievably broken that it should be nuked from 
> orbit, just to be sure.
>
> The data you were given was 100 Euros. If your system is incapable of 
> reading that as 100 Euros, and errors out, then at least to know that it 
> is brain-damaged and useless.
>
> But if instead it silently changes the data to $100 (US dollars? 
> Australian dollars? Zimbabwe dollars? the gods only know what a system 
> that broken might do...) then it is not only broken but *dangerously* 
> broken.
>
Locale based currency transformations are defined as a number to/from a
text string.

The number CAN'T say 100 Euros (can you give me what bit pattern you
would use for such a number).
The currency is encoded in the locale used for the conversion, so if it
is using en-US, the currency value would ALWAYS be US$ (which the
general locale format is just $). As such 100€ is an invalid input to a
system getting a Locale based input for a currency if the locale is not
one from a country that uses the euro. What the input sees is '1', '0',
'0',  some funny character (or maybe 2 of them). A poorly designed
input, or one being intentionally generous on input acceptance would
return 100, which would be implied US Dollars. A better error checking
routine would give an error. It is IMPOSSIBLE for it to return a number
that would be 100 euros. I suppose a very smart system might see that it
was in a different currency and try to convert it, but unless time
reference point to use for the currency, you are likely to get a wrong
answer, but in any case, the answer will NOT be 100 euros, but some
equivalent value in Dollars.

Now, if you want to define a perhaps more general currency input routine
that tries to detect a pan-locale currency input, and returned both a
value and a currency type, that could be more useful in some contexts.
But you then run into the interesting (and difficult) problem that if
you see the input of 123.456€ what is that value, is it a value around a
hundred euros specified to 3 decimal places, or is it a number just over
100 thousand euros.
>
> [...]
>> Locale predates UCS-2, it was the early attempt to provide
>> internationalization to C code so even programmers who didn't think
>> about it could add the line setlocale(LC_ALL, "") and make their code
>> work at least mostly right in more places. A single global was quick and
>> simple, and since threads didn't exist, not an issue.
> Threads were first used in 1967, five years before C even existed.
>
> https://en.wikipedia.org/wiki/Thread_%28computing%29#History
>
Threads did NOT exist (at least to the Standard) in C when locales were
added, and the C language did nothing to support threading at that time.
Looking back, it was perhaps a regrettable decision to implement locales
globally the way there were, but it is what it is.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Ben Finney]

Richard Damon  writes:

> On 6/23/18 11:27 AM, Steven D'Aprano wrote:
> >> On 6/23/18 9:05 AM, Marko Rauhamaa wrote:
> >>> Richard Damon wrote:
> >>> > Data presented to the user should normally use his locale
> >>> > (unless he has specified something different).
> >>>
> >>> Ok. Here's a value for you:
> >>>
> >>> 100€
> >>>
> > […]
> > The data you were given was 100 Euros. If your system is incapable
> > of reading that as 100 Euros, and errors out, then at least to know
> > that it is brain-damaged and useless.
> >
> > But if instead it silently changes the data to $100 (US dollars?
> > Australian dollars? Zimbabwe dollars? the gods only know what a
> > system that broken might do...) then it is not only broken but
> > *dangerously* broken.
> >
> […]
>
> The number CAN'T say 100 Euros (can you give me what bit pattern you
> would use for such a number).

That is (I believe) the point being made: The data is *not* a number. It
is a value that must encapsulate more than only the number 100, but also
and simultaneously the curency “Euro”.

> The currency is encoded in the locale used for the conversion, so if it
> is using en-US, the currency value would ALWAYS be US$ (which the
> general locale format is just $). As such 100€ is an invalid input to a
> system getting a Locale based input for a currency if the locale is not
> one from a country that uses the euro.

The value is 100 Euro, a quantity of a particular currency and not
something trivially converted to US$ (for many reasons, including the
obvious one that we don't know the exact exchange rate to use, and it
will be different at a different time).

You appear to be arguing that this value must either be arbitrarily
converted to the user's local currency, something we agree is impossible
to do given the data, or the value is simply invalid.

So the rule you assert – “Data presented to the user should normally use
his locale” – fails to usefuly handle the very normal case of data that
represents a quantity of some foreign currency. Any system following
your asserted rule will give either the wrong answer, or an error. We
had better hope the rule you assert is not in effect.

-- 
 \ “DRM doesn't inconvenience [lawbreakers] — indeed, over time it |
  `\ trains law-abiding users to become [lawbreakers] out of sheer |
_o__)frustration.” —Charles Stross, 2010-05-09 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Richard Damon]

On 6/23/18 5:31 PM, Ben Finney wrote:
> Richard Damon  writes:
>
>> On 6/23/18 11:27 AM, Steven D'Aprano wrote:
 On 6/23/18 9:05 AM, Marko Rauhamaa wrote:
> Richard Damon wrote:
>> Data presented to the user should normally use his locale
>> (unless he has specified something different).
> Ok. Here's a value for you:
>
> 100€
>
>>> […]
>>> The data you were given was 100 Euros. If your system is incapable
>>> of reading that as 100 Euros, and errors out, then at least to know
>>> that it is brain-damaged and useless.
>>>
>>> But if instead it silently changes the data to $100 (US dollars?
>>> Australian dollars? Zimbabwe dollars? the gods only know what a
>>> system that broken might do...) then it is not only broken but
>>> *dangerously* broken.
>>>
>> […]
>>
>> The number CAN'T say 100 Euros (can you give me what bit pattern you
>> would use for such a number).
> That is (I believe) the point being made: The data is *not* a number. It
> is a value that must encapsulate more than only the number 100, but also
> and simultaneously the curency “Euro”.
If you have more than just a number representing a value in the locale
currency, you can't ask the locale how to present/accept it.
>
>> The currency is encoded in the locale used for the conversion, so if it
>> is using en-US, the currency value would ALWAYS be US$ (which the
>> general locale format is just $). As such 100€ is an invalid input to a
>> system getting a Locale based input for a currency if the locale is not
>> one from a country that uses the euro.
> The value is 100 Euro, a quantity of a particular currency and not
> something trivially converted to US$ (for many reasons, including the
> obvious one that we don't know the exact exchange rate to use, and it
> will be different at a different time).
>
> You appear to be arguing that this value must either be arbitrarily
> converted to the user's local currency, something we agree is impossible
> to do given the data, or the value is simply invalid.
>
> So the rule you assert – “Data presented to the user should normally use
> his locale” – fails to usefuly handle the very normal case of data that
> represents a quantity of some foreign currency. Any system following
> your asserted rule will give either the wrong answer, or an error. We
> had better hope the rule you assert is not in effect.
>
If the user wants to talk in Euro using software that uses locales, then
he should specify a locale that uses Euros.

If you have a field to enter a foreign currency, then you can NOT make
that a LC_CURRENCY field, or you need to make that field use a different
locale than the local locale. This isn't the fault of locales, but in a
misuse of the system.

This original question came when it was asked what do I see with 100€ in
MY locale LC_CURRENCY, well MY locale doesn't have a LC_CURRENCY that is
euros, so it can't express that. It is a bit like asking how to draw a
circle with 4 straight lines or get to the moon in a boat. It is a
question with an improper premise.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Ben Bacarisse]

Bart  writes:

> On 23/06/2018 21:13, Chris Angelico wrote:
>> On Sat, Jun 23, 2018 at 10:41 PM, Bart  wrote:
>
>>> (At what point would that happen anyway; if you do this:
>
>> NONE of your examples are taking copies of the function. They all are
>> making REFERENCES to the same function. That is all.
>
> This is about your notion that invocations of the same function via
> different references, should maintain their own versions of the
> function's 'static' data.
>
> Since these references are created via the return g statement here:
>
> def f():
> def g():
> 
> return g
>
> (say to create function references i and j like this:
>
> i = f()
> j = f()
> )
>
> I'm assuming that something special must be happening. Otherwise, how
> does f() know which reference it's being called via?
>
> What is different, what extra bit of information is provided when f()
> is invoked via i() or j()?

f is not being invoked by either i() or j().  i = f() binds i to the
function returned by f.  That's a newly minted function.  In languages
like Python, executing def creates a function.  In your example, i and j
refer to different functions.  If the function temporarily named g has
"own" variables ("static" in C), then each such function should have its
own.  That was the point of the example much further up.

The effect can simulated like this:

def make_counter():
def c():
c.x += 1
return c.x
c.x = 0
return c

i = make_counter()
j = make_counter()

print(i(), i(), j(), i())


-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: syntax difference

[Bart]

On 23/06/2018 20:52, boB Stepp wrote:

I've finally found time to examine this rather long, rambling thread.



There is a place for various levels of programming language. I'm saying that 
Python which is always touted as a 'simple' language suitable for beginners, is 
missing a surprising number of basics.



I still feel like a rank beginner, but on the Tutor list some
disagree.


The first programming exercise I ever did involved asking for three 
numbers, then determining whether those numbers could form the sides of 
a triangle.


Then [40 years ago], the easy part was reading the three numbers. Now 
that would be the more challenging part.


This is one of the basics that is missing. Getting around it is not 
hard, but it's some messing about and it's a distraction. But 40 years 
ago it was just 'readln a,b,c'; it was just taken for granted.


(It make seem quaint in these days of GUIs, gestures, and voice 
recognition to be reading a line at a time, but you will need it still 
for text file i/o.)



Anyway, so far Python has not lacked for anything I have needed so
far.


I'd be surprised if Python lacked anything; there can't be anything that 
someone has thought of that is either built-in or bolted on, if not 
always that elegantly or that efficiently.


However, imagine having to use a language which didn't have assignments 
as you are used to, and that you would expect to exist. Then you might 
well remark that it's missing something that you regard as a basic, 
while the proponents of that language point out that it doesn't stop you 
writing programs; it just needs a different approach.


(I believe that you can write any program using just IF-GOTO statements 
and ASSIGNMENT statements, and no other flow control (given suitable 
data-types and means of I/O). But if you wanted to try out an 
interesting experiment along those lines (eg. transcribe any flowchart 
to code), a large number of languages, including Python, make it hard 
because 'goto' is missing.)



All I can say is I have yet to find much at all in Python cumbersome
or bewildering.


No? How many ways are there in Python, including third party add-ons, of 
working with record-like objects?



As an aside to Bart, if you strongly feel that Python is missing a
really useful feature, then why don't you do the usual thing, start a
very specific thread about just that feature (Not just a collection of
things you like in one of your languages.), and if you manage to
persuade the community of its usefulness, then write up a PEP about
it?  Just saying ... ~(:>))


I'm not a user. My interest is in design and implementation, especially 
of interpreters, and especially of efficient ones. A lot of things that 
Python could do with are made very difficult by the existing design of 
that language. Being so dynamic has a lot to answer for.


So I don't envy the job of the people who really have to move the 
language forward. That doesn't mean I can't argue with people who say 
that Python doesn't really need (say) Switch. (I guess the Blub paradox 
works both ways...)


--
bart
--
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Bart]

On 23/06/2018 23:25, Ben Bacarisse wrote:

Bart  writes:


On 23/06/2018 21:13, Chris Angelico wrote:

On Sat, Jun 23, 2018 at 10:41 PM, Bart  wrote:



(At what point would that happen anyway; if you do this:



NONE of your examples are taking copies of the function. They all are
making REFERENCES to the same function. That is all.


This is about your notion that invocations of the same function via
different references, should maintain their own versions of the
function's 'static' data.

Since these references are created via the return g statement here:

 def f():
 def g():
 
 return g

(say to create function references i and j like this:

 i = f()
 j = f()
)

I'm assuming that something special must be happening. Otherwise, how
does f() know which reference it's being called via?

What is different, what extra bit of information is provided when f()
is invoked via i() or j()?


f is not being invoked by either i() or j().  i = f() binds i to the
function returned by f.  That's a newly minted function.  In languages
like Python, executing def creates a function.


Do you mean that if the same 'def' block is re-executed, it will create 
a different instance of the function? (Same byte-code, but a different 
set of everything else the function uses.)


Wow. (Just think of all the times you write a function containing a neat 
bunch of local functions, every time it's called it has to create a new 
function instances for each of those functions, even if they are not used.)


Anyway just for regular statics, the following appears to work. Not 
ideal, but simpler than some alternatives:


def f():
if not hasattr(f,'x'): f.x=0
f.x += 1
return f.x

--
bart.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Chris Angelico]

On Sun, Jun 24, 2018 at 9:37 AM, Bart  wrote:
> On 23/06/2018 23:25, Ben Bacarisse wrote:
>>
>> Bart  writes:
>>
>>> On 23/06/2018 21:13, Chris Angelico wrote:

 On Sat, Jun 23, 2018 at 10:41 PM, Bart  wrote:
>>>
>>>
> (At what point would that happen anyway; if you do this:
>>>
>>>
 NONE of your examples are taking copies of the function. They all are
 making REFERENCES to the same function. That is all.
>>>
>>>
>>> This is about your notion that invocations of the same function via
>>> different references, should maintain their own versions of the
>>> function's 'static' data.
>>>
>>> Since these references are created via the return g statement here:
>>>
>>>  def f():
>>>  def g():
>>>  
>>>  return g
>>>
>>> (say to create function references i and j like this:
>>>
>>>  i = f()
>>>  j = f()
>>> )
>>>
>>> I'm assuming that something special must be happening. Otherwise, how
>>> does f() know which reference it's being called via?
>>>
>>> What is different, what extra bit of information is provided when f()
>>> is invoked via i() or j()?
>>
>>
>> f is not being invoked by either i() or j().  i = f() binds i to the
>> function returned by f.  That's a newly minted function.  In languages
>> like Python, executing def creates a function.
>
>
> Do you mean that if the same 'def' block is re-executed, it will create a
> different instance of the function? (Same byte-code, but a different set of
> everything else the function uses.)
>
> Wow. (Just think of all the times you write a function containing a neat
> bunch of local functions, every time it's called it has to create a new
> function instances for each of those functions, even if they are not used.)

... do you even understand what closures are?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: syntax difference

[boB Stepp]

On Sat, Jun 23, 2018 at 5:35 PM Bart  wrote:
>
> On 23/06/2018 20:52, boB Stepp wrote:

> The first programming exercise I ever did involved asking for three
> numbers, then determining whether those numbers could form the sides of
> a triangle.
>
> Then [40 years ago], the easy part was reading the three numbers. Now
> that would be the more challenging part.
>
> This is one of the basics that is missing. Getting around it is not
> hard, but it's some messing about and it's a distraction. But 40 years
> ago it was just 'readln a,b,c'; it was just taken for granted.

I don't think there are any *basics* missing.  One of the differences
between then and now is that trying to live in a simpler
ASCII/extended ASCII world is no longer possible.  The Internet and
open source movements among other things have brought in more (most?)
of the world as participants in the programming endeavor.  The many
peoples of this world and their many, varied cultures is quite rich
and complex, and they have real needs to express their thoughts as
naturally as they can.  I would contend that the basics are there, but
the complexity of the data and ideas that need to be dealt with these
days is what makes the programming journey more challenging but also
more interesting than in the days of yore.  When I first went to
college in the seventies, it was all punch cards, main frames, a few
minicomputers, time sharing IBM selectrics, etc.  I would *not* want
to go back to that!  I find today's challenges and opportunities much
more fascinating!!

> > Anyway, so far Python has not lacked for anything I have needed so
> > far.
>
> I'd be surprised if Python lacked anything; there can't be anything that
> someone has thought of that is either built-in or bolted on, if not
> always that elegantly or that efficiently.

I am not qualified to speak to issues of Python's language design, but
as a user of Python I have yet to be disappointed.  But I imagine all
languages, including yours, have accumulated cruft that in retrospect
the creators would like to remove.  But the difference, as far as I
can tell, between your languages and Python is that Python has a huge
number of active users with who knows how many LOC in production
environments, but your languages do not.  So you have the benefit, as
their creator and essentially sole user, to keep polishing and
refining your ideas.  The Python devs don't have that luxury.  But
they are obviously, even to me, constantly not only adding new
functionality, but trying to minimize the cruft as best they can
without compromising people's production code.  I imagine that the
transition from version 2 to 3 was not undertaken halfheartedly, but
only after much thought and discussion since it did break backwards
compatibility.  But now having used both Python 2 (Which I still have
to code in at work.) and 3, even I with my limited knowledge and
capabilities appreciate the positive benefits of the changes.

> However, imagine having to use a language which didn't have assignments
> as you are used to, and that you would expect to exist. Then you might
> well remark that it's missing something that you regard as a basic,
> while the proponents of that language point out that it doesn't stop you
> writing programs; it just needs a different approach.

But the beauty of things as they are today, is that if someone has
thought of it, it is probably accessible and available to use.  Not
only that, but one can cruise through the many different programming
languages and see what others have imagined and thought to be useful
and grow one's knowledge and insight much more easily than ever
before.  And if I have a need that Python or any other language I use
does not address, I can surely find one online.

> (I believe that you can write any program using just IF-GOTO statements
> and ASSIGNMENT statements, and no other flow control (given suitable
> data-types and means of I/O). But if you wanted to try out an
> interesting experiment along those lines (eg. transcribe any flowchart
> to code), a large number of languages, including Python, make it hard
> because 'goto' is missing.)

I have done the "GOTO" thing ages ago.  I would not want to go back to
that bad place.  There are too many better alternatives now for flow
control.  With today's hardware I don't think any benefit from any
theoretical optimizations would be of any benefit in practice.  As a
person who does not have much time for studying and writing programs,
I have a hard enough time understanding what I wrote after a break
from programming.  I would not want to put any more stumbling blocks
in my code for later maintainability, not to imagine having to do so
for other people's code.

> > All I can say is I have yet to find much at all in Python cumbersome
> > or bewildering.
>
> No? How many ways are there in Python, including third party add-ons, of
> working with record-like objects?

If I am understanding you correctly by your reference to "recor

Re: Static variables [was Re: syntax difference]

[Ben Bacarisse]

Bart  writes:

> On 23/06/2018 23:25, Ben Bacarisse wrote:
>> Bart  writes:
>>
>>> On 23/06/2018 21:13, Chris Angelico wrote:
 On Sat, Jun 23, 2018 at 10:41 PM, Bart  wrote:
>>>
> (At what point would that happen anyway; if you do this:
>>>
 NONE of your examples are taking copies of the function. They all are
 making REFERENCES to the same function. That is all.
>>>
>>> This is about your notion that invocations of the same function via
>>> different references, should maintain their own versions of the
>>> function's 'static' data.
>>>
>>> Since these references are created via the return g statement here:
>>>
>>>  def f():
>>>  def g():
>>>  
>>>  return g
>>>
>>> (say to create function references i and j like this:
>>>
>>>  i = f()
>>>  j = f()
>>> )
>>>
>>> I'm assuming that something special must be happening. Otherwise, how
>>> does f() know which reference it's being called via?
>>>
>>> What is different, what extra bit of information is provided when f()
>>> is invoked via i() or j()?
>>
>> f is not being invoked by either i() or j().  i = f() binds i to the
>> function returned by f.  That's a newly minted function.  In languages
>> like Python, executing def creates a function.
>
> Do you mean that if the same 'def' block is re-executed, it will
> create a different instance of the function? (Same byte-code, but a
> different set of everything else the function uses.)

Logically, yes.

> Wow. (Just think of all the times you write a function containing a
> neat bunch of local functions, every time it's called it has to create
> a new function instances for each of those functions, even if they are
> not used.)

I am surprised that this surprises you, and equally surprised that you
seem to think it's going to be in some way grossly inefficient.

-- 
Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Gregory Ewing]

Bart wrote:
Wow. (Just think of all the times you write a function containing a neat 
bunch of local functions, every time it's called it has to create a new 
function instances for each of those functions, even if they are not used.)


Fortunately, function objects are small and cheap, essentially
just a couple of object references. The overhead of creating one
is probably about the same as creating an empty list. If your
function is complicated enough to benefit from local functions,
the cost is going to be swamped by the rest of the work being
done.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Steven D'Aprano]

On Sat, 23 Jun 2018 21:44:00 +0100, Bart wrote:

> Since these references are created via the return g statement here:
> 
>  def f():
>  def g():
>  
>  return g
> 
> (say to create function references i and j like this:
> 
>  i = f()
>  j = f()
> )
> 
> I'm assuming that something special must be happening. Otherwise, how
> does f() know which reference it's being called via?

You assume wrong.


> What is different, what extra bit of information is provided when f() is
> invoked via i() or j()?

For somebody who has been using Python for a few years now, and is 
constantly telling us how we're doing it wrong, you sure are ignorant 
about the language and how it works.

For somebody who has been programming for so long, you seem awfully 
unaware of how dynamic languages with first-class functions and closures 
work.

No extra bit of information is provided. i refers to one function, j, 
refers to another function, and there is no overlap.

Each invocation of f() results in a brand new function being created, 
dynamically, and assigned to the appropriate name.



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: syntax difference

[Steven D'Aprano]

On Sat, 23 Jun 2018 23:26:43 +0100, Bart wrote:

> Then [40 years ago], the easy part was reading the three numbers. Now
> that would be the more challenging part.

# Get three numbers, separated by spaces, with no error-recovery.
# If you try to read bad data, the process will fail.
n1, n2, n3 = [float(s) for s in input("Enter three numbers: ").split()]

No more verbose or difficult (certainly not "challenging") than:

var:
  # need to declare these otherwise how will readln
  # know if it is reading floats or ints or strings?
  n1, n2, n3: float  
print("Enter three numbers: ")
n1, n2, n3 = readln()


If the use of a list comprehension is too advanced for a beginner in day 
one, how about this:

n1 = float(input("Enter a number: "))
n2 = float(input("Enter a number: "))
n3 = float(input("Enter a number: "))



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: syntax difference

[Gregory Ewing]

Bart wrote:
But 40 years 
ago it was just 'readln a,b,c'; it was just taken for granted.


The problem with something like that is that it's really only
useful for throwaway code. For any serious application, you need
to deal with the possibility of malformed input, producing helpful
diagnostics, etc. And often the input isn't going to be in such a
uniform format that you know exactly what to expect next.

So it's debable whether it's a good idea to put something with such
limited applicability into the core language or standard library.

Handling input well is fundamentally much more complicated than
producing output. I don't think this is as "basic" as you make out.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Steven D'Aprano]

On Sun, 24 Jun 2018 00:37:36 +0100, Bart wrote:

> Do you mean that if the same 'def' block is re-executed, it will create
> a different instance of the function? (Same byte-code, but a different
> set of everything else the function uses.)

That's not as slow as you think it is. Everything that can be is pre-
prepared and assembling them is pretty fast:

py> import dis
py> dis.dis(lambda x: lambda n: x*n)
  1   0 LOAD_CLOSURE 0 (x)
  3 BUILD_TUPLE  1
  6 LOAD_CONST   1 ( at
0xb78b6430, file "",
line 1>)
  9 LOAD_CONST   2 ('..')
 12 MAKE_CLOSURE 0
 15 RETURN_VALUE


How else can you get functions where the definition is not known until 
runtime, unless you assemble them at runtime?


> Wow. (Just think of all the times you write a function containing a neat
> bunch of local functions, every time it's called it has to create a new
> function instances for each of those functions, even if they are not
> used.)


That's why Pascal-style static nested functions are hardly ever used in 
Python. 99% of nested functions are closures.




-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Steven D'Aprano]

On Sat, 23 Jun 2018 17:05:17 -0400, Richard Damon wrote:

> On 6/23/18 11:27 AM, Steven D'Aprano wrote:
>> On Sat, 23 Jun 2018 09:42:29 -0400, Richard Damon wrote:
>>
>>> On 6/23/18 9:05 AM, Marko Rauhamaa wrote:
 Ok. Here's a value for you:

 100€

[...]
> Locale based currency transformations are defined as a number to/from a
> text string.
> 
> The number CAN'T say 100 Euros (can you give me what bit pattern you
> would use for such a number).

You're joking, right? You can't possibly be so ignorant as to actually 
believe that. You have, right in front of you, a news post or email 
containing the text string "100€", and yet you are writing apparently in 
full seriousness that it is impossible to get that text string in a file.

Okay, you want a bit-pattern. In hex:

'0x313030e282ac'

I'll leave the question of how I generated that as an exercise. (Hint: it 
was a one-liner, involving two method calls and a function call, all 
builtins in Python.)


> The currency is encoded in the locale used for the conversion, so if it
> is using en-US, the currency value would ALWAYS be US$ (which the
> general locale format is just $).

I cannot imagine for a second why you think any of this is even a tiny 
bit relevant to the question of how one should read a data file 
containing currency in Euro.

You seem to have heard about the locale and decide it is the One True 
Hammer than all nails must be hammered with.


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Steven D'Aprano]

On Sat, 23 Jun 2018 17:52:55 -0400, Richard Damon wrote:

> If you have more than just a number representing a value in the locale
> currency, you can't ask the locale how to present/accept it.

You're the only one saying that it has to be handled by the locale.


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Chris Angelico]

On Sun, Jun 24, 2018 at 12:44 PM, Steven D'Aprano
 wrote:
> You're joking, right? You can't possibly be so ignorant as to actually
> believe that. You have, right in front of you, a news post or email
> containing the text string "100€", and yet you are writing apparently in
> full seriousness that it is impossible to get that text string in a file.
>
> Okay, you want a bit-pattern. In hex:
>
> '0x313030e282ac'
>
> I'll leave the question of how I generated that as an exercise. (Hint: it
> was a one-liner, involving two method calls and a function call, all
> builtins in Python.)

Hmm. Actually, I'm a bit confused.

>>> hex("100€".encode())
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'bytes' object cannot be interpreted as an integer

Nope, that's not it. Needs something to turn the bytes into an integer
first. But I can't find a way to do that. Best I can find is:

>>> "100€".encode().hex()
'313030e282ac'

No "0x" prefix, no function call. So, I'm stuck. How did you create your one?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Static variables [was Re: syntax difference]

[Steven D'Aprano]

On Sat, 23 Jun 2018 18:29:51 +0100, MRAB wrote:

> You can already do something similar like this:
> 
> def f():
>   f.x += 1
>   return f.x
> f.x = 0
> 
> [snip]

You can, but only as an illustration, not as a serious implementation.

The whole point of static local variables is that they are:

(1) Local variables, and hence FAST;

(2) fully encapsulated inside the function.

Using f.x fails on both accounts:

- it requires a global name lookup for "f" (hence slow);

- followed by an attribute lookup for "x" (hence even slower);

- f.x is not encapsulated inside the function. It requires initialisation 
outside the function. The attribute f.x is easily visible to the caller.

(Technically, so probably would static variables, but only by inspecting 
the function's internals. People know they're on thin-ice if they mess 
with them. And if we really needed to, we could protect the static 
storage inside the function from writing, if not reading, as we do with 
closures. f.x looks like an ordinary attribute.)

I have used the f.x trick before, for things where speed was not 
critical, but I've never really loved the idea, and it certainly cannot 
be used as a replacement of the def func(arg, len=len, int=int) 
optimization trick.


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Steven D'Aprano]

On Sun, 24 Jun 2018 12:53:49 +1000, Chris Angelico wrote:

[...]
>> Okay, you want a bit-pattern. In hex:
>>
>> '0x313030e282ac'
[...]

> Hmm. Actually, I'm a bit confused.
> 
 hex("100€".encode())
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: 'bytes' object cannot be interpreted as an integer
> 
> Nope, that's not it. Needs something to turn the bytes into an integer
> first. But I can't find a way to do that. Best I can find is:
> 
 "100€".encode().hex()
> '313030e282ac'

Dammit, that was what I was looking for, but I only looked on *strings*, 
not bytes.

 
> No "0x" prefix, no function call. So, I'm stuck. How did you create your
> one?

py> hex(int.from_bytes("100€".encode("utf-8"), 'big'))
'0x313030e282ac'



-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: translating foreign data

[Chris Angelico]

On Sun, Jun 24, 2018 at 1:23 PM, Steven D'Aprano
 wrote:
> On Sun, 24 Jun 2018 12:53:49 +1000, Chris Angelico wrote:
>
> [...]
>>> Okay, you want a bit-pattern. In hex:
>>>
>>> '0x313030e282ac'
> [...]
>
>> Hmm. Actually, I'm a bit confused.
>>
> hex("100€".encode())
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> TypeError: 'bytes' object cannot be interpreted as an integer
>>
>> Nope, that's not it. Needs something to turn the bytes into an integer
>> first. But I can't find a way to do that. Best I can find is:
>>
> "100€".encode().hex()
>> '313030e282ac'
>
> Dammit, that was what I was looking for, but I only looked on *strings*,
> not bytes.
>
>
>> No "0x" prefix, no function call. So, I'm stuck. How did you create your
>> one?
>
> py> hex(int.from_bytes("100€".encode("utf-8"), 'big'))
> '0x313030e282ac'

Ahhh thanks, that's the part I couldn't find (and didn't remember).

Anyhow, encoding to UTF-8 and then to bytes is pretty easy.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Quick survey: locals in comprehensions (Python 3 only)

[Steven D'Aprano]

I'd like to run a quick survey. There is no right or wrong answer, since 
this is about your EXPECTATIONS, not what Python actually does.

Given this function:


def test():
a = 1
b = 2
result = [value for key, value in locals().items()]
return result




what would you expect the result of calling test() to be? Is that the 
result you think is most useful? In your opinion, is this a useful 
feature, a misfeature, a bug, or "whatever"?

I'm only looking for answers for Python 3. (The results in Python 2 are 
genuinely weird :-)


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Chris Angelico]

On Sun, Jun 24, 2018 at 3:03 PM, Steven D'Aprano
 wrote:
> I'd like to run a quick survey. There is no right or wrong answer, since
> this is about your EXPECTATIONS, not what Python actually does.
>
> Given this function:
>
>
> def test():
> a = 1
> b = 2
> result = [value for key, value in locals().items()]
> return result
>
>
>
>
> what would you expect the result of calling test() to be? Is that the
> result you think is most useful? In your opinion, is this a useful
> feature, a misfeature, a bug, or "whatever"?
>
> I'm only looking for answers for Python 3. (The results in Python 2 are
> genuinely weird :-)

Personally, I think it should give you [1, 2], the two values from the
function's locals. But genexps introduce some problems. Compare:

def test():
a = 1
b = 2
result = list(value for key, value in locals().items())
return result

def test_helper():
a = 1
b = 2
gen = (value for key, value in locals().items())
return gen
def test():
return list(test_helper())

Guido has stated that he wants the list comp to be equivalent to
calling list() immediately on the genexp. And since a genexp has to be
the same thing whether you call list() on it straight away or not,
that means that all these need to behave the same way. Which, in turn,
means that the genexp needs some form of closure semantics. Thus it is
executed in an implicit nested function, and thus the list comp also
needs to be, for consistency.

So! With that in mind, I would consider bizarre behaviour of locals()
inside comprehensions to be a wart. It's not a normal thing to do, and
if it behaves weirdly, so be it. There are other things that I would
prefer to see tidied up before that.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Steven D'Aprano]

On Sun, 24 Jun 2018 15:18:49 +1000, Chris Angelico wrote:

> Personally, I think it should give you [1, 2], the two values from the
> function's locals.

Thank you, that's the sort of answer I'm looking for.

(I'm not saying I didn't read your long and involved analysis, only that 
I'm not looking for long involved analyses at the moment :-)


-- 
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Jim Lee]

On 06/23/2018 10:03 PM, Steven D'Aprano wrote:

I'd like to run a quick survey. There is no right or wrong answer, since
this is about your EXPECTATIONS, not what Python actually does.

Given this function:


def test():
 a = 1
 b = 2
 result = [value for key, value in locals().items()]
 return result




what would you expect the result of calling test() to be? Is that the
result you think is most useful? In your opinion, is this a useful
feature, a misfeature, a bug, or "whatever"?

I'm only looking for answers for Python 3. (The results in Python 2 are
genuinely weird :-)



I would *expect* [1, 2, None], though I haven't actually tried running it.

-Jim


--
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Chris Angelico]

On Sun, Jun 24, 2018 at 3:44 PM, Jim Lee  wrote:
>
>
> On 06/23/2018 10:03 PM, Steven D'Aprano wrote:
>>
>> I'd like to run a quick survey. There is no right or wrong answer, since
>> this is about your EXPECTATIONS, not what Python actually does.
>>
>> Given this function:
>>
>>
>> def test():
>>  a = 1
>>  b = 2
>>  result = [value for key, value in locals().items()]
>>  return result
>>
>>
>>
>>
>> what would you expect the result of calling test() to be? Is that the
>> result you think is most useful? In your opinion, is this a useful
>> feature, a misfeature, a bug, or "whatever"?
>>
>> I'm only looking for answers for Python 3. (The results in Python 2 are
>> genuinely weird :-)
>>
>>
> I would *expect* [1, 2, None], though I haven't actually tried running it.
>

Interesting. Where do you get the None from? Suppose it had been "key
for..." instead of "value", what would the third key have been? ["a",
"b", ...]

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Jim Lee]

On 06/23/2018 11:02 PM, Chris Angelico wrote:

On Sun, Jun 24, 2018 at 3:44 PM, Jim Lee  wrote:


On 06/23/2018 10:03 PM, Steven D'Aprano wrote:

I'd like to run a quick survey. There is no right or wrong answer, since
this is about your EXPECTATIONS, not what Python actually does.

Given this function:


def test():
  a = 1
  b = 2
  result = [value for key, value in locals().items()]
  return result




what would you expect the result of calling test() to be? Is that the
result you think is most useful? In your opinion, is this a useful
feature, a misfeature, a bug, or "whatever"?

I'm only looking for answers for Python 3. (The results in Python 2 are
genuinely weird :-)



I would *expect* [1, 2, None], though I haven't actually tried running it.


Interesting. Where do you get the None from? Suppose it had been "key
for..." instead of "value", what would the third key have been? ["a",
"b", ...]

ChrisA
There are three locals:  a, b, and result.  Since result cannot be 
assigned a value until the list comp has been evaluated, I would expect 
the comp to return a value of "None" for result.  An argument could also 
be made for [1, 2, []], but one thing I would *not* expect is [1, 2] or 
[2, 1]...


-Jim


--
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Chris Angelico]

On Sun, Jun 24, 2018 at 4:08 PM, Jim Lee  wrote:
> There are three locals:  a, b, and result.  Since result cannot be assigned
> a value until the list comp has been evaluated, I would expect the comp to
> return a value of "None" for result.  An argument could also be made for [1,
> 2, []], but one thing I would *not* expect is [1, 2] or [2, 1]...

Ahh, I see what you mean. Thing is, there's a definite difference
between "this is None" and "this doesn't have a value". The latter
situation is indicated by simply not having the local.

def f():
print("; ".join("%s=%r" % x for x in locals().items()))
a = 1
print("; ".join("%s=%r" % x for x in locals().items()))
b = 2
print("; ".join("%s=%r" % x for x in locals().items()))

The results may surprise you, or may not.

This part has nothing to do with the behaviour of locals inside a
comprehension, though. The important part is that, like me, you would
like comprehensions to represent a block of code inside the current
function, not an implicit nested function.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick survey: locals in comprehensions (Python 3 only)

[Jim Lee]

On 06/23/2018 11:16 PM, Chris Angelico wrote:

On Sun, Jun 24, 2018 at 4:08 PM, Jim Lee  wrote:

There are three locals:  a, b, and result.  Since result cannot be assigned
a value until the list comp has been evaluated, I would expect the comp to
return a value of "None" for result.  An argument could also be made for [1,
2, []], but one thing I would *not* expect is [1, 2] or [2, 1]...

Ahh, I see what you mean. Thing is, there's a definite difference
between "this is None" and "this doesn't have a value". The latter
situation is indicated by simply not having the local.

def f():
 print("; ".join("%s=%r" % x for x in locals().items()))
 a = 1
 print("; ".join("%s=%r" % x for x in locals().items()))
 b = 2
 print("; ".join("%s=%r" % x for x in locals().items()))

The results may surprise you, or may not.

This part has nothing to do with the behaviour of locals inside a
comprehension, though. The important part is that, like me, you would
like comprehensions to represent a block of code inside the current
function, not an implicit nested function.

ChrisA
That's why I said an argument could be made for [1, 2, []].  I realize 
that NoneType is not the same as no type, but, having not studied the 
internals of any particular Python implementation,  I don't know how it 
builds or initializes its local symbol table. But, I expected "result" 
to exist before the list comprehension was evaluated simply due to 
left->right parsing.


-Jim

--
https://mail.python.org/mailman/listinfo/python-list