Re: Encapsulation in Python
Rick Johnson writes: > On Friday, March 11, 2016 at 3:28:40 AM UTC-6, Steven D'Aprano wrote: > ... > Are you sure about that? Heck, i posted code quite a few > years back that "seg faulted like a mutha". Do you want to > retract your statement, or will i need to search the > archives, and then stuff the link down your big fat mouth? ^^^ What happened that you use language like this? Obviously, you disagree with Steven - but this should not make you so angry to use agressive language. -- https://mail.python.org/mailman/listinfo/python-list
Re: Perl to Python again
On Fri, 11 Mar 2016 19:15:48 -0500, Fillmore wrote: > On 3/11/2016 7:12 PM, Martin A. Brown wrote: >> >> Aside from your csv question today, many of your questions could be >> answered by reading through the manual documenting the standard >> datatypes (note, I am assuming you are using Python 3). >> >> > are you accusing me of being lazy? > > if that's your accusation, then guilty as charged, but I not sure if you were being accused of being lazy as such but actually being given the suggestion that there are other places that you can find these answers that are probably better for a number of reasons 1) Speed, you don't have to wait for someone to reply although i hope you are continuing your research whilst waiting 2) Accuracy. I have not seen it here but there are some people who would consider it fun to provide an incorrect or dangerous solution to someone they though was asking too basic a question 3) Collateral learning, whilst looking for the solution it is highly likely that you will unearth other information that answers questions you have yet to raise. -- Badges? We don't need no stinking badges. -- https://mail.python.org/mailman/listinfo/python-list
Re: issue with CVS module
On Fri, 11 Mar 2016 16:26:02 -0500, Fillmore wrote: > On 3/11/2016 4:15 PM, Mark Lawrence wrote: >> >> https://docs.python.org/3/library/csv.html#csv.Dialect.doublequote >> >> > thanks, but my TSV is not using any particular dialect as far as I > understand... > > Thank you, anyway Every variation of a language/format is a dialect even if (at present) there is only one. CSV/TSV has many - (TSV is simply a dialect of CSV or vice versa) you also have the variable of quoting (as you have found) and line separation (which may or may not be an issue) Whenever you are processing an external file it is essential to know the exact format it is using to avoid errors -- So, you better watch out! You better not cry! You better not pout! I'm telling you why, Santa Claus is coming, to town. He knows when you've been sleeping, He know when you're awake. He knows if you've been bad or good, He has ties with the CIA. So... -- https://mail.python.org/mailman/listinfo/python-list
Re: Hello
On Fri, 11 Mar 2016 17:53:45 -0500, Larry Martell wrote: > On Fri, Mar 11, 2016 at 4:49 AM, Steven D'Aprano > wrote: > >> On Fri, 11 Mar 2016 02:28 pm, [email protected] wrote: >> >> > I am having trouble installing the Python software. >> >> Make sure your computer is turned on. I can't tell you how many times >> I've tried to install Python, and I type commands and click icons and >> nothing happens. It's really frustrating when you finally realise that >> the reason nothing is working is because the computer is turned off! (I >> thought I just had the screen brightness turned way down.) > > > Many years ago (c. 1985) I was at a job interview and the interviewer > asked me what the first thing I would do when I am presented with a new > problem that I had to code up. I gave all sorts of answers like 'do a > top down analysis of the problem,' and 'get the specs and requirements,' > and 'write a flow chart.' Each time the interviewer said no even before > that. Finally I said, what, what would do first? He said "Turn the > computer on." I decided then and there I did not want to work for that > guy. Then not only was he an arse but he is also wrong. for a complex problem it is best to have a plan on how to implement it before reaching for the keyboard. Your answer of get the spec & requirements certainly does not need a computer (although one could help for note taking if your hand writing is as bad as mine). I guess he was so caught up with his clever catch answer that he did not bother to analyse yours & realise he had missed something "I decided then and there I did not want to work for that guy." Shame I would have enjoyed reading the results on www.thedailywtf.com :-) -- He who laughs last -- missed the punch line. -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 to create a record using a FK
Aaron Christensen writes:
> I am running the following versions of software:
>
> Python 3.5
> psycopg2==2.6.1
> Postgres 9.4.5
>
> I have 2 tables. Table User has UserId (serial PK), LastName, FirstName,
> Gender, DateOfBirth, and DateEnrolled. Table UserProfile has UserProfileId
> (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity. There is
> a one-to-many relationship.
>
> The following PostgreSQL works and ultimately creates a record in
> UserProfile with an associated UserId (FK).
>
> \set last_name '''Sara'''
> \set first_name '''Jackson'''
> \set gender '''F'''
> \set dob '''1941-1-12'''
> \set fave_number '''3'''
> \set activity '''volleyball'''
>
>
> WITH ins_user AS (
> INSERT INTO User
> (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
> VALUES (:last_name, :first_name, :gender, :dob, now())
> RETURNING UserId)
> INSERT INTO UserProfile
> (UserId, DateEntered, FaveNumber, Activity)
> VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
>
> How can I build a psycopg2 cur.execute query that will accomplish the above
> PostgreSQL? I've read documentation but can't seem to get a handle on how
> I should structure this command.
>
> My starting point is:
>
> cur.execute( \
> """INSERT INTO User \
> (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
> VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
> (last_name, first_name, gender, date_of_birth, now(), ??...??)
You can add "returning UserId" to this SQL command to get back
the id of the created user in your Python program. You can
then use this "UserId" to create the row in your dependent table.
I use it like this in one of my programs:
cursor.execute("insert into service(...) "
"values (...) returning id",
(...)
)
id = cursor.fetchone()[0]
cursor.execute(
"insert into product(..., f_service_id) "
"values (..., %s) returning id",
(..., id)
)
id = cursor.fetchone()[0]
Likely, there is also a way to bind the "UserId" inside SQL (maybe
via "SET") and use it in a second "INSERT" in the same call
to "cur.execute". Check the Postgres documentation for this.
> Also, I have a second question. Is it possible to extract that value
> derived from "RETURNING UserId" so that it can be used in a later query?
Sure -- see above.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On Fri, 11 Mar 2016 22:24:45 +, BartC wrote: > On 11/03/2016 21:59, Mark Lawrence wrote: >> On 11/03/2016 18:57, BartC wrote: > >> def test(): >> s="" >> for i in range(1000): >> s+="*" >> print (len(s)) >> >> test() > >> The minor snag that you might like to correct with your microbenchmark, >> which any experienced Python programmer knows, is that you *NEVER, >> EVER* >> create strings like this. > > Why not? Chris said his version runs much faster (even allowing for > different machines), and might have a special optimisation for it. > > And I think it can be optimised if, for example, there are no other > references to the string that s refers to. > > So what's wrong with trying to fix it rather that using a workaround? because the "workarround" is not a workarround it is the correct way to do it the code above is a workarround for somone who does not know the pythonic method to do this S= "*"*1000 -- A little kid went up to Santa and asked him, "Santa, you know when I'm bad right?" And Santa says, "Yes, I do." The little kid then asks, "And you know when I'm sleeping?" To which Santa replies, "Every minute." So the little kid then says, "Well, if you know when I'm bad and when I'm good, then how come you don't know what I want for Christmas?" -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 to create a record using a FK
Aaron Christensen wrote:
> Hello,
>
> I am running the following versions of software:
>
> Python 3.5
> psycopg2==2.6.1
> Postgres 9.4.5
>
> I have 2 tables. Table User has UserId (serial PK), LastName, FirstName,
> Gender, DateOfBirth, and DateEnrolled. Table UserProfile has
> UserProfileId
> (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity. There
> is a one-to-many relationship.
>
> The following PostgreSQL works and ultimately creates a record in
> UserProfile with an associated UserId (FK).
>
> \set last_name '''Sara'''
> \set first_name '''Jackson'''
> \set gender '''F'''
> \set dob '''1941-1-12'''
> \set fave_number '''3'''
> \set activity '''volleyball'''
>
>
> WITH ins_user AS (
> INSERT INTO User
> (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
> VALUES (:last_name, :first_name, :gender, :dob, now())
> RETURNING UserId)
> INSERT INTO UserProfile
> (UserId, DateEntered, FaveNumber, Activity)
> VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
>
> How can I build a psycopg2 cur.execute query that will accomplish the
> above
> PostgreSQL? I've read documentation but can't seem to get a handle on how
> I should structure this command.
I have not tried it, but wouldn't the straight-forward
cur.execute("""
WITH ins_user AS (
INSERT INTO User
(LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
VALUES (:last_name, :first_name, :gender, :dob, now())
RETURNING UserId)
INSERT INTO UserProfile
(UserId, DateEntered, FaveNumber, Activity)
VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
""",
dict(
first_name="Sara",
last_name="Jackson",
gender="F",
dob=datetime.date(1941, 1, 12),
fave_number=3,
activity="volleyball"
))
work?
> My starting point is:
>
> cur.execute( \
> """INSERT INTO User \
> (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
> VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
> (last_name, first_name, gender, date_of_birth, now(), ??...??)
>
>
> Also, I have a second question. Is it possible to extract that value
> derived from "RETURNING UserId" so that it can be used in a later query?
>
> Thank you for your time!
> Aaron
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On 12/03/2016 10:06, alister wrote: On Fri, 11 Mar 2016 22:24:45 +, BartC wrote: On 11/03/2016 21:59, Mark Lawrence wrote: On 11/03/2016 18:57, BartC wrote: def test(): s="" for i in range(1000): s+="*" print (len(s)) test() The minor snag that you might like to correct with your microbenchmark, which any experienced Python programmer knows, is that you *NEVER, EVER* create strings like this. Why not? Chris said his version runs much faster (even allowing for different machines), and might have a special optimisation for it. And I think it can be optimised if, for example, there are no other references to the string that s refers to. So what's wrong with trying to fix it rather that using a workaround? because the "workarround" is not a workarround it is the correct way to do it the code above is a workarround for somone who does not know the pythonic method to do this S= "*"*1000 This is a benchmark that measures the cost of adding to a string a character at a time. In practice the final length of the string is not known, and the characters added at each stage are not known. Although the strings won't usually be that big; the benchmark exaggerates here to highlight a possible issue with += on strings. And it worked, as there can be big difference with or without the += optimisation in place. It also showed a possible bug or problem with PyPy. You can't demonstrate all this by just writing s="*"*1000. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On 12/03/2016 01:15, Michael Torrie wrote: On 03/11/2016 03:24 PM, BartC wrote: On 11/03/2016 21:59, Mark Lawrence wrote: On 11/03/2016 18:57, BartC wrote: def test(): s="" for i in range(1000): s+="*" print (len(s)) test() The minor snag that you might like to correct with your microbenchmark, which any experienced Python programmer knows, is that you *NEVER, EVER* create strings like this. Why not? Chris said his version runs much faster (even allowing for different machines), and might have a special optimisation for it. And I think it can be optimised if, for example, there are no other references to the string that s refers to. So what's wrong with trying to fix it rather that using a workaround? The act of "fixing" it, as you say, would change the semantics of the language in a fundamental and major way. Strings by definition are immutable in Python. Yet INPLACE_ADD is a valid byte-code even when operating on strings. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On Sat, Mar 12, 2016 at 9:34 PM, BartC wrote: >> The act of "fixing" it, as you say, would change the semantics of the >> language in a fundamental and major way. Strings by definition are >> immutable in Python. > > > Yet INPLACE_ADD is a valid byte-code even when operating on strings. That's because INPLACE_ADD is a valid byte-code regardless of what it's working on. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On 12/03/2016 10:31, BartC wrote: On 12/03/2016 10:06, alister wrote: On Fri, 11 Mar 2016 22:24:45 +, BartC wrote: On 11/03/2016 21:59, Mark Lawrence wrote: On 11/03/2016 18:57, BartC wrote: def test(): s="" for i in range(1000): s+="*" print (len(s)) test() The minor snag that you might like to correct with your microbenchmark, which any experienced Python programmer knows, is that you *NEVER, EVER* create strings like this. Why not? Chris said his version runs much faster (even allowing for different machines), and might have a special optimisation for it. And I think it can be optimised if, for example, there are no other references to the string that s refers to. So what's wrong with trying to fix it rather that using a workaround? because the "workarround" is not a workarround it is the correct way to do it the code above is a workarround for somone who does not know the pythonic method to do this S= "*"*1000 This is a benchmark that measures the cost of adding to a string a character at a time. In practice the final length of the string is not known, and the characters added at each stage are not known. Although the strings won't usually be that big; the benchmark exaggerates here to highlight a possible issue with += on strings. And it worked, as there can be big difference with or without the += optimisation in place. It also showed a possible bug or problem with PyPy. You can't demonstrate all this by just writing s="*"*1000. There is no possible issue with += on strings, there is a known issue. What to do about has all ready been discussed by myself, Dennis Lee Bieber, Michael Torrie and Steven D'Aprano, but you've chosen to ignore what we've written. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 06:48, Marko Rauhamaa wrote:
Chris Angelico :
Definitely agree with this. Having a way to declare that a name is
"truly constant" would be extremely handy;
I don't think it would be all that handy. I'm afraid all this type
hinting will turn Python into a poor man's Java.
It's not type hinting. Otherwise you can say that using 'def' or 'class'
is a form of type hinting. Think of 'const' as operating like the latter
and declaring something a little different.
Although the volatility of the names so defined is still the problem.
Maybe, but I honestly don't miss 'switch' all that often - and when I
do, it's usually because I want a range.
I don't consider the switch statement an optimization technique but
rather, a readability technique.
Note that Scheme has a "switch statement" (a "case form") despite being
a highly dynamic language.
Yes, you can have simpler forms of switch, that have the same overall
structure, but do a series of sequential tests rather than using any
form of table indexed by the value being tested.
The advantage here is that that test-value need only be evaluated once.
It stays on the stack until some match is found, or the statement comes
to an end.
It won't have as dramatic an impact on performance, but enhances
readability as you say.
Compile-time macros are actually a conceptual compromise that violate
full-fledged dynamism: once the compiler has expanded the macro, its
definition can't change.
What's big deal with dynamism anyway? I could never understand Python's
obsession with it.
For me, 'dynamic' means that a variable has a dynamic type; that's all.
But you know at compile-time (or when looking at source code) whether a
name is a variable, or a function, class, module, named constant and so on.
If you need a variable-function, then you just have a variable contain
the name of a function (ie a reference to it). You can bolt on dynamism
/when you need it/.
OK, mini-rant over...
>> You're not mistaken. There are no "character constants" in Python.
>> (Note that the definition would be Unicode codepoints, rather than
>> ASCII values.) I don't often miss them, though.
Yes, a complete non-issue.
Really? The issue as I see it is this:
Writing: a=65 generates this byte-code for the right-hand-side:
LOAD_CONST 1 (65) An integer
But writing instead: a=ord('A') generates this:
LOAD_GLOBAL 0 (ord)
LOAD_CONST 1 ('A') A string
CALL_FUNCTION 1
You might be right: doing an unnecessary global name lookup and
executing a function call are unlikely to have any impact on performance...
The problem here is that 'ord' is dynamic, so this operation cannot
simply be done at compile-time. Even when you try and optimise by
assuming that ord is immutable, you don't really want to be doing any
runtime checks. It might be faster than calling LOAD_GLOBAL and
CALL_FUNCTION, but not quite as fast as just doing LOAD_CONST.
--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 11:08, BartC wrote:
On 12/03/2016 06:48, Marko Rauhamaa wrote:
Chris Angelico :
Definitely agree with this. Having a way to declare that a name is
"truly constant" would be extremely handy;
I don't think it would be all that handy. I'm afraid all this type
hinting will turn Python into a poor man's Java.
It's not type hinting. Otherwise you can say that using 'def' or 'class'
is a form of type hinting. Think of 'const' as operating like the latter
and declaring something a little different.
Although the volatility of the names so defined is still the problem.
Maybe, but I honestly don't miss 'switch' all that often - and when I
do, it's usually because I want a range.
I don't consider the switch statement an optimization technique but
rather, a readability technique.
Note that Scheme has a "switch statement" (a "case form") despite being
a highly dynamic language.
Yes, you can have simpler forms of switch, that have the same overall
structure, but do a series of sequential tests rather than using any
form of table indexed by the value being tested.
The advantage here is that that test-value need only be evaluated once.
It stays on the stack until some match is found, or the statement comes
to an end.
It won't have as dramatic an impact on performance, but enhances
readability as you say.
Compile-time macros are actually a conceptual compromise that violate
full-fledged dynamism: once the compiler has expanded the macro, its
definition can't change.
What's big deal with dynamism anyway? I could never understand Python's
obsession with it.
For me, 'dynamic' means that a variable has a dynamic type; that's all.
But you know at compile-time (or when looking at source code) whether a
name is a variable, or a function, class, module, named constant and so on.
If you need a variable-function, then you just have a variable contain
the name of a function (ie a reference to it). You can bolt on dynamism
/when you need it/.
OK, mini-rant over...
>> You're not mistaken. There are no "character constants" in Python.
>> (Note that the definition would be Unicode codepoints, rather than
>> ASCII values.) I don't often miss them, though.
Yes, a complete non-issue.
Really? The issue as I see it is this:
Writing: a=65 generates this byte-code for the right-hand-side:
LOAD_CONST 1 (65) An integer
But writing instead: a=ord('A') generates this:
LOAD_GLOBAL 0 (ord)
LOAD_CONST 1 ('A') A string
CALL_FUNCTION 1
You might be right: doing an unnecessary global name lookup and
executing a function call are unlikely to have any impact on performance...
Function calls are hugely expensive in Python.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
BartC :
> What's big deal with dynamism anyway? I could never understand
> Python's obsession with it.
>
> For me, 'dynamic' means that a variable has a dynamic type; that's
> all. But you know at compile-time (or when looking at source code)
> whether a name is a variable, or a function, class, module, named
> constant and so on.
*I* am obsessed with dynamism. It means I don't have to declare object
data members but can write ad hoc:
def some_method(self):
self.update_state()
self.state_specific_info = "Kilroy was here"
The "state_specific_info" attribute didn't exist before I wished it into
existence. No bureaucracy, just add it where it belongs.
I also have a high level of method/attribute transparency. It doesn't
matter if I declare:
def this_or_that(self):
if self.that:
self.that()
else:
self.this()
[...]
self.that = True
or:
self.this_or_that = self.that
Somewhat related, every method is an automatic delegate. Defining
callbacks is a breeze:
def clickety_click(self, x, y):
[...]
[...]
window.register_mouse_click_callback(self.clickety_click)
> On 12/03/2016 06:48, Marko Rauhamaa wrote:
>> Chris Angelico :
>>> You're not mistaken. There are no "character constants" in Python.
>>> (Note that the definition would be Unicode codepoints, rather than
>>> ASCII values.) I don't often miss them, though.
>
>> Yes, a complete non-issue.
>
> Really? The issue as I see it is this:
>
> Writing: a=65 generates this byte-code for the right-hand-side:
>
> LOAD_CONST 1 (65) An integer
>
> But writing instead: a=ord('A') generates this:
>
> LOAD_GLOBAL 0 (ord)
> LOAD_CONST 1 ('A') A string
> CALL_FUNCTION 1
>
> You might be right: doing an unnecessary global name lookup and
> executing a function call are unlikely to have any impact on
> performance...
Simply put: I don't use "ord()" almost at all. I couldn't find any
example in any of my code.
> The problem here is that 'ord' is dynamic, so this operation cannot
> simply be done at compile-time. Even when you try and optimise by
> assuming that ord is immutable, you don't really want to be doing any
> runtime checks. It might be faster than calling LOAD_GLOBAL and
> CALL_FUNCTION, but not quite as fast as just doing LOAD_CONST.
That optimization wouldn't have any effect on any of my code.
More generally, every method call in Python is such an elaborate
exercise that dabbling with character constants is going to be a drop in
the ocean.
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 02:20, Chris Angelico wrote: On Sat, Mar 12, 2016 at 12:16 PM, BartC wrote: 'Switch' testing benchmark. The little program show below reads a text file (I used the entire CPython C sources, 6MB), and counts the number of characters of each category in upper, lower, digit and other. (Note there are other ways to approach this task, but a proper 'lexer' usually does more than count. 'Switch' then becomes invaluable.) Are you assuming that the files are entirely ASCII? (They're not.) Or are you simply declaring that all non-ASCII characters count as "other"? Once again, you cannot ignore Unicode and pretend that everything's ASCII, or eight-bit characters, or something. Asking if a character is upper/lower/digit/other is best done with the unicodedata module. If you're looking at fast processing of language source code (in a thread partly about efficiency), then you cannot ignore the fact that the vast majority of characters being processed are going to have ASCII codes. Language syntax could anyway stipulate that certain tokens can only consist of characters within the ASCII range. So I'm not ignoring Unicode, but being realistic. (My benchmark was anyway just demonstrating a possible use for 'switch' that more or less matched your own example!) -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, Mar 12, 2016 at 10:08 PM, BartC wrote:
>>> You're not mistaken. There are no "character constants" in Python.
>>> (Note that the definition would be Unicode codepoints, rather than
>>> ASCII values.) I don't often miss them, though.
>
>> Yes, a complete non-issue.
>
>
> Really? The issue as I see it is this:
>
> Writing: a=65 generates this byte-code for the right-hand-side:
>
> LOAD_CONST 1 (65) An integer
>
> But writing instead: a=ord('A') generates this:
>
> LOAD_GLOBAL 0 (ord)
> LOAD_CONST 1 ('A') A string
> CALL_FUNCTION 1
I think the "non-issue" here is the difference between ASCII and
Unicode. Either way, there's no way to say "the integer with the
codepoint of this character" as a literal. But that's actually not
even all that necessary, because subscripting a text string yields
one-character strings - you almost never need the ordinals.
Subscripting a byte string in Py3 yields integers, so you might need
ordinals for ASCII byte values. But you can get them the same way:
>>> dis.dis(lambda: b"a"[0])
1 0 LOAD_CONST 3 (97)
3 RETURN_VALUE
>>> dis.dis(lambda: u"a"[0])
1 0 LOAD_CONST 3 ('a')
3 RETURN_VALUE
Whichever one you need, you can get as a compile-time constant.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
BartC :
> If you're looking at fast processing of language source code (in a
> thread partly about efficiency), then you cannot ignore the fact that
> the vast majority of characters being processed are going to have
> ASCII codes.
I don't know why you would optimize for inputting program source code.
Text in general has left ASCII behind a long time ago. Just go to
Wikipedia and click on any of the other languages.
Why, look at the *English* page on Hillary Clinton:
Hillary Diane Rodham Clinton /ˈhɪləri daɪˈæn ˈrɒdəm ˈklɪntən/ (born
October 26, 1947) is an American politician.
https://en.wikipedia.org/wiki/Hillary_Clinton>
You couldn't get past the first sentence in ASCII.
> Language syntax could anyway stipulate that certain tokens can only
> consist of characters within the ASCII range.
Many programming languages do stipulate that. Nowadays, the main reason
for the limitation is that all keyboards can produce ASCII and no
keyboard can produce all of Unicode.
Actually, when I was in college, not all keyboards could produce ASCII.
That's why the Pascal programming language offers digraphs:
(* here is a comment *)
for:
{ here is a comment }
and:
someArray(.7,3.)
for:
someArray[7,3]
(The weird American symbols {}[]\|#$^~ were abandoned and replaced with
something more relevant on European keyboards. Even the Brits would have
£ instead of #.)
In fact, the current C standard supports trigraphs for the same reason:
??= #
??/ \
??' ^
??( [
??) ]
??! |
??< {
??> }
??- ~
[...]
To safely place two consecutive question marks within a string
literal, the programmer can use string concatenation "...?""?..."
https://en.wikipedia.org/wiki/Digraphs_and_trigraphs#C>
So be careful out there...
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: argparse
On 12.03.2016 00:18, Fillmore wrote: Playing with ArgumentParser. I can't find a way to override the -h and --help options so that it provides my custom help message. I remember everything being a lot easier using argh instead of argparse. https://pypi.python.org/pypi/argh#examples The doc string of a function basically is the help string which is true for arguments as well. I hope that helps even though you asked for argparse explicitly. :-) Best, Sven -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, 12 Mar 2016 11:10 pm, Chris Angelico wrote:
> On Sat, Mar 12, 2016 at 10:08 PM, BartC wrote:
You're not mistaken. There are no "character constants" in Python.
(Note that the definition would be Unicode codepoints, rather than
ASCII values.) I don't often miss them, though.
>>
>>> Yes, a complete non-issue.
>>
>>
>> Really? The issue as I see it is this:
>>
>> Writing: a=65 generates this byte-code for the right-hand-side:
>>
>> LOAD_CONST 1 (65) An integer
>>
>> But writing instead: a=ord('A') generates this:
>>
>> LOAD_GLOBAL 0 (ord)
>> LOAD_CONST 1 ('A') A string
>> CALL_FUNCTION 1
>
> I think the "non-issue" here is the difference between ASCII and
> Unicode. Either way, there's no way to say "the integer with the
> codepoint of this character" as a literal. But that's actually not
> even all that necessary, because subscripting a text string yields
> one-character strings - you almost never need the ordinals.
>
> Subscripting a byte string in Py3 yields integers, so you might need
> ordinals for ASCII byte values. But you can get them the same way:
>
dis.dis(lambda: b"a"[0])
> 1 0 LOAD_CONST 3 (97)
> 3 RETURN_VALUE
dis.dis(lambda: u"a"[0])
> 1 0 LOAD_CONST 3 ('a')
> 3 RETURN_VALUE
>
> Whichever one you need, you can get as a compile-time constant.
Chris, what you're looking at is the result of the CPython keyhole optimizer
doing constant folding. That is **NOT** a language promise. Other
implementations of Python may lack the keyhole optimizer. Future versions
may remove it, or allow the user to disable it.
Python the language has no feature that *guarantees* that you can write 'a'
and get 97 as a compile-time constant.
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, Mar 12, 2016 at 10:50 PM, BartC wrote: > On 12/03/2016 02:20, Chris Angelico wrote: >> >> On Sat, Mar 12, 2016 at 12:16 PM, BartC wrote: > > >>> 'Switch' testing benchmark. The little program show below reads a text >>> file >>> (I used the entire CPython C sources, 6MB), and counts the number of >>> characters of each category in upper, lower, digit and other. >>> >>> (Note there are other ways to approach this task, but a proper 'lexer' >>> usually does more than count. 'Switch' then becomes invaluable.) >> >> >> Are you assuming that the files are entirely ASCII? (They're not.) Or >> are you simply declaring that all non-ASCII characters count as >> "other"? > > >> Once again, you cannot ignore Unicode and pretend that everything's >> ASCII, or eight-bit characters, or something. Asking if a character is >> upper/lower/digit/other is best done with the unicodedata module. > > > If you're looking at fast processing of language source code (in a thread > partly about efficiency), then you cannot ignore the fact that the vast > majority of characters being processed are going to have ASCII codes. > > Language syntax could anyway stipulate that certain tokens can only consist > of characters within the ASCII range. > > So I'm not ignoring Unicode, but being realistic. > > (My benchmark was anyway just demonstrating a possible use for 'switch' that > more or less matched your own example!) Generally languages these days are built using ASCII tokens, because they can be dependably typed on all keyboards. But there's no requirement for that, and I understand there's a Chinese Python that has all the language keywords translated. And identifiers can - and most definitely SHOULD - be defined in terms of Unicode characters and their types. So ultimately, the lexer needs to be Unicode-aware. But in terms of efficiency, yes, you can't ignore that most files will be all-ASCII. And since 3.3, Python has had an optimization for such strings. So the performance question isn't ignored - but it's an invisible optimization within a clearly-defined semantic, namely that Python source code is a sequence of Unicode characters. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, Mar 12, 2016 at 11:28 PM, Steven D'Aprano wrote:
> On Sat, 12 Mar 2016 11:10 pm, Chris Angelico wrote:
>
>> On Sat, Mar 12, 2016 at 10:08 PM, BartC wrote:
> You're not mistaken. There are no "character constants" in Python.
> (Note that the definition would be Unicode codepoints, rather than
> ASCII values.) I don't often miss them, though.
>>>
Yes, a complete non-issue.
>>>
>>>
>>> Really? The issue as I see it is this:
>>>
>>> Writing: a=65 generates this byte-code for the right-hand-side:
>>>
>>> LOAD_CONST 1 (65) An integer
>>>
>>> But writing instead: a=ord('A') generates this:
>>>
>>> LOAD_GLOBAL 0 (ord)
>>> LOAD_CONST 1 ('A') A string
>>> CALL_FUNCTION 1
>>
>> I think the "non-issue" here is the difference between ASCII and
>> Unicode. Either way, there's no way to say "the integer with the
>> codepoint of this character" as a literal. But that's actually not
>> even all that necessary, because subscripting a text string yields
>> one-character strings - you almost never need the ordinals.
>>
>> Subscripting a byte string in Py3 yields integers, so you might need
>> ordinals for ASCII byte values. But you can get them the same way:
>>
> dis.dis(lambda: b"a"[0])
>> 1 0 LOAD_CONST 3 (97)
>> 3 RETURN_VALUE
> dis.dis(lambda: u"a"[0])
>> 1 0 LOAD_CONST 3 ('a')
>> 3 RETURN_VALUE
>>
>> Whichever one you need, you can get as a compile-time constant.
>
> Chris, what you're looking at is the result of the CPython keyhole optimizer
> doing constant folding. That is **NOT** a language promise. Other
> implementations of Python may lack the keyhole optimizer. Future versions
> may remove it, or allow the user to disable it.
>
> Python the language has no feature that *guarantees* that you can write 'a'
> and get 97 as a compile-time constant.
Very true. However, the peephole optimizer MUST use only what can be
compile-time optimized - unlike a function call. So in terms of
optimizations, it's proof that it's safe.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Perl to Python again
On 03/12/2016 04:40 AM, alister wrote: On Fri, 11 Mar 2016 19:15:48 -0500, Fillmore wrote: I not sure if you were being accused of being lazy as such but actually being given the suggestion that there are other places that you can find these answers that are probably better for a number of reasons 1) Speed, you don't have to wait for someone to reply although i hope you are continuing your research whilst waiting 2) Accuracy. I have not seen it here but there are some people who would consider it fun to provide an incorrect or dangerous solution to someone they though was asking too basic a question 3) Collateral learning, whilst looking for the solution it is highly likely that you will unearth other information that answers questions you have yet to raise. Alister, you are right, of course. The reality is that I discovered this trove of a newsgroup and I am rather shamelessly taking advantage of it. Rest assured that I cross check and learn what is associated with each and every answer I get. So nothing is wasted. Hopefully your answers are also useful to others who may find them at a later stage through foofle groups. Also very important, I am very grateful for the support I am getting from you and from others. -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 12:13, Marko Rauhamaa wrote: BartC : If you're looking at fast processing of language source code (in a thread partly about efficiency), then you cannot ignore the fact that the vast majority of characters being processed are going to have ASCII codes. I don't know why you would optimize for inputting program source code. Text in general has left ASCII behind a long time ago. Just go to Wikipedia and click on any of the other languages. Why, look at the *English* page on Hillary Clinton: Hillary Diane Rodham Clinton /ˈhɪləri daɪˈæn ˈrɒdəm ˈklɪntən/ (born October 26, 1947) is an American politician. https://en.wikipedia.org/wiki/Hillary_Clinton> You couldn't get past the first sentence in ASCII. I saved that page locally as a .htm file in UTF-8 encoding. I ran a modified version of my benchmark, and it appeared that 99.7% of the bytes had ASCII codes. The other 0.3% presumably were multi-byte sequences, so that the actual proportion of Unicode characters would be even less. I then saved the Arabic version of the page, which visually, when rendered, consists of 99% Arabic script. But the .htm file was still 80% ASCII! So what were you saying about ASCII being practically obsolete ... ? -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, Mar 13, 2016 at 12:18 AM, BartC wrote: > On 12/03/2016 12:13, Marko Rauhamaa wrote: >> >> BartC : >> >>> If you're looking at fast processing of language source code (in a >>> thread partly about efficiency), then you cannot ignore the fact that >>> the vast majority of characters being processed are going to have >>> ASCII codes. >> >> >> I don't know why you would optimize for inputting program source code. >> Text in general has left ASCII behind a long time ago. Just go to >> Wikipedia and click on any of the other languages. >> >> Why, look at the *English* page on Hillary Clinton: >> >> Hillary Diane Rodham Clinton /ˈhɪləri daɪˈæn ˈrɒdəm ˈklɪntən/ (born >> October 26, 1947) is an American politician. >> https://en.wikipedia.org/wiki/Hillary_Clinton> >> >> You couldn't get past the first sentence in ASCII. > > > I saved that page locally as a .htm file in UTF-8 encoding. I ran a modified > version of my benchmark, and it appeared that 99.7% of the bytes had ASCII > codes. The other 0.3% presumably were multi-byte sequences, so that the > actual proportion of Unicode characters would be even less. > > I then saved the Arabic version of the page, which visually, when rendered, > consists of 99% Arabic script. But the .htm file was still 80% ASCII! > > So what were you saying about ASCII being practically obsolete ... ? Now take the same file and save it as plain text. See how much smaller it is. If you then take that text and embed it in a 10GB file consisting of nothing but byte value 246, it will be plainly obvious that ASCII is almost completely obsolete, and that we should optimize our code for byte 246. Or maybe, all you've proven is that *the framing around the text* is entirely ASCII, which makes sense, since HTML is trying to be compatible with a wide range of messy encodings (many of them eight-bit ASCII-compatible ones). The text itself may also consist primarily of ASCII characters, but that's a separate point. In the Arabic version, that is far less likely to be true (there'll still be a good number of ASCII characters in it, as U+0020 SPACE is heavily used in Arabic text, but a far smaller percentage). But neither of those says that ASCII is "practically obsolete", any more than you could say that the numbers from 1 to 10 become obsolete once a child learns to count further than that. The ASCII characters are an important part of the Unicode set; you can't ignore the rest of Unicode, but you certainly can't ignore ASCII, and there'll be very few pieces of human-language text which include no ASCII characters whatsoever. That's why UTF-8 is so successful; even Chinese text is often more compact in UTF-8 than in UTF-16 (despite many characters fitting into a single UTF-16 code unit, but requiring three bytes in UTF-8), when framed in HTML. However, once again, we have a sharp distinction: semantically, you support all Unicode characters equally, but then you optimize for the common ones. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation in Python
On Sat, Mar 12, 2016 at 8:42 PM, dieter wrote: > Rick Johnson writes: >> On Friday, March 11, 2016 at 3:28:40 AM UTC-6, Steven D'Aprano wrote: >> ... >> Are you sure about that? Heck, i posted code quite a few >> years back that "seg faulted like a mutha". Do you want to >> retract your statement, or will i need to search the >> archives, and then stuff the link down your big fat mouth? > ^^^ > > What happened that you use language like this? > > Obviously, you disagree with Steven - but this should not make you so angry > to use agressive language. You're responding to someone who calls himself "Ranting Rick". Don't bother expecting courtesy or civility. He's a resident troll, tolerated because he does make good points now and then, but generally ignored because of his irksome style. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 11:51, Marko Rauhamaa wrote: BartC : What's big deal with dynamism anyway? I could never understand Python's obsession with it. For me, 'dynamic' means that a variable has a dynamic type; that's all. But you know at compile-time (or when looking at source code) whether a name is a variable, or a function, class, module, named constant and so on. *I* am obsessed with dynamism. It means I don't have to declare object data members but can write ad hoc: def some_method(self): self.update_state() self.state_specific_info = "Kilroy was here" The "state_specific_info" attribute didn't exist before I wished it into existence. No bureaucracy, just add it where it belongs. I was talking about 'top-level' names more than attributes (names that follow a dot). Ad-hoc attributes I don't have as much of a problem with, as they can be handy. But predefined ones also have their points. (For one thing, I know how to implement those efficiently.) However, when you have a function call like this: M.F(), where M is an imported module, then it is very unlikely that the functions in M are going to be created, modified, deleted or replaced while the program runs. [I mean, after the usual process of executing each 'def' statement.] Why then should it have to suffer the same overheads as looking up arbitrary attributes? And on every single call? I also have a high level of method/attribute transparency. It doesn't matter if I declare: def this_or_that(self): if self.that: self.that() else: self.this() [...] self.that = True or: self.this_or_that = self.that This example, I don't understand. Do you mean that when your write X.Y, that Y can be an attribute of X one minute, and a method the next? (In which case I wouldn't want to have to maintain your code!) Somewhat related, every method is an automatic delegate. Defining callbacks is a breeze: def clickety_click(self, x, y): [...] [...] window.register_mouse_click_callback(self.clickety_click) I don't follow this either. What's the advantage of dynamism here? That optimization wouldn't have any effect on any of my code. More generally, every method call in Python is such an elaborate exercise that dabbling with character constants is going to be a drop in the ocean. When you dabble with lots of little things, then they can add up. To the point where an insignificant optimisation can become significant. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
BartC : > On 12/03/2016 12:13, Marko Rauhamaa wrote: >> BartC : >> >>> If you're looking at fast processing of language source code (in a >>> thread partly about efficiency), then you cannot ignore the fact >>> that the vast majority of characters being processed are going to >>> have ASCII codes. >> >> I don't know why you would optimize for inputting program source >> code. Text in general has left ASCII behind a long time ago. Just go >> to Wikipedia and click on any of the other languages. >> >> Why, look at the *English* page on Hillary Clinton: >> >> Hillary Diane Rodham Clinton /ˈhɪləri daɪˈæn ˈrɒdəm ˈklɪntən/ >> (born October 26, 1947) is an American politician. > https://en.wikipedia.org/wiki/Hillary_Clinton> >> >> You couldn't get past the first sentence in ASCII. > > I saved that page locally as a .htm file in UTF-8 encoding. I ran a > modified version of my benchmark, and it appeared that 99.7% of the > bytes had ASCII codes. The other 0.3% presumably were multi-byte > sequences, so that the actual proportion of Unicode characters would > be even less. > > I then saved the Arabic version of the page, which visually, when > rendered, consists of 99% Arabic script. But the .htm file was still > 80% ASCII! > > So what were you saying about ASCII being practically obsolete ... ? Yes, HTML markup is all ASCII. However, as you say, the text content is often anything but. What I'm saying is that if you are designing a new programming language and associated ecosystem, you are well advised to take Unicode into account from the start. Take advantage of the hindsight; Python, Linux, C, Java and Windows were not so lucky. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
BartC : > Ad-hoc attributes I don't have as much of a problem with, as they can > be handy. But predefined ones also have their points. (For one thing, > I know how to implement those efficiently.) I wonder how large a proportion of all references are top-level. My hunch is that it is well below 10% in a normal Python program. > However, when you have a function call like this: M.F(), where M is an > imported module, then it is very unlikely that the functions in M are > going to be created, modified, deleted or replaced while the program > runs. [I mean, after the usual process of executing each 'def' > statement.] There M is top-level, M.F is a level below. Even adjusting for the meaning of "top-level," most accesses in a typical Python program are attribute references due to object orientation. > Why then should it have to suffer the same overheads as looking up > arbitrary attributes? And on every single call? It doesn't much matter because the "top level" probably plays an insignificant role in the performance. >> I also have a high level of method/attribute transparency. It doesn't >> matter if I declare: >> >> def this_or_that(self): >> if self.that: >> self.that() >> else: >> self.this() >> >> [...] >> >> self.that = True >> >> or: >> >> self.this_or_that = self.that > > This example, I don't understand. Do you mean that when your write X.Y, > that Y can be an attribute of X one minute, and a method the next? (In > which case I wouldn't want to have to maintain your code!) What I wanted to say above is that I have a choice of implementing a dynamic method using attribute assignment or a method definition. For all practical purposes, an object method is an attribute. You can write it using the handy "def" syntax or you can use an assignment. Python has defined it in a slightly more convoluted manner, but you would almost never sense the difference. Here's a whiff of the convolution: >>> four = 4 >>> four.__str__ is four.__str__ What happens here is that Python first sees if the builtin object 4 has an attribute named "__str__". It doesn't. Python then proceeds to 4's class, int, and finds int.__str__, which is the underlying method in the class. Python then constructs a delegate function like this: lambda *x, **y: int.__str__(4, *x, **y) and returns it. >> Somewhat related, every method is an automatic delegate. Defining >> callbacks is a breeze: >> >> def clickety_click(self, x, y): >> [...] >> >> [...] >> window.register_mouse_click_callback(self.clickety_click) > > I don't follow this either. What's the advantage of dynamism here? Dynamism doesn't play a direct role here, but the convolution I describe above makes the magic possible. (There would be a less convoluted way to come up with almost identical semantics, but that would cost memory and object instantiation time.) Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation in Python
On Saturday, March 12, 2016 at 3:43:16 AM UTC-6, dieter wrote: > > archives, and then stuff the link down your big fat mouth? > ^^^ > > What happened that you use language like this? Obviously, > you disagree with Steven - but this should not make you so > angry to use aggressive language. Yes, i did go too far here, and for that, i do humbly apologize to Steven and all the members of this group. There is no excuse for aggressive, or fowl language here. Steven has a bad habit of playing devils advocate, and he has on many occasions, made contradictory statements. When he does this, it angers me, because i know he's too intelligent to do these things by accident. In fact, all the members of this group are highly intelligent, and that is why i censor my language when I post here. When i am posting in other groups, groups which have set the "intelligence bar" far lower, i don't censor my words at all, because ignorant people don't deserve any decency from me. Intelligence is a rarity in this world, and much less so in many of the Usenet groups out there. Most of which seem to be nothing more than arenas for personal attacks, fowl language, and a contest to see who has the *LEAST* amount of intelligence. Very sad. I come here not only because of Python, but to engage my equals in civil discourse, and to soak up the vast amounts of general knowledge that are shared generously on a daily basis. Python-list an an oasis of intelligence in a sea of ignorance. And as such, demands that we maintain a high level of decency and mutual respect in our posts. I don't always agree with everyone here, and i may never agree with some, but I will always carry a deep respect for each and every member of this fine group. -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 12:10, Chris Angelico wrote:
On Sat, Mar 12, 2016 at 10:08 PM, BartC wrote:
Writing: a=65 generates this byte-code for the right-hand-side:
LOAD_CONST 1 (65) An integer
But writing instead: a=ord('A') generates this:
LOAD_GLOBAL 0 (ord)
LOAD_CONST 1 ('A') A string
CALL_FUNCTION 1
I think the "non-issue" here is the difference between ASCII and
Unicode. Either way, there's no way to say "the integer with the
codepoint of this character" as a literal. But that's actually not
even all that necessary, because subscripting a text string yields
one-character strings - you almost never need the ordinals.
That explains why you rarely use integers, if you prefer to use strings
even when there is a choice!
However, I was going to revise my benchmark to use strings instead of
integers, to show how much slower they would be. But the program was 10%
faster with strings!
I don't know what to make of that. Clearly comparing an integer with an
integer is a very fast operation, or ought to be, while comparing
strings is much less efficient at machine level.
So there's something funny going on. Either string operations are
super-fast or integer operations are somehow crippled. Or maybe there so
many other overheads, that the difference between strings and ints is lost.
That doesn't explain why ints are slower though (and this was the case
even on Python 2 where the 'long' type is not involved).
--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, 12 Mar 2016 11:08:25 +, BartC wrote:
>
> >> You're not mistaken. There are no "character constants" in Python.
> >> (Note that the definition would be Unicode codepoints, rather than
> >> ASCII values.) I don't often miss them, though.
>
>> Yes, a complete non-issue.
>
> Really? The issue as I see it is this:
>
> Writing: a=65 generates this byte-code for the right-hand-side:
>
> LOAD_CONST 1 (65) An integer
>
> But writing instead: a=ord('A') generates this:
>
> LOAD_GLOBAL 0 (ord)
> LOAD_CONST 1 ('A') A string CALL_FUNCTION 1
>
> You might be right: doing an unnecessary global name lookup and
> executing a function call are unlikely to have any impact on
> performance...
>
> The problem here is that 'ord' is dynamic, so this operation cannot
> simply be done at compile-time. Even when you try and optimise by
> assuming that ord is immutable, you don't really want to be doing any
> runtime checks. It might be faster than calling LOAD_GLOBAL and
> CALL_FUNCTION, but not quite as fast as just doing LOAD_CONST.
Why te constant focus on speed?
the majority of code is IO bound, either waiting for user input or
waitimng for file IO
a couple of extra microsecconds here and there between keypresses is
imaterial.
if you are wrighting truly CPU bound performance critical code then
python is probably not the correct language for these portions of the
code.
of course this is whre the 'C' based modules you so dislike can come to
your aid. meaning the performance hog can still run quickly but giving
you an easiy user interface to make the best of it.
as always there is no one language that is all things to all tasks,
python is just another tool in the box
use the correct tool for the correct job or you will invariably end up
with an unmanageable mess
--
In 1914, the first crossword puzzle was printed in a newspaper. The
creator received $4000 down ... and $3000 across.
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, Mar 13, 2016 at 2:12 AM, BartC wrote: > That explains why you rarely use integers, if you prefer to use strings even > when there is a choice! > > However, I was going to revise my benchmark to use strings instead of > integers, to show how much slower they would be. But the program was 10% > faster with strings! > > I don't know what to make of that. Clearly comparing an integer with an > integer is a very fast operation, or ought to be, while comparing strings is > much less efficient at machine level. > > So there's something funny going on. Either string operations are super-fast > or integer operations are somehow crippled. Or maybe there so many other > overheads, that the difference between strings and ints is lost. > > That doesn't explain why ints are slower though (and this was the case even > on Python 2 where the 'long' type is not involved). Or maybe they're all actually *object* comparisons, and what you know about assembly language has no relationship to what's going on here. This is why we keep advising you to get to know *Python*, rather than trying to use it as "assembly language with different syntax", especially when you start making performance estimates. Even with C code, I've stopped trying to predict what performance will be. (I'm not sure whether I could guess at assembly language performance; I stopped writing any at all when C compilers got provably better than me in all circumstances - which happened a good while ago.) The ONLY way to know whether X is faster than Y is to try them both. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On Sat, 12 Mar 2016 10:31:39 +, BartC wrote: > On 12/03/2016 10:06, alister wrote: >> On Fri, 11 Mar 2016 22:24:45 +, BartC wrote: >> >>> On 11/03/2016 21:59, Mark Lawrence wrote: On 11/03/2016 18:57, BartC wrote: >>> def test(): s="" for i in range(1000): s+="*" print (len(s)) test() >>> The minor snag that you might like to correct with your microbenchmark, which any experienced Python programmer knows, is that you *NEVER, EVER* create strings like this. >>> >>> Why not? Chris said his version runs much faster (even allowing for >>> different machines), and might have a special optimisation for it. >>> >>> And I think it can be optimised if, for example, there are no other >>> references to the string that s refers to. >>> >>> So what's wrong with trying to fix it rather that using a workaround? >> >> because the "workarround" is not a workarround it is the correct way to >> do it the code above is a workarround for somone who does not know the >> pythonic method to do this >> >> S= "*"*1000 > > This is a benchmark that measures the cost of adding to a string a > character at a time. > > In practice the final length of the string is not known, and the > characters added at each stage are not known. > > Although the strings won't usually be that big; the benchmark > exaggerates here to highlight a possible issue with += on strings. And > it worked, as there can be big difference with or without the += > optimisation in place. > > It also showed a possible bug or problem with PyPy. > > You can't demonstrate all this by just writing s="*"*1000. So you are bench marking python performance on a programming paradigm that is not good python practice. A pointless exercise what may be the best way to achieve a rsult in one language is not necessarily the best way to achieve it in another. if you wish to compare performance between one versio of python and another at least try to start with pythonic code -- There are two ways of disliking poetry; one way is to dislike it, the other is to read Pope. -- Oscar Wilde -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, 12 Mar 2016 01:20 pm, Chris Angelico wrote: >>> Definitely agree with this. Having a way to declare that a name is >>> "truly constant" would be extremely handy; there currently isn't a >>> way, and I'm not sure whether FAT Python is looking into this or not. "Constants" would be a new language feature, not an optimization. Unless CPython adds a constant feature, FAT Python isn't going to do it. >>> I think it's more focusing on the situation where something simply has >>> never been rebound, which is probably the better optimization; but >>> sometimes it'd be nice to be able to assert that something *will not* >>> be rebound, to help with self-documenting code. (Note that "constant" >>> implies both "never rebound" and "immutable". I disagree that constant necessarily implies immutable, at least in the context of Python. At least, what *I* want from a const keyword is a way to make a certain name "write-once": you can bind to it the first time, and then never again for the life of the scope. It shouldn't matter whether it is a list or a float. >> The 'const' prefix here is intended to define a named constant (a numeric >> literal with a convenient alias) rather than some 'read-only attribute >> for a conventional variable. >> >> But introducing that into Python would be a can of worms. (A named >> constant needs a compile-time expression on the right-hand-side. The >> compiler needs to be able to see named constants which are in an imported >> module, and which are accessed via attributes, and so on.) I disagree with that too. It is a limitation of older languages like Pascal and C that they can only define "consts" at compile time. But we can dynamically create constants at runtime too. Here's a proof-of-concept. Let's put efficiency aside and consider a naive "bind-once" const for a language like Python. Every namespace has a mapping of name:value, as we have now, plus a list of "constant names". Then every binding operation: x = value import x from module import x del x def x(): ... class x: ... for x in seq: ... with expr as x: ... except error as x: ... first checks the list of constant names for the name (e.g. "x"). If the name is not found, then the binding operation is allowed, just like it is today. If it is found, then the namespace is checked to see if the name already exists. If it does, the operation raises a TypeError (or perhaps ConstError?). If not, then the operation continues as normal. > Not sure about that. Consider: > > MAX_SIZE = 1<<16 > def some_func(blah): > data = some_file.read(MAX_SIZE) > > Currently, this disassembles to show that MAX_SIZE is being fetched > with LOAD_GLOBAL. If, instead, it were LOAD_CONST, this would mean > that rebinding MAX_SIZE after function definition would fail; I don't think it would fail in the sense of raising an explicit exception. I think it would just be hard to understand, giving you strange and mysterious behaviour: there's a name which I can successfully rebind, but some functions accessing it still see the old value, while others see the new value. But then, that's not far from the current behaviour of "early binding" of default values. > but it > would run faster. That's the advantage of a "declared constant", even > without it being any sort of compile-time constant. As long as it can > be finalized before the function is defined, it can be called > constant. Constants should be treated as constant even outside of functions. Maybe the secret is to make modules more like functions in their internal details? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 15:30, Chris Angelico wrote: On Sun, Mar 13, 2016 at 2:12 AM, BartC wrote: However, I was going to revise my benchmark to use strings instead of integers, to show how much slower they would be. But the program was 10% faster with strings! So there's something funny going on. Either string operations are super-fast or integer operations are somehow crippled. Or maybe there so many other overheads, that the difference between strings and ints is lost. Or maybe they're all actually *object* comparisons, Yeah, that explains it! and what you know about assembly language has no relationship to what's going on here. This is why we keep advising you to get to know *Python*, I'm not sure /my/ knowing Python better is going to help it get any faster. I discovered something that might be a clue to what's going on, but you're content to just brush it under the carpet. OK. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation in Python
On Fri, Mar 11, 2016 at 7:39 PM, Rick Johnson wrote: > At run-time, i don't care how large a "module namespace" may > be. Sometimes a module namespace will be small, with only a > few exposed symbols, but sometimes, a module namespace will > expose thousands of symbols. Thousands, really? What system do you use to ensure that symbols don't accidentally collide with one another? Because, you know, Python won't check this for you even within a single file, unlike say C++ which does allow splitting namespaces across multiple files. A linter might, but not at runtime, and I don't expect it will be smart enough to notice if the definitions are in different files. > (2) Create one file that will be the "mother-ship module", > and N files that will be the "satellite modules", and from > inside the mother-ship, import all the symbols from all > the satellites. Ha, but in reality, it's not that simple, > because state does not "magically" travel between modules! > > ## > # foo.py # > ## > FOO_SHARED_STATE = "foo" > import _fooSatilite1 > _fooSatilite1.FOO_SHARED_STATE = FOO_SHARED_STATE > from _fooSatilite1 import * > import _fooSatilite2 > _fooSatilite2.FOO_SHARED_STATE = FOO_SHARED_STATE > from _fooSatilite2 import * > print 'This is the mother-ship called foo' > ... > > > # _fooSatilite1.py # > > from _fooConstants import * > print 'This is foo-fighter1, reporting for duty' > print FOO_SHARED_STATE > ... > > > # _fooSatilite2.py # > > from _fooConstants import * > print 'This is foo-fighter2, reporting for duty' > print FOO_SHARED_STATE > ... > > But i find both to be absurd. Writing all code in a single > file might be fine for a toy module that contains a handful > of functions or classes or vars, but once we start creating > anything in the "professional size range", it will become > an "editing nightmare" of epic proportions! > > But option two is no better, because once we cut and paste > portions of the code into satellite files, we lose the > ability to "easily share state". Then we're forced to start > "hacking at the weeds" with import contortions and monkey > patches, all the while, fearing the dreaded circular import. > > > NO, THIS IS INSANITY! WHAT WE NEED IS AN OPTION 3! > > (3) Allow a programmer to define module space at will > > ## > # foo.py # > ## > module foo > FOO_SHARED_STATE = "foo" > print 'This is the mother-ship called foo' > ... > > > # _fooSatilite1.py # > > module foo > print 'This is foo-fighter1, reporting for duty' > print FOO_SHARED_STATE # NO MP REQUIRED! > ... > > > # _fooSatilite2.py # > > module foo > print 'This is foo-fighter2, reporting for duty' > print FOO_SHARED_STATE # NO MP REQUIRED! > ... Is this meant to be equivalent to the other example? In that example, you have the two "satellite" modules being imported between the definition of FOO_SHARED_STATE and the print. In this example, how on Earth is Python supposed to know that after the FOO_SHARED_STATE line it's supposed to switch to executing these other two files before continuing on to the next statement? More generally, what order do you expect these files to be executed in when the "module" is imported? There must be some defined order, because Python isn't magically capable of executing all three of them simultaneously. If there's a defined order, then you should be able to easily add explicit import statements at the tops of modules that declare that order. So this is exactly the same as your "import contortions" except that you want to make the reader (and the Python interpreter) have to guess at the dependencies instead of making them explicit. > No need for import contortions, no need for monkey patching, > but most importantly, no need for professional programmers > to feel as though they are penchant little children, who > need their "module diapers" wrapped for them by mommy > Python, who, after popping one too many "mother's little > helpers", has a nasty habit of wrapping our diapers too damn > tight -- what a drag, it is, getting old... > >> 2) Clearly define which module is to be imported first. Then follow >> it. When module b is imported, it should import module a. When module >> a is imported, it *shouldn't* import module b until it's defined all >> of its own members first. If module a depends on anything from module >> b at import time, then refactor so it doesn't. This doesn't require >> "contortions". > > That sounds simple in theory, but it breaks down quickly > when you create "professional sized programs" I find in interesting that you seem to be equating "very large" with "professional". In
Re: Encapsulation in Python
On Friday, March 11, 2016 at 6:52:42 PM UTC-6, Gregory Ewing wrote:
> Rick Johnson wrote:
> > I have witnessed the mayhem that occurs when a language does
> > not mandate module encapsulation (Ruby, i'm looking directly
> > at you), and while i agree with the Python designers
> > that modules must *ALWAYS* be mandatory, i am not convinced
> > that module space should be so strictly confined to source
> > files.
>
> Well, I am. When looking at someone else's code, it's
> very useful to be able to take a name used in one source
> file and easily find the file where it's defined. That's
> very easy to do in Python, and very hard to do in languages
> that don't relate namespaces and source files.
You're essentially saying that all you have to do is scroll
up to the top of the current file, and find the line that
imports the symbol, and that line will tell you the name of
the module that contains the source code for that symbol.
Sure, that's reliable in most cases, but your argument
assumes that the actual source code for the symbol exists in
the module from which it was imported, when it could just as
well exist N-levels below that that module, due to chained
imports.
Imagine this scenario:
# currentModule.py #
from modX import foo
def bar():
return foo()
###
# modX.py #
###
from modY import foo
###
# modY.py #
###
from modZ import foo
###
# modZ.py #
###
def foo():
return 'O:-)'
I'll admit this is a highly contrived example, but it is not
invalid in anyway, and could therefore exist in reality. So
even though you will *EVENTUALLY* find the source code for
"foo", you would have to follow a long chain of imports to
get there. A more efficient method of finding the source,
would be to search your library for "def foo(", or, even
better, to utilize the tools available in the stdlib.
PY> import Tkinter as tk
PY> import inspect
PY> inspect.getsourcefile(tk.Frame)
C:\Python27\lib\lib-tk\Tkinter.py
Your editor should have tools for doing both of these
searches, and, if it's worth it's weight in salt, it will
even open the file for you.
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, 13 Mar 2016 12:42 am, BartC wrote: > Ad-hoc attributes I don't have as much of a problem with, as they can be > handy. But predefined ones also have their points. (For one thing, I > know how to implement those efficiently.) > > However, when you have a function call like this: M.F(), where M is an > imported module, then it is very unlikely that the functions in M are > going to be created, modified, deleted or replaced while the program > runs. [I mean, after the usual process of executing each 'def' statement.] What do you consider "very unlikely"? And how do you know what people will choose to do? > Why then should it have to suffer the same overheads as looking up > arbitrary attributes? And on every single call? Because they *are* arbitrary attributes of the module. There's only one sort of attribute in Python. Python doesn't invent multiple lookup rules for attributes-that-are-functions, attributes-that-are-classes, attributes-that-are-ints, attributes-that-are-strings, and so on. They are all the same. You gain a simpler implementation, a simpler execution model, simpler rules for users to learn, and the ability to perform some pretty useful dynamic tricks on those occasions where it is useful. For example, monkey-patching a module for testing or debugging purposes. In languages where functions are different from other values, you have to recognise ahead of time "some day, I may need to dynamically replace this function with another" and write your code specially to take that into account, probably using some sort of "Design Pattern". In Python, you just write your code in the normal fashion. >> I also have a high level of method/attribute transparency. It doesn't >> matter if I declare: >> >> def this_or_that(self): >> if self.that: >> self.that() >> else: >> self.this() >> >> [...] >> >> self.that = True >> >> or: >> >> self.this_or_that = self.that > > This example, I don't understand. Do you mean that when your write X.Y, > that Y can be an attribute of X one minute, and a method the next? (In > which case I wouldn't want to have to maintain your code!) Again, methods *are* attributes. Notice that Python doesn't have two complete sets of functions: getattr, setattr, delattr getmethod, setmethod, delmethod Again, as above, there is *one* look-up rule, that applies equally to attributes-that-are-methods and attributes-that-are-floats (say). What distinguishes the two cases is whether or not the attribute is a method object or a float object, not how you access it or define it. In practice, people rarely do something like: x.y = method # later x.y = 999 # later still x.y = method # again because that would be a strange thing to do, not because it would be impossible. They don't do it because there's no reason to do so, not because they can't: the same reason we don't walk around with pots of honey carefully nestled in our hair. >> Somewhat related, every method is an automatic delegate. Defining >> callbacks is a breeze: >> >> def clickety_click(self, x, y): >> [...] >> >> [...] >> window.register_mouse_click_callback(self.clickety_click) > > I don't follow this either. What's the advantage of dynamism here? I think that Marko's point is that because obj.clickety_click is just a regular attribute that returns a method object, you don't have to invent special syntax for referring to method *without* calling them. You just use the same attribute access syntax, but don't call the result. >> That optimization wouldn't have any effect on any of my code. >> >> More generally, every method call in Python is such an elaborate >> exercise that dabbling with character constants is going to be a drop in >> the ocean. > > When you dabble with lots of little things, then they can add up. To the > point where an insignificant optimisation can become significant. Of course. Reduced runtime efficiency is the cost you pay for the flexibility gained by significant dynamism. It's a trade-off between efficiency, convenience, simplicity, etc. It's quite legitimate for language designers to choose to put that trade-off in different places, or indeed for the trade-off to change over time. For example, Java's strictness was found to be too limiting and static, and so reflection was added to the language to add some dynamism. Here's the description from a Stackoverflow answer: http://stackoverflow.com/questions/37628/what-is-reflection-and-why-is-it-useful For example, say you have an object of an unknown type in Java, and you would like to call a 'doSomething' method on it if one exists. Java's static typing system isn't really designed to support this unless the object conforms to a known interface, but using reflection, your code can look at the object and find out if it has a method called 'doSomethi
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 16:42, BartC wrote: On 12/03/2016 15:30, Chris Angelico wrote: On Sun, Mar 13, 2016 at 2:12 AM, BartC wrote: However, I was going to revise my benchmark to use strings instead of integers, to show how much slower they would be. But the program was 10% faster with strings! So there's something funny going on. Either string operations are super-fast or integer operations are somehow crippled. Or maybe there so many other overheads, that the difference between strings and ints is lost. Or maybe they're all actually *object* comparisons, Yeah, that explains it! and what you know about assembly language has no relationship to what's going on here. This is why we keep advising you to get to know *Python*, I'm not sure /my/ knowing Python better is going to help it get any faster. I discovered something that might be a clue to what's going on, but you're content to just brush it under the carpet. OK. For a language that is apparently so slow that is unusable, it somehow has managed to get a following. From https://www.python.org/about/success/ Python is part of the winning formula for productivity, software quality, and maintainability at many companies and institutions around the world. Here are 41 real-life Python success stories, classified by application domain. So I am clearly not the only programmer in the world who couldn't care less about speed. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Descriptors vs Property
Veek. M wrote:
> Veek. M wrote:
>> class TypedProperty(object):
>> def __init__(self,name,type,default=None):
>> self.name = "_" + name
>> self.type = type
>> self.default = default if default else type()
>>
>> def __get__(self,instance,cls):
>> return getattr(instance,self.name,self.default)
>> […]
>> class Foo(object):
>> name = TypedProperty("name",str)
>> num = TypedProperty("num",int,42)
>> […]
>> When you do f.name you are actually doing:
>>
>> f.name.__get__(self, instance, cls)
Correct.
>> What the heck??
> As in, why is he passing instance, cls and who is populating those vars?
RTFM:
> When you do f.name you just have self to pass into
> name.__whatever__(self)
No.
> I haven't read the descriptor protocol as yet.
You should. You should also trim your quotations to the relevant minimum,
and post using your real name.
--
PointedEars
Twitter: @PointedEars2
Please do not cc me. / Bitte keine Kopien per E-Mail.
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 16:56, Steven D'Aprano wrote: On Sun, 13 Mar 2016 12:42 am, BartC wrote: Ad-hoc attributes I don't have as much of a problem with, as they can be handy. But predefined ones also have their points. (For one thing, I know how to implement those efficiently.) However, when you have a function call like this: M.F(), where M is an imported module, then it is very unlikely that the functions in M are going to be created, modified, deleted or replaced while the program runs. [I mean, after the usual process of executing each 'def' statement.] What do you consider "very unlikely"? And how do you know what people will choose to do? Common sense tells you it is unlikely. Why then should it have to suffer the same overheads as looking up arbitrary attributes? And on every single call? Because they *are* arbitrary attributes of the module. There's only one sort of attribute in Python. Python doesn't invent multiple lookup rules for attributes-that-are-functions, attributes-that-are-classes, attributes-that-are-ints, attributes-that-are-strings, and so on. They are all the same. You gain a simpler implementation, (Have you tried looking at the CPython sources? I tried last year and couldn't head or tail of them. What was the layout of the pyObject struct? I couldn't figure it out, the source being such a mess of conditional code and macros within macros. So I wouldn't like to see a complex implementation!) a simpler execution model, simpler rules for users to learn, and the ability to perform some pretty useful dynamic tricks on those occasions where it is useful. For example, monkey-patching a module for testing or debugging purposes. In languages where functions are different from other values, you have to recognise ahead of time "some day, I may need to dynamically replace this function with another" and write your code specially to take that into account, probably using some sort of "Design Pattern". No it's very easy. In Python terms: def f(): return "One" def g(): return "Two" h=f h() returns "One". Later you do h=g, and h() returns "Two". No need for f and g themselves to be dynamic. h just needs to be a variable. The same with modules: import A import B M=A Now M.F() calls A.F(). Do M=B, and M.B now calls B.F(). No need for module names to be dynamic either! Just variables. When you dabble with lots of little things, then they can add up. To the point where an insignificant optimisation can become significant. Of course. Reduced runtime efficiency is the cost you pay for the flexibility gained by significant dynamism. It's a trade-off between efficiency, convenience, simplicity, etc. It's quite legitimate for language designers to choose to put that trade-off in different places, or indeed for the trade-off to change over time. Maybe the designer(s) of Python didn't know how popular it would get. Do you think some of the design decisions would be different now? -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
BartC : > No it's very easy. In Python terms: > > def f(): return "One" > def g(): return "Two" > > h=f > > h() returns "One". Later you do h=g, and h() returns "Two". No need > for f and g themselves to be dynamic. h just needs to be a variable. Well, what do you make of this: >>> def f(): return 1 ... >>> g = f >>> def f(): return 2 ... >>> g() 1 >>> f() 2 Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 18:07, Marko Rauhamaa wrote: BartC : No it's very easy. In Python terms: def f(): return "One" def g(): return "Two" h=f h() returns "One". Later you do h=g, and h() returns "Two". No need for f and g themselves to be dynamic. h just needs to be a variable. Well, what do you make of this: >>> def f(): return 1 ... >>> g = f >>> def f(): return 2 ... >>> g() 1 >>> f() 2 def f001(): return 1 f = f001 g = f def f002(): return 2 f=f002; print(g()) print(f()) Same results, but now we've agains removed the need for the function names (f001 and f002) to be mutable. -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sat, 12 Mar 2016 16:42:47 +, BartC wrote: > On 12/03/2016 15:30, Chris Angelico wrote: >> On Sun, Mar 13, 2016 at 2:12 AM, BartC wrote: > >>> However, I was going to revise my benchmark to use strings instead of >>> integers, to show how much slower they would be. But the program was >>> 10% faster with strings! > >>> So there's something funny going on. Either string operations are >>> super-fast or integer operations are somehow crippled. Or maybe there >>> so many other overheads, that the difference between strings and ints >>> is lost. > >> Or maybe they're all actually *object* comparisons, > > Yeah, that explains it! > > and what you know >> about assembly language has no relationship to what's going on here. >> This is why we keep advising you to get to know *Python*, > > I'm not sure /my/ knowing Python better is going to help it get any > faster. > how will you konw untill you start writing python rather than C or assembler? > I discovered something that might be a clue to what's going on, but > you're content to just brush it under the carpet. > > OK. -- Caf'e Minimalism: To espouse a philosophy of minimalism without actually putting into practice any of its tenets. -- Douglas Coupland, "Generation X: Tales for an Accelerated Culture" -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism
Marko Rauhamaa wrote: > […] HTML markup is all ASCII. Wrong. I am creating HTML documents whose source code contains Unicode characters every day. Also, the two of you fail to differentiate between US-ASCII, a 7-bit character encoding, and 8-bit or longer encodings which can *also* encode characters that can be *encoded with* US-ASCII. -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism
Marko Rauhamaa wrote: > […] all keyboards can produce ASCII and no keyboard can produce all of > Unicode. Both claims are wrong. -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
BartC wrote: > On 12/03/2016 12:13, Marko Rauhamaa wrote: >> Why, look at the *English* page on Hillary Clinton: >> >> Hillary Diane Rodham Clinton /ˈhɪləri daɪˈæn ˈrɒdəm ˈklɪntən/ (born >> October 26, 1947) is an American politician. >> https://en.wikipedia.org/wiki/Hillary_Clinton> >> >> You couldn't get past the first sentence in ASCII. > > I saved that page locally as a .htm file in UTF-8 encoding. I ran a > modified version of my benchmark, and it appeared that 99.7% of the > bytes had ASCII codes. That is a contradiction in terms. Obviously you do not know what ASCII is. -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. -- https://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation in Python
On Sun, Mar 13, 2016 at 3:49 AM, Rick Johnson
wrote:
> Imagine this scenario:
>
>
> # currentModule.py #
>
> from modX import foo
>
> def bar():
> return foo()
>
> ###
> # modX.py #
> ###
> from modY import foo
>
> ###
> # modY.py #
> ###
> from modZ import foo
>
> ###
> # modZ.py #
> ###
> def foo():
> return 'O:-)'
>
> I'll admit this is a highly contrived example, but it is not
> invalid in anyway, and could therefore exist in reality.
I've never seen public symbols deliberately being imported through
multiple levels like this. The most I've seen is __init__.py pulling
stuff in from one of its modules (eg "from .constants import *"), or a
Python module importing from its accelerator ("from _socket import
*"). In theory, I suppose you could have both at once, but that's the
most you'd ever get, and the three would be very tightly coupled.
Otherwise, this simply doesn't happen.
Also, if currentModule.py is pulling foo from modX, then modZ.py is an
implementation detail. You don't necessarily want to go straight
there; tracing the chain is more likely to be the correct behaviour.
Suppose modX.py actually looks like this:
if 'posix' in some_magic:
import posixY as modY
elif 'nt' in some_magic:
import ntY as modY
else:
raise ImportError
The correct way to find out where modX.foo comes from is to look at
this block, not to jump right through it. Law of Demeter - you take
things one step at a time (unless there's a good reason).
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism
On Sun, Mar 13, 2016 at 6:24 AM, Thomas 'PointedEars' Lahn wrote: > Marko Rauhamaa wrote: > >> […] HTML markup is all ASCII. > > Wrong. I am creating HTML documents whose source code contains Unicode > characters every day. > > Also, the two of you fail to differentiate between US-ASCII, a 7-bit > character encoding, and 8-bit or longer encodings which can *also* encode > characters that can be *encoded with* US-ASCII. Where are the non-ASCII characters in your HTML documents? Are they in the *markup* of HTML, or in the *text*? This is the difference. And I'm not conflating those two. When I say ASCII, I am referring to the 128 characters that have Unicode codepoints U+ through U+007F. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, Mar 13, 2016 at 3:22 AM, Steven D'Aprano wrote: > On Sat, 12 Mar 2016 01:20 pm, Chris Angelico wrote: > > Definitely agree with this. Having a way to declare that a name is "truly constant" would be extremely handy; there currently isn't a way, and I'm not sure whether FAT Python is looking into this or not. > > "Constants" would be a new language feature, not an optimization. Unless > CPython adds a constant feature, FAT Python isn't going to do it. Yes, they would. Having a way to *declare*, in your source code, is what I'm talking about. Although FAT Python does have facilities for noticing that something hasn't changed, this is talking about demanding that it _never_ change. I think it's more focusing on the situation where something simply has never been rebound, which is probably the better optimization; but sometimes it'd be nice to be able to assert that something *will not* be rebound, to help with self-documenting code. (Note that "constant" implies both "never rebound" and "immutable". > > I disagree that constant necessarily implies immutable, at least in the > context of Python. At least, what *I* want from a const keyword is a way to > make a certain name "write-once": you can bind to it the first time, and > then never again for the life of the scope. It shouldn't matter whether it > is a list or a float. I mean that the word generally implies that. It's perfectly possible to have a Python keyword "constant" that means "this will never be rebound" (ie "constant by identity"), without having "constant value"; but there will be people who are confused by it, just as (and for the same reason as) they're confused by mutable default arguments. There'll be posts all over the place, "never use mutable constant values, or they stop being constant". I completely agree with you that the keyword should mean "write-once" or "never rebind". > Let's put efficiency aside and consider a naive "bind-once" const for a > language like Python. Every namespace has a mapping of name:value, as we > have now, plus a list of "constant names". Then every binding operation: > >x = value >import x >from module import x >del x >def x(): ... >class x: ... >for x in seq: ... >with expr as x: ... >except error as x: ... > > first checks the list of constant names for the name (e.g. "x"). If the name > is not found, then the binding operation is allowed, just like it is today. > If it is found, then the namespace is checked to see if the name already > exists. If it does, the operation raises a TypeError (or perhaps > ConstError?). If not, then the operation continues as normal. IMO, ConstError should probably be its own thing. The nearest equivalent I can find is attempting to assign to a read-only property, which raises AttributeError; this is broader than that. I'd make this much, much simpler. This statement: constant x declares that 'x' will never, from this point on, be assigned to. And this statement: constant x = y (re)binds x to the value of y, and then declares that x will never be assigned to. All assignments simply check this list; if the name is on the list, raise error. So if you're doing a "constant import", or something else that does a special form of bind, you would spell it differently: from module import x constant x As a convenience, I would like to see a few common constructs, such as "constant def" and "constant class", but we don't need "constant for" or "constant with" (and definitely not "constant except", since that has an implicit rebind and unbind at the end). >> Not sure about that. Consider: >> >> MAX_SIZE = 1<<16 >> def some_func(blah): >> data = some_file.read(MAX_SIZE) >> >> Currently, this disassembles to show that MAX_SIZE is being fetched >> with LOAD_GLOBAL. If, instead, it were LOAD_CONST, this would mean >> that rebinding MAX_SIZE after function definition would fail; > > I don't think it would fail in the sense of raising an explicit exception. I > think it would just be hard to understand, giving you strange and > mysterious behaviour: there's a name which I can successfully rebind, but > some functions accessing it still see the old value, while others see the > new value. Yeah, it would fail in the sense that it would do the wrong thing. If you're expecting "module.MAX_SIZE = 1<<10" to reduce the maximum, that would silently fail to do what you expect. Basically, what I'm talking about here is the difference between "import x" and "from x import y". Normally, globals are effectively "my_module.MAX_SIZE", but the LOAD_CONST transformation turns them into "from my_module import MAX_SIZE". Rebinding the module name has no effect on something that already imported it by value. > But then, that's not far from the current behaviour of "early binding" of > default values. Exactly; the only difference is that you can't rebind it in any way (including inside the function; when you us
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
Chris Angelico : > I completely agree with you that the keyword should mean "write-once" > or "never rebind". That would be possible. I'm afraid that would result in people sprinkling these "constant" keywords everywhere to make the program supposedly run faster. -- Something like that has happened with the "final" keyword in some Java houses. That would prevent the ad hoc installation of wrappers, debugging tools etc. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 19:26, Thomas 'PointedEars' Lahn wrote: BartC wrote: On 12/03/2016 12:13, Marko Rauhamaa wrote: Why, look at the *English* page on Hillary Clinton: Hillary Diane Rodham Clinton /ˈhɪləri daɪˈæn ˈrɒdəm ˈklɪntən/ (born October 26, 1947) is an American politician. https://en.wikipedia.org/wiki/Hillary_Clinton> You couldn't get past the first sentence in ASCII. I saved that page locally as a .htm file in UTF-8 encoding. I ran a modified version of my benchmark, and it appeared that 99.7% of the bytes had ASCII codes. That is a contradiction in terms. Obviously you do not know what ASCII is. What does your own analysis show of that page? If you had it in memory as fully expanded 32-bit Unicode values, what proportion of those would have values below 128? -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, Mar 13, 2016 at 9:10 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> I completely agree with you that the keyword should mean "write-once" >> or "never rebind". > > That would be possible. I'm afraid that would result in people > sprinkling these "constant" keywords everywhere to make the program > supposedly run faster. -- Something like that has happened with the > "final" keyword in some Java houses. > > That would prevent the ad hoc installation of wrappers, debugging tools > etc. Hmm. I wonder if it should be like "assert" - nobody ever should depend 100% on it, but it's a hint back to the interpreter that you should never be rebinding this. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
Chris Angelico : > On Sun, Mar 13, 2016 at 9:10 AM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> I completely agree with you that the keyword should mean >>> "write-once" or "never rebind". >> >> That would be possible. I'm afraid that would result in people >> sprinkling these "constant" keywords everywhere to make the program >> supposedly run faster. -- Something like that has happened with the >> "final" keyword in some Java houses. >> >> That would prevent the ad hoc installation of wrappers, debugging >> tools etc. > > Hmm. I wonder if it should be like "assert" - nobody ever should > depend 100% on it, but it's a hint back to the interpreter that you > should never be rebinding this. Just wait and see what will happen. BTW, Java's "final" keyword is additionally used to declare that a method is not overridden. I once worked in a Java company where the policy was to declare *everything* final until you knew that you needed to inherit. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 22:10, Marko Rauhamaa wrote: Chris Angelico : I completely agree with you that the keyword should mean "write-once" or "never rebind". That would be possible. I'm afraid that would result in people sprinkling these "constant" keywords everywhere to make the program supposedly run faster. -- Something like that has happened with the "final" keyword in some Java houses. I use 'const' everywhere in other languages, most often in the form of sophisticated sets of enums. A single project might have 1000 or even 2000. (Example that defines a set of byte-codes: http://pastebin.com/q1UwjKmK) How does Python manage without them? Is it really necessary to declare hundreds of individual variables and assign a value to each? (And risk someone assigning a new value to them.) That they might lead to more efficient code is secondary, but definitely a bonus (essential though when used in a switch statement). -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 12/03/2016 23:57, BartC wrote: On 12/03/2016 22:10, Marko Rauhamaa wrote: Chris Angelico : I completely agree with you that the keyword should mean "write-once" or "never rebind". That would be possible. I'm afraid that would result in people sprinkling these "constant" keywords everywhere to make the program supposedly run faster. -- Something like that has happened with the "final" keyword in some Java houses. I use 'const' everywhere in other languages, most often in the form of sophisticated sets of enums. A single project might have 1000 or even 2000. (Example that defines a set of byte-codes: http://pastebin.com/q1UwjKmK) How does Python manage without them? Is it really necessary to declare hundreds of individual variables and assign a value to each? (And risk someone assigning a new value to them.) That they might lead to more efficient code is secondary, but definitely a bonus (essential though when used in a switch statement). It is 2016. Programmer time, and hence money, are far more important than runtime speed in the vast majority of cases. There are plenty of working recipes for switch in Python. I'll leave you to quote a few as you are such an expert in the Python programming language. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, 13 Mar 2016 04:02 am, Mark Lawrence wrote: > So I am clearly not the only programmer in the world who couldn't care > less about speed. For somebody who couldn't care less, you sure do keep going on and on and on and on and on and on and on about it. Yeah, we get it. The CPython interpreter is plenty fast enough for whatever you use it for. Good for you! Millions aren't so lucky. They are looking for something faster, because for them, you can never have too much speed. When you are trying to solve a 7000x7000 system of equations, you don't want to wait three days for a solution. Or three hours with Numpy. And that's why projects like PyPy, Numba, Cython, Pyjion, Pyston, Theano, FAT Python, Unladen Swallow, Nuitka, Pyrex, Psyco, Parakeet, SWIG and more exist. For a language which is fast enough, it sure seems that a lot of people are working to ways to speed up Python applications. If Bart is such a horrible person for wanting Python to be faster, what does that make Brett Cannon (Pyjion), Victor Stinner (FAT Python), Stefan Behnel (Cython), Armin Rigo (Psycho and PyPy), to mention just a few? And if speed goes against the Python philosophy so badly, why doesn't GvR stop them, or at least discourage them more? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 13/03/2016 01:20, Steven D'Aprano wrote: On Sun, 13 Mar 2016 04:02 am, Mark Lawrence wrote: So I am clearly not the only programmer in the world who couldn't care less about speed. For somebody who couldn't care less, you sure do keep going on and on and on and on and on and on and on about it. Yeah, we get it. The CPython interpreter is plenty fast enough for whatever you use it for. Good for you! Millions aren't so lucky. They are looking for something faster, because for them, you can never have too much speed. When you are trying to solve a 7000x7000 system of equations, you don't want to wait three days for a solution. Or three hours with Numpy. And that's why projects like PyPy, Numba, Cython, Pyjion, Pyston, Theano, FAT Python, Unladen Swallow, Nuitka, Pyrex, Psyco, Parakeet, SWIG and more exist. For a language which is fast enough, it sure seems that a lot of people are working to ways to speed up Python applications. If Bart is such a horrible person for wanting Python to be faster, what does that make Brett Cannon (Pyjion), Victor Stinner (FAT Python), Stefan Behnel (Cython), Armin Rigo (Psycho and PyPy), to mention just a few? And if speed goes against the Python philosophy so badly, why doesn't GvR stop them, or at least discourage them more? Python works and is extremely successful. If it were made even faster it would be even more successful. BartC would still be a complete idiot, as the complete crap that he's been spewing over the last days puts him into the same league as the RUE, possibly worse. It is so obvious that he knows nothing about Python that I find it stunning that you can even contemplate supporting him, or is he paying you? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, Mar 13, 2016 at 12:32 PM, Mark Lawrence wrote: > > Python works and is extremely successful. If it were made even faster it > would be even more successful. BartC would still be a complete idiot, as > the complete crap that he's been spewing over the last days puts him into > the same league as the RUE, possibly worse. It is so obvious that he knows > nothing about Python that I find it stunning that you can even contemplate > supporting him, or is he paying you? Be fair. BartC knows quite a lot about Python; his knowledge is imperfect, but whose isn't? I will debate strongly on some of the points where we disagree, and seek to educate on matters of Pythonic code, but he does know what he's doing to at least some degree. He is not trolling; perhaps he places more emphasis on performance than most of us do (and he definitely gives a lot more weight to microbenchmarks than I consider wise), and perhaps he doesn't give enough emphasis to Unicode support (but that's a VERY common programmer error - "I don't need that, it's fine"), but he is an intelligent programmer and deserves your respect. Thanks. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On Sun, 13 Mar 2016 10:57 am, BartC wrote: > I use 'const' everywhere in other languages, most often in the form of > sophisticated sets of enums. A single project might have 1000 or even > 2000. (Example that defines a set of byte-codes: > http://pastebin.com/q1UwjKmK) > > How does Python manage without them? Is it really necessary to declare > hundreds of individual variables and assign a value to each? (And risk > someone assigning a new value to them.) Copying from your code, you define a bunch of constants in a table (many of which apparently have the same value?): global tabledata() cmdnames, cmdfmt = (kzero=0, $, (0,0,0,0)), (knop, $, (0,0,0,0)), (klabel,$, (l,0,0,0)), (kcodestart,$, (p,h,0,0)), (kcodeend, $, (0,0,0,0)), (kendmodule,$, (0,0,0,0)), (kendprogram, $, (0,0,0,0)), ... end I don't understand the syntax. You seem to have three columns in the table, a name, a $ whatever that is for, and some data (four values in a comma-separated parenthesised list), but the table appears to only define two columns (cmdnames, cmdfmt). Mysterious. In Python, we might similarly define a table, using a dict: tabledata = dict( kzero=(0,0,0,0), knop=(0,0,0,0)), klabel=(l,0,0,0)), kcodestart=(p,h,0,0)), kcodeend=(0,0,0,0)), kendmodule=(0,0,0,0)), kendprogram=(0,0,0,0)), ... ) So the amount of typing is comparable. If you have 200 symbolic names with associated data, one way or the other you have to enter 200 symbolic names and their associated data into your source code, regardless of whether they are called "constants", "enums", "variables" or "symbols". The dict solution does have the rather sad consequence that we then have to use quoted names for the symbols: x = tabledata['klabel'] although one can make a simple wrapper around the dict to fix this: class table: pass table.__dict__ = tabledata x = table.klabel And if you want to make these "constants" available as top-level names as well as via the table: globals.update(tabledata) What about the risk that somebody might modify them? The Pythonic answer is "well don't do that". The names are clearly marked with a leading "k" for "konstant", so unless documented otherwise they're not variables. We don't need the compiler to enforce that rule. Some of us might like a more traditional rule. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
Mark Lawrence writes: > BartC would still be a complete idiot, as the complete crap that he's > been spewing over the last days puts him into the same league as the > RUE, possibly worse. It is so obvious that he knows nothing about > Python that I find it stunning that you can even contemplate > supporting him, or is he paying you? Stop this bullying and harrassment, Mark. This isn't acceptable behaviour in a Python community forum. -- \ “My house is on the median strip of a highway. You don't really | `\notice, except I have to leave the driveway doing 60 MPH.” | _o__) —Steven Wright | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 13/03/2016 02:01, Steven D'Aprano wrote:
On Sun, 13 Mar 2016 10:57 am, BartC wrote:
I use 'const' everywhere in other languages, most often in the form of
sophisticated sets of enums. A single project might have 1000 or even
2000. (Example that defines a set of byte-codes:
http://pastebin.com/q1UwjKmK)
How does Python manage without them? Is it really necessary to declare
hundreds of individual variables and assign a value to each? (And risk
someone assigning a new value to them.)
Copying from your code, you define a bunch of constants in a table (many of
which apparently have the same value?):
global tabledata() cmdnames, cmdfmt =
(kzero=0, $, (0,0,0,0)),
(knop, $, (0,0,0,0)),
(klabel,$, (l,0,0,0)),
(kcodestart,$, (p,h,0,0)),
(kcodeend, $, (0,0,0,0)),
(kendmodule,$, (0,0,0,0)),
(kendprogram, $, (0,0,0,0)),
...
end
I don't understand the syntax. You seem to have three columns in the table,
a name, a $ whatever that is for, and some data (four values in a
comma-separated parenthesised list), but the table appears to only define
two columns (cmdnames, cmdfmt). Mysterious.
(Obviously my comments didn't work.. The table above is roughly
equivalent to the following Python:
kzero = 0 # define a bunch of enums (with auto-increment)
knop= 1
klabel = 2
kcodestart = 3
kcodeend= 4
kendmodule = 5
kendprogram = 6
...
#define a list of matching names (the $ is a cheap gimmick to avoid
#duplicating each name. The language ought to take care of accessing
#the names, but it doesn't):
cmdnames = ("kzero",
"knop",
"klabel",
"kcodestart",
"kcodeend",
"kendmodule",
"kendprogram",...)
#define matching format codes (the single letter codes are constants
#defined previously):
cmfmt = ( (0,0,0,0),
(0,0,0,0),
etc.
Clearly it is possible to write the above, but it's harder to maintain
and read (inserting or deleting enum names, keeping the parallel lists
aligned, and trying to read off which fmt corresponds to which enum).)
In Python, we might similarly define a table, using a dict:
tabledata = dict(
kzero=(0,0,0,0),
knop=(0,0,0,0)),
klabel=(l,0,0,0)),
kcodestart=(p,h,0,0)),
kcodeend=(0,0,0,0)),
kendmodule=(0,0,0,0)),
kendprogram=(0,0,0,0)),
...
)
So the amount of typing is comparable. If you have 200 symbolic names with
associated data, one way or the other you have to enter 200 symbolic names
and their associated data into your source code, regardless of whether they
are called "constants", "enums", "variables" or "symbols".
There are all sorts of ways to do it in any language. But I found my
method invaluable. And you do end up with actual compile-time constants
for the codes on the left.
A simpler example:
enum (red,green,blue)
which is equivalent to:
const red=1
const green=2
const blue=3
I've seen half a dozen ways of doing enums in Python, but some look
really complicated for something so simple. And you still end up with a
LOAD_GLOBAL for that 'red' value!
Worse if red, green, blue are defined in a class as then you need a
lookup too. My example above defined 'open' enums, but they can be
assigned to their own type:
type lights = (red, amber, green)
Now it's necessary to say lights.green, but this is just the constant 3!
See, get rid of some of the dynamics, and lots of things become very
easy. (But I don't think that's going to happen.)
--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 to create a record using a FK
On Sat, Mar 12, 2016 at 5:26 AM, Peter Otten <[email protected]> wrote: > Aaron Christensen wrote: > > > Hello, > > > > I am running the following versions of software: > > > > Python 3.5 > > psycopg2==2.6.1 > > Postgres 9.4.5 > > > > I have 2 tables. Table User has UserId (serial PK), LastName, FirstName, > > Gender, DateOfBirth, and DateEnrolled. Table UserProfile has > > UserProfileId > > (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity. There > > is a one-to-many relationship. > > > > The following PostgreSQL works and ultimately creates a record in > > UserProfile with an associated UserId (FK). > > > > \set last_name '''Sara''' > > \set first_name '''Jackson''' > > \set gender '''F''' > > \set dob '''1941-1-12''' > > \set fave_number '''3''' > > \set activity '''volleyball''' > > > > > > WITH ins_user AS ( > > INSERT INTO User > > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) > > VALUES (:last_name, :first_name, :gender, :dob, now()) > > RETURNING UserId) > > INSERT INTO UserProfile > > (UserId, DateEntered, FaveNumber, Activity) > > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity); > > > > How can I build a psycopg2 cur.execute query that will accomplish the > > above > > PostgreSQL? I've read documentation but can't seem to get a handle on > how > > I should structure this command. > > I have not tried it, but wouldn't the straight-forward > > cur.execute(""" > WITH ins_user AS ( > INSERT INTO User > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) > VALUES (:last_name, :first_name, :gender, :dob, now()) > RETURNING UserId) > INSERT INTO UserProfile > (UserId, DateEntered, FaveNumber, Activity) > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity); > """, > dict( > first_name="Sara", > last_name="Jackson", > gender="F", > dob=datetime.date(1941, 1, 12), > fave_number=3, > activity="volleyball" > )) > > work? > Hi Peter, Yes, thank you. It turns out that I can pass a dictionary as an argument. I have combined your response with Dieter's and am running into some issues now. I need to figure out how to pass a dictionary AND the returned Id. > > > My starting point is: > > > > cur.execute( \ > > """INSERT INTO User \ > > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \ > > VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \ > > (last_name, first_name, gender, date_of_birth, now(), ??...??) > > > > > > Also, I have a second question. Is it possible to extract that value > > derived from "RETURNING UserId" so that it can be used in a later query? > > > > Thank you for your time! > > Aaron > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 to create a record using a FK
On Sat, Mar 12, 2016 at 5:03 AM, dieter wrote:
> Aaron Christensen writes:
> > I am running the following versions of software:
> >
> > Python 3.5
> > psycopg2==2.6.1
> > Postgres 9.4.5
> >
> > I have 2 tables. Table User has UserId (serial PK), LastName, FirstName,
> > Gender, DateOfBirth, and DateEnrolled. Table UserProfile has
> UserProfileId
> > (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity. There
> is
> > a one-to-many relationship.
> >
> > The following PostgreSQL works and ultimately creates a record in
> > UserProfile with an associated UserId (FK).
> >
> > \set last_name '''Sara'''
> > \set first_name '''Jackson'''
> > \set gender '''F'''
> > \set dob '''1941-1-12'''
> > \set fave_number '''3'''
> > \set activity '''volleyball'''
> >
> >
> > WITH ins_user AS (
> > INSERT INTO User
> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled)
> > VALUES (:last_name, :first_name, :gender, :dob, now())
> > RETURNING UserId)
> > INSERT INTO UserProfile
> > (UserId, DateEntered, FaveNumber, Activity)
> > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity);
> >
> > How can I build a psycopg2 cur.execute query that will accomplish the
> above
> > PostgreSQL? I've read documentation but can't seem to get a handle on
> how
> > I should structure this command.
> >
> > My starting point is:
> >
> > cur.execute( \
> > """INSERT INTO User \
> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \
> > VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \
> > (last_name, first_name, gender, date_of_birth, now(), ??...??)
>
> You can add "returning UserId" to this SQL command to get back
> the id of the created user in your Python program. You can
> then use this "UserId" to create the row in your dependent table.
>
> I use it like this in one of my programs:
>
> cursor.execute("insert into service(...) "
> "values (...) returning id",
> (...)
> )
> id = cursor.fetchone()[0]
> cursor.execute(
> "insert into product(..., f_service_id) "
> "values (..., %s) returning id",
> (..., id)
> )
> id = cursor.fetchone()[0]
>
>
>
> Likely, there is also a way to bind the "UserId" inside SQL (maybe
> via "SET") and use it in a second "INSERT" in the same call
> to "cur.execute". Check the Postgres documentation for this.
>
>
> > Also, I have a second question. Is it possible to extract that value
> > derived from "RETURNING UserId" so that it can be used in a later query?
>
> Sure -- see above.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
Hi Dieter,
Thanks for the response. I managed to get it working and also combined it
with Peter's suggestion of passing a dictionary as an argument. However, I
am trying to figure out how I can get the RETURNING ID to be used in the
next cur.execute(). Here is what I have been working on but have found
many ways for it not to work. My latest response is that the tuple indices
must be integers or slices.
# Here I initialize the dictionaries. I will use each dictionary as an
input into each cur.execute()
user_input = dict(
last_name = 'Jackson',
first_name = 'Sara',
gender = 'F',
date_of_birth = '1941-1-12'
)
user_profile_input = dict(
fave_number = 3,
activity = 'volleyball'
)
# Create record in User // cur.execute(query, user_input)
cur.execute("""
INSERT INTO User
(LastName, FirstName, Gender, DateOfBirth)
VALUES (%(last_name)s, %(first_name)s, %(gender)s, %(date_of_birth))
RETURNING UserId""",
user_input)
conn.commit()
UserId = cur.fetchone()[0]#< -- HERE is the UserId
print("UserId = %s" % UserId)
# Create record in UserProfile // cur.execute(query, user_profile_input)
cur.execute("""
INSERT INTO UserProfile
(FaveNumber, Activity, UserId)
VALUES (%(fave_number)s, %(activity)s, %s)< I tried
following your format
RETURNING UserProfileId""",
(user_profile_input, UserId) # < This is what I'm trying
to figure out.. How to pass the UserId.
)
conn.commit()
UserProfileId = cur.fetchone()[0]
print("UserProfileId = %s" % UserProfileId)
--
https://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation in Python
On Saturday, March 12, 2016 at 10:45:43 AM UTC-6, Ian wrote: > On Fri, Mar 11, 2016 at 7:39 PM, Rick Johnson > wrote: > > At run-time, i don't care how large a "module namespace" may > > be. Sometimes a module namespace will be small, with only a > > few exposed symbols, but sometimes, a module namespace will > > expose thousands of symbols. > > Thousands, really? What system do you use to ensure that symbols don't > accidentally collide with one another? I use words, mostly. > > [SNIP: Two Examples From Rick] > > Is this meant to be equivalent to the other example? In that example, > you have the two "satellite" modules being imported between the > definition of FOO_SHARED_STATE and the print. In this example, how on > Earth is Python supposed to know that after the FOO_SHARED_STATE line > it's supposed to switch to executing these other two files before > continuing on to the next statement? The examples i provided are too simplistic to convey such details, because they were never meant to convey such details. Of *COURSE* you'll encounter issues in *real code* if you don't organize it's execution correctly. Spreading the contents of a module across multiple source files does not prevent NameErrors from raising, and it shouldn't. Heck, you can easily break a python module today, by referencing a symbol too early: ## # example.py # ## bar = Bar() class Bar(object): pass > > That sounds simple in theory, but it breaks down quickly > > when you create "professional sized programs" > > I find in interesting that you seem to be equating "very large" with > "professional". In my view, it's exactly the opposite. Anything very > large and so unorganized looks rather sloppy and unprofessional. I find it interesting that you seem to be equating "very large" with "sloppy". Not all large software's are sloppy, and not all small software's are neat. Dealing in absolutes is never a good idea. > >> 3) Just put all your damn shared state in a class. There's nothing > >> stopping you from spreading out your class method definitions over > >> multiple files if you really want to, and it solves your import issue > >> by allowing everything to be imported before you even begin to set up > >> the shared state. > > > > Your words tell me that you have not written anything substantial. > > I'll take your lack of substantive reply to mean "Huh, I never thought > of this approach that both solves my problem and improves the > structure of my code at the same time. Better hurl an insult to save > face!" Compliment accepted. Why must I invoke the awesome "instancing power" of a SharedStateObject, when a singleton would be a more appropriate design? But even a singleton would be a nasty hack for: sharing state across a "module space" that is defined in multiple source files. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pyhon 2.x or 3.x, which is faster?
On Sun, 13 Mar 2016 02:36 am, alister wrote about building up strings by repeated concatenation: > So you are bench marking python performance on a programming paradigm > that is not good python practice. > > A pointless exercise > what may be the best way to achieve a rsult in one language is not > necessarily the best way to achieve it in another. "Pointless"? I don't think so. The whole *point* is see how a straightforward and simple idiom compares between one implementation (or language) and another. There's nothing wrong with that, unless you think that the only code you should benchmark is code you think will be fast. If your aim is to *hide the fact* that vanilla Python performs poorly on repeated concatenation, then by all means this benchmark is a bad benchmark. You might also decide that it's a "bad" benchmark if you simply don't care about the results. But why would you do that? Repeated string concatenation is not only legal python code, but CPython actually includes special code to optimize that case. Using CPython 2.7 on Linux, contrast the time taken bythe optimized concatenation example: py> with Stopwatch(): ... s = '' ... for i in range(10**6): ... s = s + '%' ... time taken: 0.361933 seconds with an unoptimized variation in all it's quadratic horror: py> with Stopwatch(): ... s = '' ... for i in range(10**6): ... s = '%' + s ... time taken: 1075.427070 seconds (Source for Stopwatch available on request.) Somebody took the time and effort to optimize the first case, and you think it is "pointless" to see if it actually works? I disagree. What we conclude from the results of the benchmark is a separate issue from the results themselves. We might legitimate conclude "well don't write your code like that". We might conclude "this is a problem that needs fixing". We might punt, as indeed the Python core developers have done for this specific one, and decided that the language Python need not make any performance guarantees about repeated string concatenation, but implementations are allowed to optimize it or not, as they prefer. The speed of repeated string concatenation is a *quality of implementation* issue. Exactly the sort of thing which, arguable, ought to be benchmarked. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Encapsulation in Python
On Saturday, March 12, 2016 at 3:10:49 PM UTC-6, Chris Angelico wrote: > Also, if currentModule.py is pulling foo from modX, then modZ.py is an > implementation detail. You don't necessarily want to go straight > there; tracing the chain is more likely to be the correct behaviour. > Suppose modX.py actually looks like this: > > if 'posix' in some_magic: > import posixY as modY > elif 'nt' in some_magic: > import ntY as modY > else: > raise ImportError Ha, well, along with my example, this is another good example of a very bad design ;-). It's a terrible sin to import a symbol through multiple modules, but to import a symbol through multiple modules *AND* redefine it along the way, well, that should be punishable by the fatal exception. No judge, no jury. Just swift, and absolute, justice. :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 to create a record using a FK
On Sat, Mar 12, 2016 at 9:57 PM, Aaron Christensen < [email protected]> wrote: > > > On Sat, Mar 12, 2016 at 5:03 AM, dieter wrote: > >> Aaron Christensen writes: >> > I am running the following versions of software: >> > >> > Python 3.5 >> > psycopg2==2.6.1 >> > Postgres 9.4.5 >> > >> > I have 2 tables. Table User has UserId (serial PK), LastName, >> FirstName, >> > Gender, DateOfBirth, and DateEnrolled. Table UserProfile has >> UserProfileId >> > (serial, PK), UserId (FK), DateEntered, FaveNumber, and Activity. >> There is >> > a one-to-many relationship. >> > >> > The following PostgreSQL works and ultimately creates a record in >> > UserProfile with an associated UserId (FK). >> > >> > \set last_name '''Sara''' >> > \set first_name '''Jackson''' >> > \set gender '''F''' >> > \set dob '''1941-1-12''' >> > \set fave_number '''3''' >> > \set activity '''volleyball''' >> > >> > >> > WITH ins_user AS ( >> > INSERT INTO User >> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) >> > VALUES (:last_name, :first_name, :gender, :dob, now()) >> > RETURNING UserId) >> > INSERT INTO UserProfile >> > (UserId, DateEntered, FaveNumber, Activity) >> > VALUES ( (SELECT UserId FROM ins_user), now(), :fave_number :activity); >> > >> > How can I build a psycopg2 cur.execute query that will accomplish the >> above >> > PostgreSQL? I've read documentation but can't seem to get a handle on >> how >> > I should structure this command. >> > >> > My starting point is: >> > >> > cur.execute( \ >> > """INSERT INTO User \ >> > (LastName, FirstName, Gender, DateOfBirth, DateEnrolled) \ >> > VALUES (%s, %s, %s, %s, %s) RETURNING UserId;""", \ >> > (last_name, first_name, gender, date_of_birth, now(), ??...??) >> >> You can add "returning UserId" to this SQL command to get back >> the id of the created user in your Python program. You can >> then use this "UserId" to create the row in your dependent table. >> >> I use it like this in one of my programs: >> >> cursor.execute("insert into service(...) " >> "values (...) returning id", >> (...) >> ) >> id = cursor.fetchone()[0] >> cursor.execute( >> "insert into product(..., f_service_id) " >> "values (..., %s) returning id", >> (..., id) >> ) >> id = cursor.fetchone()[0] >> >> >> >> Likely, there is also a way to bind the "UserId" inside SQL (maybe >> via "SET") and use it in a second "INSERT" in the same call >> to "cur.execute". Check the Postgres documentation for this. >> >> >> > Also, I have a second question. Is it possible to extract that value >> > derived from "RETURNING UserId" so that it can be used in a later query? >> >> Sure -- see above. >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > > > Hi Dieter, > > Thanks for the response. I managed to get it working and also combined it > with Peter's suggestion of passing a dictionary as an argument. However, I > am trying to figure out how I can get the RETURNING ID to be used in the > next cur.execute(). Here is what I have been working on but have found > many ways for it not to work. My latest response is that the tuple indices > must be integers or slices. > > # Here I initialize the dictionaries. I will use each dictionary as an > input into each cur.execute() > user_input = dict( > last_name = 'Jackson', > first_name = 'Sara', > gender = 'F', > date_of_birth = '1941-1-12' > ) > > user_profile_input = dict( > fave_number = 3, > activity = 'volleyball' > ) > > > > # Create record in User // cur.execute(query, user_input) > cur.execute(""" > INSERT INTO User > (LastName, FirstName, Gender, DateOfBirth) > VALUES (%(last_name)s, %(first_name)s, %(gender)s, %(date_of_birth)) > RETURNING UserId""", > user_input) > conn.commit() > UserId = cur.fetchone()[0]#< -- HERE is the UserId > print("UserId = %s" % UserId) > > # Create record in UserProfile // cur.execute(query, user_profile_input) > cur.execute(""" > INSERT INTO UserProfile > (FaveNumber, Activity, UserId) > VALUES (%(fave_number)s, %(activity)s, %s)< I tried > following your format > RETURNING UserProfileId""", > (user_profile_input, UserId) # < This is what I'm trying > to figure out.. How to pass the UserId. > ) > conn.commit() > UserProfileId = cur.fetchone()[0] > print("UserProfileId = %s" % UserProfileId) > Dieter, I think I finally figured it out! In this section: UserId = cur.fetchone()[0]#< -- HERE is the UserId print("UserId = %s" % UserId) I added user_id to the dictionary and passed the dictionary to the cur.execute(): user_id = cur.fetchone()[0] user_profile_input['user_id'] = user_id cur.execute(""" INSERT INTO UserProfile (...) VALUES (...) RETURNING UserProfileId""", (user_profile_input) That seemed to have done the trick. However, I hope that the way my code is structured is a decent design with the multiple conn.commit()s. Should I need to close the cursor af
Re: Encapsulation in Python
On Sun, Mar 13, 2016 at 2:36 PM, Rick Johnson wrote: > On Saturday, March 12, 2016 at 3:10:49 PM UTC-6, Chris Angelico wrote: >> Also, if currentModule.py is pulling foo from modX, then modZ.py is an >> implementation detail. You don't necessarily want to go straight >> there; tracing the chain is more likely to be the correct behaviour. >> Suppose modX.py actually looks like this: >> >> if 'posix' in some_magic: >> import posixY as modY >> elif 'nt' in some_magic: >> import ntY as modY >> else: >> raise ImportError > > Ha, well, along with my example, this is another good > example of a very bad design ;-). It's a terrible sin to > import a symbol through multiple modules, but to import a > symbol through multiple modules *AND* redefine it along the > way, well, that should be punishable by the fatal exception. > No judge, no jury. Just swift, and absolute, justice. :-) Good. Go complain to the os module about how it's doing the wrong thing. This is exactly what it does to provide os.path. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)
On 3/12/2016 8:20 PM, Steven D'Aprano wrote: Yeah, we get it. The CPython interpreter is plenty fast enough for whatever you use it for. Good for you! Millions aren't so lucky. They are looking for something faster, because for them, you can never have too much speed. When you are trying to solve a 7000x7000 system of equations, you don't want to wait three days for a solution. Or three hours with Numpy. There is a positive form of Parkinson's Law at work. People patience is about the same as ever. So as computation gets faster, people attempt larger problems. I remember when the idea of solving such a problem was ludicrous. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Descriptors vs Property
Thomas 'PointedEars' Lahn wrote: >> I haven't read the descriptor protocol as yet. > > You should. You should also trim your quotations to the relevant > minimum, and post using your real name. > I don't take advice from people on USENET who DON'T have a long history of helping ME - unless I'm blatantly wrong to the point that someone might actually die :) but thanks anyhow I shall endeavor to oblige. -- https://mail.python.org/mailman/listinfo/python-list
