Re: How to streamingly read text file and display whenever updated text
On 07.10.2013 03:54, [email protected] wrote: https://docs.google.com/file/d/0B2D69u2pweEvelh1T25ra19oZEU/edit?usp=sharing For the readers who don't bother clicking on the link above: It's a short video where the OP demonstrates how her/his usage of tail doesn't work. no matter call tail directly in python or using the script of tail all failed it seems it can not read next line In your video you use gedit to write some file and "tail -f " to follow it. But "tail -f" will follow the file descriptor. Usually, editors like gedit won't save your changes to the original file but create a new temporary file and rename it later to the original file name after deleting the original one. Thus tail will follow an already deleted file. See also this blog post: http://tech.shantanugoel.com/2009/12/23/continuous-monitor-tail-fails.html For your example you will have to use "tail -F " which will follow the file name. Alternatively you could write a simple script to simulate a continously growing file like import time for i in range(1000): with open("test.txt", "a") as f: f.write(str(i) + '\n') time.sleep(1) which should work with "tail -f". Bye, Andreas -- https://mail.python.org/mailman/listinfo/python-list
Re: Goodbye: was JUST GOT HACKED
On Thu, 03 Oct 2013 11:35:00 +, Steven D'Aprano wrote: > On Thu, 03 Oct 2013 09:21:08 +0530, Ravi Sahni wrote: > >> On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry >> wrote: >>> Ding ding! Nikos is simply trolling. It's easy enough to killfile him >>> but inconvenient to skip all the answers to his lengthy threads. If >>> only people would just ignore him! >> >> Hello Walter Hurry please wait! >> >> Did I do/say something wrong?! > > Don't worry about it Ravi, you haven't done anything wrong. > > Walter is not a regular here. At best he is a lurker who neither asks > Python questions nor answers them. In the last four months, I can see > four posts from him: three are complaining about Nikos, and one is a > two- > line "Me to!" response to a post about defensive programming. > > > >> If one of us should go it should be me -- Im just a newbie here. > > No, you are welcome here. You've posted more in just a few days than > Walter has in months. We need more people like you. Steven, You make a fair point. I have posted very little recently, for the following reasons: a) I'm not really competent enough to answer python questions, at least not yet. b) I try not to post my own Python questions unless as a last resort. I prefer to try to solve my own problems by reading the fine documentation, and DuckDuckGoing. However, I do lurk assiduously and have learned much by reading excellent 'answering' posts from many such as you. The 'Goodbye' post was made in rather a fit of pique, for which I apologise. If I am allowed a second chance, there is actually something puzzling me at the moment. It's a UnicodeDecodeError, but I shall start a separate thread about it. Sorry again. Walter -- https://mail.python.org/mailman/listinfo/python-list
Re: Goodbye: was JUST GOT HACKED
On Mon, Oct 7, 2013 at 11:26 PM, Walter Hurry wrote: > The 'Goodbye' post was made in rather a fit of pique, for which I > apologise. If I am allowed a second chance, there is actually something > puzzling me at the moment. It's a UnicodeDecodeError, but I shall start > a separate thread about it. You're allowed a second chance, and a third, and a seventh, and a seventy-times-seventh (or a seventy-seventh, depending on which translation you read). Welcome back. And yes, starting a separate thread for the actual question is definitely the best way :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Goodbye: was JUST GOT HACKED
On 2013-10-07 12:26, Walter Hurry wrote: > The 'Goodbye' post was made in rather a fit of pique, for which I > apologise. If I am allowed a second chance, there is actually > something puzzling me at the moment. It's a UnicodeDecodeError, but > I shall start a separate thread about it. Indeed, the list eagerly welcomes folks who want to learn Python (not just have the answers spoon-fed to them **ehem**), so based on your "I prefer to try to solve my own problems by reading the fine documentation, and DuckDuckGoing," you're the ideal person we *want* here. You've already done the grunt work debugging/diagnosis, and leave the interesting problems for the mailing-list to have fun with. :-) So UnicodeDecodeErrors? Bring it on! :-) -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: Goodbye: was JUST GOT HACKED
On Mon, Oct 7, 2013 at 5:56 PM, Walter Hurry wrote: > On Thu, 03 Oct 2013 11:35:00 +, Steven D'Aprano wrote: > >> On Thu, 03 Oct 2013 09:21:08 +0530, Ravi Sahni wrote: >> >>> On Thu, Oct 3, 2013 at 2:43 AM, Walter Hurry >>> wrote: Ding ding! Nikos is simply trolling. It's easy enough to killfile him but inconvenient to skip all the answers to his lengthy threads. If only people would just ignore him! >>> >>> Hello Walter Hurry please wait! >>> >>> Did I do/say something wrong?! >> >> Don't worry about it Ravi, you haven't done anything wrong. >> >> Walter is not a regular here. At best he is a lurker who neither asks >> Python questions nor answers them. In the last four months, I can see >> four posts from him: three are complaining about Nikos, and one is a >> two- >> line "Me to!" response to a post about defensive programming. >> >> >> >>> If one of us should go it should be me -- Im just a newbie here. >> >> No, you are welcome here. You've posted more in just a few days than >> Walter has in months. We need more people like you. > > Steven, > > You make a fair point. I have posted very little recently, for the > following reasons: > > a) I'm not really competent enough to answer python questions, at least > not yet. > > b) I try not to post my own Python questions unless as a last resort. I > prefer to try to solve my own problems by reading the fine documentation, > and DuckDuckGoing. > > However, I do lurk assiduously and have learned much by reading excellent > 'answering' posts from many such as you. > > The 'Goodbye' post was made in rather a fit of pique, for which I > apologise. If I am allowed a second chance, there is actually something > puzzling me at the moment. It's a UnicodeDecodeError, but I shall start > a separate thread about it. > > Sorry again. Thanks! For changing decision! -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
> What makes Matplotlib so professional? > > Assuming that "professional" packages necessarily do the right thing is > an unsafe assumption. Many packages have *lousy* interfaces. Not that it's a complete explanation for matplotlib's interfaces, but it did start out as a Python-based replacement for MATLAB. I seem to recall that John Hunter started the project because the lab he worked in as a postdoc only had a single MATLAB license, so it wasn't always available when he needed it. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
esmaspäev, 7. oktoober 2013 4:27.44 UTC+3 kirjutas Piet van Oostrum: > [email protected] writes: > > > > > problem is : Traceback (most recent call last): > > > File "C:\Users\Marko\Desktop\hacker.org\XOR cypher.py", line 35, in > > > > > print("Key-" + str(võti) + ": " + str("".join(tulemus2))) > > > TypeError: sequence item 0: expected str instance, bytes found > > > > > > If i take away the join command i get this: > > > Key-: [b'u', b'o', b'\x00', b'\x1d', b' ', b'|', b'N', b'\x0f', > > b'9', b'j', b'K', b'J', b'&', b'#', b'A', b'K', b'5', b'k', b'_', b'\x1e', > > b',', b'j', b'\x0c', b'\x08', b'i', b'(', b'\x06', b'\\', b'r', b'3', > > b'\x1f', b'V', b's', b'9', b'\x1d'] > > > > > > the Key- is the key im using to decrypt the code. everything else > > is generated by the decrytion process and the unhexlify command. So my > > guess is, the join command cant handle the b"u" type of format. how can i > > get rid of the b. > > > > > > Or does anyone have a better idea how to translate HEX into ASCII and sort > > out the lines that make sense > > > > Why do you post the same question twice under different subjects? > > -- Because i was told so, cause the subject change so it cant be under there anymore. This is the code i came up with: from teisendaja import * from operator import * import binascii teisendus = teisendus() kood = input("Kood: ") key = input("Võti: ") chunksize = 2 vastus = [teisendus.teisendus3(16,2,kood[i: (i + chunksize)]) for i in range(0, len(kood),chunksize)] vastus = ["0"*(8-len(x)) + x for x in vastus] #key = teisendus.teisendus3(10,2,int(key)) getBin = lambda x, n: x >= 0 and str(bin(x))[2:].zfill(n) or "-" + str(bin(x))[3:].zfill(n) def dekrüpteeria(vastus, key): XOR = [] tulemus = [] for i in range(len(vastus)): for j in range(8): XOR.append(str(ixor(int(vastus[i][j]), int(key[j] tulemus.append("".join(XOR)) key = "".join(XOR) XOR = [] return tulemus tulemus2= [] if key == "": for i in range(256): võti = getBin(i,8) tulemus = [teisendus.teisendus3(2,16,i) for i in dekrüpteeria(vastus, võti)] tulemus = ["0"*(2-len(x)) + x for x in tulemus] # for j in range(len(tulemus)): tulemus2.append(binascii.unhexlify("".join(tulemus))) print("Key-" + str(võti) + ": " + str(tulemus2)) tulemus2 = [] #tulemus = [teisendus.teisendus3(2,16,i) for i in dekrüpteeria(vastus, key)] #print(":".join(tulemus)) #751a6f1d3d5c3241365321016c05620a7e5e34413246660461412e5a2e412c49254a24 Although this is quite ugly atm. But it serves me well, until i reach the unhexlify part. The number and lette r string at the wery end is the mesage im trying to decypher. -- https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. -- https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
On 07/10/2013 14:54, [email protected] wrote: I forgot to tell. The teisendaja module that i have imported, is a number converter that allow to convert numbers from one base to another. i mostly use it for HEX to BIN and vice versa, but it supports other bases too. That's nice to know, but what has it got to do with the market price of oranges in Timbuktu? Or to put it another way, you're forcing volunteers to go and find your original message as once again you don't quote any context. Please make life easier for everybody, including yourself, by quoting something from the original. Thanks in anticipation. -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: HEX to ASCII
[email protected] writes: > This is the code i came up with: > from teisendaja import * > from operator import * > import binascii > > teisendus = teisendus() > kood = input("Kood: ") > key = input("Võti: ") > > chunksize = 2 > vastus = [teisendus.teisendus3(16,2,kood[i: (i + chunksize)]) for i in > range(0, len(kood),chunksize)] > vastus = ["0"*(8-len(x)) + x for x in vastus] > #key = teisendus.teisendus3(10,2,int(key)) > getBin = lambda x, n: x >= 0 and str(bin(x))[2:].zfill(n) or "-" + > str(bin(x))[3:].zfill(n) Instead of boolean and expr1 or expr2 in current Python you can better write: expr1 if boolean else expr2. and I think using def getBin(x, n):would be more clear. But I have another observation: You use getBin only for positive ints, so the whole '-' case isn't necessary. Actually it would be damaging, as the rest of the code assumes that the key that results from getBin is 8 characters long, whereas in the negative case it would be 9 characters. Also you use int(key[j]) and if key[0] == '-' this would give an error. If you just want to get 8 binary digits for an int using format would be simpler: getBin = lambda x, n: '{0:={1}b}'.format(x, n) or getBin = lambda x, n: '{0:=0{1}b}'.format(x, n) if you want also negatives (but this would give you 7 or eight binary digits, not always 8.) > def dekrüpteeria(vastus, key): > XOR = [] > tulemus = [] > for i in range(len(vastus)): > for j in range(8): > XOR.append(str(ixor(int(vastus[i][j]), int(key[j] > tulemus.append("".join(XOR)) > key = "".join(XOR) > XOR = [] > return tulemus You can use list comprehension: def dekrüpteeria(vastus, key): tulemus = [] for i in range(len(vastus)): XOR = [(str(ixor(int(vastus[i][j]), int(key[j] for j in range(8)] tulemus.append("".join(XOR)) key = "".join(XOR) return tulemus and then because you only use "".join(XOR), not XOR itself: def dekrüpteeria(vastus, key): tulemus = [] for i in range(len(vastus)): XOR = "".join([(str(ixor(int(vastus[i][j]), int(key[j] for j in range(8)]) tulemus.append(XOR)) key = XOR return tulemus and then you could rewrite this also to use a list comprehension for tulemus, but that may make it in a too big one liner. Also note that you always use int() on the elements of key and vastus, so it might be simpler to store these as int arrays (lists) instead of strings > > tulemus2= [] > if key == "": > for i in range(256): > võti = getBin(i,8) > tulemus = [teisendus.teisendus3(2,16,i) for i in dekrüpteeria(vastus, > võti)] > tulemus = ["0"*(2-len(x)) + x for x in tulemus] Look at the zfill method for the above 2 line Probably tulemus = [teisendus.teisendus3(2,16,i).zfill(2) for i in dekrüpteeria(vastus, võti)] will do the same ># for j in range(len(tulemus)): > tulemus2.append(binascii.unhexlify("".join(tulemus))) > print("Key-" + str(võti) + ": " + str(tulemus2)) > tulemus2 = [] > #tulemus = [teisendus.teisendus3(2,16,i) for i in dekrüpteeria(vastus, key)] > #print(":".join(tulemus)) > > #751a6f1d3d5c3241365321016c05620a7e5e34413246660461412e5a2e412c49254a24 > > Although this is quite ugly atm. But it serves me well, until i reach the > unhexlify part. The number and lette r string at the wery end is the mesage > im trying to decypher. The result of unhexlify is a byte string. So tulemus2 is a list of byte strings, which you can joint with xxx = b''.join(tulemus2), and then you have another byte string. If you are sure this is ASCII (which means all bytes are < 128), the you can convert it to a string with str(xxx, 'ascii') or xxx.decode('ascii'). If there are bytes > 127 then you have to know which encoding it is to be able to make a string out of it. Is this some known encryption method? If so why not use a standard solution for it? If it is a home brew encryption: are you sure it is safe? Most home brew solutions in encryption are not. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
Thanks, everyone, for your replies. Perhaps I have complicated things unnecessarily? I was just trying to do some error-checking on the arguments supplied to the class constructor. Perhaps Python already implements automatically what I am trying to accomplish manually? I'll tinker around with some minimal code, try to provoke some errors, and see what I get. Here is one more detail which may be relevant. The base class for the family of classes I am developing is a numpy.ndarray. The numpy.ndarray is a C extension type (and if I understand correctly, that means it is immutable by ordinary Python methods). Subclassing ndarray can get a bit complicated (see http://docs.scipy.org/doc/numpy/user/basics.subclassing.html). The ndarray.__init__ method is inaccessible, instead one overrides ndarray.__new__. Making further subclasses of a subclassed numpy.ndarray, each of which may have their own arguments, is what I am trying to accomplish while adhering to the "DRY" principle. -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
On Monday, October 7, 2013 9:26:51 AM UTC-7, I wrote: > Here is one more detail which may be relevant. The base class for the family > of classes I am developing is a numpy.ndarray. The numpy.ndarray is a C > extension type (and if I understand correctly, that means it is immutable by > ordinary Python methods). Subclassing ndarray can get a bit complicated (see > http://docs.scipy.org/doc/numpy/user/basics.subclassing.html). I've just been reading the above page, which is pretty new. It supersedes a now-defunct page, http://www.scipy.org/Subclasses. The unusual subclassing needs of an ndarray apparently arise, not from the fact that an ndarray is a C extension type, but because of numpy's special view casting and slicing requirements. While I don't believe this is highly relevant to the args/kwargs issues I have here, I thought that I should correct my earlier remark. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Terry Reedy writes:
> On 10/4/2013 5:49 AM, Alain Ketterlin wrote:
>
>> I think allowing rebinding of function names is extremely strange,
>
> Steven already countered the 'is extremely strange' part by showing
> that such rebinding is common, generally useful, and only occasionally
> dodgy and a candidate for being blocked.
I was talking about rebinding a "function name", not about rebinding a
name to a function. Steve's message was pretty clear on this: iirc, his
first two cases were "binding a new name to an existing callable", the
last case (rebinding len) was in line with what I meant (except his code
"saved" the original function).
My example was: the code of "fact" contains a call to "fact", which is
not guaranteed to be bound to the function it appears in. And this
prevents any kind of optimization.
> I want to consider here what it would mean to concretely implement the
> abstract notion 'disallow rebinding of function names' and show what
> would be behind calling the idea 'not feasible'.
Again, I'm more concerned about the function than about the name.
And the fact that "disallow rebinding of function names" is not feasible
in Python is a design choice, not an essential characteristic of every
programming language.
That's fine. My point was: you can't at the same time have full
dynamicity *and* procedural optimizations (like tail call opt).
Everybody should be clear about the trade-off.
> 1. A 'name' is not a 'function name' unless the name is bound, at
> runtime, to a 'function'.
>
> 2. A 'function' in Python is not just one class but any callable
> object -- with with a __call__ methods. That comprises an unbounded
> set.
Right. Then when you do:
def myfunc(...): ...
myfunc is bound to an callable object. In my example, I was doing the
equivalent to rebinding myfunc, losing the last reference to that piece
of code:
myfunc = somethingelse
BTW, does the original callable object have a ref counter? Is it garbage
collected in that case? If not, would it be considered a bug?
> 3. Python does not mandate how namespaces are implemented. CPython
> uses both dicts and, for function local namespaces, internal C arrays.
> So 'names' in code can become either string keys for dicts or integer
> indexes for arrays.
Well, yes, but that's an implementation detail, no?
> 4. Rebinding can be explicit ('='), implicit ('import', 'class',
> def'), *or hidden* (globals('a') = 1; ob.__dict__('a') = 1). The
> hidden' methods are intentional as they are sometimes needed*. In
> CPython, these forms remain different in the byte code, but it could
> be otherwise. The point is that is may or may not be possible for the
> interpreter to even recognize a 'rebinding' in order to apply any
> rebinding blocking rule.
Sure (that's exactly why I said it is easier to implement). If you were
to disallow rebinding of global function names, you would need a proper
notion of global scope and various categories of names, something almost
all compiled languages have.
> * There is no trick (that I know of) for hidden rebinding of function
> locals and nonlocals (short of using ctypes), but I am not sure that
> this is a language guarantee. However, I an sure that the absence is
> intentional.
>
>> even though it's easier to implement.
>
> No kidding.
(See above).
-- Alain.
--
https://mail.python.org/mailman/listinfo/python-list
Mysql's mysql module
I just noticed this: http://dev.mysql.com/doc/connector-python/en/index.html What are the thoughts on this vs. the MySQLdb with which I'm familiar? I also noticed MySQLdb2. I was wondering whether to use this version on a new project that is likely to take some months to develop. Thanks, Tobiah -- https://mail.python.org/mailman/listinfo/python-list
Re: Mysql's mysql module
On Mon, Oct 7, 2013 at 1:08 PM, Tobiah wrote: > I just noticed this: > > > http://dev.mysql.com/doc/connector-python/en/index.html * Does it adhere to the Python database API? http://www.python.org/dev/peps/pep-0249/ * Is source available? * Does it have a reasonable open source license? These questions come immediately to mind because at work we briefly considered, then rejected, a similar offering from the Sybase folks for connecting to (big surprise) Sybase. We never needed to ask the first and third questions, because the answer to the second was, "no", and they only offered a version built against Python 2.6. Since we use Python 2.4 and 2.7, that was an immediate nonstarter. Skip -- https://mail.python.org/mailman/listinfo/python-list
Newbie: installation difficulties [webapp2 / babel]
Hi,
I'm following webapp2 documentation (release 2.1).
I made a mistake in following the text.
I typed "pip install babel" and this led to errors in the installation.
As that user is not in sudo list, I changed users, typed "sudo pip install
babel" and everything seemed right.
Further on, the manual says "$ pybabel " and when I use that command, the
response is "-bash: pybabel: command not found" (in both user environments).
I'd like to solve this and learn at the same time ;)
I'm on OS X 10.8.5, python 2.7.5.
I found a "babel" in /Library/Python/2.7/site-packages.
I found a "babel" in the response to "python -c "help('modules')" | grep babel".
>From which I conclude that babel is on the machine and is known to python.
However, I gather from the webapp2 documentation that babel should also be
known to the shell.
I don't yet know how to check this, nor how to repair.
find /usr -name '*babel*'
and
find /bin -name '*babel*'
return no values, but
find /Library -name '*babel*'
does (as expected).
I've tried to find similar situations in documentation, forums, Google but
found nothing helpful.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Newbie: installation difficulties [webapp2 / babel]
On Mon, Oct 7, 2013 at 3:12 PM, BobAalsma wrote:
> Hi,
>
> I'm following webapp2 documentation (release 2.1).
>
> I made a mistake in following the text.
> I typed "pip install babel" and this led to errors in the installation.
> As that user is not in sudo list, I changed users, typed "sudo pip install
> babel" and everything seemed right.
>
> Further on, the manual says "$ pybabel " and when I use that command, the
> response is "-bash: pybabel: command not found" (in both user environments).
>
> I'd like to solve this and learn at the same time ;)
>
> I'm on OS X 10.8.5, python 2.7.5.
>
> I found a "babel" in /Library/Python/2.7/site-packages.
> I found a "babel" in the response to "python -c "help('modules')" | grep
> babel".
> From which I conclude that babel is on the machine and is known to python.
>
> However, I gather from the webapp2 documentation that babel should also be
> known to the shell.
> I don't yet know how to check this, nor how to repair.
>
> find /usr -name '*babel*'
> and
> find /bin -name '*babel*'
> return no values, but
> find /Library -name '*babel*'
> does (as expected).
>
> I've tried to find similar situations in documentation, forums, Google but
> found nothing helpful.
> --
> https://mail.python.org/mailman/listinfo/python-list
If you can run $python babel then I think you need to set babel to be
executable to run it without invoking python first.
--
Joel Goldstick
http://joelgoldstick.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Mysql's mysql module
Skip Montanaro wrote: > On Mon, Oct 7, 2013 at 1:08 PM, Tobiah wrote: >> I just noticed this: >> >> >> http://dev.mysql.com/doc/connector-python/en/index.html > > * Does it adhere to the Python database API? > http://www.python.org/dev/peps/pep-0249/ > > * Is source available? > > * Does it have a reasonable open source license? > > These questions come immediately to mind because at work we briefly > considered, then rejected, a similar offering from the Sybase folks > for connecting to (big surprise) Sybase. We never needed to ask the > first and third questions, because the answer to the second was, "no", > and they only offered a version built against Python 2.6. Since we use > Python 2.4 and 2.7, that was an immediate nonstarter. > > Skip Based on a quick look at the link given, I think the answers to questions 1 and 3 are yes and no respectively. No idea about #2. -- Duncan Booth http://kupuguy.blogspot.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Mysql's mysql module
On Oct 7, 2013 9:36 PM, "Duncan Booth" wrote: > > Skip Montanaro wrote: > > > On Mon, Oct 7, 2013 at 1:08 PM, Tobiah wrote: > >> I just noticed this: > >> > >> > >> http://dev.mysql.com/doc/connector-python/en/index.html > > > > * Does it adhere to the Python database API? > > http://www.python.org/dev/peps/pep-0249/ > > > > * Is source available? > > > > * Does it have a reasonable open source license? > > > > These questions come immediately to mind because at work we briefly > > considered, then rejected, a similar offering from the Sybase folks > > for connecting to (big surprise) Sybase. We never needed to ask the > > first and third questions, because the answer to the second was, "no", > > and they only offered a version built against Python 2.6. Since we use > > Python 2.4 and 2.7, that was an immediate nonstarter. > > > > Skip > > Based on a quick look at the link given, I think the answers to questions 1 > and 3 are yes and no respectively. No idea about #2. Number two is "yes", unless there are deceptive statements on that page. > > -- > Duncan Booth http://kupuguy.blogspot.com > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Newbie: installation difficulties [webapp2 / babel]
Well Joel, umm, I'm not sure if I understand you correctly. $ python babel /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'babel': [Errno 2] No such file or directory And $ python Python 2.7.5 (v2.7.5:ab05e7dd2788, May 13 2013, 13:18:45) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import babel >>> Does this help? Bob -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Op 07-10-13 19:15, Alain Ketterlin schreef: I want to consider here what it would mean to concretely implement the abstract notion 'disallow rebinding of function names' and show what would be behind calling the idea 'not feasible'. Again, I'm more concerned about the function than about the name. And the fact that "disallow rebinding of function names" is not feasible in Python is a design choice, not an essential characteristic of every programming language. That's fine. My point was: you can't at the same time have full dynamicity *and* procedural optimizations (like tail call opt). Everybody should be clear about the trade-off. Your wrong. Full dynamics is not in contradiction with tail call optimisation. Scheme has already done it for years. You can rebind names to other functions in scheme and scheme still has working tail call optimisatiosn. -- Antoon Pardon -- https://mail.python.org/mailman/listinfo/python-list
Re: Newbie: installation difficulties [webapp2 / babel]
On Mon, Oct 7, 2013 at 4:00 PM, BobAalsma wrote: > Well Joel, umm, I'm not sure if I understand you correctly. > > $ python babel > /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: > can't open file 'babel': [Errno 2] No such file or directory > > And > > > $ python > Python 2.7.5 (v2.7.5:ab05e7dd2788, May 13 2013, 13:18:45) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. import babel > > Does this help? > > Bob > -- > https://mail.python.org/mailman/listinfo/python-list Bob, I'm not a pro at getting python stuff to run on Apple, but you might learn something here: http://docs.python.org/2/using/mac.html#using-python-on-a-macintosh >From the looks of it, babel doesn't seem to be in a directory where the mac looks for executables. You may want to cd yourself over to where babel is and try to run it. If that works, you have to figure out how to get babel in your executable path. I can't help you there. -- Joel Goldstick http://joelgoldstick.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On 10/7/2013 1:15 PM, Alain Ketterlin wrote: Terry Reedy writes: 3. Python does not mandate how namespaces are implemented. CPython uses both dicts and, for function local namespaces, internal C arrays. So 'names' in code can become either string keys for dicts or integer indexes for arrays. Well, yes, but that's an implementation detail, no? That is why I switched from 'Python' to 'CPython'. But I note the following: in 2.x, 'from mod import *' in a function meant that the compile time mapping of name to index could not be used and that a fallback to dict was necessary. So another implementation might take the easier path and always use a dict for function locals. In 3.x, such import are limited to module scope so that functions can always use an array and indexes for function locals. So other implementations can take the hint and do the same without needing a dict fallback. In other words, 3.x changed the language to facilitate the implementation detail. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
On Saturday, October 5, 2013 9:04:25 PM UTC-7, John Ladasky wrote: > Hi folks, > > > > I'm trying to make some of Python class definitions behave like the ones I > find in professional packages, such as Matplotlib. A Matplotlib class can > often have a very large number of arguments -- some of which may be optional, > some of which will assume default values if the user does not override them, > etc. > > > > I have working code which does this kind of thing. I define required > arguments and their default values as a class attribute, in an OrderedDict, > so that I can match up defaults, in order, with *args. I'm using > set.issuperset() to see if an argument passed in **kwargs conflicts with one > which was passed in *args. I use set.isdisjoint() to look for arguments in > **kwargs which are not expected by the class definition, raising an error if > such arguments are found. > > > > Even though my code works, I'm finding it to be a bit clunky. And now, I'm > writing a new class which has subclasses, and so actually keeps the "extra" > kwargs instead of raising an error... This is causing me to re-evaluate my > original code. > > > > It also leads me to ask: is there a CLEAN and BROADLY-APPLICABLE way for > handling the *args/**kwargs/default values shuffle that I can study? Or is > this sort of thing too idiosyncratic for there to be a general method? > > > > Thanks for any pointers! "Subclassing ndarray can get a bit complicated" Another software pattern idea is "encapsulate don't inherit". When a class is really messy to subclass, start fresh with a new class that wraps the messy class. Create redirect methods for whatever is needed, then subclass from the class you created. In fact, I'd go so far as to say you should only subclass from classes that were designed with subclassing in mind. If you find yourself bending over backwards to make subclassing work, it means you should be wrapping and redirecting instead. This is perhaps more true in C#/Java than Python, but still something to think about. -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
On Mon, 07 Oct 2013 09:26:51 -0700, John Ladasky wrote:
> Thanks, everyone, for your replies. Perhaps I have complicated things
> unnecessarily? I was just trying to do some error-checking on the
> arguments supplied to the class constructor. Perhaps Python already
> implements automatically what I am trying to accomplish manually? I'll
> tinker around with some minimal code, try to provoke some errors, and
> see what I get.
It's really hard to make definitive judgements without actually seeing
your code and understanding your use-case. I can only suggest that, you
*may* be complicating things unnecessarily. On the other hand, there's
always the chance that your requirements are sufficiently unusual that
you have done exactly what needs to be done.
But I suspect even in this case, there may be a more elegant way to solve
the problem of "I'm finding it to be a bit clunky", to quote your
original post. Clunky code can sometimes be smoothed out by refactoring
the complexity by use of decorators. Can you post an example of your code?
One thought -- often, people turn to subclassing as the only tool in
their toolbox. Have you considered that it may be easier/better to work
with delegation and composition instead?
> Here is one more detail which may be relevant. The base class for the
> family of classes I am developing is a numpy.ndarray. The numpy.ndarray
> is a C extension type (and if I understand correctly, that means it is
> immutable by ordinary Python methods). Subclassing ndarray can get a
> bit complicated (see
> http://docs.scipy.org/doc/numpy/user/basics.subclassing.html). The
> ndarray.__init__ method is inaccessible, instead one overrides
> ndarray.__new__.
Don't forget ndarray.__array_finalize__, __array_wrap__ and
__array_prepare__.
I am not an expert on numpy, but reading that page just makes me think
they're doing it all wrong, adding far too much complication. (I've
written code like that myself, but thank goodness I've had the sense to
throw it away and start again...). I'm trying to give them the benefit of
the doubt, but I've never liked the amount of DWIM cleverness in numpy,
and I think they would have been *much* better off having a clean
separation between the three ways of creating an array:
- the normal Python __new__ and __init__ mechanism
- creating a view into an array
- templating
instead of conflating the three into a single mechanism. I suspect that
the fundamental confusion comes about because numpy doesn't have a clean
distinction between views into an array, and actual arrays. Although I
must admit I've not done more than dip my toe into numpy, so you should
take my criticisms with a generous pinch of salt.
> Making further subclasses of a subclassed numpy.ndarray, each of which
> may have their own arguments, is what I am trying to accomplish while
> adhering to the "DRY" principle.
The usual way of doing this is to accept only keyword arguments for any
additional args:
class Base:
def __new__(cls, doc, grumpy, happy, sleepy, bashful, sneezy, dopey):
...
class Subclass(Base):
def __new__(cls, *args, **kwargs):
# grab the additional arguments
sneaky = kwargs.pop('sneaky', True) # optional
grabby = kwargs.pop('grabby') # mandatory
touchy = kwargs.pop('touchy')
feely = kwargs.pop('feely')
instance = super(Subclass, cls).__new__(cls, *args, **kwargs)
# process additional arguments
instance.apply_extras(sneaky, grabby, touchy, feely)
return instance
# In Python 3, I can do this to make it even cleaner:
class Subclass(Base):
def __new__(cls, *args, sneaky=True, grabby, touchy, feely, **kwargs):
instance = super(Subclass, cls).__new__(cls, *args, **kwargs)
# process additional arguments
instance.apply_extras(sneaky, grabby, touchy, feely)
return instance
In general, you should aim to use either __new__ or __init__ but not
both, although that's not a hard law, just a guideline.
Can you adapt this pattern to ndarray?
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On Mon, Oct 7, 2013, at 13:15, Alain Ketterlin wrote: > That's fine. My point was: you can't at the same time have full > dynamicity *and* procedural optimizations (like tail call opt). > Everybody should be clear about the trade-off. Let's be clear about what optimizations we are talking about. Tail call optimization, itself, doesn't care _what_ is being called. It can just as easily mean "erase its own stack frame and replace it with that of another function" as "reassign the arguments and jump to the top of this function". Some people have introduced the idea of _further_ optimizations, transforming "near" tail recursion (i.e. return self()+1) into tail recursion, and _that_ depends on knowing the identity of the function (though arguably that could be accounted for at the cost of including dead code for the path that assumes it may have been changed), but tail call optimization itself does not. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On Sat, Oct 5, 2013, at 3:39, Antoon Pardon wrote: > What does this mean? > > Does it mean that a naive implementation would arbitrarily mess up > stack traces and he wasn't interested in investigating more > sophisticated implementations? > > Does it mean he just didn't like the idea a stack trace wouldn't be a > 100% represenatation of the active call history? > > Does it mean he investigated more sphisticated implementations but found > them to have serious short comings that couldn't be remedied easily? The entire point of tail call optimization requires not keeping the intervening stack frames around, in _any_ form, so as to allow arbitrarily deep recursion without ever having the possibility of a stack overflow. An implementation which reduced but did not eliminate the space used per call would not be worthwhile because it would not enable the recursive functional programming patterns that mandatory tail call optimization allows. You could require that an "optimizable tail call" be made explicit. Actually, I suspect it might be possible to do this now, by abusing exception handling as a control flow mechanism. -- https://mail.python.org/mailman/listinfo/python-list
Re: Newbie: installation difficulties [webapp2 / babel]
On Tue, Oct 8, 2013 at 7:00 AM, BobAalsma wrote: > Well Joel, umm, I'm not sure if I understand you correctly. > > $ python babel > /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: > can't open file 'babel': [Errno 2] No such file or directory If the file's called babel.py, you have to invoke it that way - Python won't add an implicit file extension. Not sure if that helps your problem or not - I don't know babel, nor Mac OS. Try: $ python babel.py ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On 07/10/2013 18:57, Antoon Pardon wrote: Op 07-10-13 19:15, Alain Ketterlin schreef: I want to consider here what it would mean to concretely implement the abstract notion 'disallow rebinding of function names' and show what would be behind calling the idea 'not feasible'. Again, I'm more concerned about the function than about the name. And the fact that "disallow rebinding of function names" is not feasible in Python is a design choice, not an essential characteristic of every programming language. That's fine. My point was: you can't at the same time have full dynamicity *and* procedural optimizations (like tail call opt). Everybody should be clear about the trade-off. Your wrong. Full dynamics is not in contradiction with tail call optimisation. Scheme has already done it for years. You can rebind names to other functions in scheme and scheme still has working tail call optimisatiosn. Consider this code: def fact(n, acc=1): if n <= 1: return acc return fact(n-1, n*acc) It compiles to this: >>> dis.dis(fact) 2 0 LOAD_FAST0 (n) 3 LOAD_CONST 1 (1) 6 COMPARE_OP 1 (<=) 9 POP_JUMP_IF_FALSE 16 3 12 LOAD_FAST1 (acc) 15 RETURN_VALUE 4 >> 16 LOAD_GLOBAL 0 (fact) 19 LOAD_FAST0 (n) 22 LOAD_CONST 1 (1) 25 BINARY_SUBTRACT 26 LOAD_FAST0 (n) 29 LOAD_FAST1 (acc) 32 BINARY_MULTIPLY 33 CALL_FUNCTION2 (2 positional, 0 keyword pair) 36 RETURN_VALUE I think that CALL_FUNCTION immediately followed by RETURN_VALUE could be tail-call optimised by dropping the caller's stack frame provided that (like in this case) the callee doesn't refer to any name in the callers stack frame (nonlocal). You could also consider replacing the caller's stack frame with a smaller pseudo-frame, perhaps compressing multiple pseudo-frames or repeated sequences of pseudo-frames into a single pseudo-frame, so that it could still generate the same traceback as before (or an compressed traceback like what was discussed on python-ideas in the threads "Compressing excepthook output" and "Idea: Compressing the stack on the fly"). -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable arguments (*args, **kwargs): seeking elegance
Wow, Steven, that was a great, detailed reply. I hope you will forgive me for
shortcutting to the end, because I've been hacking away for a few hours and
came to this very conclusion:
On Monday, October 7, 2013 2:13:10 PM UTC-7, Steven D'Aprano wrote:
> In general, you should aim to use either __new__ or __init__ but not
> both, although that's not a hard law, just a guideline.
My problems were solved by adhering to using only __new__ in my ndarray
subclasses, and avoiding __init__. (If I used both methods, my arguments were
passed to the object twice, once through each method. That's weird! It messed
me up! And I'm not sure what purpose it serves.) The __new__ methods of my
subclasses now call super().__new__ to handle the attributes and error checking
which are common to all the classes, then handle the subclass-specific
variables.
One wrinkle that I had to comprehend was that super().__new__ would be
returning me a half-baked object on which I had to do more work. I'm used to
__init__, of course, which works on self.
OK, as for some other points:
> Don't forget ndarray.__array_finalize__, __array_wrap__ and
> __array_prepare__.
I handle __array_finalize__ in my base class. Also __reduce_ex__ and
__setstate__, so that I can pickle and unpickle my array objects (which is
necessary for multiprocessing work). I haven't had time to deal with
__array_wrap__ or __array_prepare__ yet, but so far my downstream code is
working without these methods (crossing fingers).
> I am not an expert on numpy, but reading that page just makes me think
> they're doing it all wrong, adding far too much complication. (I've
> written code like that myself, but thank goodness I've had the sense to
> throw it away and start again...). I'm trying to give them the benefit of
> the doubt, but I've never liked the amount of DWIM cleverness in numpy,
> and I think they would have been *much* better off having a clean
> separation between the three ways of creating an array:
>
> - the normal Python __new__ and __init__ mechanism
> - creating a view into an array
> - templating
>
> instead of conflating the three into a single mechanism.
I agree, I always find it complicated to wrap my head around these
complexities. But I simply can't live without numpy!
And finally:
> sneaky = kwargs.pop('sneaky', True) # optional
I don't know whether to be excited or embarrassed that I can still learn things
about the basics of Python... I've never used the optional argument of
dict.pop(). Cool! Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Alain Ketterlin writes: > BTW, does the original callable object have a ref counter? Is it garbage > collected in that case? If not, would it be considered a bug? In CPython ALL objects have ref counters. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
>> That's fine. My point was: you can't at the same time have full >> dynamicity *and* procedural optimizations (like tail call opt). >> Everybody should be clear about the trade-off. > > Your wrong. Full dynamics is not in contradiction with tail call > optimisation. Scheme has already done it for years. You can rebind > names to other functions in scheme and scheme still has working > tail call optimisatiosn. Yeah, and this is where two models of computation have been conflated, creating magical effects, confusing everybody. I challenge you to get down to the machine code in scheme and formally describe how it's doing both. -- MarkJ Tacoma, Washington -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On Mon, 07 Oct 2013 15:47:26 -0700, Mark Janssen wrote: > I challenge you to get > down to the machine code in scheme and formally describe how it's doing > both. For which machine? Or are you assuming that there's only one machine code that runs on all computing devices? Frankly, asking somebody to *formally* describe a machine code implementation strikes me as confused. Normally formal descriptions are given in terms of abstract operations, often high level operations, sometimes *very* high level, and rarely in terms of low-level "flip this bit, copy this byte" machine code operations. I'm not sure how one would be expected to generate a formal description of a machine code implementation. But even putting that aside, even if somebody wrote such a description, it would be reductionism gone mad. What possible light on the problem would be shined by a long, long list of machine code operations, even if written using assembly mnemonics? Far more useful would be a high-level description of Scheme's programming model. If names can be rebound on the fly, how does Scheme even tell whether something is a recursive call or not? def foo(arg): do stuff here foo(arg-1) # how does Scheme know that this is the same foo? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
> Only that you've got a consistent, stable (and therefore, > formalizable) translation from your language to the machine. That's > all. Everything else is magic. Do you know that the Warren > Abstraction Engine used to power the predicate logic in Prolog into > machien code for a VonNeumann machine is so complex, no one has > understood it (or perhaps even verified it)? Sorry, I mean the Warren Abstraction Machine (or WAM). I refer you to www.cvc.uab.es/shared/teach/a25002/wambook.pdf. Now, one can easily argue that I've gone too far to say "no one has understood it" (obviously), so it's very little tongue-in-cheek, but really, when one tries to pretend that one model of computation can be substituted for another (arguing *for* the Church-Turing thesis), they are getting into troubling territory (it is only a thesis, remember) -- MarkJ Tacoma, Washington -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On Mon, Oct 7, 2013 at 4:50 PM, Steven D'Aprano wrote: > On Mon, 07 Oct 2013 15:47:26 -0700, Mark Janssen wrote: >> I challenge you to get >> down to the machine code in scheme and formally describe how it's doing >> both. > > For which machine? Right, I should stop assuming a modern implementation of vonNeumann architecture (even though that, too, is ambiguous) since I'm talking about theory, but yet it is relevant. My demarcation point for arguments between "the scheme way" and other procedural languages (which, apart from Pascal variants, I blithely all "the C way") gets down to differing models of computation which shouldn't get conflated, even though everyone thinks and lumps it all as "computation". They simply can't get *practically* translated between one and the other, even though they are *theoretically* translated between each other all the time. Humans, of course know how to translate, but that doesn't count from the pov of computer *science*. > Frankly, asking somebody to *formally* describe a machine code > implementation strikes me as confused. Normally formal descriptions are > given in terms of abstract operations, often high level operations, > sometimes *very* high level, and rarely in terms of low-level "flip this > bit, copy this byte" machine code operations. I'm not sure how one would > be expected to generate a formal description of a machine code > implementation. It's like this: there *should* be one-to-one mappings between the various high-level constructs to the machine code, varying only between different chips (that is the purpose of the compiler after all), yet for some operations, in languages like scheme, well... I cannot say what happens... hence my challenge. > But even putting that aside, even if somebody wrote such a description, > it would be reductionism gone mad. What possible light on the problem > would be shined by a long, long list of machine code operations, even if > written using assembly mnemonics? Only that you've got a consistent, stable (and therefore, formalizable) translation from your language to the machine. That's all. Everything else is magic. Do you know that the Warren Abstraction Engine used to power the predicate logic in Prolog into machien code for a VonNeumann machine is so complex, no one has understood it (or perhaps even verified it)? One hardly knows where these things originate. But here it gets into dark arts best not entered into too deeply. It will turn you mad, like that guy in the movie "pi". -- MarkJ Tacoma, Washington -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
On Mon, 07 Oct 2013 17:16:35 -0700, Mark Janssen wrote:
> It's like this: there *should* be one-to-one mappings between the
> various high-level constructs to the machine code, varying only between
> different chips (that is the purpose of the compiler after all), yet for
> some operations, in languages like scheme, well... I cannot say what
> happens... hence my challenge.
>
>> But even putting that aside, even if somebody wrote such a description,
>> it would be reductionism gone mad. What possible light on the problem
>> would be shined by a long, long list of machine code operations, even
>> if written using assembly mnemonics?
>
> Only that you've got a consistent, stable (and therefore, formalizable)
> translation from your language to the machine.
You are mistaken to think that there is a single, one-to-one, mapping
between high-level code and machine code.
In practice, only if you use the exact same source code, the exact same
compiler, the exact same version of that compiler, with the exact same
compiler options, and the exact same version of the language and all its
libraries, then and only then will you will get the same machine code.
And even that is not guaranteed.
Take, for example, the single high-level operation:
sort(alist)
What machine code will be executed? Obviously that will depend on the
sort algorithm used. There are *dozens*. Here are just a few:
http://en.wikipedia.org/wiki/Category:Sorting_algorithms
Now sorting is pretty high level, but the same principle applies to even
simple operations like "multiply two numbers". There are often multiple
algorithms for performing the operation, and even a single algorithm can
often be implemented in slightly different ways. Expecting all compilers
to generate the same machine code is simply naive.
We can use Python to discuss this, since Python includes a compiler. It
generates byte code, which just means machine code for a virtual machine
instead of a hardware machine, but the principle is the same. Python even
has a disassembler, for taking those raw byte codes and turning them into
human-readable pseudo-assembly for the Python Virtual Machine.
Here is some "machine code" corresponding to the high-level instructions:
x = 23
y = 42
z = x + y
del x, y
py> code = compile('x = 23; y = 42; z = x + y; del x, y', '', 'exec')
py> code.co_code
'd\x00\x00Z\x00\x00d\x01\x00Z\x01\x00e\x00\x00e\x01\x00\x17Z\x02\x00[\x00
\x00[\x01\x00d\x02\x00S'
Translated into "assembly":
py> from dis import dis
py> dis(code)
1 0 LOAD_CONST 0 (23)
3 STORE_NAME 0 (x)
6 LOAD_CONST 1 (42)
9 STORE_NAME 1 (y)
12 LOAD_NAME0 (x)
15 LOAD_NAME1 (y)
18 BINARY_ADD
19 STORE_NAME 2 (z)
22 DELETE_NAME 0 (x)
25 DELETE_NAME 1 (y)
28 LOAD_CONST 2 (None)
31 RETURN_VALUE
You should be able to see that there are all sorts of changes that the
compiler could have made, which would have lead to different "machine
code" but with the exact same behaviour. This particular compiler emits
code for x=23; y=42 in the order that they were given, but that's not
actually required in this case. The compiler might have reversed the
order, generating something like:
0 LOAD_CONST 1 (42)
3 STORE_NAME 1 (y)
6 LOAD_CONST 0 (23)
9 STORE_NAME 0 (x)
or even:
0 LOAD_CONST 1 (42)
3 LOAD_CONST 0 (23)
6 STORE_NAME 1 (y)
9 STORE_NAME 0 (x)
without changing the behaviour of the code. Or it might have even
optimized the entire thing to:
0 LOAD_CONST 0 (65)
3 STORE_NAME 0 (z)
since x and y are temporary variables and a smart compiler could perform
the computation at compile time and throw them away. (Nitpicks about
"what if x and y already existed?" aside.) CPython even does this sort of
optimization, although in a more limited fashion:
py> dis(compile('x = 23 + 42', '', 'exec'))
1 0 LOAD_CONST 3 (65)
3 STORE_NAME 0 (x)
6 LOAD_CONST 2 (None)
9 RETURN_VALUE
This is called keyhole optimization. It's not beyond possibility for a
smarter compiler to look beyond the keyhole and make optimizations based
on the whole program.
So you can see that there is no one-to-one correspondence from high-level
source code to low-level machine code, even for a single machine. The
same is even more true for languages such as C, Fortran, Pascal, Lisp,
Scheme, Haskell, Java, etc. where people have spent years or decades
working on compiler technology. Compilers differ in the quality and
efficiency of the machine code
Re: Tail recursion to while iteration in 2 easy steps
Mark Janssen writes: > Yeah, and this is where two models of computation have been conflated, > creating magical effects, confusing everybody. I challenge you to get > down to the machine code in scheme and formally describe how it's > doing both. Which two models of computation are you talking about? And what magica; effects? AFAIK there is no magic in computer science, although every sufficiently advanced ... -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
Steven D'Aprano writes: > Far more useful would be a high-level description of Scheme's programming > model. If names can be rebound on the fly, how does Scheme even tell > whether something is a recursive call or not? Maybe it doesn't have to tell. If you do tail call optimization there is no need to do tail recursion optimization. -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Formal-ity and the Church-Turing thesis
On Tuesday, October 8, 2013 5:54:10 AM UTC+5:30, zipher wrote: > Now, one can easily argue that I've gone too far to say "no one has > understood it" (obviously), so it's very little tongue-in-cheek, but > really, when one tries to pretend that one model of computation can be > substituted for another (arguing *for* the Church-Turing thesis), they > are getting into troubling territory (it is only a thesis, > remember) The CT thesis is scientific and provable in one sense and vague/philosophical in another. The Science: Turing computability and lambda-computability are equivalent. The proofs just consist of writing interpreters for one in terms of the other. The philosophy: *ALL* computational models are turing equivalent (and therefore lambda-equivalent) or weaker. The Idea (note not proof) is that for equivalence one can write pair-interpreters like above. For the 'weaker' case, (eg DFA and TMs) one proves that TMs can interpret DFAs and disproves the possibility of the other direction. This must remain an idea (aka thesis) and not a proof because one cannot conceive of all possible computational models. It is hard science however for all the models that anyone has so far come up with. Now there are caveats to this which I wont go into for now. As for: > I challenge you to get down to the machine code in scheme and formally > describe how it's doing both. I can only say how ironic it sounds to someone who is familiar with the history of our field: Turing was not a computer scientist (the term did not exist then) but a mathematician. And his major contribution was to create a form of argument so much more rigorous than what erstwhile mathematicians were used to that he was justified in calling that math as a machine. The irony is that today's generation assumes that 'some-machine' implies its something like 'Intel-machine'. To get out of this confusion ask yourself: Is it finite or infinite? If the TM were finite it would be a DFA If the Intel-machine (and like) were infinite they would need to exist in a different universe. And so when you understand that TMs are just a kind of mathematical rewrite system (as is λ calculus as are context free grammars as is school arithmetic etc etc) you will not find the equivalence so surprising -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
>>> But even putting that aside, even if somebody wrote such a description, >>> it would be reductionism gone mad. What possible light on the problem >>> would be shined by a long, long list of machine code operations, even >>> if written using assembly mnemonics? >> >> Only that you've got a consistent, stable (and therefore, formalizable) >> translation from your language to the machine. > > You are mistaken to think that there is a single, one-to-one, mapping > between high-level code and machine code. It's not mistaken. Given a stable and formalized language definition, there should only be continued optimization of the lexical and procedural constructs into better machine code. In the case of an "interpreted" language like Python (which I'll define as a language which includes a layer of indirection between the user and the machine, encouraging the nice benefits of interactivity), such optimization isn't really apropos, because it's not the purpose of python to be optimal to the machine as much as "optimal to the programmer". In any case, while such optimization can continue over time, they generally create new compiler releases to indicate such changes. The one-to-one mapping is held by the compiler. Such determinism *defines* the machine, otherwise you might as well get rid of the notion of computer *science*. All else is error, akin to cosmic rays or magic. Unless the source code changes, all else remaining equal, the machine code is supposed to be the same, no matter how many times it is compiled. >[Only if you use the exact source, compiler, switches, etc]] will the output >be the same. > And even that is not guaranteed. Oh, and what would cause such non-determinism? > Take, for example, the single high-level operation: > > sort(alist) > > What machine code will be executed? Obviously that will depend on the > sort algorithm used. There are *dozens*. Here are just a few: Well, since you didn't specify your programming language, you're then merely stating an English construct. As such, there can be no single mapping from English into the machine, which is why there are so many different languages and experiments that map your [English] concepts into source code. > Now sorting is pretty high level, but the same principle applies to even > simple operations like "multiply two numbers". There are often multiple > algorithms for performing the operation, and even a single algorithm can > often be implemented in slightly different ways. Expecting all compilers > to generate the same machine code is simply naive. You are both over-simplifying and complexifying things at once. Pick one. -- MarkJ Tacoma, Washington -- https://mail.python.org/mailman/listinfo/python-list
Re: Tail recursion to while iteration in 2 easy steps
>> Yeah, and this is where two models of computation have been conflated, >> creating magical effects, confusing everybody. I challenge you to get >> down to the machine code in scheme and formally describe how it's >> doing both. > > Which two models of computation are you talking about? And what magica; > effects? Well, I delineate all computation involving predicates (like lambda calculus) between those using digital logic (like C). These realms of computation are so different, they are akin to mixing the complex numbers with the real. Yet hardly anyone points it out (I've concluded that hardly anyone has ever noticed -- the Church-Turing thesis has lulled the whole field into a shortcut in thinking which actually doesn't pan out in practice). > AFAIK there is no magic in computer science, although every sufficiently > advanced ... Ha! That's very good. I'm glad you catch the spirit of my rant. "Any sufficiently advanced compiler can be substituted with magic to the neophyte without a change in output." A mini Liskov substitution. -- MarkJ Tacoma, Washington -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tue, Oct 8, 2013 at 8:47 AM, rusi wrote: > I can only say how ironic it sounds to someone who is familiar with the > history of our field: > Turing was not a computer scientist (the term did not exist then) but a > mathematician. And his major contribution was to create a form of argument > so much more rigorous than what erstwhile mathematicians were used to that he > was justified in calling that math as a machine. > > The irony is that today's generation assumes that 'some-machine' implies its > something like 'Intel-machine'. > To get out of this confusion ask yourself: Is it finite or infinite? > If the TM were finite it would be a DFA > If the Intel-machine (and like) were infinite they would need to exist in a > different universe. With due respect Sir, you saying that Turing machine not a machine? Very confusion Sir!!! > > And so when you understand that TMs are just a kind of mathematical rewrite > system (as is λ calculus as are context free grammars as is school arithmetic > etc etc) you will not find the equivalence so surprising -- Ravi -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
> On Tuesday, October 8, 2013 5:54:10 AM UTC+5:30, zipher wrote: >> Now, one can easily argue that I've gone too far to say "no one has >> understood it" (obviously), so it's very little tongue-in-cheek, but >> really, when one tries to pretend that one model of computation can be >> substituted for another (arguing *for* the Church-Turing thesis), they >> are getting into troubling territory (it is only a thesis, >> remember) > > The CT thesis is scientific and provable in one sense and vague/philosophical > in another. > The Science: Turing computability and lambda-computability are equivalent. > The proofs just consist of writing interpreters for one in terms of the other. Ah, good, a fellow theoretician. Now it's nice that you use language that makes it seem quite clear, but understand that there's a hidden, subconscious, *cultural* encoding to your *statement*. The use of the term "equivalent", for example. Equivalent for the programmer, or for the machine? (etc., et cetera), and further: "writing interpreters for one in terms of the other", but again, this will change depending on your pragmatic requirements. To the theorist, you've accomplished something, but then that is a self-serving kind of accomplishment. To the programmer, operating under different requirements, you haven't accomplished anything. I don't have an infinite stack to implement lambda calculus, but I can treat my computer's memory as a TM and *practically* infinite and only rarely hit against the limits of physicality. This is just being "respectful"... ;^) (For the purposes of discussion, if I make a word in CamelCase, I am referring to a page on the WikiWikiWeb with the same name: http://c2.com/cgi/wiki?WikiWikiWeb.) > The philosophy: *ALL* computational models are turing equivalent (and > therefore lambda-equivalent) or weaker. > The Idea (note not proof) is that for equivalence one can write > pair-interpreters like above. For the 'weaker' case, (eg DFA and TMs) one > proves that TMs can interpret DFAs and disproves the possibility of the other > direction. > > This must remain an idea (aka thesis) and not a proof because one cannot > conceive of all possible computational models. Why not? I can "conceive" of all possible integer numbers even if I never "pictured" them. Is there not an inductive way to conceive of and define computation? I mean, I observe that the field seems to define several ModelsOfComputation. Intuitively I see two primary domains > It is hard science however for all the models that anyone has so far come up > with. And what of "interactive computation"? > As for: > >> I challenge you to get down to the machine code in scheme and formally >> describe how it's doing both. > > I can only say how ironic it sounds to someone who is familiar with the > history of our field: > Turing was not a computer scientist (the term did not exist then) but a > mathematician. And his major contribution was to create a form of argument > so much more rigorous than what erstwhile mathematicians were used to that he > was justified in calling that math as a machine. Hmm, I'm wondering if my use of the word "formally" is confusing you. In mathematics, this word has a subtly differing meaning, I think, than in computer science. Turing was "justified in calling that math as a machine" because he was using a definition (the translation table + finite dictionary) such that it remained perfectly deterministic. And here, again, one can easily gets mixed up using the same lexicon across two different domains: that of math and that of CS. I advise you to look at the dialog at ConfusedComputerScience. > The irony is that today's generation assumes that 'some-machine' implies its > something like 'Intel-machine'. > To get out of this confusion ask yourself: Is it finite or infinite? But this only gets us out of the confusion for the mathematicians. For the programmer and perhaps even the computer scientist (the one's coming from physics), it is something different. > If the TM were finite it would be a DFA But this is not a useful formalism. Any particular Program implements a DFA, even as it runs on a TM. The issue of whether than TM is finite or not can be dismissed because a simple calculation can usually suffice, or at least establish a range "usefulness" so as not to "run out of memory". > If the Intel-machine (and like) were infinite they would need to exist in a > different universe. Ha, yeah. Let us dismiss with that. > And so when you understand that TMs are just a kind of mathematical rewrite > system (as is λ calculus as are context free grammars as is school arithmetic > etc etc) you will not find the equivalence so surprising It's not that it's surprising, it's that it's *practically* a problem. The translation between one PL and another which assumes a different model of computation can get intractible. Maybe that makes sense MarkJ Tacoma, Washington -- https://
Re: Formal-ity and the Church-Turing thesis
On Tuesday, October 8, 2013 10:46:50 AM UTC+5:30, Ravi Sahni wrote: > With due respect Sir, you saying that Turing machine not a machine? > Very confusion Sir!!! Thanks Ravi for the 'due respect' though it is a bit out of place on a list like this :-) Thanks even more for the 'very confusion'. I can tell you being a teacher for 25 years that the most terrifying thing is the Dunning Kruger effect -- students who are utterly clueless and dont have a frigging clue about that. Ive seen PhDs in CS ask questions in compiler-related degree projects that they would not ask if they really understood the halting problem. [Or were you suggestig that I am the confused? :-) ] So yes, its a big thing to be confused and to accept that. To explain at length will be too long and OT (off-topic) for this list. I'll just give you a link and you tell me what you make of it: http://sloan.stanford.edu/mousesite/Secondary/Whorfframe2.html [mainly concentrate on the first section] -- https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On Tuesday, October 8, 2013 10:49:11 AM UTC+5:30, zipher wrote: > I don't have an infinite stack to implement > lambda calculus, but... And then > But this is not a useful formalism. Any particular Program implements > a DFA, even as it runs on a TM. The issue of whether than TM is > finite or not can be dismissed because a simple calculation can > usually suffice, or at least establish a range "usefulness" so as not > to "run out of memory". Having it both ways aren't you? -- https://mail.python.org/mailman/listinfo/python-list
Code golf challenge: XKCD 936 passwords
Who's up for some fun? Implement an XKCD-936-compliant password
generator in Python 3, in less code than this:
print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4))
Second challenge: Use it for generating all your passwords :)
[1] https://en.wikipedia.org/wiki/Code_golf
[2] http://xkcd.com/936/
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Formal-ity and the Church-Turing thesis
On 08/10/2013 06:44, rusi wrote: On Tuesday, October 8, 2013 10:46:50 AM UTC+5:30, Ravi Sahni wrote: With due respect Sir, you saying that Turing machine not a machine? Very confusion Sir!!! Thanks Ravi for the 'due respect' though it is a bit out of place on a list like this :-) With due respect Sir I'd like to point out that this appears to have very little to do (directly) with Python, so to go completely off topic I'll point out that my nephew is currently working on the film about the life of said Alan Turing :) -- Roses are red, Violets are blue, Most poems rhyme, But this one doesn't. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote:
> Who's up for some fun? Implement an XKCD-936-compliant password
>
> generator in Python 3, in less code than this:
>
>
>
> print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4))
>
>
>
> Second challenge: Use it for generating all your passwords :)
>
>
>
> [1] https://en.wikipedia.org/wiki/Code_golf
>
> [2] http://xkcd.com/936/
>
>
>
> ChrisA
Well, here's a start:
import random as r
print(*r.sample(open("/usr/share/dict/words").readlines(),4))
Shaves off 6 characters.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Monday, October 7, 2013 8:45:39 PM UTC-10, [email protected] wrote: > On Monday, October 7, 2013 8:17:21 PM UTC-10, Chris Angelico wrote: > > > Who's up for some fun? Implement an XKCD-936-compliant password > > > > > > generator in Python 3, in less code than this: > > > > > > > > > > > > print(*__import__("random").sample(open("/usr/share/dict/words").read().split("\n"),4)) > > > > > > > > > > > > Second challenge: Use it for generating all your passwords :) > > > > > > > > > > > > [1] https://en.wikipedia.org/wiki/Code_golf > > > > > > [2] http://xkcd.com/936/ > > > > > > > > > > > > ChrisA > > > > Well, here's a start: > > > > import random as r > > print(*r.sample(open("/usr/share/dict/words").readlines(),4)) > > > > Shaves off 6 characters. And if we were actually trying then that filename should just be "/w". Would get rid of another 19 chars. -- https://mail.python.org/mailman/listinfo/python-list
Re: Code golf challenge: XKCD 936 passwords
On Tue, Oct 8, 2013 at 5:48 PM, wrote: > And if we were actually trying then that filename should just be "/w". Would > get rid of another 19 chars. I'm working this on the assumption that the dictionary file already exists (that's where it is on my Debian Linux systems, for instance) and shouldn't be moved :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
