Re: [Tutor] garbage collecting
On 08/01/2014 04:41, Keith Winston wrote:
Iirc, Python periodically cleans memory of bits & pieces that are no
longer being used. I periodically do something stupid -- I mean
experimental -- and end up with a semi-locked up system. Sometimes it
comes back, sometimes everything after that point runs very slowly, etc.
I just saw where I could do os.system('python'), but in restarting the
interpreter I'd lose everything currently loaded: my real question
involves merely pushing the garbage collector into action, I think.
--
Keith
I've never played with the garbage collector in 10+ years of using
Pythom, so I think it unlikely you need it now. Far more likely is that
you've written an infinite loop, fix that and all other problems go
away. Also where would you call the GC? Surely until you know where in
your code the problem is, you don't know where to make the call. Even
then, if memory is being allocated and Python thinks it's still in use,
calling the GC won't have any impact anyway.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] OverflowError: cannot fit 'long' into an index-sized integer
On 08/01/2014 03:36, Keith Winston wrote: Hey Steven, That's a cool primes module, I'm going to look that over more carefully. I can see that you've thought through this stuff before ;) And yeah, I'd like to see your Stopwatch code... I haven't looked at "with" yet, that's interesting. As usual, I don't totally get it... Keith Search for python context managers, then read to your hearts's content :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python 3.3 split method confusion
Danny Yoo wrote:
> One of the common cases for split() is to break a line into a list of
> words, for example.
>
> #
'hello this is a test'.split()
> ['hello', 'this', 'is', 'a', 'test']
> #
>
> The Standard Library can not do everything that we can conceive of as
> being useful, because that set is fairly large.
>
> If the Standard Library doesn't do it, we'll probably need to do it
> ourselves, or find someone who has done it already.
>
>
> ##
def mysplit(s, delim):
> ... start = 0
> ... while True:
> ... index = s.find(delim, start)
> ... if index != -1:
> ... yield s[start:index]
> ... yield delim
> ... start = index + len(delim)
> ... else:
> ... yield s[start:]
> ... return
> ...
list(mysplit("this,is,a,test", ","))
> ['this', ',', 'is', ',', 'a', ',', 'test']
> ##
The standard library does provide a way to split a string and preserve the
delimiters:
>>> import re
>>> re.split("(,)", "this,is,a,test")
['this', ',', 'is', ',', 'a', ',', 'test']
It is very flexible...
>>> re.split("([-+*/])", "alpha*beta/gamma-delta")
['alpha', '*', 'beta', '/', 'gamma', '-', 'delta']
but you need to learn a mini-language called "regular expressions" and it
takes some time to get used to them and to avoid the pitfalls (try to swap
the "-" and "+" in my second example).
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
well, fair enough. Generally, the issue involves instances when Python will come back, but it might take several minutes or much longer. And weird behaviour ensues: like timers I put on the program don't actually time the amount of time it's busy (by a very, very large margin). Also, often several minutes after such a thing (sometimes quite a bit later), things will suddenly start working quickly again. Also, on my laptop I can actually tell when it's struggling, b/c the fan turns on and/or speeds up, and in many of these cases it will go into all-out mode, even though I'm not doing anything. But your point about having no lever with which to move the world is a good one. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On 08/01/2014 09:30, Keith Winston wrote: well, fair enough. Generally, the issue involves instances when Python will come back, but it might take several minutes or much longer. And weird behaviour ensues: like timers I put on the program don't actually time the amount of time it's busy (by a very, very large margin). Also, often several minutes after such a thing (sometimes quite a bit later), things will suddenly start working quickly again. Also, on my laptop I can actually tell when it's struggling, b/c the fan turns on and/or speeds up, and in many of these cases it will go into all-out mode, even though I'm not doing anything. But your point about having no lever with which to move the world is a good one. Typical symptoms of hammering a system particularly when you start swapping memory to and from disk. The usual solution is to find a better algorithm or buy more ram :) A slight aside you've managed to lose all of my context, more problems with gmail I take it? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] OverflowError: cannot fit 'long' into an index-sized integer
Yes, I did read the context manager stuff, but I can only absorb a bit of it, I just don't have the background. It's getting better though. Plus I think most things will come only after I play with them, and I haven't really played much with wrappers, decorators and the like. Which is fine, the year is young. I've just knocked off the first 11 Euler project problems, which forced me to do a bunch of things for the first time (mostly quite uglily). Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On 08/01/14 08:15, Mark Lawrence wrote:
I just saw where I could do os.system('python'), but in restarting the
interpreter I'd lose everything currently loaded: my real question
involves merely pushing the garbage collector into action, I think.
Don't do that!
You are not restarting the interpreter you are starting a brand new
interpreter *in addition* to the one that's already running. Now you
have even less resources to play with. It will make things worse not better.
os.system() is really for running shell commands that hopefully don't
take long, do a job that doesn't need any input and produces output that
you don't need to process(or a file that you can). Printing
a file, converting a file with lame of ffmpeg, say, that kind of thing.
It's actually deprecated now anyway and the subprocess module is preferred.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On 08/01/2014 10:41, Alan Gauld wrote:
On 08/01/14 08:15, Mark Lawrence wrote:
Did I? :)
I just saw where I could do os.system('python'), but in restarting the
interpreter I'd lose everything currently loaded: my real question
involves merely pushing the garbage collector into action, I think.
Don't do that!
You are not restarting the interpreter you are starting a brand new
interpreter *in addition* to the one that's already running. Now you
have even less resources to play with. It will make things worse not
better.
os.system() is really for running shell commands that hopefully don't
take long, do a job that doesn't need any input and produces output that
you don't need to process(or a file that you can). Printing
a file, converting a file with lame of ffmpeg, say, that kind of thing.
It's actually deprecated now anyway and the subprocess module is preferred.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Tue, Jan 07, 2014 at 11:41:53PM -0500, Keith Winston wrote:
> Iirc, Python periodically cleans memory of bits & pieces that are no longer
> being used. I periodically do something stupid -- I mean experimental --
> and end up with a semi-locked up system. Sometimes it comes back,
> sometimes everything after that point runs very slowly, etc.
You can control the garbage collector with the gc module, but you
shouldn't have to. It runs automatically, whenever your code is done
with an object, the garbage collector reclaims the memory used. In
particular, the garbage collector is not the solution to your problem.
As Mark says, the symptoms you describe suggest you have run out of
memory and are hammering the swap space. This will slow everything
down by a lot. Virtual memory is *millions* of times slower than real
memory (RAM).
None of this is specific to Python, but here's a quick (and simplified)
explanation, as best I understand it myself. Modern operating systems,
which include Windows, Linux and OS X, have a concept of "virtual
memory", which you can consider to be a great big file on your hard
disk. (To be pedantic, it's a single file in Windows, usually a separate
partition on Linux, and I don't know about OS X.) When an application
requests some memory, if there is not enough memory available, the OS
will grab some chunk of memory which is not in use (we'll call it "B"),
copy it to the swap file (the virtual memory), then free it up for the
application to use. Then when the app tries to use memory B back again,
the OS sees that it's in swap, grab another chunk of real memory, copy
it to swap, then move memory B back into RAM.
So think of virtual memory (swap space) as a great big cheap but
astonishingly slow storage area. When you run out of space in RAM, stuff
gets copied to swap. When you try to use that stuff in swap, something
else has to get moved to swap first, to free up space in RAM to move the
first chunk back into RAM. The OS is always doing this, and you hardly
ever notice.
Except, when an app asks for a lot of memory, or lots and lots and lots
of apps ask for a little bit all at the same time, you can end up with a
situtation where the OS is spending nearly all its time just copying
stuff from RAM to swap and back again. So much time is being spent doing
this that the application itself hardly gets any time to run, and so
everything slows down to a crawl. You can often actually *hear this
happening*, as the hard drive goes nuts jumping backwards and forwards
copying data to and fro swap. This is called "thrashing", and it isn't
fun for anyone.
The only cure for a thrashing system is to sit back and wait, possibly
shut down a few applications (which may temporarily make the thrashing
worse!) and eventually the whole thing will settle down again, in five
minutes or five days.
I'm not exaggerating -- I once made a mistake with a list, ran
something stupid like mylist = [None]*(10**9), and my system was
thrashing so hard the operating system stopped responding. I left it for
16 hours and it was still thrashing, so I shut the power off. That's a
fairly extreme example though, normally a thrashing system won't be
*that* unresponsive, it will just be slow.
If you are using Linux, you can use the ulimit command to limit how much
memory the OS will allow an application to use before shutting it off.
So for example:
[steve@ando ~]$ ulimit -v 1
[steve@ando ~]$ python
Python 2.7.2 (default, May 18 2012, 18:25:10)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> l = range(20)
Traceback (most recent call last):
File "", line 1, in
MemoryError
> I just saw where I could do os.system('python'), but in restarting the
> interpreter I'd lose everything currently loaded: my real question involves
> merely pushing the garbage collector into action, I think.
As Alan has said, os.system('python') doesn't restart the interpreter,
it pauses it and starts up a new one.
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On 08/01/14 10:50, Mark Lawrence wrote: On 08/01/2014 10:41, Alan Gauld wrote: On 08/01/14 08:15, Mark Lawrence wrote: Did I? :) Oops, one too many lines deleted. Sorry Mark. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] OverflowError: cannot fit 'long' into an index-sized integer
On Tue, Jan 07, 2014 at 10:36:56PM -0500, Keith Winston wrote:
> And yeah, I'd like to see your Stopwatch code... I haven't looked at "with"
> yet, that's interesting. As usual, I don't totally get it...
Okay, let's talk about a common pattern in Python code:
try:
do some stuff
finally:
clean up
You have a "try" block which runs some code. Whatever happens inside the
block, regardless of whether it succeeds or fails, when it is done,
Python runs the "finally" block. Let's see this in action, first with a
successful computation, then an unsuccessful one:
py> try:
... print(1 + 1)
... finally:
... print("=== Done ===")
...
2
=== Done ===
py> try:
... print(1 + "1")
... finally:
... print("=== Done ===")
...
=== Done ===
Traceback (most recent call last):
File "", line 2, in
TypeError: unsupported operand type(s) for +: 'int' and 'str'
This is such a common pattern, that a few versions back (Python 2.5, I
believe) Python introduced new syntax for it: context managers and the
"with" statement.
The new syntax is:
with obj as name:
block
When Python runs the "with" statement, it expects that obj will have two
special methods:
* obj.__enter__ is run when the "with" statement begins, just
before the block is run;
* obj.__exit__ is run when the block exits, regardless of
whether it exited successfully, or due to an error.
There are more complications, of course, but in a nutshell that's it. A
with statement is not terribly different from this:
name = obj.__enter__()
try:
block
finally:
obj.__exit__(*args)
(The args passed to __exit__ have to do with any exceptions raised
inside the block.)
The Stopwatch code is quite simple: it has an __enter__ method which
starts the timer, and an __exit__ method which stops it and reports how
long things took.
=== cut ===
import gc
import sys
from functools import wraps
class Stopwatch:
"""Time hefty or long-running block of code using a ``with`` statement:
>>> with Stopwatch(): #doctest: +SKIP
... do_this()
... do_that()
...
time taken: 1.234567 seconds
The Stopwatch class takes four optional arguments:
timer::
Timer function; by default, the default timer from the
timeit module is used.
allow_gc::
If true (the default), the garbage collector is free to
operate while the code block is running, otherwise, the
garbage collector is temporarily disabled.
verbose::
If a true value (the default), the timer result is
reported after the code block completes, and a warning
displayed if the elapsed time is too small.
cutoff::
If None, elapsed time warnings are disabled; otherwise,
the amount of time in seconds below which a warning is
displayed. Defaults to 0.001.
For non-interactive use, you can retrieve the time taken using the
``interval`` attribute:
>>> with Stopwatch(verbose=False) as sw: #doctest: +SKIP
... do_this()
... do_that()
...
>>> print(sw.interval) #doctest: +SKIP
1.234567
"""
def __init__(self, timer=None, allow_gc=True, verbose=True, cutoff=0.001):
if timer is None:
from timeit import default_timer as timer
self.timer = timer
self.allow_gc = allow_gc
self.verbose = verbose
self.cutoff = cutoff
self.start = self.end = self.interval = None
def __enter__(self):
if not self.allow_gc:
self._gc_state = gc.isenabled()
gc.disable()
self.interval = None
self.start = self.timer()
return self
def __exit__(self, *args):
self.end = self.timer()
if not self.allow_gc and self._gc_state:
gc.enable()
self.interval = self.end - self.start
self._report()
def _report(self):
if not self.verbose:
return
if self.cutoff is not None and self.interval < self.cutoff:
print("elapsed time is very small; consider using timeit.Timer"
" for micro-timings of small code snippets")
print('time taken: %f seconds' % self.interval)
=== cut ===
And that's all there is to it!
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On 01/08/2014 10:30 AM, Keith Winston wrote: well, fair enough. Generally, the issue involves instances when Python will come back, but it might take several minutes or much longer. And weird behaviour ensues: like timers I put on the program don't actually time the amount of time it's busy (by a very, very large margin). Also, often several minutes after such a thing (sometimes quite a bit later), things will suddenly start working quickly again. Also, on my laptop I can actually tell when it's struggling, b/c the fan turns on and/or speeds up, and in many of these cases it will go into all-out mode, even though I'm not doing anything. But your point about having no lever with which to move the world is a good one. It smells like a logical bug, or lack of simplicity / clarity, leading to a behaviour with "exponential" cost (in cpu usage, thus time, more or less in proportion), in some given circumstances. Or maybe just an unavoidable exponential cost routine, but the quantity of iput shoulde remain small, while in some (buggy) cases it is big. [Just wildly guessing, don't know your app & code. Can you reduce it to a minimal still manifesting similar bug?] Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Wed, Jan 08, 2014 at 12:43:06PM +0100, spir wrote: > [Just wildly guessing, don't know your app & code. Can you reduce it to a > minimal still manifesting similar bug?] Try (or rather, *don't* try): import sys list(range(sys.maxsize)) That ought to do it :-) (Note: depending on your operating system, that may simply fail with MemoryError, or lock up your computer. Don't blame me for any data loss or inconvenience if you try it.) -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
Well, thanks everyone. I get the picture. And there's nothing subtle going on here: I'm playing around, trying to factor million-digit numbers and the like. No biggie, this was interesting. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] need help how to run it on python 3.3
need help how to run it on python 3.3, or change it to python 3.3 when i run it
says syntax error if i run it on python 2.7 it works.
# Import statements
import random
import datetime
#Arrays to store the definitions and keywords read from the file
keywords=[];
definition=[];
correctAnswer=[];
#Counter for the wrong Answer and counter of number of definition in
correctAnswerCounter=0
wrongAnswer=0;
counter=0;
# Taking User input for accepting the file name to be read
filename= raw_input("Enter the File Name with extension")
#Reading the file from start to the end
for line in open(filename,'r').readlines():
if(counter%2==0):
keywords.append(line);
counter=counter+1;
correctAnswer.append(0)
else:
definition.append(line);
keys=[];
keys=line.split(" ");
counter=counter+1;
# Running two while loops to make the pattern recursive
while True:
# Starting the time for quiz and also, creating variables to make sure that
same sequences and answers are not repeated
a = datetime.datetime.now().replace(microsecond=0)
prevWord=0
prevSeq=0
# While loop to run the code till each answer is correctly answered
while correctAnswer.count(2)!=(counter/2):
#While loop to generate an different random number from one the
generated previously
while True:
word=random.randint(0,(counter/2)-1)
if(correctAnswer[word]!=2):
break;
if(prevWord==word):
continue;
# Displaying the new keyword each time.
print "Please Select the number which is the correct definition of the
word:" ,keywords[word]
#Generating an new sequence each time different from previous one
while True:
sequences =random.randint(0,2)
if(prevSeq==sequences):
continue;
else:
break
#Generating an new incorrect answer each time different from previous
one
while True:
incorrectAnswer=random.randint(0,len(correctAnswer)-1)
if(incorrectAnswer==word):
continue;
else :
break
#Generating an new incorrect answer each time different from previous
one
while True:
incorrectAnswerSecond=random.randint(0,len(correctAnswer)-1);
if (incorrectAnswer==incorrectAnswerSecond):
continue
if(incorrectAnswerSecond==word):
continue
else:
break
# Displaying the options to the user based on the sequence number
generated
if (sequences==0):
print "1.",definition[word]
print "2.",definition[incorrectAnswer]
print "3.",definition[incorrectAnswerSecond]
elif (sequences==1):
print "1.",definition[incorrectAnswer]
print "2.",definition[incorrectAnswerSecond]
print "3.",definition[word]
elif (sequences==2):
print "1.",definition[incorrectAnswerSecond]
print "2.",definition[word]
print "3.",definition[incorrectAnswer]
#Taking the answer from user
answer = raw_input("Enter the Correct Number between 1 to 3")
# Assign the seq and word to preseq and word
prevSeq=sequences
prevWord=word
#Checking the answer if they are corret.
if(0 == sequences):
if(answer == "1"):
print "success"
correctAnswer[word]=correctAnswer[word]+1
correctAnswerCounter=correctAnswerCounter+1
else:
print "Wrong Answer"
print "Correct Answer: " ,definition[word]
wrongAnswer=wrongAnswer+1;
elif(1 == sequences):
if(answer == "3"):
print "success"
correctAnswer[word]=correctAnswer[word]+1
correctAnswerCounter=correctAnswerCounter+1
else:
print "Wrong Answer"
print "Correct Answer: " ,definition[word]
wrongAnswer=wrongAnswer+1;
elif(2 == sequences):
if(answer == "2"):
print "success"
correctAnswer[word]=correctAnswer[word]+1
correctAnswerCounter=correctAnswerCounter+1
else:
print "Wrong Answer"
print "Correct Answer: " ,definition[word]
wrongAnswer=wrongAnswer+1
# Stopping the time of the clock
b = datetime.datetime.now().replace(microsecond=0)
# displaying number of wrong answer and total quiz time
print "Total Number of Wrong Answer:", wrongAnswer
print "Total Quiz Time", (b-a)
print "Total Number of correct Answer", correctAnswerCounter
#asking user to reenter
restart= raw_input("Do You want to start the quiz again Yes or No")
if(restart=="no"):
print "Thanks for qui
[Tutor] recursion depth
I've been playing with recursion, it's very satisfying. However, it appears that even if I sys.setrecursionlimit(10), it blows up at about 24,000 (appears to reset IDLE). I guess there must be a lot of overhead with recursion, if only 24k times are killing my memory? I'm playing with a challenge a friend gave me: add each number, up to 1000, with it's reverse, continuing the process until you've got a palindrome number. Then report back the number of iterations it takes. There's one number, 196, that never returns, so I skip it. It's a perfect place to practice recursion (I do it in the adding part, and the palindrome checking part), but apparently I can't help but blow up my machine... -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Jan 8, 2014 5:51 PM, "Keith Winston" wrote: > > Well, thanks everyone. I get the picture. And there's nothing subtle going on here: I'm playing around, trying to factor million-digit numbers and the like. No biggie, this was interesting. Million digit numbers will use up some memory. Perhaps a megabyte per number. If you just hold one in memory and try to factor it then it shouldn't lead to the problems you described. Or are you holding a list of many million digit numbers? Or are they several hundred million digits? The garbage collector has nothing to do with the memory usage of immutable types like ints. There are deallocated instantly when the last reference you hold is cleared (in CPython). So if you run out of memory because of them then it is because you're keeping them alive in your own code. Running the garbage collector with gc.collect cannot help with that. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] need help how to run it on python 3.3
On 08/01/2014 18:19, S Tareq wrote: need help how to run it on python 3.3, or change it to python 3.3 when i run it says syntax error if i run it on python 2.7 it works. if(counter%2==0): Please remove those unneeded brackets, this is a Python list, not C :) print "Please Select the number which is the correct definition of the word:" ,keywords[word] Please add the needed brackets, print is a function in Python 3, not a statement. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On 1/8/2014 12:25 PM, Keith Winston wrote: I've been playing with recursion, it's very satisfying. However, it appears that even if I sys.setrecursionlimit(10), it blows up at about 24,000 (appears to reset IDLE). I guess there must be a lot of overhead with recursion, if only 24k times are killing my memory? Yes -- the docs warn specifically about that: sys.setrecursionlimit(limit)¶ Set the maximum depth of the Python interpreter stack to limit. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python. The highest possible limit is platform-dependent. A user may need to set the limit higher when she has a program that requires deep recursion and a platform that supports a higher limit. This should be done with care, because a too-high limit can lead to a crash. I'm playing with a challenge a friend gave me: add each number, up to 1000, with it's reverse, continuing the process until you've got a palindrome number. Then report back the number of iterations it takes. There's one number, 196, that never returns, so I skip it. It's a perfect place to practice recursion (I do it in the adding part, and the palindrome checking part), but apparently I can't help but blow up my machine... Without seeing your code it's hard to be specific, but it's obvious you'll need to rethink your approach. :) Emile ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] need help how to run it on python 3.3
Hi S Tareq, You probably want to review the "What's new in Python 3" document, with close attention to the "Common Stumbing Blocks" section: http://docs.python.org/3.0/whatsnew/3.0.html#common-stumbling-blocks Have you read this document already? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Wed, Jan 8, 2014 at 3:30 PM, Oscar Benjamin wrote: > The garbage collector has nothing to do with the memory usage of immutable > types like ints. There are deallocated instantly when the last reference > you hold is cleared (in CPython). So if you run out of memory because of > them then it is because you're keeping them alive in your own code. Running > the garbage collector with gc.collect cannot help with that. > Well that's sort of interesting... so it's different for mutables? huh. Anyway, I think I got what I can reasonably hope to get from this question, thanks to you and everyone! -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On Wed, Jan 8, 2014 at 3:42 PM, Emile van Sebille wrote: > > Without seeing your code it's hard to be specific, but it's obvious you'll > need to rethink your approach. :) Yes, it's clear I need to do the bulk of it without recusion, I haven't really thought about how to do that. I may or may not ever get around to doing it, since this was primarily an exercise in recursion, for me... Thanks for your thoughts. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Jan 8, 2014 9:11 PM, "Keith Winston" wrote: > > > On Wed, Jan 8, 2014 at 3:30 PM, Oscar Benjamin wrote: >> >> The garbage collector has nothing to do with the memory usage of immutable types like ints. There are deallocated instantly when the last reference you hold is cleared (in CPython). So if you run out of memory because of them then it is because you're keeping them alive in your own code. Running the garbage collector with gc.collect cannot help with that. > > > Well that's sort of interesting... so it's different for mutables? huh. Anyway, I think I got what I can reasonably hope to get from this question, thanks to you and everyone! > It's not that it's different for mutables specifically. The primary deallocation method is by reference count. This fails if there are reference cycles so CPython has a cyclic garbage collector that deals with those. Reference cycles can only emerge because of mutable containers. A slight correction to what I wrote above is that if an immutable object such as an int is referenced by an mutable one that is itself in a reference cycle then it wouldn't be deallocated until the cyclic garbage collector runs. Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On Wed, Jan 8, 2014 at 3:25 PM, Keith Winston wrote: > I've been playing with recursion, it's very satisfying. > > However, it appears that even if I sys.setrecursionlimit(10), it blows > up at about 24,000 (appears to reset IDLE). I guess there must be a lot of > overhead with recursion, if only 24k times are killing my memory? CPython recursion is limited by the thread's stack size, since evaluating a Python frame requires calling PyEval_EvalFrameEx. The default stack size on Windows is 1 MiB, and on Linux RLIMIT_STACK is typically set at 8 MiB (inspect this w/ the stdlib's resource module). You can create a worker thread with a larger stack using the threading module. On Windows the upper limit is 256 MiB, so give this a try: import sys import threading MiB = 2 ** 20 threading.stack_size(256 * MiB) sys.setrecursionlimit(10) t = threading.Thread(target=your_function) t.start() I'm not saying this is a good solution in general. It's just something to play around with, and may help in a pinch. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] need help how to run it on python 3.3
You might want to check out the '2to3' program to convert Python 2.x
code to Python 3.x code.
The following command worked to change your code to runnable Python 3.x
code:
2to3 -w
--
Steve Mayer
smaye...@me.com
On 8 Jan 2014, at 10:19, S Tareq wrote:
need help how to run it on python 3.3, or change it to python 3.3 when
i run it says syntax error if i run it on python 2.7 it works.
# Import statements
import random
import datetime
#Arrays to store the definitions and keywords read from the file
keywords=[];
definition=[];
correctAnswer=[];
#Counter for the wrong Answer and counter of number of definition in
correctAnswerCounter=0
wrongAnswer=0;
counter=0;
# Taking User input for accepting the file name to be read
filename= raw_input("Enter the File Name with extension")
#Reading the file from start to the end
for line in open(filename,'r').readlines():
if(counter%2==0):
keywords.append(line);
counter=counter+1;
correctAnswer.append(0)
else:
definition.append(line);
keys=[];
keys=line.split(" ");
counter=counter+1;
# Running two while loops to make the pattern recursive
while True:
# Starting the time for quiz and also, creating variables to make sure
that same sequences and answers are not repeated
a = datetime.datetime.now().replace(microsecond=0)
prevWord=0
prevSeq=0
# While loop to run the code till each answer is correctly
answered
while correctAnswer.count(2)!=(counter/2):
#While loop to generate an different random number from
one the generated previously
while True:
word=random.randint(0,(counter/2)-1)
if(correctAnswer[word]!=2):
break;
if(prevWord==word):
continue;
# Displaying the new keyword each time.
print "Please Select the number which is the correct
definition of the word:" ,keywords[word]
#Generating an new sequence each time different from
previous one
while True:
sequences =random.randint(0,2)
if(prevSeq==sequences):
continue;
else:
break
#Generating an new incorrect answer each time different
from previous one
while True:
incorrectAnswer=random.randint(0,len(correctAnswer)-1)
if(incorrectAnswer==word):
continue;
else :
break
#Generating an new incorrect answer each time different
from previous one
while True:
incorrectAnswerSecond=random.randint(0,len(correctAnswer)-1);
if (incorrectAnswer==incorrectAnswerSecond):
continue
if(incorrectAnswerSecond==word):
continue
else:
break
# Displaying the options to the user based on the sequence
number generated
if (sequences==0):
print "1.",definition[word]
print "2.",definition[incorrectAnswer]
print "3.",definition[incorrectAnswerSecond]
elif (sequences==1):
print "1.",definition[incorrectAnswer]
print "2.",definition[incorrectAnswerSecond]
print "3.",definition[word]
elif (sequences==2):
print "1.",definition[incorrectAnswerSecond]
print "2.",definition[word]
print "3.",definition[incorrectAnswer]
#Taking the answer from user
answer = raw_input("Enter the Correct Number between 1 to
3")
# Assign the seq and word to preseq and word
prevSeq=sequences
prevWord=word
#Checking the answer if they are corret.
if(0 == sequences):
if(answer == "1"):
print "success"
correctAnswer[word]=correctAnswer[word]+1
correctAnswerCounter=correctAnswerCounter+1
else:
print "Wrong Answer"
print "Correct Answer: " ,definition[word]
wrongAnswer=wrongAnswer+1;
elif(1 == sequences):
if(answer == "3"):
print "success"
correctAnswer[word]=correctAnswer[word]+1
correctAnswerCounter=correctAnswerCounter+1
else:
print "Wrong Answer"
print "Correct Answer: " ,definition[word]
wrongAnswer=wrongAnswer+1;
elif(2 == sequences):
if(answer == "2"):
print "success"
correctAnswer[word]=correctAnswer[word]+1
correctAnswerCounter=correctAnswerCounter+1
else:
print "Wrong Answer"
print "Correct Answer: " ,definition[word]
wrongAnswer=wrongAnswer+1
# Stopping the time of the clock
b = datetime.datetime.now().replace(microsecond=0)
# displaying number of wrong answer and total quiz time
print "Total Number o
Re: [Tutor] need help how to run it on python 3.3
Hi S Tareq, If you can, next time please just copy and paste the error message as plain text. Please avoid screenshots unless they really are relevant to the problem at hand. There are good reasons why you should prefer plain text when asking questions on a mailing list like this one. (1) Screenshots are often useful but resource-intensive. (2) Screenshots are difficult to search and index: folks who are using non-graphical email clients have no way of helping you, and search engines become less useful when run across our archives in the presence of non-textual content. Anyway, the error message you're running into: NameError: name 'raw_input' is not defined is because Python 2's 'raw_input()' has been renamed to Python 3's 'input()'. Again, please _closely_ read the document: http://docs.python.org/3.0/whatsnew/3.0.html#builtins which lists this and other backwards-incompatible changes to the language. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On 01/08/2014 10:11 PM, Keith Winston wrote:
On Wed, Jan 8, 2014 at 3:42 PM, Emile van Sebille wrote:
Without seeing your code it's hard to be specific, but it's obvious you'll
need to rethink your approach. :)
Yes, it's clear I need to do the bulk of it without recusion, I haven't
really thought about how to do that. I may or may not ever get around to
doing it, since this was primarily an exercise in recursion, for me...
Thanks for your thoughts.
Funny and useful exercise in recursion: write a func that builds str and repr
expressions of any object, whatever its attributes, inductively. Eg with
obj.__repr__() = Type(attr1, attr2...) # as in code
obj.__str__() = {id1:attr1 id2:attr2...} # nicer
Denis
PS: Don't knwo why it's not builtin, would be very useful for debugging,
testing, any kind of programmer feedback. Guess it has to do with cycles, but
there are ways to do that; and python manages cycles in list expressions:
spir@ospir:~$ python3
Python 3.3.1 (default, Sep 25 2013, 19:29:01)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
l1 = [1]
l2 = [1, l1]
l1.extend([l2,l1])
l1
[1, [1, [...]], [...]]
l2
[1, [1, [...], [...]]]
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On Wed, Jan 8, 2014 at 4:23 PM, eryksun wrote: > You can create a worker thread with a larger stack using the threading > module. On Windows the upper limit is 256 MiB, so give this a try: > quite excellent, mwahaha... another shovel to help me excavate out the bottom of my hole... I'll play with this someday, but maybe not today. I seem to be pushing some dangerous limits. Which does happen to be a hobby of mine. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On Wed, Jan 8, 2014 at 5:15 PM, spir wrote: > Funny and useful exercise in recursion: write a func that builds str and > repr expressions of any object, whatever its attributes, inductively. Eg > with > Hmm, can't say I get the joke. I haven't really played with repr, though I think I understand it's use. Could you give an example, I'm not sure I understand the goal? -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] need help how to run it on python 3.3
On 08/01/2014 20:54, S Tareq wrote: On Wednesday, 8 January 2014, 20:46, Danny Yoo wrote: Hi S Tareq, You probably want to review the "What's new in Python 3" document, with close attention to the "Common Stumbing Blocks" section: http://docs.python.org/3.0/whatsnew/3.0.html#common-stumbling-blocks Have you read this document already? Please don't attach 118kb screenshot files when you could have used cut and paste for the four lines that are relevant. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On Wed, 8 Jan 2014 16:23:06 -0500, eryksun wrote: On Wed, Jan 8, 2014 at 3:25 PM, Keith Winston wrote: > I've been playing with recursion, it's very satisfying. > > However, it appears that even if I sys.setrecursionlimit(10), it blows > up at about 24,000 (appears to reset IDLE). I guess there must be a lot of > overhead with recursion, if only 24k times are killing my memory? I can't see the bodies of any of your messages (are you perchance posting in html? ), but I think there's a good chance you're abusing recursion and therefore hitting the limit much sooner than necessary. I've seen some code samples here using recursion to fake a goto, for example. One question to ask is whether each time you recurse, are you now solving a simpler problem. For example, when iterating over a tree you should only recurse when processing a SHALLOWER subtree. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] need help how to run it on python 3.3
On Wed, 8 Jan 2014 20:54:09 + (GMT), S Tareq wrote: ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor This is a text newsgroup. Html messages and attachments of any kind (especially binary ones) are out of place. Guessing that you're trying to show an error message, I'd suggest copy/paste not screen shot. And if your terminal type doesn't seem to support the clipboard, tell us what you're using and somebody will probably have a suggestion. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] recursion depth
On Wed, Jan 8, 2014 at 6:16 PM, Dave Angel wrote:
> I can't see the bodies of any of your messages (are you perchance posting
> in html? ), but I think there's a good chance you're abusing recursion and
> therefore hitting the limit much sooner than necessary. I've seen some code
> samples here using recursion to fake a goto, for example. One question to
> ask is whether each time you recurse, are you now solving a simpler
> problem.
> For example, when iterating over a tree you should only recurse when
> processing a SHALLOWER subtree.
>
Hi Dave: I've been taken to task so often here about having unnecessary
chaff in my email replies, that I started getting in the habit of deleting
everything (since gmail by (unadjustable) default quotes the entire message
string unless you highlight/reply). Since I look at messages in a threaded
manner, I wasn't really realizing how much of a pain that was for others.
I'm trying to re-establish a highlight/reply habit, like this.
I don't THINK I'm misusing recursion, I think I'm just recursing ridiculous
things. The problem came in creating palindrome numbers. Apparently, if you
add a number to it's reversal (532 + 235), it will be a palindrome, or do
it again (with the first result)... with the only mysterious exception of
196, as I understand it. Interestingly, most numbers reach this palindrome
state rather quickly: in the first 1000 numbers, here are the number of
iterations it takes (numbers don't get credit for being palindromes before
you start):
{0: 13, 1: 291, 2: 339, 3: 158, 4: 84, 5: 33, 6: 15, 7: 18, 8: 10, 10: 2,
11: 8, 14: 2, 15: 8, 16: 1, 17: 5, 19: 1, 22: 2, 23: 8, 24: 2}
Zero stands for where I ran out of recursion depth, set at the time at
9900. Except for the first zero, which is set at 196. It's sort of
fascinating: those two 24's both occur in the first 100.
So it hardly ever takes any iterations to palindromize a number, except
when it takes massive numbers. Except in the single case of 196, where it
never, ever happens apparently (though I understand this to not be proven,
merely tested out to a few million places).
--
Keith
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Wed, Jan 8, 2014 at 3:30 PM, Oscar Benjamin wrote: > The garbage collector has nothing to do with the memory usage of immutable > types like ints. There are deallocated instantly when the last reference you > hold is cleared (in CPython). So if you run out of memory because of them > then it is because you're keeping them alive in your own code. Running the > garbage collector with gc.collect cannot help with that. Object deallocation in CPython doesn't necessarily free memory to the system. A deallocated object can go on a freelist, or it could have been allocated out of an obmalloc arena that persists. Either way this can lead to to a heap high-water mark. 3.3 avoids this wrt obmalloc by using POSIX mmap instead of malloc, and 3.4 uses Windows VirtualAlloc (whereas malloc uses HeapAlloc). It doesn't need malloc to manage small blocks; that's its job after all. Calling gc.collect(generation=2) will clear freelists for several built-in types, which may enable the heap to shrink if there's a high-water mark lingering in a freelist. This is mostly a concern prior to 3.3. float was the last holdout. It used the original freelist design until 3.3. Per issue 14435 it was transitioned to using obmalloc. The old float freelist was the same design as the one for 2.x int (PyInt, not PyLong), which grows without bound. The design also allocates objects in 1 KiB blocks (approx. size). glibc's malloc will use the heap for a block that's this small. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] garbage collecting
On Thu, Jan 9, 2014 at 12:27 AM, eryksun wrote: > The old float freelist was the same design as the one for 2.x int > (PyInt, not PyLong), which grows without bound. The design also > allocates objects in 1 KiB blocks (approx. size). glibc's malloc will > use the heap for a block that's this small. > Thanks Eryksun, pretty sure that's more than I could have known to ask about garbage collection. But who knows where it will come in handy? -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
