Re: [Tutor] MemoryError

2015-12-30 Thread Alan Gauld
On 30/12/15 06:26, Satya Luzy wrote: > Dear everyone, thank you for all of your support. > To Alan : > It is true that my code still need some improvement on the efficiency part, > and thanks for pointing out which part that needs to be improved. My suggestions won't really do much for efficiency

Re: [Tutor] MemoryError

2015-12-30 Thread Satya Luzy
Thanks for testing out more of the program. Yes, as you can see, I'm using the continued fraction method on n, which in your case is 459. During the continued fraction process (referring to function cfract(n,boundary)), I set the boundary to store only 50 value of the first iteration. The result of

Re: [Tutor] MemoryError

2015-12-30 Thread Satya Luzy
Dear everyone, thank you for all of your support. To Alan : It is true that my code still need some improvement on the efficiency part, and thanks for pointing out which part that needs to be improved. Speaking of the nested loops, with my ability, I don't think I could simplify it, as the nested l

Re: [Tutor] MemoryError

2015-12-29 Thread Steven D'Aprano
On Wed, Dec 30, 2015 at 12:00:02AM +0700, Satya Luzy wrote: > Hello, > I am currently working on a program to find the prime factor of n, in which > n is a big integer. Using the Continued Fraction factorization method (I > will provide the source code below, please don't mind the variables). [...]

Re: [Tutor] MemoryError

2015-12-29 Thread Martin A. Brown
Hello there Satya, >I am currently working on a program to find the prime factor of n, in which >n is a big integer. Using the Continued Fraction factorization method (I >will provide the source code below, please don't mind the variables). I do not know the Continued Fraction factorization meth

Re: [Tutor] MemoryError

2015-12-29 Thread Danny Yoo
> To be honest this is probably a bit beyond the scope of the tutor > list which is aimed at questions about the Python language and > standard library. However I'll make a few observations (and assume > your logic is OK since you did test it on some smaller data first) Just to add to Alan's comm

Re: [Tutor] MemoryError

2015-12-29 Thread Steven D'Aprano
On Wed, Dec 30, 2015 at 12:00:02AM +0700, Satya Luzy wrote: > Hello, > I am currently working on a program to find the prime factor of n, in which > n is a big integer. Using the Continued Fraction factorization method (I > will provide the source code below, please don't mind the variables). Are

Re: [Tutor] MemoryError

2015-12-29 Thread Alan Gauld
On 29/12/15 17:00, Satya Luzy wrote: > Hello, > I am currently working on a program to find the prime factor of n, in which > n is a big integer. Using the Continued Fraction factorization method To be honest this is probably a bit beyond the scope of the tutor list which is aimed at questions ab

Re: [Tutor] MemoryError !!! Help Required

2008-04-07 Thread Andreas Kostyrka
Hint: MemoryError suggests that his dicts have filled up his address space (probably). 1-3GB on Linux, 2?GB on Windows. At least for 32bit versions. So storing the whole URL in memory is probably out of question, storing it only in some form of files might be slightly slow, so one compromise would

Re: [Tutor] MemoryError !!! Help Required

2008-04-07 Thread W W
I don't have a lot of experience, but I would suggest dictionaries (which use hash values). A possible scenario would be somthing similar to Andreas' visited = dict() url = "http://www.monty.com"; file = "/spam/holyhandgrenade/three.html" visited[url] = file unvisited = dict() url = "http://

Re: [Tutor] MemoryError !!! Help Required

2008-04-06 Thread Andreas Kostyrka
Am Montag, den 07.04.2008, 00:32 -0500 schrieb Luke Paireepinart: > devj wrote: > > Hi, > > I am making a web crawler using Python.To avoid dupliacy of urls,i have to > > maintain lists of downloaded urls and to-be-downloaded urls ,of which the > > latter grows exponentially,resulting in a MemoryE

Re: [Tutor] MemoryError !!! Help Required

2008-04-06 Thread Luke Paireepinart
devj wrote: > Hi, > I am making a web crawler using Python.To avoid dupliacy of urls,i have to > maintain lists of downloaded urls and to-be-downloaded urls ,of which the > latter grows exponentially,resulting in a MemoryError exception .What are > the possible ways to avoid this ?? > get more R

Re: [Tutor] MemoryError

2004-12-10 Thread Liam Clarke
Yeah, that's a bit of a brute force approach alright. Mainly all the (\n)?(=\n)?... I, personally, intend to have words with a Mr William Gates regarding just what horrors Frontpage inflicts. I also intend to discuss this with gmail.google.com, as they parsed the text attachment I sent myself, add

Re: [Tutor] MemoryError

2004-12-10 Thread Kent Johnson
Well, that's a regex only a mother could love. :-) I can see why you were happy to find the (?P) form of grouping. You should compile textObj and urlObj outside of getVals. You could just pull the four lines that create textObj and urlObj outside of the function (into global scope). You just hav

Re: [Tutor] MemoryError

2004-12-10 Thread Liam Clarke
Hi Kent, Thanks for the help, it worked third time around! The final product is here if you have an interest - http://www.rafb.net/paste/results/XCYthC70.html But, I think I found a new best friend for this sort of thing - (?P.*?) Being able to label stuff is brilliant. But yeah, thanks fo

Re: [Tutor] MemoryError

2004-12-09 Thread Kent Johnson
Liam, Here's a nifty re trick for you. The sub() method can take a function as the replacement parameter. Instead of replacing with a fixed string, the function is called with the match object. Whatever string the function returns, is substituted for the match. So you can simplify your code a bit

Re: [Tutor] MemoryError

2004-12-09 Thread Liam Clarke
Hi all, Yeah, I should've written this in functions from the get go, but I thought it would be a simple script. :/ I'll come back to that script when I've had some sleep, my son was recently born and it's amazing how dramatically lack of sleep affects my acuity. But, I want to figure out what's

Re: [Tutor] MemoryError

2004-12-09 Thread Jeff Shannon
Liam Clarke wrote: So, I'm going to throw caution to the wind, and try an re approach. It can't be any more unwieldy and ugly than what I've got going at the moment. If you're going to try a new approach, I'd strongly suggest using a proper html/xml parser instead of re's. You'll almost certainly

Re: [Tutor] MemoryError

2004-12-09 Thread Kent Johnson
Liam, I'm sorry to hear you so discouraged. It sourds like your program has gotten too big for you to clearly understand what it is doing. An earlier poster suggested that you break your program up into functions. This is a very good idea, especially if you do it from the start. A very powerful

Re: [Tutor] MemoryError

2004-12-09 Thread Liam Clarke
Well, I figured out the memory error relatively easily once I poked some stuff, I was using codeSt.find instead of codeSt.index, so it was returning no matches as -1, index raises an error for me, and I wasn't catering for that -1. The MemoryError occurred when Python wanted to slice from 160,000 t

Re: [Tutor] MemoryError

2004-12-08 Thread Jeff Shannon
Liam Clarke wrote: I'm not sure why you're getting the MemoryError, but it'd be easier to figure out if you posted the entire text of the traceback. Traceback: Line 39: seg=codeSt[element:endInd+len(endStr] MemoryError Hehe. Helpful, no? Actually, it was the "usual bit about in module" that might

Re: [Tutor] MemoryError

2004-12-08 Thread Alan Gauld
> Traceback: > > Line 39: seg=codeSt[element:endInd+len(endStr] > MemoryError > > Hehe. Helpful, no? Sometimes seeing the whole traceback gives clues as to whats throwing the wobbly, without the full stack its hard to tell. However before worrying about that I'd add a print statement to pri

Re: [Tutor] MemoryError

2004-12-08 Thread Liam Clarke
> I'm not sure why you're getting the MemoryError, but it'd be easier to > figure out if you posted the entire text of the traceback. Traceback: Line 39: seg=codeSt[element:endInd+len(endStr] MemoryError Hehe. Helpful, no? I think I'll sprinkle print statements throughout as suggested, and wr

Re: [Tutor] MemoryError

2004-12-08 Thread Liam Clarke
Oh, and I never knew about .read() for a file object. I always just used readline/readlines. Silly, really. On Thu, 9 Dec 2004 11:31:40 +1300, Liam Clarke <[EMAIL PROTECTED]> wrote: > > I'm not sure why you're getting the MemoryError, but it'd be easier to > > figure out if you posted the entire

Re: [Tutor] MemoryError

2004-12-08 Thread Jeff Shannon
Liam Clarke wrote: Hi all, I'm playing with a file, and attempting to replace a section with a string, and using the following command - seg=codeSt[element:endInd+len(endStr] codeSt=codeSt.replace(seg, hrefString) At the replace, I get a MemoryError. It's part of a for loop, but it gives the erro

Re: [Tutor] MemoryError

2004-12-08 Thread Michael Janssen
[forgot to cc to the list] -- Forwarded message -- From: Michael Janssen <[EMAIL PROTECTED]> Date: Wed, 8 Dec 2004 14:40:46 +0100 Subject: Re: [Tutor] MemoryError To: Liam Clarke <[EMAIL PROTECTED]> On Wed, 8 Dec 2004 23:29:38 +1300, Liam Clarke <[EMAIL PR