Re: [Tutor] Recursion depth exceeded in python web crawler

2018-06-14 Thread Mark Lawrence
On 14/06/18 19:32, Daniel Bosah wrote: I am trying to modify code from a web crawler to scrape for keywords from certain websites. However, Im trying to run the web crawler before I modify it, and I'm running into issues. When I ran this code - *import threading* *from Queue import Queue* *

Re: [Tutor] Recursion depth exceeded in python web crawler

2018-06-14 Thread Steven D'Aprano
On Thu, Jun 14, 2018 at 02:32:46PM -0400, Daniel Bosah wrote: > I am trying to modify code from a web crawler to scrape for keywords from > certain websites. However, Im trying to run the web crawler before I > modify it, and I'm running into issues. > > When I ran this code - [snip enormous co

Re: [Tutor] recursion depth

2014-01-09 Thread Dave Angel
On Thu, 9 Jan 2014 13:02:30 -0500, Keith Winston wrote: Well, hopefully this is plain text. It all looks the same to me, so if gmail switches back, it might go unnoticed for a while. Sorry for the incessant hassle. That looks great, thanks. -- DaveA _

Re: [Tutor] recursion depth

2014-01-09 Thread Keith Winston
On Thu, Jan 9, 2014 at 5:41 AM, Steven D'Aprano wrote: > > Keith, if you are able, and would be so kind, you'll help solve this > issue for Dave if you configure your mail client to turn so-called "rich > text" or formatted text off, at least for this mailing list. Well, hopefully this is plain t

Re: [Tutor] recursion depth

2014-01-09 Thread Dave Angel
On Thu, 9 Jan 2014 21:41:41 +1100, Steven D'Aprano wrote: I presume that your question is aimed at Keith. Yes, Keith's emails have a HTML part and a text part. A half-decent mail client should be able to read the text part even if the HTML part exists. But I believe you're reading this fro

Re: [Tutor] recursion depth

2014-01-09 Thread Steven D'Aprano
On Wed, Jan 08, 2014 at 06:16:03PM -0500, Dave Angel wrote: > On Wed, 8 Jan 2014 16:23:06 -0500, eryksun wrote: > >On Wed, Jan 8, 2014 at 3:25 PM, Keith Winston > wrote: > >> I've been playing with recursion, it's very satisfying. > >> > >> However, it appears that even if I sys.setrecursionlimi

Re: [Tutor] recursion depth

2014-01-08 Thread Keith Winston
On Wed, Jan 8, 2014 at 6:16 PM, Dave Angel wrote: > I can't see the bodies of any of your messages (are you perchance posting > in html? ), but I think there's a good chance you're abusing recursion and > therefore hitting the limit much sooner than necessary. I've seen some code > samples here

Re: [Tutor] recursion depth

2014-01-08 Thread Dave Angel
On Wed, 8 Jan 2014 16:23:06 -0500, eryksun wrote: On Wed, Jan 8, 2014 at 3:25 PM, Keith Winston wrote: > I've been playing with recursion, it's very satisfying. > > However, it appears that even if I sys.setrecursionlimit(10), it blows > up at about 24,000 (appears to reset IDLE). I gue

Re: [Tutor] recursion depth

2014-01-08 Thread Keith Winston
On Wed, Jan 8, 2014 at 5:15 PM, spir wrote: > Funny and useful exercise in recursion: write a func that builds str and > repr expressions of any object, whatever its attributes, inductively. Eg > with > Hmm, can't say I get the joke. I haven't really played with repr, though I think I understand

Re: [Tutor] recursion depth

2014-01-08 Thread Keith Winston
On Wed, Jan 8, 2014 at 4:23 PM, eryksun wrote: > You can create a worker thread with a larger stack using the threading > module. On Windows the upper limit is 256 MiB, so give this a try: > quite excellent, mwahaha... another shovel to help me excavate out the bottom of my hole... I'll play wi

Re: [Tutor] recursion depth

2014-01-08 Thread spir
On 01/08/2014 10:11 PM, Keith Winston wrote: On Wed, Jan 8, 2014 at 3:42 PM, Emile van Sebille wrote: Without seeing your code it's hard to be specific, but it's obvious you'll need to rethink your approach. :) Yes, it's clear I need to do the bulk of it without recusion, I haven't reall

Re: [Tutor] recursion depth

2014-01-08 Thread eryksun
On Wed, Jan 8, 2014 at 3:25 PM, Keith Winston wrote: > I've been playing with recursion, it's very satisfying. > > However, it appears that even if I sys.setrecursionlimit(10), it blows > up at about 24,000 (appears to reset IDLE). I guess there must be a lot of > overhead with recursion, if o

Re: [Tutor] recursion depth

2014-01-08 Thread Keith Winston
On Wed, Jan 8, 2014 at 3:42 PM, Emile van Sebille wrote: > > Without seeing your code it's hard to be specific, but it's obvious you'll > need to rethink your approach. :) Yes, it's clear I need to do the bulk of it without recusion, I haven't really thought about how to do that. I may or may

Re: [Tutor] recursion depth

2014-01-08 Thread Emile van Sebille
On 1/8/2014 12:25 PM, Keith Winston wrote: I've been playing with recursion, it's very satisfying. However, it appears that even if I sys.setrecursionlimit(10), it blows up at about 24,000 (appears to reset IDLE). I guess there must be a lot of overhead with recursion, if only 24k times are