I've worked with a share of big log files and seen a lot of others on this list who have. Memory issues *do* come up from time to time. In fact there's a whole doc page about it (http://analog.cx/docs/lowmem.html). It seems to me that issues of referrer, or host tables become too large for memory long before the DNS file does. I suppose that DNS/Host is the same thing. Why don't you try it and see what happens. Maybe some use of HOSTLOWMEM command can help if you run out of memory.
-- Jeremy Wadsack Wadsack-Allen Digital Group S Collis <[EMAIL PROTECTED]> (Tuesday, February 17, 2004 10:29 AM): > My dns cache at work is about 8MB. My home one is about 1.5MB, and that's 18,824 > lookups. That's an average of 82 characters a lookup, which means my work one is > roughly 103,200 lookups. Each > report takes about 1/4 hour, and analog pushes the 64MB mark on WinNT task manager > (which is a bit economical with the truth in terms of missing out dlls and > libraries, but with analog, I think we > can trust it.) > So I'd say that a 20-million line DNSFILE *might* cause analog to choke, depending > on the amount of lines you're running the report on. > Personally, I'd use open source Interbase rather than mysql, but I guess I'm just an > old Borland bigot... :) > SC > *********** REPLY SEPARATOR *********** > On 16/02/2004 at 14:57 Cliff Addy wrote: >>I've slogged through the site/faq/archive and couldn't find an answer on >>this. How does analog handle the DNSFILE? Does it load it entirely into >>memory? Create it's own hash? Just do a linear search? >> >>The reason I'd like to know is that I used analog on a small hosting >>company, where I set up a system to reverse large number of IP addresses >>and keep them in a mysql database. I then patched analog to use this >>database for lookups. There are millions and millions of reversed IP >>addresses in this database and mysql is getting a little dodgy handling >>it. I'm setting up a new stats system for another company and I'm >>wondering if a 20-million line DNSFILE will cause analog to choke or eat a >>rediculous amount of memory. If so, I have a plan for a different >>database scheme that should be more elegant, but a lot of work. >> >>So, any experience with really big dns cache files? >> >>Cliff >> +------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +------------------------------------------------------------------------
