Dear Terry,
Ok... perfect... You convinced me...
But doesn' t exist, in every configuration you could immagine, a machine
with a storage capability so big (2 teras or more) to store inside a so huge
database... For this reason it is important and necessary a cluster (for
processing operations even more), in wich distribuite the database. I don' t
know BerkeleyDB, but I don' t think it has a RDBS in it that permits
clustered databases...
Please correct me...
Ciao
tomi
----- Original Message -----
From: Terry Luedtke <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Wednesday, June 28, 2000 2:53 PM
Subject: Re: [htdig3-dev] Creating a SQL backend...
>>> "tomi" <[EMAIL PROTECTED]> 28-Jun-00 07:57:40 >>>
<SNIP>
> There were about 20,000,000 of html
> and hypertextual documents in all the servers (this would make me assume
> they' d be much more, because a great deal of the non reached where in
.ps,
> .pdf, or other format non followble).
> I did not test ht://Dig working to index this great ammount of datas, but
> everything let me thing that BerkeleyDB is not the appropriate way to do
it.
BerkeleyDB can handle databases of up to 281E12 bytes (2^48). See
http://www.sleepycat.com/featdetail.html#big_dbs Of course you would need a
fairly healthy machine to do that (lots of memory, large raid etc).
There are other ways to improve the speed while still using BerkeleyDB (or
any other db for that matter). The ability to run concurrent digs into the
same database for one. An htsearch that stays in memory, similar to
fast-cgi programs, for another.
Terry Luedtke
National Library of Medicine
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
