Re: solr tuple/tag store

2007-10-09 Thread Pieter Berkel
On 10/10/2007, Ryan McKinley <[EMAIL PROTECTED]> wrote: > > Without seeing the actual queries that are slow, it's difficult to > determine > > what the problem is. Have you tried using EXPLAIN ( > > http://dev.mysql.com/doc/refman/5.0/en/explain.html) to check if your > query > > is using the tab

Re: solr tuple/tag store

2007-10-09 Thread Ryan McKinley
the most basic stuff, and copyField things around. With SOLR-139, to rebuild an index you simply reconfigure the copyField settings and basically `touch` each document to reindex it. had not thought of that... yes, that would work Yonik has some pretty prescient design ideas here: <

Re: solr tuple/tag store

2007-10-09 Thread Ryan McKinley
Pieter Berkel wrote: Given that the tables are of type InnoDB, I think it's safe to assume that you're not planning to use MySQL full-text search (only supported on MyISAM tables). I am only using SQL for the base store - it is only accessed for updating and generating solr documents. All s

Re: solr tuple/tag store

2007-10-09 Thread Lance Norskog
You could just make a separate Lucene index with the document ID unique and with multiple tag values. Your schema would have the entryID as the unique field and multiple tag values per entryID. I just made a phrase-suggesting clone of the Spellchecker class that is almost exactly the same. It ind

Re: solr tuple/tag store

2007-10-09 Thread Pieter Berkel
Given that the tables are of type InnoDB, I think it's safe to assume that you're not planning to use MySQL full-text search (only supported on MyISAM tables). If you are not concerned about transactional integrity provided by InnoDB, perhaps you could try using MyISAM tables (although most people

Re: solr tuple/tag store

2007-10-09 Thread Erik Hatcher
On Oct 9, 2007, at 3:14 PM, Ryan McKinley wrote: 2. Figure out how to keep the base Tuple store in solr. I think this will require finishing up SOLR-139. This would keep the the core data in solr - so there is no good way to 'rebuild' the index. With SOLR-139, cool stuff can be done to 'r

RE: solr tuple/tag store

2007-10-09 Thread Lance Norskog
ngle-record results are what Lucene does best. Lance -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 09, 2007 12:14 PM To: solr-user@lucene.apache.org Subject: solr tuple/tag store Hello- I am running into some scaling performance problems with SQL

solr tuple/tag store

2007-10-09 Thread Ryan McKinley
Hello- I am running into some scaling performance problems with SQL that I hope a clever solr solution could fix. I've already gone through a bunch of loops, so I figure I should solicit advice before continuing to chase my tail. I have a bunch of things (100K-500K+) that are defined by a s