How are you using it? Streaming the files to Solr via HTTP? You can use Tika on the client to extract the various bits from the structured documents, and use SolrJ to assemble various bits of that data Tika exposes into a Solr document that you then send to Solr. At the point you're transferring data from the Tika parse to the Solr document, you could add any data from your database that you wanted.
The result is that you'd be indexing the complete Solr document only once. You're right that updating a document in Solr overwrites the previous version and any data in the previous version is lost.... Best Erick On Thu, Jun 9, 2011 at 1:20 PM, Greg Georges <greg.geor...@biztree.com> wrote: > Hello Erick, > > Thanks for the response. No, I am using the extract handler to extract the > data from my text files. In your second approach, you say I could use a DIH > to update the index which would have been created by the extract handler in > the first phase. I thought that lets say I get info from the DB and update > the index with the document ID, will I overwrite the data and lose the > initial data from the extract handler phase? Thanks > > Greg > > -----Original Message----- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: 9 juin 2011 12:15 > To: solr-user@lucene.apache.org > Subject: Re: Indexing data from multiple datasources > > Hmmm, when you say you use Tika, are you using some custom Java code? Because > if you are, the best thing to do is query your database at that point > and add whatever information > you need to the document. > > If you're using DIH to do the crawl, consider implementing a > Transformer to do the database > querying and modify the document as necessary.... This is pretty > simple to do, we can > chat a bit more depending on whether either approach makes sense. > > Best > Erick > > > > On Thu, Jun 9, 2011 at 10:43 AM, Greg Georges <greg.geor...@biztree.com> > wrote: >> Hello all, >> >> I have checked the forums to see if it is possible to create and index from >> multiple datasources. I have found references to SOLR-1358, but I don't >> think this fits my scenario. In all, we have an application where we upload >> files. On the file upload, I use the Tika extract handler to save metadata >> from the file (_attr, literal values, etc..). We also have a database which >> has information on the uploaded files, like the category, type, etc.. I >> would like to update the index to include this information from the db in >> the index for each document. If I run a dataimporthandler after the extract >> phase I am afraid that by updating the doc in the index by its id will just >> cause that I overwrite the old information with the info from the DB (what I >> understand is that Solr updates its index by ID by deleting first then >> recreating the info). >> >> Anyone have any pointers, is there a clean way to do this, or must I find a >> way to pass the db metadata to the extract handler and save it as literal >> fields? >> >> Thanks in advance >> >> Greg >> >
