Hoss I see what you mean. I am trying to implement a CustomUpdateProcessor
checking out here:
http://wiki.apache.org/solr/UpdateRequestProcessor
What is confusing me now is that I have to implement my logic in
processComit as you said:
>>you'll still need the "double commit" (once so you can see the
>>main changes, and once so the rest of the world can see your
>>modifications) but you can execute them both directly in your
>>processCommit(CommitUpdateCommand)
I have noticed that in the processAdd you have acces to the concrete
SolrInpuntDocument you are going to add:
SolrInputDocument doc = cmd.getSolrInputDocument();
But in processCommit, having access to the core I can get the IndexReader
but I still don't know how to get the IndexWriter and SolrInputDocuments in
there.
My idea is to do something like:
@Override
public void processCommit(CommitUpdateCommand cmd) throws IOException {
//first commit that show me modification
//open and iterate over the reader and create solrDocuments list
//close reader
//openwriter and update the docs in the list
//close writer and second commit that shows my changes to the world
if (next != null)
next.processCommit(cmd);
}
As I understood the process, the commitCommand will be sent to the
DirectUpdateHandler2. that will proper do the commit via
UpdateRequestProcessor.
Am I in the right way? I haven't dealed with CustomUpdateProcessor for
doing something after a commit is executed so I am a bit confused...
Thanks in advance.
hossman wrote:
>
>
> This thread all sounds really kludgy ... among other things the
> newSearcher listener is going to need to some how keep track of when it
> was called as a result of a "real" commit, vs when it was called as the
> result of a commit it itself triggered to make changes.
>
> wouldn't an easier place to implement this logic be in an UpdateProcessor?
> you'll still need the "double commit" (once so you can see the
> main changes, and once so the rest of the world can see your
> modifications) but you can execute them both directly in your
> processCommit(CommitUpdateCommand) method (so you don't have to worry
> about being able to tell them apart)
>
> : Date: Thu, 30 Jul 2009 10:14:16 +0530
> : From:
> :
> =?UTF-8?B?Tm9ibGUgUGF1bCDgtKjgtYvgtKzgtL/gtLPgtY3igI0gIOCkqOCli+CkrOCljeCk
> : s+CljQ==?= <noble.p...@corp.aol.com>
> : Reply-To: solr-user@lucene.apache.org, noble.p...@gmail.com
> : To: solr-user@lucene.apache.org
> : Subject: Re: update some index documents after indexing process is done
> with
> : DIH
> :
> : If you make your EventListener implements SolrCoreAware you can get
> : hold of the core on inform. use that to get hold of the
> : SolrIndexWriter
> :
> : On Wed, Jul 29, 2009 at 9:20 PM, Marc Sturlese<marc.sturl...@gmail.com>
> wrote:
> : >
> : > From the newSearcher(..) of a CustomEventListener which extends of
> : > AbstractSolrEventListener can access to SolrIndexSearcher and all
> core
> : > properties but can't get a SolrIndexWriter. Do you now how can I get
> from
> : > there a SolrIndexWriter? This way I would be able to modify the
> documents (I
> : > need to modify them depending on values of other documents, that's why
> I
> : > can't do it with DIH delta-import).
> : > Thanks in advance
> : >
> : >
> : > Noble Paul നോബിള് नोब्ळ्-2 wrote:
> : >>
> : >> On Tue, Jul 28, 2009 at 5:17 PM, Marc
> Sturlese<marc.sturl...@gmail.com>
> : >> wrote:
> : >>>
> : >>> That really sounds the best way to reach my goal. How could I
> invoque a
> : >>> listener from the newSearcher?Would be something like:
> : >>> <listener event="newSearcher" class="solr.QuerySenderListener">
> : >>> <arr name="queries">
> : >>> <lst> <str name="q">solr</str> <str name="start">0</str> <str
> : >>> name="rows">10</str> </lst>
> : >>> <lst> <str name="q">rocks</str> <str name="start">0</str>
> <str
> : >>> name="rows">10</str> </lst>
> : >>> <lst><str name="q">static newSearcher warming query from
> : >>> solrconfig.xml</str></lst>
> : >>> </arr>
> : >>> </listener>
> : >>> <listener event="newSearcher" class="solr.MyCustomListener">
> : >>>
> : >>> And MyCustomListener would be the class who open the reader:
> : >>>
> : >>> RefCounted<SolrIndexSearcher> searchHolder = null;
> : >>> try {
> : >>> searchHolder = dataImporter.getCore().getSearcher();
> : >>> IndexReader reader = searchHolder.get().getReader();
> : >>>
> : >>> //Here I iterate over the reader doing docuemnt
> modifications
> : >>>
> : >>> } finally {
> : >>> if (searchHolder != null) searchHolder.decref();
> : >>> }
> : >>> } catch (Exception ex) {
> : >>> LOG.info("error");
> : >>> }
> : >>
> : >> you may not be able to access the DIH API from a newSearcher event .
> : >> But the API would give you the searcher directly as a method
> : >> parameter.
> : >>>
> : >>> Finally, to access to documents and add fields to some of them, I
> have
> : >>> thought in using SolrDocument classes. Can you please point me where
> : >>> something similar is done in solr source (I mean creation of
> : >>> SolrDocuemnts
> : >>> and conversion of them to proper lucene docuements).
> : >>>
> : >>> Does this way for reaching the goal makes sense?
> : >>>
> : >>> Thanks in advance
> : >>>
> : >>>
> : >>>
> : >>> Noble Paul നോബിള് नोब्ळ्-2 wrote:
> : >>>>
> : >>>> when a core is reloaded the event fired is firstSearcher.
> newSearcher
> : >>>> is fired when a commit happens
> : >>>>
> : >>>>
> : >>>> On Tue, Jul 28, 2009 at 4:19 PM, Marc
> Sturlese<marc.sturl...@gmail.com>
> : >>>> wrote:
> : >>>>>
> : >>>>> Ok, but if I handle it in a newSearcher listener it will be
> executed
> : >>>>> every
> : >>>>> time I reload a core, isn't it? The thing is that I want to use an
> : >>>>> IndexReader to load in a HashMap some doc fields of the index and
> : >>>>> depending
> : >>>>> of the values of some field docs modify other docs. Its very
> memory
> : >>>>> consuming (I have tested it with a simple lucene script). Thats
> why I
> : >>>>> wanted
> : >>>>> to do it just after the indexing process.
> : >>>>>
> : >>>>> My ideal case would be to do it in the commit function of
> : >>>>> DirectUpdatehandler2.java just before
> : >>>>> writer.optimize(cmd.maxOptimizeSegments); is executed. But I don't
> want
> : >>>>> to
> : >>>>> mess that code... so trying to find out the best way to do that as
> a
> : >>>>> plugin
> : >>>>> instead of a hack as possible.
> : >>>>>
> : >>>>> Thanks in advance
> : >>>>>
> : >>>>>
> : >>>>> Noble Paul നോബിള് नोब्ळ्-2 wrote:
> : >>>>>>
> : >>>>>> It is best handled as a 'newSearcher' listener in solrconfig.xml.
> : >>>>>> onImportEnd is invoked before committing
> : >>>>>>
> : >>>>>> On Tue, Jul 28, 2009 at 3:13 PM, Marc
> : >>>>>> Sturlese<marc.sturl...@gmail.com>
> : >>>>>> wrote:
> : >>>>>>>
> : >>>>>>> Hey there,
> : >>>>>>> I would like to be able to do something like: After the indexing
> : >>>>>>> process
> : >>>>>>> is
> : >>>>>>> done with DIH I would like to open an indexreader, iterate over
> all
> : >>>>>>> docs,
> : >>>>>>> modify some of them depending on others and delete some others.
> I can
> : >>>>>>> easy
> : >>>>>>> do this directly coding with lucene but would like to know if
> there's
> : >>>>>>> a
> : >>>>>>> way
> : >>>>>>> to do it with Solr using SolrDocument or SolrInputDocument
> classes.
> : >>>>>>> I have thougth in using SolrJ or DIH listener onImportEnd but
> not
> : >>>>>>> sure
> : >>>>>>> if
> : >>>>>>> I
> : >>>>>>> can get an IndexReader in there.
> : >>>>>>> Any advice?
> : >>>>>>> Thanks in advance
> : >>>>>>> --
> : >>>>>>> View this message in context:
> : >>>>>>>
> http://www.nabble.com/update-some-index-documents-after-indexing-process-is-done-with-DIH-tp24695947p24695947.html
> : >>>>>>> Sent from the Solr - User mailing list archive at Nabble.com.
> : >>>>>>>
> : >>>>>>>
> : >>>>>>
> : >>>>>>
> : >>>>>>
> : >>>>>> --
> : >>>>>> -----------------------------------------------------
> : >>>>>> Noble Paul | Principal Engineer| AOL | http://aol.com
> : >>>>>>
> : >>>>>>
> : >>>>>
> : >>>>> --
> : >>>>> View this message in context:
> : >>>>>
> http://www.nabble.com/update-some-index-documents-after-indexing-process-is-done-with-DIH-tp24695947p24696872.html
> : >>>>> Sent from the Solr - User mailing list archive at Nabble.com.
> : >>>>>
> : >>>>>
> : >>>>
> : >>>>
> : >>>>
> : >>>> --
> : >>>> -----------------------------------------------------
> : >>>> Noble Paul | Principal Engineer| AOL | http://aol.com
> : >>>>
> : >>>>
> : >>>
> : >>> --
> : >>> View this message in context:
> : >>>
> http://www.nabble.com/update-some-index-documents-after-indexing-process-is-done-with-DIH-tp24695947p24697751.html
> : >>> Sent from the Solr - User mailing list archive at Nabble.com.
> : >>>
> : >>>
> : >>
> : >>
> : >>
> : >> --
> : >> -----------------------------------------------------
> : >> Noble Paul | Principal Engineer| AOL | http://aol.com
> : >>
> : >>
> : >
> : > --
> : > View this message in context:
> http://www.nabble.com/update-some-index-documents-after-indexing-process-is-done-with-DIH-tp24695947p24722111.html
> : > Sent from the Solr - User mailing list archive at Nabble.com.
> : >
> : >
> :
> :
> :
> : --
> : -----------------------------------------------------
> : Noble Paul | Principal Engineer| AOL | http://aol.com
> :
>
>
>
> -Hoss
>
>
--
View this message in context:
http://www.nabble.com/update-some-index-documents-after-indexing-process-is-done-with-DIH-tp24695947p24741985.html
Sent from the Solr - User mailing list archive at Nabble.com.
