Thanks a lot Steve!
On Wed, Jul 11, 2018 at 10:24 AM Steve Rowe <sar...@gmail.com> wrote:
> Hi Jerome,
>
> I was able to setup a configset to perform OpenNLP NER, loading the model
> files from local storage.
>
> There is a trick though[1]: the model files must be located *in a jar* or
> *in a subdirectory* under ${solr.solr.home}/lib/ or under a directory
> specified via a solrconfig.xml <lib> directive.
>
> I tested with the bin/solr cloud example, and put model files under the
> two solr home directories, at example/cloud/node1/solr/lib/opennlp/ and
> example/cloud/node1/solr/lib/opennlp/. The “opennlp/“ subdirectory is
> required, though its name can be anything else you choose.
>
> [1] As you noted, ZkSolrResourceLoader delegates to its parent classloader
> when it can’t find resources in a configset, and the parent classloader is
> set up to load from subdirectories and jar files under
> ${solr.solr.home}/lib/ or under a directory specified via a solrconfig.xml
> <lib> directive. These directories themselves are not included in the set
> of directories from which resources are loaded; only their children are.
>
> --
> Steve
> www.lucidworks.com
>
> > On Jul 9, 2018, at 10:10 PM, Jerome Yang <jey...@pivotal.io> wrote:
> >
> > Hi Steve,
> >
> > Put models under " ${solr.solr.home}/lib/ " is not working.
> > I check the "ZkSolrResourceLoader" seems it will first try to find modes
> in
> > config set.
> > If not find, then it uses class loader to load from resources.
> >
> > Regards,
> > Jerome
> >
> > On Tue, Jul 10, 2018 at 9:58 AM Jerome Yang <jey...@pivotal.io> wrote:
> >
> >> Thanks Steve!
> >>
> >>
> >> On Tue, Jul 10, 2018 at 5:20 AM Steve Rowe <sar...@gmail.com> wrote:
> >>
> >>> Hi Jerome,
> >>>
> >>> See the ref guide[1] for a writeup of how to enable uploading files
> >>> larger than 1MB into ZooKeeper.
> >>>
> >>> Local storage should also work - have you tried placing OpenNLP model
> >>> files in ${solr.solr.home}/lib/ ? - make sure you do the same on each
> node.
> >>>
> >>> [1]
> >>>
> https://lucene.apache.org/solr/guide/7_4/setting-up-an-external-zookeeper-ensemble.html#increasing-the-file-size-limit
> >>>
> >>> --
> >>> Steve
> >>> www.lucidworks.com
> >>>
> >>>> On Jul 9, 2018, at 12:50 AM, Jerome Yang <jey...@pivotal.io> wrote:
> >>>>
> >>>> Hi guys,
> >>>>
> >>>> In Solrcloud mode, where to put the OpenNLP models?
> >>>> Upload to zookeeper?
> >>>> As I test on solr 7.3.1, seems absolute path on local host is not
> >>> working.
> >>>> And can not upload into zookeeper if the model size exceed 1M.
> >>>>
> >>>> Regards,
> >>>> Jerome
> >>>>
> >>>> On Wed, Apr 18, 2018 at 9:54 AM Steve Rowe <sar...@gmail.com> wrote:
> >>>>
> >>>>> Hi Alexey,
> >>>>>
> >>>>> First, thanks for moving the conversation to the mailing list.
> >>> Discussion
> >>>>> of usage problems should take place here rather than in JIRA.
> >>>>>
> >>>>> I locally set up Solr 7.3 similarly to you and was able to get things
> >>> to
> >>>>> work.
> >>>>>
> >>>>> Problems with your setup:
> >>>>>
> >>>>> 1. Your update chain is missing the Log and Run update processors at
> >>> the
> >>>>> end (I see these are missing from the example in the javadocs for the
> >>>>> OpenNLP NER update processor; I’ll fix that):
> >>>>>
> >>>>> <processor class="solr.LogUpdateProcessorFactory" />
> >>>>> <processor class="solr.RunUpdateProcessorFactory" />
> >>>>>
> >>>>> The Log update processor isn’t strictly necessary, but, from <
> >>>>>
> >>>
> https://lucene.apache.org/solr/guide/7_3/update-request-processors.html#custom-update-request-processor-chain
> >>>>>> :
> >>>>>
> >>>>> Do not forget to add RunUpdateProcessorFactory at the end of any
> >>>>> chains you define in solrconfig.xml. Otherwise update requests
> >>>>> processed by that chain will not actually affect the indexed
> >>> data.
> >>>>>
> >>>>> 2. Your example document is missing an “id” field.
> >>>>>
> >>>>> 3. For whatever reason, the pre-trained model "en-ner-person.bin"
> >>> doesn’t
> >>>>> extract anything from text “This is Steve Jobs 2”. It will extract
> >>> “Steve
> >>>>> Jobs” from text “This is Steve Jobs in white” e.g. though.
> >>>>>
> >>>>> 4. (Not a problem necessarily) You may want to use a multi-valued
> >>> “string”
> >>>>> field for the “dest” field in your update chain, e.g. “people_str”
> >>> (“*_str”
> >>>>> in the default configset is so configured).
> >>>>>
> >>>>> --
> >>>>> Steve
> >>>>> www.lucidworks.com
> >>>>>
> >>>>>> On Apr 17, 2018, at 8:23 AM, Alexey Ponomarenko <
> >>> alex1989s...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi once more I am trying to implement named entities extraction
> using
> >>>>> this
> >>>>>> manual
> >>>>>>
> >>>>>
> >>>
> https://lucene.apache.org/solr/7_3_0//solr-analysis-extras/org/apache/solr/update/processor/OpenNLPExtractNamedEntitiesUpdateProcessorFactory.html
> >>>>>>
> >>>>>> I am modified solrconfig.xml like this:
> >>>>>>
> >>>>>> <updateRequestProcessorChain name="multiple-extract">
> >>>>>> <processor
> >>>>> class="solr.OpenNLPExtractNamedEntitiesUpdateProcessorFactory">
> >>>>>> <str name="modelFile">opennlp/en-ner-person.bin</str>
> >>>>>> <str name="analyzerFieldType">text_opennlp</str>
> >>>>>> <str name="source">description_en</str>
> >>>>>> <str name="dest">content</str>
> >>>>>> </processor>
> >>>>>> </updateRequestProcessorChain>
> >>>>>>
> >>>>>> But when I was trying to add data using:
> >>>>>>
> >>>>>> *request:*
> >>>>>>
> >>>>>> POST
> >>>>>>
> >>>>>
> >>>
> http://localhost:8983/solr/numberplate/update?version=2.2&wt=xml&update.chain=multiple-extract
> >>>>>>
> >>>>>> <add><doc><field name="description_en">This is Steve Jobs 2
> >>>>>> </field><field name="content_pos">This is text 2</field><field
> >>>>>> name="content">This is text for content 2</field></doc></add>
> >>>>>>
> >>>>>> *response*
> >>>>>>
> >>>>>> <?xml version="1.0" encoding="UTF-8"?>
> >>>>>> <response>
> >>>>>> <lst name="responseHeader">
> >>>>>> <int name="status">0</int>
> >>>>>> <int name="QTime">3</int>
> >>>>>> </lst>
> >>>>>> </response>
> >>>>>>
> >>>>>> But I don't see any data inserted to *content* field and in any
> other
> >>>>> field.
> >>>>>>
> >>>>>> *If you need some additional data I can provide it.*
> >>>>>>
> >>>>>> Can you help me? What have I done wrong?
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> Pivotal Greenplum | Pivotal Software, Inc. <https://pivotal.io/>
> >>>
> >>>
> >>
> >> --
> >> Pivotal Greenplum | Pivotal Software, Inc. <https://pivotal.io/>
> >>
> >>
> >
> > --
> > Pivotal Greenplum | Pivotal Software, Inc. <https://pivotal.io/>
>
>
--
Pivotal Greenplum | Pivotal Software, Inc. <https://pivotal.io/>
