Re: problem with solr plugin

[davidphilip cherian]

Hi Sara,

The error is clear: class not found exception, which means solr couldn't
locate that jar file.

If you are not using solr-cloud then place that custom jar under
solr_home/lib folder.
You can also hard code the path of this jar file in solrconfig.xml under
/lib element.

If you are using solr-cloud, I think you should upload that zookeeper.



On Sat, Dec 19, 2015 at 5:58 PM, sara hajili  wrote:

> hi i wanna to have own normalization .
> i write 2 class one class form normalization filter factort that extends
> token filter factory and imoplement multiTermAwarecomponent
> and an other one class is normalization factory that extends token filter.
> then i create a jar from this classes with dependencies .
> and i add this jar file to solr_home/dist and
> solr_home/contrib/extraction/lib
> and i use this class in schema in this way:
>  positionIncrementGap="100">
> 
>  
>  class="com.ponila.set.textanalyzer.PersianCustomNormalizerFilterFactory"/>
>
> 
> but i get this error when i added a core to solr:
>
> org.apache.solr.common.SolrException: Could not load conf for core
> post: Plugin init failure for [schema.xml] fieldType
> "sample_normalizer": Plugin init failure for [schema.xml]
> analyzer/filter: Error loading class
> 'com.ponila.set.textanalyzer.PersianCustomNormalizerFilterFactory'.
> Schema file is D:\solr-5.3.1\example\poinila\solr\post\conf\schema.xml
> at
> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:80)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:721)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:697)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:629)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:675)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.solr.common.SolrException: Plugin init failure
> for [schema.xml] fieldType "sample_normalizer": Plugin init failure
> for [schema.xml] analyzer/filter: Error loading class
> 'com.ponila.set.textanalyzer.PersianCustomNormalizerFilterFactory'.
> Schema file is D:\solr-5.3.1\example\poinila\solr\post\conf\schema.xml
> at
> org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:596)
> at org.apache.solr.schema.IndexSchema.(IndexSchema.java:175)
> at
> org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
> at
> org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
> at
> org.apache.solr.core.C

Re: Having replica will slow down Solr?

[davidphilip cherian]

Few questions:

What are your memory configs: xms and xmx values, available ram size?

Were you also indexing at the time when the search response was so slow? In
the sense, were you committing often with openSearcher=true, in any case?

What is the QTime for consequent searches? Are the caches enabled?






On Wed, Dec 30, 2015 at 1:50 PM, Binoy Dalal  wrote:

> If I understand the cloud setup correctly having a replica will serve for
> redundancy and won't slow your searches down.
> In your case for the slow response times, are you returning any large
> stored fields?
>
> On Wed, 30 Dec 2015, 13:26 Zheng Lin Edwin Yeo 
> wrote:
>
> > Hi,
> >
> > I would like to find out, will having a replica slow down the search for
> > Solr?
> >
> > Currently, I'm having 1 shard and a replicationFactor of 2 using Solr
> > 5.3.0. I'm running SolrCloud, with 3 external ZooKeeper using ZooKeeper
> > 3.4.6, and my index size is 183GB.
> >
> > I have been getting QTime of more than 3000ms for my basic search
> function,
> > even without adding other things like faceting or highlighting.
> >
> > Regards,
> > Edwin
> >
> --
> Regards,
> Binoy Dalal
>

Re: Using post.jr for indexing in Solr 5.4.0

[davidphilip cherian]

Did you try using the latest post jar (shipped with 5.4 version) to index
same docs?
What happened? Did you get the same error or no?



On Thu, Dec 31, 2015 at 11:12 AM, Zheng Lin Edwin Yeo 
wrote:

> Hi,
>
> Would like to find out, is there any changes to the post.jar for Solr 5.4?
> I tried to use the post.jar from Solr 5.3.0, and it gives me the following
> error during indexing:
>
> java.lang.NoSuchMethodError:
> org.apache.solr.h
>
> andler.extraction.ExtractingDocumentLoader.(Lorg/apache/solr/request
>
> /SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcessor;Lorg/
>
> apache/tika/config/TikaConfig;Lorg/apache/solr/handler/extraction/ParseContextCo
>
> nfig;Lorg/apache/solr/handler/extraction/SolrContentHandlerFactory;)V name="trace">java.lang.RuntimeException: java.lang.NoSuchMethodError:
> org.apache
>
> .solr.handler.extraction.ExtractingDocumentLoader.(Lorg/apache/solr/
>
> request/SolrQueryRequest;Lorg/apache/solr/update/processor/UpdateRequestProcesso
>
> r;Lorg/apache/tika/config/TikaConfig;Lorg/apache/solr/handler/extraction/ParseCo
>
> ntextConfig;Lorg/apache/solr/handler/extraction/SolrContentHandlerFactory;)V
> at
> org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:611)
>
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:472)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
> r.java:222)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
> r.java:181)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
> Handler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java
> :585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
> ava:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav
> a:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl
> er.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
> er.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
> 515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle
> r.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
> r.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
> ava:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Cont
> extHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColl
> ection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
> .java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav
> a:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java
> :540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo
> l.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool
> .java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.solr.handler.extraction.Extra
>
> ctingDocumentLoader.(Lorg/apache/solr/request/SolrQueryRequest;Lorg/
>
> apache/solr/update/processor/UpdateRequestProcessor;Lorg/apache/tika/config/Tika
>
> Config;Lorg/apache/solr/handler/extraction/ParseContextConfig;Lorg/apache/solr/h
> andler/extraction/SolrContentHandlerFactory;)V
> at
> org.apache.solr.handler.extraction.ExtractingRequestHandler.newLoader
> (ExtractingRequestHandler.java:130)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
> ntentStreamHandlerBase.java:59)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
> erBase.java:156)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:457)
> ... 22 more
> 500
> 
>
> Regards,
> Edwin
>

Re: Querying with action parameter included in URL

[davidphilip cherian]

Hi Vidya,

I think you are confused with solr search queries/requests with solr other
restful apis to perform CRUD operations on collections.

Sample of search queries are list here with standard query parser :
https://cwiki.apache.org/confluence/display/solr/The+Standard+Query+Parser

Solr collection apis to perform crud operations on collections :
https://cwiki.apache.org/confluence/display/solr/Collections+API



On Tue, Jan 5, 2016 at 12:37 PM, Binoy Dalal  wrote:

> I think that all this will do is create a config file with the name
> booksConfig based on a template. This and other calls like these are solr's
> core admin api calls that you make through http requests.
> You don't need to make any changes to your schema or solrconfig files in
> order to execute such calls.
>
> On Tue, 5 Jan 2016, 11:57 vidya  wrote:
>
> > Hi
> >
> > I am pretty new to solr and when i am going through the tutorials , I
> came
> > across urls for querying like
> > "
> >
> http://localhost:8983/solr/admin/configs?action=CREATE&name=booksConfig&baseConfigSet=genericTemplate
> > "
> > .
> > I wanted to know how to implement the same by doing changes in schema.xml
> > or
> > solrconfig.xml. Where should i make changes when an "action=" is
> specified.
> >
> > Thanks in advance
> >
> >
> >
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Querying-with-action-parameter-included-in-URL-tp4248576.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
> --
> Regards,
> Binoy Dalal
>

Re: solr 5.2.0 need to build high query response

[davidphilip cherian]

You should use solr softcommit for this use case. So, by setting softcommit
to 5 seconds and autoCommit to minute with openSearcher=false should do the
work.

 
 6
false
 


2000


Reference link-
https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching

To know more about solr hard and soft commits, have a look at this blog :
https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin  wrote:

> Hi guys,
>
> I'm having trouble to figure what would be idle solr config for where:
>
> I'm doing hard commit in every minute   for very few number of users
> because I have to show those docs in search results quickly when user save
> the changes.
>
> It is causing the response in around  2 secs to show even I am getting only
> 10 records.
>
> Could you able to give some idea where to look at.
>
>
> Thanks in advance,
> Novin
>

Re: Configuring cores to persist in the event of Solr restart

[davidphilip cherian]

What do you mean by cores getting deleted? Files created on filesystem for
these cores disappear?
How are you starting and stopping solr? Is this solr cloud or standalone
mode?


On Sun, Jan 10, 2016 at 2:59 PM, Allan Kamau  wrote:

> We are able to load several cores into Solr 5.3.1.
> The problem is that after a restart of the server, these cores seem to get
> deleted.
> Is there a way to make cores loaded in Solr 5.x survive a server restart.
> Could there be a setting in solr.xml or perhaps the "core.properties" files
> that would enable core(s) survivability in the event of server restart.
> I have searched for a solution to this issue but found none, though I feel
> that it has already been answered.
>
>
> Solr version
> 5.3.1
>
> Java version
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
> OS version
> CentOS release 6.5 (Final)
>
>
>
> Allan.
>

Re: Error while reloading collection

[davidphilip cherian]

You should probably ask this question here

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.7.0/Cloudera-Manager-Introduction/cmi_getting_help_and_support.html

On Thu, Jan 14, 2016 at 12:11 PM, vidya  wrote:

> Hi
>  I am using solrcloud on cloudera cluster. I have created collections using
> solrctl command which is supported by cloudera search tool. I included one
> class of java in schema.xml for creating a field type which is dependent on
> a jar that i have included in solrconfig.xml. But when i reload that
> collection, I am getting an error that ERROR LOADING THAT CLASS what i
> included in schema.xml. What else do i need to include ?
>
> In solrconfig.xml :  regex=".*\.jar" />
>
> Error while relaoding in command line interface :
>
> 
>
> 
>
> 
> 
> 0
> 
> 197
> 
> 
> 
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> handling 'reload' action
> 
>
> 
>
>
>
> Please help me on this.Thanks in advance
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Error-while-reloading-collection-tp4250635.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Solr node 'Gone' status

[davidphilip cherian]

Hi,

Solr-admin cloud view page has got another new radio button indicating
status of node :  'Gone' status. What does that mean?  One of my collection
is in that state and it is not serving any request. How to bring that up?

Re: Close and open vendor sorting

[davidphilip cherian]

Hi Irshad,

So, assuming that each vendor information is one solr document, you will
have information regarding the vendors open-close hours correct? You should
be indexing this content in one of the fields, isn't it? If yes, then you
should try something as explained:
When the user searches, *capture the current time and search across this
field by boosting it with high weightage.  "date-time" field type can be
helpful with range queries. *




On Fri, Jan 22, 2016 at 3:44 PM, irshad siddiqui 
wrote:

> Hi ,
> Thanks prateek for your reply.
>
> My query is i have multiple opening and closing hours, within the same day.
> how to manage index and search query to get all opening  first than close
>
> i don't think below url will solve my problem.
> https://wiki.apache.org/solr/SpatialForTimeDurations
>
> please suggest
>
> Regards,
> Irshad
>
>
> On Fri, Jan 22, 2016 at 2:30 PM, Prateek Jain J <
> prateek.j.j...@ericsson.com
> > wrote:
>
> >
> > According to me this is what you are looking for
> >
> > https://wiki.apache.org/solr/QueryElevationComponent
> >
> >
> > Regards,
> > Prateek Jain
> > Team: Totoro
> > Mobile: +353 894 391716
> >
> >
> > -Original Message-
> > From: irshad siddiqui [mailto:irshad.s...@gmail.com]
> > Sent: 22 January 2016 07:32 AM
> > To: solr-user@lucene.apache.org
> > Subject: Close and open vendor sorting
> >
> > Hi,
> >
> > I am using solr 5.1 in my search . my search are working fine
> >
> > today i got one new requirement that show first all open vendor and then
> > close vendor.
> >
> > All vendor have different time oping and closing for a day
> >
> > Example:*"Royal China"*  vendor on Friday have 3 time slot for opening
> and
> > closing.
> >
> >
> >- 11am to 2 pm
> >- 5pm to 8pm
> >- 10pm to 11:30pm
> >
> >
> >
> > Kindly let me know how to show first open vendor and than close vendor in
> > single search query. please help
> >
> > Thanks
> >
> > Regards,
> > Irshad
> >
>

hitratio vs cumulative_hitratio

[davidphilip cherian]

Solr caching : What does it mean have lookup=0, hits=0 and hitratio=0 but
cumulative_hitratio=0.75 and cumulative_lookups >100,000 with
cumulative_inserts >20k and cumulative_evictions =0,  maxSize of cache
objects is 512

Re: understanding phonetic matching

[davidphilip cherian]

The "phonetic_en" analyzer definition available in solr-schema does return
documents having "Jon", "JN", "John" when search term is "John". Checkout
screen shot here : http://imgur.com/0R6SvX2

This wiki page explains how phonetic matching works :
https://cwiki.apache.org/confluence/display/solr/Phonetic+Matching#PhoneticMatching-DoubleMetaphone


Hope that helps.



On Wed, Mar 23, 2016 at 11:18 AM, Alexandre Rafalovitch 
wrote:

> I'd start by putting LowerCaseFF before the PhoneticFilter.
>
> But then, you say you were using Analysis screen and what? Do you get
> the matches when you put your sample text and the query text in the
> two boxes in the UI? I am not sure what "look at my solr data" means
> in this particular context.
>
> Regards,
>Alex.
> 
> Newsletter and resources for Solr beginners and intermediates:
> http://www.solr-start.com/
>
>
> On 23 March 2016 at 16:27, Jay Potharaju  wrote:
> > Hi,
> > I am trying to do name matching using the phonetic filter factory. As
> part
> > of that I was analyzing the data using analysis screen in solr UI. If i
> > search for john, any documents containing john or jon should be found.
> >
> > Following is my definition of the custom field that I use for indexing
> the
> > data. When I look at my solr data I dont see any similar sounding names
> in
> > my solr data, even though I have set inject="true". Is that not how it is
> > supposed to work?
> > Can someone explain how phonetic matching works?
> >
> >   positionIncrementGap
> > ="100">
> >
> >  
> >
> > 
> >
> >  encoder="DoubleMetaphone"
> > inject="true" maxCodeLength="5"/>
> >
> > 
> >
> >  
> >
> > 
> >
> > --
> > Thanks
> > Jay
>

Contrib module for Document Clustering

[davidphilip cherian]

Hi,

Is there any contribution(open source contrib module) that routes documents
to shards based on document similarity technique? Or any suggestions that
integrates mahout to solr for this use case?

>From what I know, currently there are two document route strategies as
explained here
https://lucidworks.com/blog/2013/06/13/solr-cloud-document-routing/. But Is
there anything else that I'm missing?




Thanks.

Re: Contrib module for Document Clustering

[davidphilip cherian]

Any thoughts?


On Tue, Apr 5, 2016 at 9:05 PM, davidphilip cherian <
davidphilipcher...@gmail.com> wrote:

> Hi,
>
> Is there any contribution(open source contrib module) that routes
> documents to shards based on document similarity technique? Or any
> suggestions that integrates mahout to solr for this use case?
>
> From what I know, currently there are two document route strategies as
> explained here
> https://lucidworks.com/blog/2013/06/13/solr-cloud-document-routing/. But
> Is there anything else that I'm missing?
>
>
>
>
> Thanks.
>
>
>

Re: Contrib module for Document Clustering

[davidphilip cherian]

Hi Joel,

Right now, we are (web) crawling almost 85millions of documents and this
can increase to double. Collection is plainly divided into shards and so
while searching, its search across all shards.
If it is possible for a system to distributed documents into shards based
on documents similarity, and at search time, analyze the query and search
across these shards, it can improve search time performance and reduce
resource utilization as well.  Let me know your thoughts. Use Case: Since
this is a web search kind of data, both false positives and false negatives
to an extent should be fine.



On Wed, Apr 6, 2016 at 11:18 PM, Joel Bernstein  wrote:

> I don't know of any contrib or module that does this. Can you describe why
> you'd want to route documents to shards based on similarity? What
> advantages would you get by using this approach?
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Wed, Apr 6, 2016 at 1:36 PM, davidphilip cherian <
> davidphilipcher...@gmail.com> wrote:
>
> > Any thoughts?
> >
> >
> > On Tue, Apr 5, 2016 at 9:05 PM, davidphilip cherian <
> > davidphilipcher...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Is there any contribution(open source contrib module) that routes
> > > documents to shards based on document similarity technique? Or any
> > > suggestions that integrates mahout to solr for this use case?
> > >
> > > From what I know, currently there are two document route strategies as
> > > explained here
> > > https://lucidworks.com/blog/2013/06/13/solr-cloud-document-routing/.
> But
> > > Is there anything else that I'm missing?
> > >
> > >
> > >
> > >
> > > Thanks.
> > >
> > >
> > >
> >
>

Re: java.lang.IllegalStateException: Too many values for UnInvertedField faceting on field content

[davidphilip cherian]

I think you should just make docValues=true and reindex. But be warned that
faceting is generally  not performed on field that are of type text and
tokenized.  They should be string if they are not numeric. What is analysis
chain of 'text_fa'?


On Mon, Jul 20, 2015 at 8:16 PM, Ali Nazemian  wrote:

> Dears,
> Hi,
> I have a collection of 1.6m documents in Solr 5.2.1. When I use facet on
> field of content this error will appear after around 30s of trying to
> return the results:
>
> null:org.apache.solr.common.SolrException: Exception during facet.field:
> content
> at
> org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:632)
> at
> org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:617)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:571)
> at
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:642)
> at
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:285)
> at
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:102)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:255)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:497)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalStateException: Too many values for
> UnInvertedField faceting on field content
> at
> org.apache.lucene.uninverting.DocTermOrds.uninvert(DocTermOrds.java:509)
> at
> org.apache.lucene.uninverting.DocTermOrds.(DocTermOrds.java:215)
> at
> org.apache.lucene.uninverting.DocTermOrds.(DocTermOrds.java:206)
> at
> org.apache.lucene.uninverting.DocTermOrds.(DocTermOrds.java:199)
> at
> org.apache.lucene.uninverting.FieldCacheImpl$DocTermOrdsCache.createValue(FieldCacheImpl.java:946)
> at
> org.apache.lucene.uninverting.FieldCacheImpl$Cache.get(FieldCacheImpl.java:190)
> at
> org.apache.lucene.uninverting.FieldCacheImpl.getDocTermOrds(FieldCacheImpl.java:933)
> at
> org.apache.lucene.uninverting.UninvertingReader.getSortedSetDocValues(UninvertingReader.java:275)
> at
> org.apache.lucene.index.FilterLeafReader.getSortedSetDocValues(FilterLeafReader.java:454)
> at
> org.apache.lucene.index.MultiDocValues.getSortedSetValues(MultiDocValues.java:356)
> at
> org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:165)
> at
> org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:72)
> at
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:490)

Collection APIs to create collection and custom cores naming

[davidphilip cherian]

How to use the 'property.name=value' in the api example[1] to modify
core.properties value of 'name'

While creating the collection with below query[2], the core names become
'aggregator_shard1_replica1' and 'aggregator_shard2_replica1'. I wanted
have specific/custom name for each of these cores. I tried passing the
params as property.name=name&name=aggregator_s1, but it did not work.

Editing the core.properties key value pair of name=aggregator_s1 after
collection is created, works! But I was looking for setting this property
with create request itself.

[2]
http://example.com:8983/solr/admin/collections?action=CREATE&name=aggregator&numShards=1&replicationFactor=2&maxShardsPerNode=1&collection.configName=aggregator_config&property.name=name&name=aggregator_s1

[1]
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api1

Re: Is cache enabled by default?

[davidphilip cherian]

Hi Nitin,

You can just set the attributes of caches to zero.   size="0"
initialSize="0"autowarmCount="0" and so on.
Why do you want to turn off caches btw? Any specific reasons?IMO, documents
are cached in OS disc cache space which you may not able to control. It is
OS specific.

I don't quite understand your question. In the first statement you said,
you want to turn off caches.  In second statement you are asking about
making search faster. That's confusing.

To make search fast even at first time, you might want to have look at
newSearcher and firstSearcher events.

https://wiki.apache.org/solr/SolrCaching#newSearcher_and_firstSearcher_Event_Listeners







On Mon, Aug 10, 2015 at 6:45 PM, Nitin Solanki  wrote:

> Hi,
>  I have commented  "querycache, filterquerycache and document
> cache". Still searching is using cache. why so?
>
> 2) First time searching a query, it takes time and afterwards, it can't due
> to cache, I know that. But how to make search always faster even first time
> searching?
>

Re: Changing solr.Date to solr.TrieDate

[davidphilip cherian]

Hi Saurabh,

You could probably try command=fetchindex functionality.
http://node:port
/solr//replication?command=fetchindex&masterUrl=http://node:port
/solr/
Master url should be the existing index solr instance url.


On Mon, Aug 10, 2015 at 6:37 PM, saurabh tewari 
wrote:

> Hi,
>
> I recently started to move my solr index from 4.8 to 5.2 via replication
> from 4.8 master to 5.2 slave. I replaced all the primitive fieldtypes to
> Trie-fields, since the original ones are removed from 5.2 . Replication
> went successful, but when I fired a query on slave, it shows a class-cast
> exception in solr.DateField.I understand that my documents are still using
> old fieldtypes and my current schema doesn't support that. Is there any way
> I can move my existing index to 5.2?
>

Re: Make search faster in Solr

[davidphilip cherian]

Hi Nitin,

32 shards for 16 million documents is too much. 2 shards should suffice
considering your document sizes are moderate. Caches are to be monitored
and tuned accordingly. You should study about caches a bit here
https://cwiki.apache.org/confluence/display/solr/Query+Settings+in+SolrConfig



On Mon, Aug 10, 2015 at 4:34 PM, Nitin Solanki  wrote:

> Hi,
> I have 32 shards and single replica of each shards having 4 nodes
> over Solr cloud.
> I have indexed 16 million documents. Without cache, total time taken to
> search a document is 0.2 second. And with cache is 0.04 second.
> I don't do anything of cache. Caches are set by default in solrconfig.xml.
>
> How to make faster search without cache? Or how to make more faster with
> cache while searching. Which cache is used for searching?
>

Re: No. of records mismatch

[davidphilip cherian]

Hi,

You should check whether there were deletions by navigating to solr admin
core admin page. Example url
http://localhost:8983/solr/#/~cores/test_shard1_replica1, check for
numDocs, maxDocs and deletedDocs. If numDocs remains equal to maxDocs, then
you confirm that there were no updations (as recommended by Upayavira)

HTH

On Mon, Aug 17, 2015 at 4:41 AM, Pattabiraman, Meenakshisundaram <
pattabiraman.meenakshisunda...@aig.com> wrote:

> " You almost certainly have a non-unique ID field."
> Yes it is not absolutely unique but do not think it is at this 1 to 6
> ratio.
>
> "Try it with a clean index, and then review the number of deleted
> documents (updates are a delete then insert action) "
> I tried on a new instance - same effect. I do not see any deletions. Is
> there a way to determine this from the logs to confirm that the behavior is
> due to non-uniqueness? This will serve as an assurance.
> Thanks
>
> 6843469
> 6843469
> 0
> 2015-08-16 21:22:24
> 
> Indexing completed. Added/Updated: 6843469 documents. Deleted 0 documents.
> 
> 2015-08-16 22:31:47
>
> Whereas '*:*'
> "params":{
>   "q":"*:*"}},
>   "response":{"numFound":1143108,"start":0,"docs":[
>
> -Original Message-
> From: Upayavira [mailto:u...@odoko.co.uk]
> Sent: Sunday, August 16, 2015 3:18 PM
> To: solr-user@lucene.apache.org
> Subject: Re: No. of records mismatch
>
> You almost certainly have a non-unique ID field. Some documents are
> overwritten during indexing. Try it with a clean index, and then review the
> number of deleted documents (updates are a delete then insert action).
> Deletes are calculated with maxDocs minus numDocs.
>
> Upayavira
>
> On Sun, Aug 16, 2015, at 07:18 PM, Pattabiraman, Meenakshisundaram
> wrote:
> > I did a dataimport with 'clean' set to false.
> > The DIH status upon completion was:
> >
> > idle
> > 
> > 
> > 1 6843427 6843427 0
> > 2015-08-16 16:50:54 
> > Indexing completed. Added/Updated: 6843427 documents. Deleted 0
> > documents.
> > 
> > Whereas when I query using 'query?q=*:*&rows=0', I get the following
> > count {
> >   "responseHeader":{
> > "status":0,
> > "QTime":1,
> > "params":{
> >   "q":"*:*",
> >   "rows":"0"}},
> >   "response":{"numFound":1616376,"start":0,"docs":[]
> >   }}
> >
> > There is a difference of 5 million records. Can anyone help me
> > understand the behavior? The logs look fine.
> > Thanks
>

Logging in solr admin page

[davidphilip cherian]

Hi,

Where are the logs fetched from on solr admin ui page?
http://localhost:8983/solr/#/~logging. I am unable to see any logs there.
Its just showing the 'loading' symbol but no logs fetched. What could be
the reason? Any logging setting that has to be made?

Thanks.

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

[davidphilip cherian]

Hi Kevin/Noble,

What is the download link to take the latest? What are the steps to compile
it, test and use?
We also have a use case to have this feature in solr too. Therefore, wanted
to test and above info would help a lot to get started.

Thanks.


On Fri, Sep 4, 2015 at 1:45 PM, Kevin Lee  wrote:

> Thanks, I downloaded the source and compiled it and replaced the jar file
> in the dist and solr-webapp’s WEB-INF/lib directory.  It does seem to be
> protecting the Collections API reload command now as long as I upload the
> security.json after startup of the Solr instances.  If I shutdown and bring
> the instances back up, the security is no longer in place and I have to
> upload the security.json again for it to take effect.
>
> - Kevin
>
> > On Sep 3, 2015, at 10:29 PM, Noble Paul  wrote:
> >
> > Both these are committed. If you could test with the latest 5.3 branch
> > it would be helpful
> >
> > On Wed, Sep 2, 2015 at 5:11 PM, Noble Paul  wrote:
> >> I opened a ticket for the same
> >> https://issues.apache.org/jira/browse/SOLR-8004
> >>
> >> On Wed, Sep 2, 2015 at 1:36 PM, Kevin Lee 
> wrote:
> >>> I’ve found that completely exiting Chrome or Firefox and opening it
> back up re-prompts for credentials when they are required.  It was
> re-prompting with the /browse path where authentication was working each
> time I completely exited and started the browser again, however it won’t
> re-prompt unless you exit completely and close all running instances so I
> closed all instances each time to test.
> >>>
> >>> However, to make sure I ran it via the command line via curl as
> suggested and it still does not give any authentication error when trying
> to issue the command via curl.  I get a success response from all the Solr
> instances that the reload was successful.
> >>>
> >>> Not sure why the pre-canned permissions aren’t working, but the one to
> the request handler at the /browse path is.
> >>>
> >>>
>  On Sep 1, 2015, at 11:03 PM, Noble Paul  wrote:
> 
>  " However, after uploading the new security.json and restarting the
>  web browser,"
> 
>  The browser remembers your login , So it is unlikely to prompt for the
>  credentials again.
> 
>  Why don't you try the RELOAD operation using command line (curl) ?
> 
>  On Tue, Sep 1, 2015 at 10:31 PM, Kevin Lee 
> wrote:
> > The restart issues aside, I’m trying to lockdown usage of the
> Collections API, but that also does not seem to be working either.
> >
> > Here is my security.json.  I’m using the “collection-admin-edit”
> permission and assigning it to the “adminRole”.  However, after uploading
> the new security.json and restarting the web browser, it doesn’t seem to be
> requiring credentials when calling the RELOAD action on the Collections
> API.  The only thing that seems to work is the custom permission “browse”
> which is requiring authentication before allowing me to pull up the page.
> Am I using the permissions correctly for the RuleBasedAuthorizationPlugin?
> >
> > {
> >   "authentication":{
> >  "class":"solr.BasicAuthPlugin",
> >  "credentials": {
> >   "admin”:” ",
> >   "user": ” "
> >   }
> >   },
> >   "authorization":{
> >  "class":"solr.RuleBasedAuthorizationPlugin",
> >  "permissions": [
> >   {
> >   "name":"security-edit",
> >   "role":"adminRole"
> >   },
> >   {
> >   "name":"collection-admin-edit”,
> >   "role":"adminRole"
> >   },
> >   {
> >   "name":"browse",
> >   "collection": "inventory",
> >   "path": "/browse",
> >   "role":"browseRole"
> >   }
> >   ],
> >  "user-role": {
> >   "admin": [
> >   "adminRole",
> >   "browseRole"
> >   ],
> >   "user": [
> >   "browseRole"
> >   ]
> >   }
> >   }
> > }
> >
> > Also tried adding the permission using the Authorization API, but no
> effect, still isn’t protecting the Collections API from being invoked
> without a username password.  I do see in the Solr logs that it sees the
> updates because it outputs the messages “Updating /security.json …”,
> “Security node changed”, “Initializing authorization plugin:
> solr.RuleBasedAuthorizationPlugin” and “Authentication plugin class
> obtained from ZK: solr.BasicAuthPlugin”.
> >
> >>>

Re: Problems upgrading to Solr 5.3.0

[davidphilip cherian]

If you have a look at solr 5.3 'changes.txt', there are some jiras
(resolved) that are related to ICU Tokenizer. You could possibly try taking
the latest jar of this tokenizer from solr 5.3 binaries and replace in 5.2
and try.




On Mon, Sep 7, 2015 at 9:49 AM, Zheng Lin Edwin Yeo 
wrote:

> Hi,
>
> I'm trying to upgrade from Solr 5.2.1 to 5.3.0. My Solr is running on
> SolrCloud and also on external zookeeper 3.4.6.
>
> When I tried to migrate the index over from Solr 5.2.1 to Solr 5.3.0, the
> Solr is not able to startup, and it just give the error saying Solr did not
> come online within 30 seconds!
> I've put the indexed under solrMain\node1\solr and solrMain\node2\solr as
> I'm running my Solr with 2 notes, and I used the following command to
> start.
> bin\solr.cmd start -cloud -p 8983 -s solrMain\node1\solr -m 12g -z
> "localhost:2181,localhost:2182,localhost:2183"
>
>
> I've also tried not to migrate the index, but to start Solr 5.3.0 without
> any cores and indexes, and tried to create a new core from there from my
> Solr 5.2.1 configurations. However, when I tried to create, I got the
> following error:
>
> org.apache.solr.common.SolrException: Could not load conf for core
> collection1_shard1_replica2: Plugin init failure for [schema.xml]
> fieldType "text_ar": Plugin init failure for [schema.xml]
> analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'.
> Schema file is /configs/collection1/schema.xml
> at
> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:80)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:725)
> at
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:701)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:629)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:675)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Plugin init failure
> for [schema.xml] fieldType "text_ar": Plugin init failure for
> [schema.xml] analyzer/tokenizer: Error loading class
> 'solr.ICUTokenizerFactory'. Schema file is
> /configs/collection1/schema.xml
> at
> org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:596)
> at org.apache.solr.schema.IndexSchema.(IndexSchema.java:175)
> at
> org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
> at
> org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
> at
> org.apache.solr.core.ConfigSetService.createI

String index out of range exception from Spell check

[davidphilip cherian]

There are irregular exceptions from spell check component. Below is the
stack trace. This is not common for all the q terms but have often seen
them occurring for specific queries after enabling spellcheck.collate
method.



String index out of range: -3



java.lang.StringIndexOutOfBoundsException: String index out of range: -3 at
java.lang.AbstractStringBuilder.replace(AbstractStringBuilder.java:789) at
java.lang.StringBuilder.replace(StringBuilder.java:266) at
org.apache.solr.spelling.SpellCheckCollator.getCollation(SpellCheckCollator.java:235)
at
org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:92)
at
org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:230)
at
org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:197)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:226)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497) at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:722)



500

Re: Sort not working as expected

[davidphilip cherian]

You can create a copy field with string type and make it copy from this
existing field, and sort on this new one.
That way, you can still continue doing text search on existing one and sort
on this new field.





On Fri, Oct 30, 2015 at 3:04 PM, Brian Narsi  wrote:

> Is there no way that the existing field can be used?
>
>
> On Fri, Oct 30, 2015 at 1:42 PM, Ray Niu  wrote:
>
> > you should use string type instead of text if you want to sort
> > alphabetically
> >
> > 2015-10-30 11:12 GMT-07:00 Brian Narsi :
> >
> > > I have a fieldtype setup as
> > >
> > >  > positionIncrementGap=
> > > "100">   > > "solr.StandardTokenizerFactory"/>  > > "solr.LowerCaseFilterFactory"/>  > class="solr.EdgeNGramFilterFactory"
> > > minGramSize="3" maxGramSize="25"/>  
> <
> > > tokenizer class="solr.StandardTokenizerFactory"/>  > > "solr.LowerCaseFilterFactory"/>  
> > >
> > >
> > > When I sort on this field type in ascending order I am not getting
> > results
> > > sorted alphabetically as expected.
> > >
> > > Why is that?
> > >
> > > What should I do to get the sort on?
> > >
> > > Thanks
> > >
> >
>

Re: Solr Keyword query on a specific field.

[davidphilip cherian]

>> "Is there any way to have a single field search use the same keyword
search logic as the default query?"
Do a phrase search, with double quotes surrounding the multiple keywords,
it should work.

Try q=title:("Test Keywords")

You could possibly try adding this q.op as local param to query as shown
below.
https://cwiki.apache.org/confluence/display/solr/Local+Parameters+in+Queries

If you are using edismax query parser, check for what is mm pram
set. q.op=AND => mm=100%; q.op=OR => mm=0%)
https://wiki.apache.org/solr/ExtendedDisMax#mm_.28Minimum_.27Should.27_Match.29


On Fri, Oct 30, 2015 at 3:27 PM, Aaron Gibbons <
agibb...@synergydatasystems.com> wrote:

> Is there any way to have a single field search use the same keyword search
> logic as the default query? I define q.op as AND in my query which gets
> applied to any main keywords but any keywords I'm trying to use within a
> field do not get the same logic applied.
> Example:
> q=(title:(Test Keywords)) the space is treated as OR regardless of q.op
> q=(Test Keywords) the space is defined by q.op which is AND
>
> Using the correct operators (AND OR * - +...) it works great as I have it
> defined. There's just this one little caveat when you use spaces between
> keywords expecting the q.op operator to be applied.
> Thanks,
> Aaron
>

SSL on Solr with CA signed certificate

[davidphilip cherian]

The doc[1] on reference guide provides steps related to setting up ssl with
self signed certificate. My employer wants me to set up and test with CA
signed certificate.
When I go to buy[2] a ssl certificate(just for testing), it asks for
specific web server name and jetty is not listed on it.

Is there something else that I need to look for, to enable ssl on solr,
with CA signed certificate? Has anyone tried doing this instead of
selfsigned one? Any further inputs? reference blogs?


[1] https://cwiki.apache.org/confluence/display/solr/Enabling+SSL
[2] https://www.instantssl.com/free-ssl-certificate.html

Re: solr-8983-console.log is huge

[davidphilip cherian]

>From mail archives

https://support.lucidworks.com/hc/en-us/articles/207072137-Solr-5-X-Console-Logging-solr-8983-console-log

On Fri, Nov 6, 2015 at 1:10 PM, Shawn Heisey  wrote:

> On 11/6/2015 9:13 AM, Alexandre Rafalovitch wrote:
> > What about the Garbage Collection output? I think we have the same
> > issue there. Frankly, I don't know how many people know what to do
> > with that in a first place.
>
> Turns out that Java has rotation capability built in to GC logging:
>
> http://stackoverflow.com/a/12277309/2665648
>
> Thanks,
> Shawn
>
>

Re: Solr Keyword query on a specific field.

[davidphilip cherian]

Nested queries might help.

http://www.slideshare.net/erikhatcher/solr-query-parsing-tips-and-tricks

On Mon, Nov 2, 2015 at 10:20 AM, Aaron Gibbons <
agibb...@synergydatasystems.com> wrote:

> The input for the title field is user based so a wide range of things can
> be entered there.  Quoting the title is not what I'm looking for.  I also
> checked and q.op is AND and MM is 100%.  In addition to the Title field the
> user can also use general keywords so setting local params (df) to
> something else would not work either to my knowledge.
>
> To give you a better idea of what I'm trying to accomplish: I have a form
> to allow users to search on Title, Keywords and add a location. The correct
> operators are applied between each of these and also for the main keywords
> themselves.  The only issue is with the default operator being applied
> within the Title sections's keywords. My goal is to have the Title keywords
> work the same as the general keywords but only be applied to the title
> field vs the default text field.
>
> On Fri, Oct 30, 2015 at 6:35 PM, davidphilip cherian <
> davidphilipcher...@gmail.com> wrote:
>
> > >> "Is there any way to have a single field search use the same keyword
> > search logic as the default query?"
> > Do a phrase search, with double quotes surrounding the multiple keywords,
> > it should work.
> >
> > Try q=title:("Test Keywords")
> >
> > You could possibly try adding this q.op as local param to query as shown
> > below.
> >
> >
> https://cwiki.apache.org/confluence/display/solr/Local+Parameters+in+Queries
> >
> > If you are using edismax query parser, check for what is mm pram
> > set. q.op=AND => mm=100%; q.op=OR => mm=0%)
> >
> >
> https://wiki.apache.org/solr/ExtendedDisMax#mm_.28Minimum_.27Should.27_Match.29
> >
> >
> > On Fri, Oct 30, 2015 at 3:27 PM, Aaron Gibbons <
> > agibb...@synergydatasystems.com> wrote:
> >
> > > Is there any way to have a single field search use the same keyword
> > search
> > > logic as the default query? I define q.op as AND in my query which gets
> > > applied to any main keywords but any keywords I'm trying to use within
> a
> > > field do not get the same logic applied.
> > > Example:
> > > q=(title:(Test Keywords)) the space is treated as OR regardless of q.op
> > > q=(Test Keywords) the space is defined by q.op which is AND
> > >
> > > Using the correct operators (AND OR * - +...) it works great as I have
> it
> > > defined. There's just this one little caveat when you use spaces
> between
> > > keywords expecting the q.op operator to be applied.
> > > Thanks,
> > > Aaron
> > >
> >
>

Re: solr-8983-console.log is huge

[davidphilip cherian]

I think, the collection reload, using collection apis[1] should do the
work. If you are not on solr-cloud setup, just navigate to coreadmin on
solr UI, and click on reload, and see if that works.


[1]
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api2
http:
//localhost:8983/solr/admin/collections?action=RELOAD&name=newCollection

On Sun, Nov 8, 2015 at 5:43 PM, CrazyDiamond  wrote:

> Thx for answers. Is there any way to reload log4j.properities without
> restarting solr?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/solr-8983-console-log-is-huge-tp4238613p4239067.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: SnapPuller Exception in Slave server

[davidphilip cherian]

Hi,

It is in the exception that 'master node' is not available, and thus
reporting IO exception.
Did you check if the master instance was  running when these exceptions
were reported? Where are these two nodes located? Is there a network
latency? Do you see any mismatch in the numFound, at master node and slave
node?


On Sun, Nov 8, 2015 at 12:17 AM, sangeetha.subraman...@gtnexus.com <
sangeetha.subraman...@gtnexus.com> wrote:

> Hi All,
>
> I am using solr 4.5.1 with Master and Slave architecture. I am seeing the
> below exception in the Slave server
>
> SnapPuller
>
> Master at:  not available. Index fetch failed. Exception:
> org.apache.solr.client.solrj.SolrServerException: IOException occured when
> talking to server at:
>
>
> I don't see replication not working due to this exception but I guess it
> slows down the replication process
> Could someone please let me know why this occurs and what needs to checked
> for ?
>
> Thanks
> Sangeetha
>

Convert output response xml into input xml format using xslt

[davidphilip cherian]

Has anyone written a sample xslt (and would like to share) that converts
output response xml of solr into its  input format, to
repost/reindex it back?

Thanks

Re: Simple web interface for queries

[davidphilip cherian]

How about velocity search UI? You can access that using /browse

https://cwiki.apache.org/confluence/display/solr/Velocity+Search+UI




On Tue, Nov 10, 2015 at 10:14 AM, Nigel Robbins <
nigel.robb...@wallstreetsystems.com> wrote:

> Hi,
>
>
>
> Just wondering if there is a simplified web user interface for users to
> issue just queries.
>
>
>
> The http://localhost:8984/solr/#/core/query URL has too many options for
> my
> end-users.
>
>
>
> For example, can the web interface be configured for queries so that only
> the “query string” and output can be seen ?
>
>
>
> Thanks,
>
> Nigel
>

Re: unsubscribe me.

[davidphilip cherian]

You should probably send an email to solr-user-unsubscr...@lucene.apache.org


Reference links

http://lucene.apache.org/solr/resources.html#community

https://wiki.apache.org/solr/Unsubscribing%20from%20mailing%20lists

On Wed, Nov 18, 2015 at 1:04 PM, Pramod  wrote:

> please unsubscribe me.
>
> Regards,
> YP
>

Re: Create Collection Admin Request - unable to specify collection configName

[davidphilip cherian]

If I'm carefully looking at your commands, the IP are different.
For upconfig "54.XX.XX.124:2181"
For downconfig, you used 54.XX.XX.84:2181

Are these zookeeper on an ensemble setup?

Anyways, to avoid confusion, what I'd recommend is, just spin one only
zookeeper first, start solr by passing respective "-z host:port"

Do a upconfig on that.

$  ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
./server/solr/configsets/scbe_public7_config -confname testForErick2 -z
host:port

And downconfig on same too

$ ./server/scripts/cloud-scripts/zkcli.sh -cmd downconfig -confdir
/tmp/erick2 -confname testForErick2 -z host:port
$ ls -1 /tmp/erick2






On Wed, Dec 2, 2015 at 10:19 AM, Erick Erickson 
wrote:

> Why are you adding the /solr to the -z parameter when you start Solr? You
> never specify it for upconfig/downconfig and the like.
>
> Of course I'm almost always in dev and have only a single ZK running.
> And when I just
> tried adding /solr the results were a completely useless admin screen
> that hung there. So
> I'm not entirely sure what's going on.
>
> My suspicion is that your ZK configs are a mish-mash of starting with
> the /solr and bare
> ZK strings and thus is hard to untangle at this point. So here's what I'd
> do:
>
> > Stop all your Zk instances and completely blow away the configs. These
> are usually in
> /var/zookeeper
>
> > make sure your Solr nodes are clean. By "clean" here I mean you don't
> have any
> cores under server/solr. You probably don't, but I'd check.
>
> > try your operations again without the /solr.
>
> Of course you could just restart solr without the /solr on the -z
> parameter for a quick
> test, but even if that works I'd still blow away the ZK data just to
> have a clean slate.
>
> Best,
> Erick
>
> On Tue, Dec 1, 2015 at 6:29 PM, Kelly, Frank  wrote:
> > Thanks Erick,
> >
> > Yes I start Solr as follows
> >
> > $ ./bin/solr start -cloud -z
> > 54.XX.XX.84:2181,54.XX.XX.124:2181,54.XX.XX.254:2181/solr -h 52.XX.XX.13
> >
> >
> > And confirmed those are the ZooKeeper nodes in question.
> >
> > Good idea on the downconfig and I confirmed it is working e.g.
> >
> > $  ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
> > ./server/solr/configsets/scbe_public7_config -confname testForErick2 -z
> > 54.XX.XX.124:2181
> > $ ./server/scripts/cloud-scripts/zkcli.sh -cmd downconfig -confdir
> > /tmp/erick2 -confname testForErick2 -z 54.XX.XX.84:2181
> > $ ls -1 /tmp/erick2
> > currency.xml
> > lang
> > protwords.txt
> > _rest_managed.json
> > schema.xml
> > solrconfig.xml
> > stopwords.txt
> > synonyms.txt
> >
> > So this is telling me ZooKeeper is at least storing the configuration but
> > something with Solr/Zk is awry.
> >
> > Cheers!
> >
> >
> > -Frank
> >
> >
> >
> >
> >
> > On 12/1/15, 9:01 PM, "Erick Erickson"  wrote:
> >
> >>bq: I have tried upconfig to all three ZooKeeper nodes (1 leader, 2
> >>followers)
> >>and they seem to work but when I look at cloud/tree it doesn’t make a
> >>difference.
> >>
> >>That doesn't answer how you start _Solr_. My question earlier was trying
> >>to insure that your Solr instance points at the same zookeeper ensemble
> >>as you hit when you upconfig.
> >>
> >>You should also be able to "downconfig" from the client and get a fresh
> >>copy
> >>from ZK to a new local directory to insure that it's really up in ZK.
> >>
> >>Best,
> >>Erick
> >>
> >>On Tue, Dec 1, 2015 at 5:21 PM, Manikandan Sivanesan
> >> wrote:
> >>> And one more thing I noticed you are specifying your ensemble as
> >>> zk.zk.zk.zk:2181 . But it should be of the form
> >>>*zk1:port,zk2:port,zk3:port
> >>> *.
> >>>
> >>> On Tue, Dec 1, 2015 at 8:12 PM, Manikandan Sivanesan
> >>>
> >>> wrote:
> >>>
> 
>  Try with the conf directory in place with schema.xml & solrconfig.xml
>  inside conf.  You should see the files being uploaded
> 
>  ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
>  *./server/solr/configsets/scbe_**public7_config/conf* -confname
>  scbe_public7 -z zk.zk.zk.zk:2181
> 
>  This is how I do
>  zkcli.sh  -zkhost $ZK_ENSEMBLE -cmd upconfig -confdir /tmp/access/conf
>  -confname  access
> 
>  You can verify if you have properly uploaded the config to either by
>  Upayvira's suggestion or using ./zkcli.sh Eg :
>  https://gist.github.com/manisnesan/52ffc84dd761365e0c22
> 
> 
> 
>  On Tue, Dec 1, 2015 at 5:22 PM, Kelly, Frank 
> wrote:
> 
> > So I have an ensemble of three Zk nodes running
> >
> > I have tried upconfig to all three ZooKeeper nodes (1 leader, 2
> >followers)
> > and they seem to work but when I look at cloud/tree it doesn’t make a
> > difference.
> > I tried switching browsers to avoid the potential browser caching
> >issue
> > and still no dice.
> >
> >
> > The only thing I can think of is that “mycollection” which I *can*
> >see is
> > a Managed Schema - does having one Manage

Re: Issue in Geospatial Search

[davidphilip cherian]

What is the fieldType of the field "latlon" in older schema as well as new
schema?
Have you confirmed that both are same?


On Tue, Dec 15, 2015 at 3:18 PM, Shenbagarajan 
wrote:

> Hello,
>
> I am trying to implement geo spatial search in solr by referring the below
> site.
> https://cwiki.apache.org/confluence/display/solr/Spatial+Search
>
> Everytime i try to execute i am getting the same error as below.
>  "msg":"The field latlon does not support spatial filtering",
>
> When i try to run the same query in Solr 4.10 its working fine without any
> issues. But in Solr 5.3 its not working properly. Direct me to figure out
> where the issue is as i am stuck with it:(
>
> Thanks,
> Shen.
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Issue-in-Geospatial-Search-tp4245441.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Solr 5 admin screen description

[davidphilip cherian]

Hi Group,

 I would like to understand the solr's dashboard screen, particularly the
screen *system* properties displayed such as Physical Memory, Swap Space,
File Desc. count and etc. 1. What is the importance of file descriptor
count?  2. Dashboard shows that it has taken 16 gb of space out of which
14.3g is used. I did not understand on what basis it allocated 16 gb? How
do I increase it?If my index sizes increase, will it automatically extend
if there is hard disc space available?  I couldn't find these info. on
reference guide. Could you please point me to any doc/blog that explains
this?

IP Address assgined to solr instance during the Cloud mode start

[davidphilip cherian]

Hi,

When I started solr in cloud mode(interactive) and chose 2 nodes, it
started and in the cloud-view screen it showed some different ip with url
169.254.5.207:7574, when clicked on that, it says page not found. When I
modified url to localhost(http://localhost:7574/solr/#/~cloud) it
worked(loaded solr admin page)
Query is, Where is this ip address picked from? How to edit them?

Start stop solr started in solr cloud mode

[davidphilip cherian]

Hi,
I started solr in cloud mode (interactive set up). 3 nodes, 3 shards and 1
replica and a collection.  I stopped it using ./solr stop -all. How do I
get the same above cloud mode setup to start? "./solr -c start"  started
the new solr cloud instance all together where as I was looking for the
previously set up instance to start?.  I am going through reference guide.
I did not find any command for this. Please help.

Re: IP Address assgined to solr instance during the Cloud mode start

[davidphilip cherian]

I think this is because of change in network ip address. I got it. Thanks.


On Thu, Mar 19, 2015 at 1:32 PM, davidphilip cherian <
davidphilipcher...@gmail.com> wrote:

> Hi,
>
> When I started solr in cloud mode(interactive) and chose 2 nodes, it
> started and in the cloud-view screen it showed some different ip with url
> 169.254.5.207:7574, when clicked on that, it says page not found. When I
> modified url to localhost(http://localhost:7574/solr/#/~cloud) it
> worked(loaded solr admin page)
> Query is, Where is this ip address picked from? How to edit them?
>

Re: IP Address assgined to solr instance during the Cloud mode start

[davidphilip cherian]

Hi Shawn,

Thanks you for the detailed explanation.

On Thu, Mar 19, 2015 at 7:31 PM, Shawn Heisey  wrote:

> On 3/19/2015 2:02 AM, davidphilip cherian wrote:
> > When I started solr in cloud mode(interactive) and chose 2 nodes, it
> > started and in the cloud-view screen it showed some different ip with url
> > 169.254.5.207:7574, when clicked on that, it says page not found. When I
> > modified url to localhost(http://localhost:7574/solr/#/~cloud) it
> > worked(loaded solr admin page)
> > Query is, Where is this ip address picked from? How to edit them?
>
> An IP address of 169.254.x.x is what Windows will assign to a machine
> when a network card configured for DHCP comes up and no DHCP response is
> received.
>
> http://packetlife.net/blog/2008/sep/24/169-254-0-0-addresses-explained/
>
> When Solr starts in SolrCloud mode and you do not provide a "host"
> property, Solr (when it is in SolrCloud mode) asks Java to ask the
> operating system "what is my IP address?"  Whatever the response is to
> that question is the default hostname that Solr will use when it
> registers itself in zookeeper.
>
> http://wiki.apache.org/solr/SolrCloud#SolrCloud_Instance_Params
>
> In a nutshell, your operating system networking is misconfigured.  Once
> you fix that (or provide a host property to Solr to override the bad
> choice), you will need to manually edit your zookeeper data to remove
> the bad node entry.  You will probably need to use the zkCli that comes
> with zookeeper itself, or perhaps something like the zookeeper plugin
> for eclipse.
>
> Thanks,
> Shawn
>
>

Re: Securing Solr 5.0.0

[davidphilip cherian]

Hi Frederik Arnold,

Could you please blog it? the steps to setup the same using apache as
reverse proxy and share it with community?

On Sun, Mar 22, 2015 at 10:16 PM, Frederik Arnold 
wrote:

> I have and I tried all sorts of things and they didn't work.
> But I figured it out now. I setup Apache as a reverse proxy and it works.
>
> 2015-03-22 17:25 GMT+01:00 Erick Erickson :
>
> > Have you looked at https://wiki.apache.org/solr/SolrSecurity?
> >
> > Best,
> > Erick
> >
> > On Sun, Mar 22, 2015 at 4:20 AM, Frederik Arnold  >
> > wrote:
> > > I followed the "Taking Solr to Production" tutorial and I now have an
> > > solr 5.0.0 instance up and running.
> > >
> > > What is the recommended way for securing solr?
> > > Searching should be available for everyone but I want authentication
> for
> > > the Solr Admin UI and also for posting and deleting files.
> >
>

Re: Contribute the Customized Phonetic Filter to Apache Solr

[davidphilip cherian]

Hi Aman,

https://wiki.apache.org/solr/HowToContribute

HTH

On Thu, Jun 18, 2015 at 12:11 PM, Aman Tandon 
wrote:

> Hi,
>
> We created the new phonetic filter, It is working great on our products,
> mostly of our suppliers are Indian, it is quite helpful for us to provide
> the exact result e.g.
>
> 1) rikshaw, still able to find the suppliers of rickshaw
> 2) telefone, still able to find the suppliers of telephone
>
> We also analyzed our search satisfaction feedback, it improved by 13% (54%
> -> 67%) just after implementing the same.
>
> And we want to contribute the same to solr, So how could I do it.
>
> With Regards
> Aman Tandon
>

Re: Solr Suggester not working.

[davidphilip cherian]

Did you have a quick look at this blog?
http://lucidworks.com/blog/solr-suggester/



On Fri, Jun 26, 2015 at 12:05 PM, ssharma7...@gmail.com <
ssharma7...@gmail.com> wrote:

> Hi,
> I am new to Solr & have installed Solr 5.1. I am trying to get the
> Suggester
> component working without any success, the following are the relevant
> details:
>
> *Schema.xml*
>  termVectors="true" termPositions="true" termOffsets="true" />
>
>  positionIncrementGap="100">
> 
>  class="solr.UAX29URLEmailTokenizerFactory"/>
>  ignoreCase="true"
> words="lang/stopwords_en.txt" />
>  class="solr.ASCIIFoldingFilterFactory"/>
>  class="solr.EnglishPossessiveFilterFactory"/>
>  class="solr.RemoveDuplicatesTokenFilterFactory"/>
> 
>  class="solr.LowerCaseFilterFactory"/>
> 
> 
>  class="solr.UAX29URLEmailTokenizerFactory"/>
>  ignoreCase="true"
> words="lang/stopwords_en.txt" />
>  class="solr.ASCIIFoldingFilterFactory"/>
>  class="solr.EnglishPossessiveFilterFactory"/>
>  class="solr.LowerCaseFilterFactory"/>
> 
> 
>
> *solr-config.xml*
>
> 
>
>   textSuggester
>   FuzzyLookupFactory
>   DocumentDictionaryFactory
>   text
>   string
>   true
>
> 
>
>  startup="lazy" >
> 
>   true
>   10
>   textSuggester
> 
> 
>   suggest
> 
>   
>
> *Query to get suggestions*
>
> http://localhost:8983/solr/portal_documents/suggest?suggest=true&suggest.dictionary=textSuggester&suggest.q=testing
>
> *Above Query Output*
> 
> 
> 0
> 31
> 
> 
> 
> 
> 0
> 
> 
> 
> 
> 
>
> The data is getting indexed properly, as can be seen from image below:
> 
>
> Regards,
> Sachin Vyas.
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Solr 5.2.1 setup zookeeper ensemble problem

[davidphilip cherian]

Hi Adrian,

What is the error that you are getting?
In order to  upload configs files, you could use zkcli.sh script that will
be shipped with solr and use the upconfig command.

./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:9983 \
   -cmd upconfig -confname my_new_config -confdir
server/solr/configsets/basic_configs/conf

https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities



On Mon, Jul 6, 2015 at 1:43 PM, Adrian Liew  wrote:

> There seems to be an issue running the following command using solr.cmd as
> below:
>
>  - bin\solr.cmd -e cloud -z localhost:2181,localhost:2182,localhost:2183
>
> Anyone can please advise.
>
> Also, Is there a way to upload a configuration file (containing schema.xml
> and solrconfig.xml) to ZooKeeper easily using solr.cmd in solr 5.2.1?
>
> Best regards,
>
> Adrian Liew |  Consultant Application Developer
> Avanade Malaysia Sdn. Bhd..| Consulting Services
> (: Direct: +(603) 2382 5668
> È: +6010-2288030
>
>
>

Re: Tokenizer and Filter Factory to index Chinese characters

[davidphilip cherian]

Hi Edwin,

Have you tried the Paoding analyzer?  It is not out of the box shipped with
Solr jars. You may have to download it and add it to solr libs.

https://stanbol.apache.org/docs/trunk/components/enhancer/nlp/paoding



2015-07-06 12:29 GMT+05:30 Zheng Lin Edwin Yeo :

> I'm now using the solr.ICUTokenizerFactory, and the searching for Chinese
> characters can work when I use the Query tab in Solr Admin UI.
>
> In the Admin UI, it converts the Chinese characters to code before passing
> it to the URL, so it looks something like this:
>
> http://localhost:8983/solr/chinese2/select?q=%E8%83%A1%E5%A7%AC%E8%8A%B1&wt=json&indent=true&hl=true
>
> "highlighting":{
>
> "chinese5":{
>
>   "text":["园将办系列活动庆祝入遗 \n 从胡姬花展到音
> 乐会，为庆祝申遗成功，植物园这个月起将举办一系列活动与公众一同庆贺。
> 本月10日开始的“新加坡植物园胡姬及其文化遗产”展览，将展出1万
> 6000株胡姬花，这是"]},
>
> "chinese3":{
>
>   "text":[" \n 原版为 马来语 《Majulah Singapura》，中文译为《 前  进吧，新加坡 》。 \n  \n
> \t  国花 \n 新加坡以一种名为 卓  锦  ·  万代  兰
> 的胡姬花为国花。东南亚通称兰花为胡姬花"]}}}
>
>
>
> However, if I enter the Chinese characters directly into the URL, the
> results I get are wrong.
>
> http://localhost:8983/solr/chinese2/select?q=胡姬花&hl=true&hl.fl=text
>
>
>   "highlighting":{
>
> "chinese1":{
>
>   "text":["1月份的制造业产值同比仅增长0 \n \n   新加坡 我国1月份的制造业产值同比仅增长0.9％。
> 虽然制造业结束连续两个月的萎缩，但比经济师普遍预估的增长3.3％疲软得多。这也意味着，我国今年第一季度的经济很可能让人失望 \n
> "]},
>
> "chinese2":{
>
>   "text":["Zheng Lin Yeo"]},
>
> "chinese3":{
>
>   "text":["Zheng Lin Yeo"]},
>
> "chinese4":{
>
>   "text":["户只要订购《联合晚报》任一种配套，就可选择下列其中一项赠品带回家。 \n 签订两年配套的读者可获得一台价值
> 199元的Lenovo TAB 2 A7-10七寸平板电脑，或者一架价值249元的Philips
> Viva"]},
>
> "chinese5":{
>
>   "text":["Zheng Lin Yeo"]}}}
>
>
>
> Why is this so?
>
>
> Regards,
>
> Edwin
>
>
> 2015-06-25 18:54 GMT+08:00 Markus Jelsma :
>
> > You may also want to try Paoding if you have enough time to spend:
> > https://github.com/cslinmiso/paoding-analysis
> >
> > -Original message-
> > > From:Zheng Lin Edwin Yeo 
> > > Sent: Thursday 25th June 2015 11:38
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Tokenizer and Filter Factory to index Chinese characters
> > >
> > > Hi, The result doesn't seems that good as well. But you're not using
> the
> > > HMMChineseTokenizerFactory?
> > >
> > > The output below is from the filters you've shown me.
> > >
> > >   "highlighting":{
> > > "chinese1":{
> > >   "id":["chinese1"],
> > >   "title":["我国1月份的制造业产值同比仅增长0"],
> > >
> >
> "content":["，但比经济师普遍预估的增长3.3％疲软得多。这也意味着，我国今年第一季度的经济很可能让人失望
> > > \n  "],
> > >   "author":["Edwin"]},
> > > "chinese2":{
> > >   "id":["chinese2"],
> > >   "content":["铜牌，让我国暂时高居奖牌荣誉榜榜首。
> > > 你看好新加坡在本届的东运会中，会夺得多少面金牌？
> > > 请在6月12日中午前，投票并留言为我国健将寄上祝语吧  \n
> > > "],
> > >   "author":["Edwin"]},
> > > "chinese3":{
> > >   "id":["chinese3"],
> > >   "content":[")组成的我国女队在今天的东运会保龄球女子三人赛中，
> > >
> >
> 以六局3963总瓶分夺冠，为新加坡赢得本届赛会第三枚金牌。队友陈诗桦（Jazreel)、梁蕙芬和陈诗静以3707总瓶分获得亚军，季军归菲律宾女队。（联合早报记者：郭嘉惠)
> > > \n  "],
> > >   "author":["Edwin"]},
> > > "chinese4":{
> > >   "id":["chinese4"],
> > >   "content":["，则可获得一架价值309元的Philips Viva
> > > Collection HD9045面包机。 \n
> > > 欲订从速，读者可登陆www.wbsub.com.sg，或拨打客服专线6319
> > > 1800订购。 \n
> > >
> >
> 此外，一年一度的晚报保健美容展，将在本月23日和24日，在新达新加坡会展中心401、402展厅举行。
> > > \n
> >
> 现场将开设《联合晚报》订阅展摊，读者当场订阅晚报，除了可获得丰厚的赠品，还有机会参与“必胜”幸运抽奖"],
> > >   "author":["Edwin"]}}}
> > >
> > >
> > > Regards,
> > > Edwin
> > >
> > >
> > > 2015-06-25 17:28 GMT+08:00 Markus Jelsma :
> > >
> > > > Hi - we are actually using some other filters for Chinese, although
> > they
> > > > are not specialized for Chinese:
> > > >
> > > > 
> > > > 
> > > > 
> > > > 
> > > >
> > > >
> > > > -Original message-
> > > > > From:Zheng Lin Edwin Yeo 
> > > > > Sent: Thursday 25th June 2015 11:24
> > > > > To: solr-user@lucene.apache.org
> > > > > Subject: Re: Tokenizer and Filter Factory to index Chinese
> characters
> > > > >
> > > > > Thank you.
> > > > >
> > > > > I've tried that, but when I do a search, it's returning much more
> > > > > highlighted results that what it supposed to.
> > > > >
> > > > > For example, if I enter the following query:
> > > > > http://localhost:8983/solr/chinese1/highlight?q=我国
> > > > >
> > > > > I get the following results:
> > > > >
> > > > > "highlighting":{
> > > > > "chinese1":{
> > > > >   "id":["chinese1"],
> > > > >
> > > >
> >
> "title":["我国1月份的制造业产值同比仅增长0"],
> > > > >
> > > >
> >
> "content":["结束连续两个月的萎缩，但比经济师普遍预估的增长3.3％疲软得多。这也意味着，我国今年第一季度的经济很可能让人失望
> > > > > \n  "],
> > > > >   "author":["Edwin"]},
> > > > > "chinese2":{
> > > > >   "id":["chinese2"],
> > > > >
> > > >
> >
> "content":["铜牌，让我国暂时高居奖牌荣誉榜榜首。
> > > > > 你看好新加坡在本届的东运会中，会夺得多少面金牌？
> > > > >
> > > >
> >
> 请在6月12日中午前，投票并留言为我国健将寄上祝语吧
> > > > >  \n  "],
> > > > >   "author":["Edwin"]},
> > > > > "chinese3":{
> > > > >   "id":["chinese3"],
> > > > >
> > > >
> >
> "content":[")组成的我国女队在今天的东运会保龄球女子三人赛中，
> > > > >
> > > >
> >

Re: Protwords in solr spellchecker

[davidphilip cherian]

The best bet is to use solr.StopFilterFactory.
Have all such words added to stopwords.txt and add this filter to your
analyzer.

Reference links
https://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.StopFilterFactory
https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-StopFilter

HTH


On Thu, Jul 9, 2015 at 11:50 AM, Kamal Kishore Aggarwal <
kkroyal@gmail.com> wrote:

> Hi Team,
>
> I am currently working with Java-1.7, Solr-4.8.1 with tomcat 7. Is there
> any feature by which I can refrain the following words to appear in spell
> suggestion.
>
> For example: Somebody searches for sexe, I does not want to show him sex as
> the spell suggestion via solr. How can I stop these type of keywords to be
> shown in suggestion.
>
> Any help is appreciated.
>
>
> Regards
> Kamal Kishore
> Solr Beginner
>

Re: Protwords in solr spellchecker

[davidphilip cherian]

Hi Kamal,

Not necessarily. You can have different filters applied at index time and
query time. (note that the order in which filters are defined matters). You
could just add the stop filter at query time.
Have your own custom data type defined (similar to 'text_en' that will be
in schem.xml) and perhaps use standard/whitespace tokenizer followed by
stop filter at query time.

Tip: Use analysis tool that is available in solr admin page to further
understand the analysis chain of data types.

HTH



On Fri, Jul 10, 2015 at 1:03 PM, Kamal Kishore Aggarwal <
kkroyal@gmail.com> wrote:

> Hi David,
>
> This one is a good suggestion. But, if add these *adult* keywords in the
> stopwords.txt file, it will be requiring the re-indexing of these keywords
> related data.
>
> How can I see the change instantly. Is there any other great suggestion
> that you can suggest me.
>
>
>
>
> On Thu, Jul 9, 2015 at 12:09 PM, davidphilip cherian <
> davidphilipcher...@gmail.com> wrote:
>
> > The best bet is to use solr.StopFilterFactory.
> > Have all such words added to stopwords.txt and add this filter to your
> > analyzer.
> >
> > Reference links
> >
> >
> https://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.StopFilterFactory
> >
> >
> https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-StopFilter
> >
> > HTH
> >
> >
> > On Thu, Jul 9, 2015 at 11:50 AM, Kamal Kishore Aggarwal <
> > kkroyal@gmail.com> wrote:
> >
> > > Hi Team,
> > >
> > > I am currently working with Java-1.7, Solr-4.8.1 with tomcat 7. Is
> there
> > > any feature by which I can refrain the following words to appear in
> spell
> > > suggestion.
> > >
> > > For example: Somebody searches for sexe, I does not want to show him
> sex
> > as
> > > the spell suggestion via solr. How can I stop these type of keywords to
> > be
> > > shown in suggestion.
> > >
> > > Any help is appreciated.
> > >
> > >
> > > Regards
> > > Kamal Kishore
> > > Solr Beginner
> > >
> >
>