Solr 1.4 - autoSuggest - is it a default service
Hi, I am trying to use autoSuggest in Solr 1.4. Is autoSugest service available by default like select? or should I configure anything? Solrconfig.xml contains the termcomponent defined. Thanks R -- View this message in context: http://www.nabble.com/Solr-1.4---autoSuggestis-it-a-default-service-tp25443128p25443128.html Sent from the Solr - User mailing list archive at Nabble.com.
autosuggest - terms componet - document details
I just started using terms/autosuggest service.Application need the document details along with result items. What params I need to use to fetch the document details. -- View this message in context: http://www.nabble.com/autosuggest---terms-componet---document-details-tp25476823p25476823.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Searching on other fields which are not in query
Hi, I have a data set with the following schema. PersonName:Text AnimalName:Text PlantName:Text < lot more attributes about each of them like nick name, animal nick name, plant generic name etc which are multually exclusive> UniqueId:long For each of the document data set, there will be only one value of the above three. In my solr query from client I am using AnimalName:German Shepard. The return result contains PersonName with 'Shepard' in it, even though I am querying on AnimalName field. Can anyone point me whats happening and how to prevent scanning other columns/fields. I appreciate your help. Thanks Ravi -- View this message in context: http://www.nabble.com/Solr-Searching-on-other-fields-which-are-not-in-query-tp20249798p20249798.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching on other fields which are not in query
Hmm, I dont have any defined in the schema.xml. Can you give the exact syntax how it looks like in schema.xml I have text. Does it mean if sufficient requested count not available, it looks for the search string in any of the text fields that are indexed? Thanks Ravi Jorge Solari wrote: > > Your query > > AnimalName:German Shepard. > > means > > AnimalName:German :Shepard. > > whichever the defaultField is > > Try with > AnimalName:"German Shepard" > > or > > AnimalName:German AND AnimalName:Shepard. > > > > On Thu, Oct 30, 2008 at 12:58 PM, Yerraguntla <[EMAIL PROTECTED]> > wrote: > >> >> Hi, >> >> I have a data set with the following schema. >> >> PersonName:Text >> AnimalName:Text >> PlantName:Text >> >> < lot more attributes about each of them like nick name, animal nick >> name, plant generic name etc which are multually exclusive> >> UniqueId:long >> >> For each of the document data set, there will be only one value of the >> above >> three. >> >> In my solr query from client >> >> I am using AnimalName:German Shepard. >> >> The return result contains >> PersonName with 'Shepard' in it, even though I am querying on AnimalName >> field. >> Can anyone point me whats happening and how to prevent scanning other >> columns/fields. >> >> I appreciate your help. >> >> Thanks >> Ravi >> >> -- >> View this message in context: >> http://www.nabble.com/Solr-Searching-on-other-fields-which-are-not-in-query-tp20249798p20249798.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/Solr-Searching-on-other-fields-which-are-not-in-query-tp20249798p20252475.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching on other fields which are not in query
Never mind, I understand now. I have text. I was searching on a string field with space in it and with no quotes. This is causing to scan for text fields(since default search field is text) in the schema. Also in my schema there is an indexed field(AnimalNameText) which is not populated which is a text field. After I populate the text field, I get only the results I am expecting. Thanks for the pointer Jorge ! Yerraguntla wrote: > > Hmm, > > I dont have any defined in the schema.xml. > Can you give the exact syntax how it looks like in schema.xml > > I have text. > Does it mean if sufficient requested count not available, it looks for the > search string in any of the text fields that are indexed? > > Thanks > Ravi > > > Jorge Solari wrote: >> >> Your query >> >> AnimalName:German Shepard. >> >> means >> >> AnimalName:German :Shepard. >> >> whichever the defaultField is >> >> Try with >> AnimalName:"German Shepard" >> >> or >> >> AnimalName:German AND AnimalName:Shepard. >> >> >> >> On Thu, Oct 30, 2008 at 12:58 PM, Yerraguntla <[EMAIL PROTECTED]> >> wrote: >> >>> >>> Hi, >>> >>> I have a data set with the following schema. >>> >>> PersonName:Text >>> AnimalName:Text >>> PlantName:Text >>> >>> < lot more attributes about each of them like nick name, animal nick >>> name, plant generic name etc which are multually exclusive> >>> UniqueId:long >>> >>> For each of the document data set, there will be only one value of the >>> above >>> three. >>> >>> In my solr query from client >>> >>> I am using AnimalName:German Shepard. >>> >>> The return result contains >>> PersonName with 'Shepard' in it, even though I am querying on AnimalName >>> field. >>> Can anyone point me whats happening and how to prevent scanning other >>> columns/fields. >>> >>> I appreciate your help. >>> >>> Thanks >>> Ravi >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Solr-Searching-on-other-fields-which-are-not-in-query-tp20249798p20249798.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> > > -- View this message in context: http://www.nabble.com/Solr-Searching-on-other-fields-which-are-not-in-query-tp20249798p20252950.html Sent from the Solr - User mailing list archive at Nabble.com.
solr admin page throwing errors
Hi, I am using solr admin page with index.jsp from < <%-- $Id: index.jsp 686780 2008-08-18 15:08:28Z yonik $ --%> I am getting these errors. Any insight will be helpful. HTTP Status 500 - javax.servlet.ServletException: java.lang.NoSuchFieldError: config org.apache.jasper.JasperException: javax.servlet.ServletException: java.lang.NoSuchFieldError: config at org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:532) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:408) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:320) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:687) at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:469) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:403) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:273) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:228) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:216) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:634) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:445) at java.lang.Thread.run(Thread.java:619) Caused by: javax.servlet.ServletException: java.lang.NoSuchFieldError: config at org.apache.jasper.runtime.PageContextImpl.doHandlePageException(PageContextImpl.java:855) at org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:784) at org.apache.jsp.admin.index_jsp._jspService(index_jsp.java:324) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:384) ... 22 more Caused -- View this message in context: http://www.nabble.com/solr-admin-page-throwing-errors-tp21375221p21375221.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr admin page throwing errors
I think its more of admin.jsp code, which id throwing errors. I can succefully use admin/luke and solr/select?q= without any problem. Grant Ingersoll-6 wrote: > > Haven't seen this before, was there anything in the log further up? I > don't even see Solr anywhere in the stack trace here. > > > On Jan 9, 2009, at 11:11 AM, Yerraguntla wrote: > >> >> Hi, >> >> I am using solr admin page with index.jsp from >> >> < <%-- $Id: index.jsp 686780 2008-08-18 15:08:28Z yonik $ --%> >> >> I am getting these errors. Any insight will be helpful. >> >> HTTP Status 500 - javax.servlet.ServletException: >> java.lang.NoSuchFieldError: config org.apache.jasper.JasperException: >> javax.servlet.ServletException: java.lang.NoSuchFieldError: config at >> org >> .apache >> .jasper >> .servlet.JspServletWrapper.handleJspException(JspServletWrapper.java: >> 532) >> at >> org >> .apache >> .jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:408) >> at >> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java: >> 320) >> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:266) >> at >> javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at >> org >> .apache >> .catalina >> .core >> .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: >> 290) >> at >> org >> .apache >> .catalina >> .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org >> .apache >> .catalina >> .core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:687) >> at >> org >> .apache >> .catalina >> .core >> .ApplicationDispatcher.processRequest(ApplicationDispatcher.java:469) >> at >> org >> .apache >> .catalina >> .core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:403) >> at >> org >> .apache >> .catalina >> .core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301) >> at >> org >> .apache >> .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:273) >> at >> org >> .apache >> .catalina >> .core >> .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: >> 235) >> at >> org >> .apache >> .catalina >> .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org >> .apache >> .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: >> 228) >> at >> org >> .apache >> .catalina.core.StandardContextValve.invoke(StandardContextValve.java: >> 175) >> at >> org >> .apache >> .catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) >> at >> org >> .apache >> .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104) >> at >> org >> .apache >> .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: >> 109) >> at >> org >> .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: >> 216) >> at >> org >> .apache.coyote.http11.Http11Processor.process(Http11Processor.java: >> 844) >> at >> org.apache.coyote.http11.Http11Protocol >> $Http11ConnectionHandler.process(Http11Protocol.java:634) >> at org.apache.tomcat.util.net.JIoEndpoint >> $Worker.run(JIoEndpoint.java:445) >> at java.lang.Thread.run(Thread.java:619) Caused by: >> javax.servlet.ServletException: java.lang.NoSuchFieldError: config at >> org >> .apache >> .jasper >> .runtime.PageContextImpl.doHandlePageException(PageContextImpl.java: >> 855) >> at >> org >> .apache >> .jasper >> .runtime.PageContextImpl.handlePageException(PageContextImpl.java:784) >> at org.apache.jsp.admin.index_jsp._jspService(index_jsp.java:324) at >> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at >> javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at >> org >> .apache >> .jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:384) >> ... 22 more Caused >> -- >> View this message in context: >> http://www.nabble.com/solr-admin-page-throwing-errors-tp21375221p21375221.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > > -- > Grant Ingersoll > > Lucene Helpful Hints: > http://wiki.apache.org/lucene-java/BasicsOfPerformance > http://wiki.apache.org/lucene-java/LuceneFAQ > > > > > > > > > > > > -- View this message in context: http://www.nabble.com/solr-admin-page-throwing-errors-tp21375221p21398748.html Sent from the Solr - User mailing list archive at Nabble.com.
Searching solr without qualified field
Hi, Basic question :) I have added new xml data after changing the example's schema.xml. When I am using admin qquery page. The example works given the value of "video". But for my data the field is looking for fieldname:value to return the results. What should I do if i want to just search just with value. Thanks Y -- View this message in context: http://www.nabble.com/Searching-solr-without-qualified-field-tp15357026p15357026.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Searching solr without qualified field
Nm, the fileds need to be defined as part of copyfield section just like in the example. silly me :( Yerraguntla wrote: > > Hi, > > Basic question :) > > I have added new xml data after changing the example's schema.xml. When > I am using admin qquery page. The example works given the value of > "video". But for my data the field is looking for fieldname:value to > return the results. What should I do if i want to just search just with > value. > > > Thanks > Y > -- View this message in context: http://www.nabble.com/Searching-solr-without-qualified-field-tp15357026p15360917.html Sent from the Solr - User mailing list archive at Nabble.com.
Bossting a token with space at the end
Hi, I have to search on Company Names, which contain multiple words. Some of the examples are Micro Image Systems, Microsoft Corp, Sun Microsystems, Advanced Micro systems. For the above example when the search is for micro, the expected results order is Micro Image Systems Advanced Micro systems Microsoft Corp Sun Microsystems What needs to be done both for field type, indexing and query . There are bunch of company names for the each of the compnay name example I mentioned. I have been trying with couple of ways with multiple queries, but I am not able retrieve Micro Image systems on the top at all. Appreciate any hints and help. --Yerra -- View this message in context: http://www.nabble.com/Bossting-a-token-with-space-at-the-end-tp15465726p15465726.html Sent from the Solr - User mailing list archive at Nabble.com.
Smart way of indexing for Better performance
Hi, I have the following use case. I could implement the solution but performance is affected. I need some smart ways of doing this. Use Case : Incoming data has two fields which have values like 'WAL MART STORES INC' and 'wal-mart-stores-inc'. Users can search the data either in 'walmart' 'wal mart' or 'wal-mart' also partially on any part of the name from the start of word like 'wal', 'walm' 'wal m' etc . I could get the solution by using two indexes, one as text field for the first field (wal mart ) column and sub word wal-mart-stores (with WordDelimiterFilterFactory filter). Is there a smart way of doing or any other techniques to boost the performance? I need to use them for a high traffic application where the response requirements are around 50 milli seconds. I have some control on modifying the incoming data. Can someone suggest better ways of implementing. I can provide more information the tokens and filters I am using. Thanks Ravi -- View this message in context: http://www.nabble.com/Smart-way-of-indexing-for-Better-performance-tp16092886p16092886.html Sent from the Solr - User mailing list archive at Nabble.com.
Smart Indexing for Better performance and functionality ??
Hi, I have the following use case. I could implement the solution but performance is affected. I need some smart ways of doing this. Use Case : Incoming data has two fields which have values like 'WAL MART STORES INC' and 'wal-mart-stores-inc'. Users can search the data either in 'walmart' 'wal mart' or 'wal-mart' also partially on any part of the name from the start of word like 'wal', 'walm' 'wal m' etc . I could get the solution by using two indexes, one as text field for the first field (wal mart ) column and sub word wal-mart-stores (with WordDelimiterFilterFactory filter). Is there a smart way of doing or any other techniques to boost the performance? I need to use them for a high traffic application where the response requirements are around 50 milli seconds. I have some control on modifying the incoming data and data set is around 100K records. Can someone suggest better ways of implementing. I can provide more information the tokens and filters I am using. Thanks Ravi -- View this message in context: http://www.nabble.com/Smart-Indexing-for-Better-performance-and-functionalitytp16121987p16121987.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Smart way of indexing for Better performance
data set(number of documents) is not large - 100k. Number of fields could max to 10 . With average size of indexed field could be 200 characters. I tried creating using multiple indexes by using copy field. Let me see how the performance will be with EdgeNGramTokenFilter or EdgeNGramTokenizer Thanks for the sugegstions. hossman wrote: > > : I have the following use case. I could implement the solution but > : performance is affected. I need some smart ways of doing this. > : Use Case : > : Incoming data has two fields which have values like 'WAL MART STORES > INC' > : and 'wal-mart-stores-inc'. > : Users can search the data either in 'walmart' 'wal mart' or 'wal-mart' > : also partially on any part of the name from the start of word like > 'wal', > : 'walm' 'wal m' etc . I could get the solution by using two indexes, > one > : as text field for the first field (wal mart ) column and sub word > : wal-mart-stores (with WordDelimiterFilterFactory filter). > > there are lots of solutions that could work, all depending on what *else* > you need to be able to match on besides just prefix queries where > whitespace/punctuation are ignored. > > One example: using KeywordTokenizer, along with a PatternReplaceFilter > that throws away non letter charagers and a LowercaseFilter and then > issuing all your queries as PrefixQueries will get w* wa* wal* and walm* > to all match "wal mart", "WALMART", "WAL-mart", etc but that won't > let "mart" match a document contain "wal mart" .. but you can always use > copyField and hit one field for the first type of query, and the other > field for "normal" queries. > > depending on the nature of your data (ie: how many documents, how common > certian prefixes are, etc...) you might get better performacne at the > expense of a larger index if you use something like the > EdgeNGramTokenFilter or EdgeNGramTokenizer to index all the prefixes of > various sizes so you don't need to do a prefix query > > The bottom line: there are *lots* of options, you'll need to experimentto > find the right solution that matches when you want to match, and doesn't > when you don't > > > > -Hoss > > > -- View this message in context: http://www.nabble.com/Smart-way-of-indexing-for-Better-performance-tp16092886p16143967.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Text Vs String
Hi, I have incoming field stored both as Text and String field in solr indexed data. When I search the following cases, string field returns documents(from Solr client) and not text fields. NAME:T - no results Name_Str:T - returns documents Similarly for the following cases - CPN*, DPS*, S, IF,AND, ARE, etc. AAA,AN,AND,ARE,BE,BUT,CCC,CPN*,DPS*,FOR,HRC*,IF,IN,INTO,IT,NOT,S,SID*,T,THE,THIS,TO Is anything with keywords for writing queries or stopwords or synonyms. Below is the definition of my field definition. - - - - -- View this message in context: http://www.nabble.com/Solr-Text-Vs-String-tp17364259p17364259.html Sent from the Solr - User mailing list archive at Nabble.com.
Fwd: Solr cloud replication factort
All, I am using Solr 5.4 in solr cloud mode in a 8 node cluster. Used the replication factor of 1 for creating the index, then switched to replication factor > 1 for redundancy. With replication factor > 1, and tried to do indexing for incremental. When the incremental indexing happens - getting a stack trace with the root cause pointing to write.lock is not available. Further analysis found that there is only one write.lock across all shards (leader and replicas). But with replication factor of 1 , I could see write.lock across all nodes. is this the expected behavior (one write.lock) in the solr cloud with replication factor > 1. If so how can the indexing be done (even though it is slow) with distributed and redundant shards? OR is there any config param that is missing to create write.lock across all shards with replication factor > 1. Appreciate your insights. Thanks Ravi
Re: Fwd: Solr cloud replication factort
Thanks Shawn. I got both the replies. Most likely we might have used some of the NFS options. I will try them early next week. Thanks Ravi On Wed, Mar 23, 2016 at 9:50 AM, Shawn Heisey wrote: > On 3/23/2016 6:00 AM, Raveendra Yerraguntla wrote: > > I am using Solr 5.4 in solr cloud mode in a 8 node cluster. Used the > > replication factor of 1 for creating the index, then switched to > > replication factor > 1 for redundancy. With replication factor > 1, and > > tried to do indexing for incremental. When the incremental indexing > > happens - getting a stack trace with the root cause pointing to > write.lock > > is not available. Further analysis found that there is only one > write.lock > > across all shards (leader and replicas). > > > > But with replication factor of 1 , I could see write.lock across all > nodes. > > Did you see my reply on the dev list, sent before I told you that your > question belonged on this list? > > In that reply I told you that replicationFactor will have no effect > after you create your collection. It also cannot cause problems with > the write.lock file. > > I also said this: > > There are three major reasons for a problem with write.lock. > 1) Solr is crashing and leaving the write.lock file behind. > 2) You are trying to share an index directory between more than one core > or Solr instance. > 3) You are trying to run with your index data on a network filesystem > like NFS. > > Thanks, > Shawn > >
Multiple calls across the distributed nodes for a query
I need help in understanding a query in solr cloud. When user issues a query , there are are two phases of query - one with the purpose(from debug info) of GET_TOP_FIELDS and another with GET_FIELDS. This is having an effect on end to end performance of the application. - what triggers (any components like facet, highlight, spellchecker ??? ) the two calls - How can I make a query to be executed only with GET_FIELDS only .
Re: Multiple calls across the distributed nodes for a query
Thank you Jeff . Let me try out how much of improvement I get out of single pass param Sent from my iPhone > On Jun 15, 2016, at 1:59 PM, Jeff Wartes wrote: > > Any distributed query falls into the two-phase process. Actually, I think > some components may require a third phase. (faceting?) > > However, there are also cases where only a single pass is required. A > fl=id,score will only be a single pass, for example, since it doesn’t need to > get the field values. > > https://issues.apache.org/jira/browse/SOLR-5768 would be a good place to read > about some of this, and provides a way to help force a one-pass even if you > need other fields. > > >> On 6/15/16, 7:31 AM, "Raveendra Yerraguntla" >> wrote: >> >> I need help in understanding a query in solr cloud. >> When user issues a query , there are are two phases of query - one with the >> purpose(from debug info) of GET_TOP_FIELDS and another with GET_FIELDS. >> >> This is having an effect on end to end performance of the application. >> >> - what triggers (any components like facet, highlight, spellchecker ??? ) >> the two calls >> - How can I make a query to be executed only with GET_FIELDS only . >
6.3 -> 6.4 Sorting responseWriter renamed
Hello All, In 6.4 (Solr-9717) SortingResponseWriter is renamed to ExportWriter and moved to a different package. For migrating to higher Solr (post 6.4) versions, I need to help with compatible functionalities. Application is using SortingResponseWriter in the searcher handlers inform method to register responseWriters for the xSort. Since the class and write methods Signature is changed, what are alternative ways to use the functionality. ThanksRavi
Re: 6.3 -> 6.4 Sorting responseWriter renamed
Hi Joel,
Thanks for the quick response.
Our current usage is below. Could you guide me in using the new class and write
method.
public class customSearchHandler extends SearchHandler {@Override
public void inform(SolrCore core)
{
super.inform(core);
….
core.registerResponseWriter("xsort", new SortingResponseWriter(){
@Override
public void write(Writer out, SolrQueryRequest req,SolrQueryResponse
response) throws IOException {
try {
if (handleResponseWriter) {
CustomController.singleton.prepareThreadForWork();
}
super.write(out, req,response);
}
finally {
CustomController.singleton.releaseFromThread();
}
}
});
}
Signature of new class isExportWriter(SolrQueryRequest req, SolrQueryResponse
res, String wt) {public void write(OutputStream os) throws IOException {
On Friday, January 11, 2019, 1:55:15 PM EST, Joel Bernstein
wrote:
The functionality should be exactly the same. The config files though need
to be changed. I would recommend adding any custom configs that you have to
the new configs following the ExportWriter changes.
Joel Bernstein
http://joelsolr.blogspot.com/
On Thu, Jan 10, 2019 at 11:21 AM Raveendra Yerraguntla
wrote:
> Hello All,
>
> In 6.4 (Solr-9717) SortingResponseWriter is renamed to ExportWriter and
> moved to a different package.
>
> For migrating to higher Solr (post 6.4) versions, I need to help with
> compatible functionalities.
>
>
> Application is using SortingResponseWriter in the searcher handlers
> inform method to register responseWriters for the xSort.
>
> Since the class and write methods Signature is changed, what are
> alternative ways to use the functionality.
>
>
> Thanks
> Ravi
>
>
Score field NaN in the expanded Results
All,
I am using the collapse plugin with the following query params
text_field:( apple OR apples) ) )
Fq : {!collapse field=string_field sort='numeric_field asc , score desc '
nullPolicy=expand}
Raw params : expand=true&expand.rows=600&expand.sort=numeric_field asc
The above query results in score value of NaN in theexpandedResults.
When the expand.sort=numeric_field asc is not usedexpanded results has score
value populated.
The value of the numeric filed is in the form ofd form. The value of
d is 1 to 5.
Any insights about why the score field is returned asNaN
Appreciate your help.
ThanksRavi
Re: Score field NaN in the expanded Results
Found the issue.I am using solr 5.4.0 version. in ExpandComponent - the
TermFieldCollector is created with the TotalDefaultScores and
TotalDefaultMaxScores were passed as false. This limits any field sorting
(other than score) makes the score to be returned as NaN.
By passing either or both of them as true, the score value is returned when the
expand.sort is done on any other field.
On Wednesday, March 13, 2019, 11:09:21 AM EDT, Raveendra Yerraguntla
wrote:
All,
I am using the collapse plugin with the following query params
text_field:( apple OR apples) ) )
Fq : {!collapse field=string_field sort='numeric_field asc , score desc '
nullPolicy=expand}
Raw params : expand=true&expand.rows=600&expand.sort=numeric_field asc
The above query results in score value of NaN in theexpandedResults.
When the expand.sort=numeric_field asc is not usedexpanded results has score
value populated.
The value of the numeric filed is in the form ofd form. The value of
d is 1 to 5.
Any insights about why the score field is returned asNaN
Appreciate your help.
ThanksRavi
Solr 7.6 optimize index size increase
when optimize command is issued, the expectation after the completion of optimization process is that the index size either decreases or at most remain same. In solr 7.6 cluster with 50 plus shards, when optimize command is issued, some of the shard's transient or older segment files are not deleted. This is happening randomly across all shards. When unnoticed these transient files makes disk full. Currently it is handled through monitors, but question is what is causing the transient/older files remains there. Are there any specific race conditions which laves the older files not being deleted? Any pointers around this will be helpful. TIA
Re: Solr 7.6 optimize index size increase
Thank you David, Walt , Eric. 1. First time bloated index generated , there is no disk space issue. one copy of index is 1/6 of disk capacity. we ran into disk capacity after more than 2 copies of bloated copies.2. Solr is upgraded from 5.*. in 5.* more than 5 segments is causing performance issue. Performance in 7.* is not measured for increasing segments. I will plan a PT to get optimum number. Application has incremental indexing multiple times in a work week. I will keep you updated on the resolution. Thanks again On Tuesday, June 16, 2020, 07:34:26 PM EDT, Erick Erickson wrote: It Depends (tm). As of Solr 7.5, optimize is different. See: https://lucidworks.com/post/solr-and-optimizing-your-index-take-ii/ So, assuming you have _not_ specified maxSegments=1, any very large segment (near 5G) that has _zero_ deleted documents won’t be merged. So there are two scenarios: 1> What Walter mentioned. The optimize process runs out of disk space and leaves lots of crud around 2> your “older segments” are just max-sized segments with zero deletions. All that said… do you have demonstrable performance improvements after optimizing? The entire name “optimize” is misleading, of course who wouldn’t want an optimized index? In earlier versions of Solr (i.e. 4x), it made quite a difference. In more recent Solr releases, it’s not as clear cut. So before worrying about making optimize work, I’d recommend that you do some performance tests on optimized and un-optimized indexes. If there are significant improvements, that’s one thing. Otherwise, it’s a waste. Best, Erick > On Jun 16, 2020, at 5:36 PM, Walter Underwood wrote: > > For a full forced merge (mistakenly named “optimize”), the worst case disk > space > is 3X the size of the index. It is common to need 2X the size of the index. > > When I worked on Ultraseek Server 20+ years ago, it had the same merge > behavior. > I implemented a disk space check that would refuse to merge if there wasn’t > enough > free space. It would log an error and send an email to the admin. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > >> On Jun 16, 2020, at 1:58 PM, David Hastings >> wrote: >> >> I cant give you a 100% true answer but ive experienced this, and what >> "seemed" to happen to me was that the optimize would start, and that will >> drive the size up by 3 fold, and if you out of disk space in the process >> the optimize will quit since, it cant optimize, and leave the live index >> pieces in tact, so now you have the "current" index as well as the >> "optimized" fragments >> >> i cant say for certain thats what you ran into, but we found that if you >> get an expanding disk it will keep growing and prevent this from happening, >> then the index will contract and the disk will shrink back to only what it >> needs. saved me a lot of headaches not needing to ever worry about disk >> space >> >> On Tue, Jun 16, 2020 at 4:43 PM Raveendra Yerraguntla >> wrote: >> >>> >>> when optimize command is issued, the expectation after the completion of >>> optimization process is that the index size either decreases or at most >>> remain same. In solr 7.6 cluster with 50 plus shards, when optimize command >>> is issued, some of the shard's transient or older segment files are not >>> deleted. This is happening randomly across all shards. When unnoticed these >>> transient files makes disk full. Currently it is handled through monitors, >>> but question is what is causing the transient/older files remains there. >>> Are there any specific race conditions which laves the older files not >>> being deleted? >>> Any pointers around this will be helpful. >>> TIA >
count mismatch with and without sort param
Hello my date query returns mismatch with and without sort parameter. The query parameter is on date field, on which records are inserted few hours ago. sort parameter is different from query parameter, which is the expected the count. Any clues if how it could be different? I am planning to do a commit (one more time) but after looking for clues. Any pointers are appreciated, Thanks Ravi
How to Add replicas in 7.6, similar to 5.4
All, We are upgrading from solr 5.4 to solr 7.6. In 5.4 each solr process based on the core.properties (shard value assigned) will be joining as either leader or replica based on the sequence of start. By following the same procedure in 7.6 the initial leader node solr process is replaced with later started solr. Also in the process core is not loaded in the later solr process (which is supposed to be replica). To have the compatibility , the legacyCloud property is set as true in the clustereprops.json Question: .With replica type is NRT to keep the transition smooth.How to add later started solr process as replicas in 7.6, without interfering the leader election process? Appreciate any pointers in understanding and resolving the issue. ThanksRavi
Re: How to Add replicas in 7.6, similar to 5.4
@eric Thanks for the reply. The upgrading process is in two phases. In the initial phase brings the cluster using new solr version binaries. In the next phase use the non-deprecated functionalities and make use of the new functionalities Let me explain the problem little different way. Environment lets say has 8 hosts, 4 leaders and 4 replicas for a 4 shard and single core cluster. On each host solr is started as something like the below from command line.(not using solr control script)Before starting the solrs on any node, collection is created using uploaded, linked in zk using zkcli.sh of the solr bundle. Once first 4 hosts for each of the shard is started, the cluster is in functional state. When the fifth node solr process is started as replica of shard1, expectation is the fifth node will join as replica for shard1 along with host1. But host1 is dropped from the admin console / cloud / graph , only host 5 is displayed and core is not loaded for host 5. Question is how to make later joining solr process for a particular shard as part of leader/replica group. It will be less of a concern, which one will be leader/replica , as along as a replicationFactor is achieved. Should the collection APIs need to be used to create a replicas? Could it be achieved with settings in some properties or with startup param in solr ? Any pointers towards this will be helpful. " /usr/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/Users/ravi/solr-7.6.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -DzkClientTimeout=15000 -DzkHost=localhost:2181 -Dsolr.log.dir=/Users/ravi/solr-7.6.0/server/logs -Djetty.port=8080 -DSTOP.PORT=7080 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/Users/ravi/solr-7.6.0/server -Dsolr.solr.home=/Users/ravi/solr-7.6.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/Users/ravi/solr-7.6.0 -Dsolr.default.confdir=/Users/ravi/solr-7.6.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.jetty.https.port=8080 -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/Users/ravi/solr-7.6.0/bin/oom_solr.sh 8080 /Users/ravi/solr-7.6.0/server/logs -jar start.jar --module=http" 'Ravi' Raveendra On Sunday, April 21, 2019, 11:36:07 AM EDT, Erick Erickson wrote: My first question would “why do you think this is important?”. The additional burden a leader has is quite small and is only present when indexing. Leaders have no role in processing queries. So unless you have, say, on the order of 100 leaders on a single Solr instance _and_ are indexing heavily, I’d be surprised if you notice any difference between that and having leaders evenly distributed. I would _strongly_ urge you, BTW, to have legacyCloud set to false. That setting will totally go away at some point, so adjusting sooner rather than later is probably a good idea. Bottom line. I’d recommend just using the defaults (including legacyCloud) and not worrying about distributing the leader role unless and until you can prove that having unbalanced leaders is really having a performance impact. In my experience, 95% of the time people spend time trying to manage which nodes are leaders the effort is wasted. Best, Erick > On Apr 20, 2019, at 2:45 PM, Raveendra Yerraguntla > wrote: > > All, > We are upgrading from solr 5.4 to solr 7.6. In 5.4 each solr process based on > the core.properties (shard value assigned) will be joining as either leader > or replica based on the sequence of start. > By following the same procedure in 7.6 the initial leader node solr process > is replaced with later started solr. Also in the process core is not loaded > in the later solr process (which is supposed to be replica). > To have the compatibility , the legacyCloud property is set as true in the > clustereprops.json > Question: .With replica type is NRT to keep the transition smooth.How to add > later started solr process as replicas in 7.6, without interfering the leader > election process? > Appreciate any pointers in understanding and resolving the issue. > ThanksRavi
