Re: Query ReRanking question

Hi, Very-2 nice mail thread. I think many people might be facing the problem of maintaining the relevance and recency both at the same time. boost=max(recip(ms(NOW/HOUR,publish_date),7.889e-10,1,1),scale(query > ($q),0,1)) Currently in our search we are using the recency without any condition.

Solr 5.2.1 highlighting results are not available

I installed solr 5.2.1 and use dih example with tika integration to search for pdf content. Everything work as expected except highlighting plugin. When I execute the query I don't even see highlighting section in results: { "responseHeader": { "status": 0, "QTime": 1, "params": {

Solr SolrEntityProcessor - can it take customer parameters?

I's like to use SolrEntityProcessor for import some documents from one solr cloud to another solr cloud. The date range is dynamic and can change. Is there a way to pass, say solr/core/data-import?&fromDate=&toDate= And then use them in query for SolrEntityProcessor q="date:[$fromDate TO $toDa

Re: SOLR 5.3

On 8/4/2015 7:28 PM, William Bell wrote: > When do we think SOLR 5.3 might come out? > > It is nearly ready? There is no simple answer to this question. There are a lot of things that could change the release date. For anyone who wants to keep up on the pulse of releases, you can join the dev m

Re: Parent/Child (Nested Document) Faceting

If I want to do a faceting on field in child document and the get the count of parent document, how should I construct my query? E.g, I want to find out the authors who have written reviews on the largest number of books, as in the example nested document structures . -- View this message in

Re: SOLR 5.3

Are you watching lucene-dev list? The discussion is happening there. In short, the preparations have started, but there are things to cleanup and no RCs have been out yet. I don't think even a branch has been cut yet. So, a while to go still. Solr Analyzers, Tokenizers, Filters, URPs and ev

SOLR 5.3

When do we think SOLR 5.3 might come out? It is nearly ready? -- Bill Bell billnb...@gmail.com cell 720-256-8076

Re: solr multicore vs sharding vs 1 big collection

On 8/4/2015 3:30 PM, Jay Potharaju wrote: > For the last few days I have been trying to correlate the timeouts with GC. > I noticed in the GC logs that full GC takes long time once in a while. Does > this mean that the jvm memory is to high or is it set to low? > 1973953.560: [GC 4474277K->33004

Re: solr multicore vs sharding vs 1 big collection

For the last few days I have been trying to correlate the timeouts with GC. I noticed in the GC logs that full GC takes long time once in a while. Does this mean that the jvm memory is to high or is it set to low? [GC 4730643K->3552794K(4890112K), 0.0433146 secs] 1973853.751: [Full GC 3552794K->

Re: TrieIntField not working in Solr 4.7 ?

Hi Upayavira, I have physically cleaned up the files under index directory, and re-index did not fix the problem. The following is an example of the field definition: and the following is the definition of tint For some reason, I keep getting the error message: Caused by: java.lang.Illega

Re: TrieIntField not working in Solr 4.7 ?

Right. So, you want to use Trie fields for search performance, and doc values for more efficient faceting/sorting. Trie fields and doc values aren't the same. On your field definition in your schema, you need to add docValues="true". You'll need to re-index from scratch in order to build the docVa

Re: TrieIntField not working in Solr 4.7 ?

Hi Upayavira, My queries has all the features: search, sorting, grouping, faceting. As I was working on the project, I noticed the response time of the query got longer and longer as I added these features. I was reading the solr-ref-guide-4.7, and the following is from page 66. I thought covert

Re: Can Apache Solr Handle TeraByte Large Data

On Tue, Aug 4, 2015 at 8:10 PM, Mugeesh Husain wrote: > Thanks you Erik, I will preferred XML files instead of csv. > On my requirement if i want to use DIH for indexing than how could i split > these operation or include java clode to DIH.. > Here is my favorite way to tweak data in DIH https://

Re: Supported languages

Hi Steve, This page may be useful: In most cases the configurations described there are the only OOTB alternative, so optimality isn’t discussed. I think the path most people take i

Re: collection mbeans: requests

I have your suggested shards.qt set up in another collection for another reason but I'll do that redirect here as well, thanks for the confirmation. On 08/04/2015 10:45 AM, Shawn Heisey wrote: On 8/4/2015 5:19 AM, David Santamauro wrote: I have a question about how the stat 'requests' is ca

Re: textgen field type on Solr 5.2

On Tue, Aug 4, 2015, at 05:50 PM, shacky wrote: > Hi. > > On an old Solr 1.4.1 installation I am using some textgen fields to > separate text in words and having an "exact word" match when I search > for something. > > Now I have to migrate to a new Solr 5.2 installation, and I realized > that

Re: TrieIntField not working in Solr 4.7 ?

On Tue, Aug 4, 2015, at 06:40 PM, Shawn Heisey wrote: > On 8/4/2015 10:15 AM, wwang525 wrote: > > I waited until the indexing process finished successfully. > > > > I also set default value for these fields and I can see from simply query > > that the data was fine. The error happened after I execu

Re: Supported languages

On Tue, Aug 4, 2015, at 05:55 PM, Steven White wrote: > Hi Everyone, > > I see Solr comes pre-configured with text analyzers for a list of > supported > languages e.g.: "text_ar", "text_bq", "text_ca", "text_cjk", "text_ckb", > "text_cz", etc. > > My questions are: > > 1) How well optimized ar

Re: Can Apache Solr Handle TeraByte Large Data

On Tue, Aug 4, 2015, at 06:13 PM, Mugeesh Husain wrote: > @Upayavira if i uses Solrj for indexing. autocommit or Softautocommit > will > work in case of SolJ There are two ways to get content into Solr: * push it in via an HTTP post. - this is what SolrJ uses, what bin/post uses, and every

Re: TrieIntField not working in Solr 4.7 ?

On 8/4/2015 10:15 AM, wwang525 wrote: > I waited until the indexing process finished successfully. > > I also set default value for these fields and I can see from simply query > that the data was fine. The error happened after I execute a faceted query. There's not enough information here to answ

Re: Can Apache Solr Handle TeraByte Large Data

@Upayavira if i uses Solrj for indexing. autocommit or Softautocommit will work in case of SolJ -- View this message in context: http://lucene.472066.n3.nabble.com/Can-Apache-Solr-Handle-TeraByte-Large-Data-tp3656484p4220796.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Can Apache Solr Handle TeraByte Large Data

Thanks you Erik, I will preferred XML files instead of csv. On my requirement if i want to use DIH for indexing than how could i split these operation or include java clode to DIH.. I have googled but not get such type of requirement. provide my any of link for it or some suggestion to do it.

Supported languages

Hi Everyone, I see Solr comes pre-configured with text analyzers for a list of supported languages e.g.: "text_ar", "text_bq", "text_ca", "text_cjk", "text_ckb", "text_cz", etc. My questions are: 1) How well optimized are those languages for general usage? This is something I need help with bec

textgen field type on Solr 5.2

Hi. On an old Solr 1.4.1 installation I am using some textgen fields to separate text in words and having an "exact word" match when I search for something. Now I have to migrate to a new Solr 5.2 installation, and I realized that the textgen field is not available, so I am looking for something

Re: Documentation for: solr.EnglishPossessiveFilterFactory

Thanks Alex. Steve On Mon, Aug 3, 2015 at 9:44 PM, Alexandre Rafalovitch wrote: > Seems simple enough that the source answers all the questions: > > https://github.com/apache/lucene-solr/blob/lucene_solr_4_9/lucene/analysis/common/src/java/org/apache/lucene/analysis/en/EnglishPossessiveFilter.j

Re: TrieIntField not working in Solr 4.7 ?

Hi Alex, I waited until the indexing process finished successfully. I also set default value for these fields and I can see from simply query that the data was fine. The error happened after I execute a faceted query. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com

Re: DateRangeField Query throws NPE

OK so: This is solr 5.2.1 (however we had the same problem with 5.1.0, which prompted us to upgrade in the first place) The schema is brand new with 5.1.0. The daterangefield is defined simply as: referring to: Through the Solr admin panel, this filter query works: sku_history.date_range

Re: TrieIntField not working in Solr 4.7 ?

Did you re-index and commit completely after the definition switch? Looks like internal representation conflict. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 4 August 2015 at 11:31, wwang525 wrote: > Hi All, > > I was try

FileNotFoundException in recovery

Hi, I sometimes get FileNotFoundExceptions from the recovery of a core in my log. Does anyone know the reason for this? As I understand Solr this may (or should) not happen. Sorry for cross-posting in wrong user group (java-user). Markus 2015-08-04 15:06:07,646|INFO|mpKPXpbUwp|org.apache.solr.u

TrieIntField not working in Solr 4.7 ?

Hi All, I was trying to switch the type definition for some fields from SortableIntField to TrieIntField so that I may be able boost the performance for the queries that use grouping, sorting, and faceting. After I switched on field for grouping, I got the following error: java.lang.IllegalState

Re: serious data loss bug in correlation with "too much data after closed"

On 8/4/2015 8:06 AM, adfel70 wrote: > I saw this post: > http://lucene.472066.n3.nabble.com/Too-much-data-after-closed-for-HttpChannelOverHttp-td4170459.html > > I tried reducing the bulk size from 1000 to 200 as the post suggests (didn't > go to runing each doc in a seperate .add call yet), with

Re: solr status 404 error

On 8/4/2015 6:28 AM, harry singh wrote: > Dear Sir > When I will type > http://localhost:8080 > I will get the tomcat page. Your tomcat log should contain at least minimal info about what went wrong. Chances are good that it's a problem with logging. I see that the web page you referenced does t

Re: Closing the IndexSearcher/IndexWriter for a core

Not what I meant the LotsOfCores stuff doesn't issue the core API unload command, it's just shutting down the searcher, getting rid of the internal config structures and the like. Best, Erick On Mon, Aug 3, 2015 at 7:12 PM, Brian Hurt wrote: > So unloading a core doesn't delete the data?

Re: collection mbeans: requests

On 8/4/2015 5:19 AM, David Santamauro wrote: > > I have a question about how the stat 'requests' is calculated. I would > really appreciate it if anyone could shed some light on the figures below. > > Assumptions: > version: 5.2.0 > layout: 8 node solrcloud, no replicas (node71-node78) > co

Re: multiple but identical suggestions in autocomplete

Hello Nutch Solr user, You are right I use DocumentDictionaryFactory as you can see in my solrconfig file suggest_publisher AnalyzingInfixLookupFactory DocumentDictionaryFactory publisher text_hu_suggest_ngram suggester_infix_dir_publisher price false false You wrote that you you have develo

Re: Solr suggester throws error on core reload.

I found existing issue here https://issues.apache.org/jira/browse/SOLR-6246 . It says fix version 5.2 but Resolution is unresolved. - Nutch Solr User "The ultimate search engine would basically understand everything in the world, and it would always give you the right thing." -- View thi

Re: multiple but identical suggestions in autocomplete

May be you are using DocumentDictionaryFactory because HighFrequencyDictionaryFactory will never return duplicate duplicate terms. We also had same problem with *DocumentDictionaryFactory + AnalyzingInfixSuggester* We have created one service between UI and Solr which groups duplicate suggestions.

Re: serious data loss bug in correlation with "too much data after closed"

Are you adding all new documents? If you are not updating documents at all, take a look at your maxDocs vs numDocs, if they are not the same, then you have overwritten some documents. You may also be right that the exception you've seen could be the cause of it, just thought the above is worth che

Solr suggester throws error on core reload.

I am using AnalyzingInfixSuggester for auto-suggest feature. but whenever I try to reload solr core following error is thrown , org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@E:\SSea

serious data loss bug in correlation with "too much data after closed"

Hello, I'm using solr 5.2.1 I'm running indexing of a collection with 20 shards. around 1.7 billion docs should be indexed. the indexer is a mapreduce job that runs on yarn, running 60 concurrent containers. I index with bulks of 1000 docs and write logs for each bulk that was indexed. each such l

Re: Highlight not working when using _query_ syntax

Hi Ahmet, I made the mistake(syntax error) ... I add to this param is your adviced query hl.q="olası (kast kasıt)"~6&hl.qparser=complexphrase is working. Thanks for advice. Emre On Tue, Aug 4, 2015 at 3:45 PM, Emre ERKEK wrote: > I wrote wrong parsed query. Correct is > > > (+((+ComplexPhra

RE: Same query fails in SolrJ, but works in the browser.

I just found that, too. It's not the exact same query if you misspell your parameter. Oh, the benefits of a second look at things. Thanks Mikhail. -Original Message- From: Mikhail Khludnev [mailto:mkhlud...@griddynamics.com] Sent: Tuesday, August 04, 2015 9:37 AM To: solr-user Subje

Re: Same query fails in SolrJ, but works in the browser.

On Tue, Aug 4, 2015 at 4:24 PM, Mark Horninger < mhornin...@grayhairsoftware.com> wrote: > Hi Solr Users, > > I am running into a problem where a more complex dismax query I am trying > to run works with no problem in the browser. The query is as follows: > http:// > :8983/solr//select?q=Fred&def

Re: Same query fails in SolrJ, but works in the browser.

What does Solr log in the 400 response situation? It’d be good to compare the logging of both browser and SolrJ requests - something is different. — Erik Hatcher, Senior Solutions Architect http://www.lucidworks.com > On Aug 4, 2015, at 9:24 AM, Mark Horninger

Same query fails in SolrJ, but works in the browser.

Hi Solr Users, I am running into a problem where a more complex dismax query I am trying to run works with no problem in the browser. The query is as follows: http://:8983/solr//select?q=Fred&defType=dismax&sort=date1%20desc&qf=Line1^2%20Line2%20Line3%20Line4%20Line5%20Line6%20Line7^2%20Line8^2%

RE: Collapsing Query Parser returns one record per shard...was not expecting this...

Joel, Thank you for that information. I had not heard of composite ID routing, and found a post (by you) on the feature that was most instructive (https://lucidworks.com/blog/solr-cloud-document-routing/). Thanks for clearing up the behavior of the collapsing query parser. Sadly, I doubt co-

Re: Highlight not working when using _query_ syntax

I wrote wrong parsed query. Correct is (+((+ComplexPhraseQuery("olası (kast kasıt)") +xml:adam) (+ComplexPhraseQuery("olası (kast kasıt)") +xml_en:adam)) () () () () () () ())/no_coord On Tue, Aug 4, 2015 at 3:38 PM, Emre ERKEK wrote: > When I changed query *_query_:"{!complexphrase df=xml}\"

Re: Highlight not working when using _query_ syntax

When I changed query *_query_:"{!complexphrase df=xml}\"olası (kast kasıt)\"~6 AND adam" OR _query_:"{!complexphrase df=xml_en}\"olası (kast kasıt)\"~6 **AND adam**" *, ' adam ' is highlighted. 'xml' field content : ... OLASI KAST OLASI KASTLA ADAM... (+((+ComplexPhraseQuery("olası (kast kasıt)"

Re: solr status 404 error

Dear Sir When I will type http://localhost:8080 I will get the tomcat page. On Tue, Aug 4, 2015 at 5:34 PM, Alexandre Rafalovitch wrote: > What do you get at just http://localhost:8080/ ? > > My guess would be that you may have already had something else on that > port and your Solr instance did

Re: Highlight not working when using _query_ syntax

Hi Ahmet, Unfortunatly, I cannot leverage hl.qparser and hl.q. I tried to queries like that: *hl.q=_query_:"{!complexphrase df=xml}\"olası (kast kasıt)\"~6" OR _query_:"{!complexphrase df=xml_en}\"olası (kast kasıt)\"~6" * is not working. *hl.q="olası (kast kasıt)"~6&hl.qparser=complexphrase *i

Warning : Import command failed . another import is running

Hi All, I have enabled auto indexing via Data Import option. I see below warning occur on a daily basis not at regular specific time. Could someone please tell me why I am seeing below warning , Aug 3, 2015 1:01:34 PM org.apache.solr.handler.dataimport.DataImporter runCmd WARNING: Import comma

Re: solr status 404 error

What do you get at just http://localhost:8080/ ? My guess would be that you may have already had something else on that port and your Solr instance did not actually start. If in doubt, I would test that by bringing your Solr instance down and trying to revisit the URL. You should get a generic ad

Re: Can Apache Solr Handle TeraByte Large Data

If you have data that only consists of id (full filename) and filename (indexed, tokenized) 40M of those will fit comfortably into a single shard provided enough RAM to operate. I know SolrJ is tossed out there a lot as a/the way to index - but if you’ve got a directory tree of files and want t

solr status 404 error

Dear all I did the configuration of solr with this tutorial https://idodevjobs.wordpress.com/2014/05/13/solr-nutch-tomcat-windows-getting-started/comment-page-1/#comment-314 When I type http://localhost:8080/solr I will get http status 404 – solr The requested resource is not available Kindly

collection mbeans: requests

I have a question about how the stat 'requests' is calculated. I would really appreciate it if anyone could shed some light on the figures below. Assumptions: version: 5.2.0 layout: 8 node solrcloud, no replicas (node71-node78) collection: col1 handler: /search stats request: /col1/a

Re: Can Apache Solr Handle TeraByte Large Data

Thank @Alexandre and Erickson ,Hatcher. I will generate ID of MD5 with help of filename using java. I can do it with help of SolrJ nicely because i am java developer apart from this The question raised that data is too large i think it will break into multiple shards(core) Using multi core inde

Re: Can Apache Solr Handle TeraByte Large Data

Yes, you are right - generally autocommit is a better way. If you are doing a one-off indexing, then a manual commit may well be the best option, but generally, autocommit is a better way. Upayavira On Mon, Aug 3, 2015, at 11:15 PM, Konstantin Gribov wrote: > Upayavira, manual commit isn't a good

Re: Highlight not working when using _query_ syntax

Hi Emre, May be you can leverage hl.qparser and hl.q ? Ahmet On Tuesday, August 4, 2015 11:00 AM, Emre ERKEK wrote: Hi, When I search with nested query such as :* {!complexphrase df=xml}\"olası (kast kasıt)\"~6* , highlighting is working. But, when I use this query *_query_:"{!complexp

Highlight not working when using _query_ syntax

Hi, When I search with nested query such as :* {!complexphrase df=xml}\"olası (kast kasıt)\"~6* , highlighting is working. But, when I use this query *_query_:"{!complexphrase df=xml}\"olası (kast kasıt)\"~6"* , highlighting is not working. Finally, I want to use query like that *_query_:"{!com