Problems with pointing to custom core directories during startup in Solr 5.3.0

Hi, I have a custom core directories in my Solr located at solrMain\node1\solr, and I set it through the -s parameter in the Solr startup script, and it looks like this: bin\solr.cmd start -cloud -p 8983 -s solrMain\node1\solr -m 12g -z "localhost:2181,localhost:2182,localhost:2183" This works f

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

On Sep 8, 2015 6:25 AM, "Erick Erickson" wrote: > > Perhaps the browser cache? What happens if you, say, use > Zookeeper client tools to bring down the the cluster state in > question? Or perhaps just refresh the admin UI when showing > the cluster status > Hello Erick. Thank you very much f

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

On Sep 8, 2015 6:25 AM, "Erick Erickson" wrote: > > Perhaps the browser cache? What happens if you, say, use > Zookeeper client tools to bring down the the cluster state in > question? Or perhaps just refresh the admin UI when showing > the cluster status > Hello Erick. Thank you very much f

Re: Different boost values for multiple parsers in Solr 5.2.1

you can add bq= inside your {!synonym_edismax} section, if you wish and it will apply to that query parser only. Upayavira On Mon, Sep 7, 2015, at 03:05 PM, dinesh naik wrote: > Please find below the detail: > > My main query is like this: > > q=(((_query_:"{!synonym_edismax qf='itemname OR it

Re: Maximum Number of entires in External Field?

If you have just 5-7 items, then an external file will work, as will the join query. You'll need to handle the 'default' case with the join query, that is, making sure you do OR so that documents matching the join are boosted above those matching the main query, rather than the join being a filte

Re: Search opening hours

BTW any idea how index speed is influenced? I used worldbounds with -1 and 1 y-axes. But figured this could also be 0. After changing to 0 indexing became a lot slower though (no exceptions in log). -- View this message in context: http://lucene.472066.n3.nabble.com/Search-opening-hours-tp422

RE: Trouble making tests with BaseDistributedSearchTestCase

Thanks! I went on using AbstractFullDistribZkTestBase and for some tests i circumvent the control core. I do sometimes get a recovery time out when starting up the tests. I have set the time out to 30 seconds, just like many other tests that extends AbstractFullDistribZkTestBase. Any thoughts o

Re: Different boost values for multiple parsers in Solr 5.2.1

Thanks Alot Upayavira. It worked as expected. On Tue, Sep 8, 2015 at 2:09 PM, Upayavira wrote: > you can add bq= inside your {!synonym_edismax} section, if you wish and > it will apply to that query parser only. > > Upayavira > > On Mon, Sep 7, 2015, at 03:05 PM, dinesh naik wrote: > > Please f

SOLR DataImportHandler - Problem with XPathEntityProcessor

Hi All I am facing a problem with XPathEntityProcessor . Objective: When I index Resource XML file using DIH XPathEntityProcessor then there should be 2 solr documents 01) Link where id is 1000 with 2 tags ABC and DEF 02) Link where id is 2000 with 3 tags GHI, JKL and MNO Solr Version: 4.10.2 P

Solr Join between two indexes taking too long.

Hi, I hope somebody can help. We have two indexes, one which holds the descriptive data and the other one which holds lists of docs which are of a certain type (called universes in our world). They need to be joined together to show a list of data from indexA where a filtered indexB (by univers

Help on Out of memory when using Cursor with sort on Unique Key

Cluster details : Solr Version : solr-4.10.4 No of nodes : 2 each 16 GB RAM Node of shards : 2 Replication : 1 Each node memory parameter : -Xms2g, -Xmx4g Collection details : No of docs in my collection : 12.31 million Indexed field per document : 2 Unique key field : tids Stored filed per doc

Re: Search opening hours

Sounds odd that the indexing times would change. Hopefully something else was going on - I've not experienced this. On Tue, Sep 8, 2015 at 4:31 AM, O. Klein wrote: > BTW any idea how index speed is influenced? > > I used worldbounds with -1 and 1 y-axes. But figured this could also be 0. > > Aft

Re: Help on Out of memory when using Cursor with sort on Unique Key

Hi Naresh 1) For 'sort by' fields, have you considered using DocValue=true for in schema definition. If you are changing schema definition, you would need redo full reindex after backing up & deleting current index from dataDir. Also note that, adding docValue=true would increase size of index.

conf Folder is not getting created while creating a collection on solr cloud

I am trying to create a collection on Solr cloud. I have created a 3 node zookeeper cluster on the same machine. using this command to start solr on three ports : bin/solr start cloud -z localhost:2181,localhost:2182,localhost:2183 -p 8983 bin/solr start cloud -z localhost:2181,localhost:2182,l

Re: SOLR DataImportHandler - Problem with XPathEntityProcessor

Both version seem to be painful in that they will retrieve the URL content multiple times. The first version is definitely wrong. The second version is probably wrong because both inner and outer entities are having the same name. I would try giving different name to the inner entity and seeing if

Re: Merging documents from a distributed search

Joel, It needs to perform. Typically users will have 1 - 5 million rows in a query, returning 10 - 15 fields. Grouping reduces the return by 50% or more normally. Responses tend be less than a half second. It sounds like the manipulation of docs at the collector level has been left to the single

Solr facets implementation question

I am trying to understand why faceting on a field with lots of unique values has a great impact on query performance. Since Googling for Solr facet algorithm did not yield anything, I looked how facets are implemented in Lucene. I found out that there are 2 methods - taxonomy-based and SortedSetDoc

Re: Search opening hours

Doesn't sound odd to me. I just expected index time to be faster with smaller "world" I used minutes as scale first, but that slows it even down a lot more. So changed to 15 minute interval to keep it reasonable. Maybe there is a setting that can speed this up. Like the precisionStep in a Triefi

Re: Problems with pointing to custom core directories during startup in Solr 5.3.0

I have found that it could be a problem in solr.cmd. It works after I replace this line in Solr 5.3.0 IF NOT "%EXAMPLE%"=="" goto run_example With this segment of the code from Solr 5.2.1 IF "%EXAMPLE%"=="" ( IF NOT "%SOLR_HOME%"=="" ( REM Absolutize a relative solr home IF EXIS

Re: conf Folder is not getting created while creating a collection on solr cloud

There is no conf file located at solr-5.3.0/server/solr/test_collection_shard1_replica1/. Inside that folder should only contain data folder and core.properties file. The conf folder is only in the solr-5.3.0/server/solr/data_ driven_schema_configs. Why do you need the conf file in solr-5.3.0/serv

Re: Solr facets implementation question

On 9/8/2015 9:10 AM, adfel70 wrote: > I am trying to understand why faceting on a field with lots of unique values > has a great impact on query performance. Since Googling for Solr facet > algorithm did not yield anything, I looked how facets are implemented in > Lucene. I found out that there are

Re: Solr facets implementation question

Every faceting implementation I’ve seen (not just Solr/Lucene) makes big in-memory lists. Lots of values means a bigger list. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Sep 8, 2015, at 8:33 AM, Shawn Heisey wrote: > On 9/8/2015 9:10 AM, adfel70

Log4J level from admin web UI

Hi All, I am using Solr 4.8.1 and when I tries to change the log level via the admin web UI It doesn't do anything and the only way to change the log level is to edit my log4j file and restart SOLR process Is it a known issue? Can you guide me what should I check?

Re: SOLR DataImportHandler - Problem with XPathEntityProcessor

Thanks Alex. Inner entity name should be different - It was a typo error in my question. Regarding using XsltUpdateRequestHandler , It's a good solution but I can not use it in my application since I need to include few more transformer and

Re: Sorting on date with multivalued False attribute

Hi, stop the solr server,delete index before indexing you should change or write schema fields then start solr and index which you want. -- View this message in context: http://lucene.472066.n3.nabble.com/Sorting-on-date-with-multivalued-False-attribute-tp4227495p4227625.html Sent from the So

Solr Cloud: Massive indexing

Hello, I am indexing lots of big documents thanks to Solr Cloud in a map reduce job: so every day it is 1 - 2 documents (avg:8Mb, max 100Mb, total ~ 100 Gb). This is done is 20 minutes. We have 5 nodes, Solr server is launched with 20 Gb of Ram (and GC1). We add in parallel around 200 S

Re: conf Folder is not getting created while creating a collection on solr cloud

When you create a collection, you specify a "configset" via the collection.configName parameter _or_ it's the same name as your collection and already uploaded _or_ it's the only configset up in ZK. Anyway, thereafter whenever a node starts up it downloads the configs from ZK. If the config direct

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

Arcadius: Hmmm. It may take a while for the cluster state to change, but I'm assuming that this state persists for minutes/hours/days. So to recap: If dump the entire ZK node from the root, you have 1> liveNodes has N nodes listed (correctly) 2> clusterstate.json has N+M nodes listed as "active"

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

I believe this is expected in the current code. From Replica.State javadoc: /** * The replica's state. In general, if the node the replica is hosted on is * not under {@code /live_nodes} in ZK, the replica's state should be * discarded. */ public enum State { /** * The r

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

Hello Erick. Yes, 1> liveNodes has N nodes listed (correctly): Correct, liveNodes is always right. 2> clusterstate.json has N+M nodes listed as "active": clusterstate.json is always empty as it's no longer being "used" in 5.3. You were probably referring to state.json which is in individual coll

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

Thanks Dan! Please let us know what you find. I’m interested to know if this is an issue with anyone else’s setup or if I have an issue in my local configuration that is still preventing it to work on start/restart. - Kevin > On Sep 5, 2015, at 8:45 AM, Dan Davis wrote: > > Kevin & Noble, >

Re: Log4J level from admin web UI

On 9/8/2015 10:02 AM, Nir Barel wrote: > Hi All, > > I am using Solr 4.8.1 and when I tries to change the log level via the admin > web UI > It doesn't do anything and the only way to change the log level is to edit my > log4j file and restart SOLR process > > Is it a known issue? > Can you guide

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

bq: You were probably referring to state.json yep, I'm never sure whether people are on the old or new ZK versions. OK, With Tomás' comment, I think it's explained... although confusing. WDYT? On Tue, Sep 8, 2015 at 10:03 AM, Arcadius Ahouansou wrote: > Hello Erick. > > Yes, > > 1> liveNodes

Re: SOLR DataImportHandler - Problem with XPathEntityProcessor

What about DIH's own XSL pre-processor? It is XSL param on https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler#UploadingStructuredDataStoreDatawiththeDataImportHandler-TheXPathEntityProcessor No other ideas, unfortunately, I don't usua

Solr score distribution usage

Hello, I would like to use the Solr score distribution to pick up most relevant documents from the search result. Rather than top n results, I am interested only in picking up the most relevant based on statistical distribution of the scores. A brief study of some sample searches (the most freque

tmp directory over load

HI Solr/Tika uses the /tmp directory to process documents. At times the directory hits 100%. This causes alarms from Nagios for us. Is there a way in Solr/Tika to limit the amount of space used in /tmp? Value could be 80% or 570MB. thanks Tom LeZotte Health I.T. - Senior Product Developer (p)

Re: Solr facets implementation question

adfel70 wrote: > I am trying to understand why faceting on a field with lots of unique values > has a great impact on query performance. Faceting in Solr is performed in different ways. String faceting different from Numerics faceting, DocValued fields different from non-DocValued, fc different

Source address of zookeeper connection

Hi, We have multihomed hosts running solr 5.2.1 as well es external zookeeper instances. In solr.in.sh, the value of SOLR_HOST is set correctly to the hostname with the correct IP address that must be used. However, in the zookeeper logs I find another IP address used as source address for the

Exception using Json Facet API with Multivalue Int field +docValues=true

Hello! I'm attempting to facet a multivalue int field that has docvalues enabled. Using the new json facet api, running 5.3.0, I get the exception here: http://pastebin.com/xNaqGJRf Here's the relevant config: Here's my facet: json.facet={"pv-44":{"type":"query","q":"ReportingDate:[2015-

Re: Exception using Json Facet API with Multivalue Int field +docValues=true

A sanity check question. Was this test done with a completely new index after you enabled docvalues? Not just "delete all" but actually deleted index directory and rebuilt from scratch? If it still happens after such a thorough cleanup, it might be a bug. Regards, Alex. Solr Analyzers, Tok

RE: Exception using Json Facet API with Multivalue Int field +docValues=true

It's a very large index, will take a couple of days to reload it from scratch. I'll post back once I have tried this with either success or failure. Thanks, Curt A sanity check question. Was this test done with a completely new index after you enabled docvalues? Not just "delete all" but

Re: Exception using Json Facet API with Multivalue Int field +docValues=true

Could you make a small index from scratch using a subset of data and see if the problem happens anyway? If yes, you have a test case. If no, you may need to do a full rebuild to be fully assured. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.sol

RE: Exception using Json Facet API with Multivalue Int field +docValues=true

The issue persists even with a new core with a couple of documents. One thing I did notice though, is that string multi-value fields do not have this problem. I can probably use that as a workaround for now, but this seems like a bug. Thanks, Curt Could you make a small index from scrat

Re: Solr Join between two indexes taking too long.

Hello Russ, It's an interesting case! Can you get a brief context? - is it possible to keep both type of data at the same core? Why not? - can you manually shard both indices by those longValues? - It seems like you query a plenty of data, don't you have another query/filter to intersect that join

Re: Exception using Json Facet API with Multivalue Int field +docValues=true

Right. It seems like a functional gap - numeric DVs out of scope for a while. https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/search/facet/FacetField.java#L147 On Tue, Sep 8, 2015 at 11:57 PM, Curtis Fehr wrote: > The issue persists even with a new core with a

Re: Source address of zookeeper connection

On 9/8/2015 3:09 PM, Jens Brandt wrote: > We have multihomed hosts running solr 5.2.1 as well es external zookeeper > instances. In solr.in.sh, the value of SOLR_HOST is set correctly to the > hostname with the correct IP address that must be used. However, in the > zookeeper logs I find another

Re: Maximum Number of entires in External Field?

> > I can provide examples if needed. Yes that will be so much helpful. Thank you so much. Then I will try both methodology. And will report the results back here. With Regards Aman Tandon On Tue, Sep 8, 2015 at 2:11 PM, Upayavira wrote: > If you have just 5-7 items, then an external file wil

Replication Sync OR Async?

We are using Solrcloud 5.3 with 2 shards and 2 replica. We observed that indexing is slower when replicas is up and running. If we stop replicas than indexing become very fast. Here is some readings for indexing of 10 documents. When replicas are running it took around 900 seconds for indexin

Solr Replication sometimes coming in log files

Hi Team, I am currently working with Java-1.7, Solr-4.8.1 with tomcat 7. The solr configuration has master & slave ( 2 Slaves) architecture. Master & Slave 2 are in same server location (say zone A) , whereas Slave 1 is in another server in different zone (say zone B). There is latency of 40 ms

Re: SolrCloud Admin UI shows node is Down, but state.json says it's active/up

Thank you Tomás for pointing to the JavaDoc http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE The Javadoc is quite clear. So this stale state.json is not an issue after all. However, it's very confusing that when a node goes down, state.json may

Re: conf Folder is not getting created while creating a collection on solr cloud

Yeah, I got the point. Thanks a lot guys. @eric @edwin On Tue, Sep 8, 2015 at 10:14 PM, Erick Erickson wrote: > When you create a collection, you specify a "configset" via the > collection.configName > parameter _or_ it's the same name as your collection and already uploaded > _or_ it's the only