Single zookeeper ensemble for multiple solr instances/clusters
Hi , I have multiple Solr clusters running(5 clusters), i would like to make use of single zookeeper ensemble contains (3 servers) . can anyone tell me what is the pros and cons of this approach? Thanks, Gomathi -- View this message in context: http://lucene.472066.n3.nabble.com/Single-zookeeper-ensemble-for-multiple-solr-instances-clusters-tp4263650.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr:Skip document from indexing when it matches specific value
Hi, How we can ignore a document from indexing into solr when a field matches particular value. Eg. we would like to ignore a document from indexing when document's field path matches value "/content". Do we have any OOTB processors to accomplish this in solr. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Skip-document-from-indexing-when-it-matches-specific-value-tp4264071.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr:Skip document from indexing when it matches specific value
thanks. i am able to do this by extending UpdateRequestProcessorFactory . -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Skip-document-from-indexing-when-it-matches-specific-value-tp4264071p4265526.html Sent from the Solr - User mailing list archive at Nabble.com.
Merge two Solr documents into One
Hi, I have 2-3 Solr documents but i would like to merge all these into one document while indexing. Something like parent-child. So do we have any utils for Solr to merge two or more SolrInputDocument into one SolrInputDocument. Thanks, Gomathi. -- View this message in context: http://lucene.472066.n3.nabble.com/Merge-two-Solr-documents-into-One-tp4265528.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Nodes appear twice in state.json
Hi Shalin, How do we edit state.json? Do we have any utility to edit state.json as we have for clusterstate.json? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Nodes-appear-twice-in-state-json-tp4274504p4275791.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Nodes appear twice in state.json
i am able to delete the down/unused cores which does not actually a core but it have an entry in state.json using the DELETEREPLICA API. /admin/collections?action=DELETEREPLICA&collection=collection name&shard=shardname&replica=(dead/unused core name listed in state.json) eg: /admin/collections?action=DELETEREPLICA&collection=collection name&shard=shardname&replica=core_node3 -- View this message in context: http://lucene.472066.n3.nabble.com/Nodes-appear-twice-in-state-json-tp4274504p4275797.html Sent from the Solr - User mailing list archive at Nabble.com.
How do we get terms suggestion from SuggestComponent?
Hi, I am using solr.SuggestComponent for auto suggestion, it works fine. But the problem is, it returns the whole field value as suggestion instead of terms. But my requirement is term needs to be returned as suggestion. How do we achieve this with solr.SuggestComponent? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-do-we-get-terms-suggestion-from-SuggestComponent-tp4283399.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How do we get terms suggestion from SuggestComponent?
Thanks Ahmet. It is working fine. Now i would like to get suggestions for multiple terms. How do i get suggestions for multiple terms? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-do-we-get-terms-suggestion-from-SuggestComponent-tp4283399p4283584.html Sent from the Solr - User mailing list archive at Nabble.com.
solr date range query
Hi, We have list of events with events start date and end date.for eg: event1 starts @ 2nd Aug 2016 ends @ 3rd Aug 2016 event2 starts @ 4th Aug 2016 ends @ 5th Aug 2016 event3 starts @ 1st Aug 2016 ends @ 7th Aug 2016 event4 starts @ 15th july 2016 ends @ 15th Aug 2016 when user selects a date range Aug 2nd to Aug 5th 2016 we are able to fetch event1 and event2 with start and end date range query (Aug 2nd TO Aug 5th ). But as event3 and event4 are also an ongoing event we need to fetch that . how this can be achieved? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-date-range-query-tp4291918.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr date range query
yes. dates are stored as a single valued date field -- View this message in context: http://lucene.472066.n3.nabble.com/solr-date-range-query-tp4291918p4291920.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr date range query
eventStartdate:[2016-08-02T00:00:00Z TO 2016-08-05T23:59:59.999Z] OR eventEnddate:[2016-08-02T00:00:00Z TO 2016-08-05T23:59:59.999Z] this is my query. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-date-range-query-tp4291918p4291922.html Sent from the Solr - User mailing list archive at Nabble.com.
overwrite the parameter query in DIH
Hi All,
We are retrieving mongodb data using Dataimport handler. We have a scenario
where we have to overwrite the mongodb query configured in data-config file.
We have to do this overwrite programmatically using solrj. For this we are
using ModifiableSolrParams to set the parameters. Here is the code snippet
used to create a dataimport http request.
String solrURL=
"http://:/solr/collectionname";
SolrClient solr = new HttpSolrClient.Builder(solrURL).build();
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("qt", "/dataimport");
params.set("command", "full-import");
params.set("query=id:{ $in: ", idlist+ " }");
QueryResponse response = solr.query(params);
Here the expectation is it should use this query parameter value given in
the code snippet instead of using the query parameter configured in
data-config file.
Is there a way to do this? .Please suggest.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: overwrite the parameter query in DIH
Thanks Erick. We are almost done with the full indexing part using DIH. So if we change the approach using Solrj now, we need to re-write the entire thing. So is there any way to overwrite the DIH data config file. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Solrcloud replication not working
Hi, We are using Solr 6.4.2 & SolrCloud setup. We have two solr instances in the solr cluster.This solrcloud running in ubuntu OS. The problem is replication is not happening between these two solr instances. sometimes it replicate 10% of the content and sometimes not. In Zookeeper ensemble we have three zookeeper instances running in a different box. thanks. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Solrcloud replication not working
Hi, We are using Solr 6.4.2 & SolrCloud setup. We have two solr instances in the solr cluster.This solrcloud running in ubuntu OS. The problem is replication is not happening between these two solr instances. sometimes it replicate 10% of the content and sometimes not. In Zookeeper ensemble we have three zookeeper instances running in a different box. thanks. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solrcloud replication not working
thanks. We dont see any error message/any message in logs. And we have enough disk space. We are running solr as root user in ubuntu box but zookeeper process running as zookeeper user.Will that cause the problem? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solrcloud replication not working
i could see different version of the below entries in Leader and replica. While doing index , in replica instance logs we could see it is keep receiving update request from leader but it says no changes, skipping commit. Master (Searching) Master (Replicable) There is no other error messages related to replication.Any idea why this is happening? Is there any API to run replication manually. - -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solrcloud replication not working
The problem was replicationFactor was set to 1.Now replication works fine while setting replicationFactor as 2. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Multiple tables data aggregation
Hi, We have set of mongodb collections which has one-many mapping/relation. So we are trying to create single document in solr from rows of different mongodb collections at the time of indexing. Can anyone suggest the best approach to achieve this? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-tables-data-aggregation-tp4331667.html Sent from the Solr - User mailing list archive at Nabble.com.
More heap usage in Solr during indexing
Hi, we have 80 million records in index now and we are indexing 800k records everyday.We have one shard and 4 replicas in 4 servers under solrcloud. Currently we have 16GB heap but during indexing sometimes it is reaching 16GB and sometimes its normal. What is the reason to use the max heap at sometimes during indexing? due to large index size(80M docs) or some large incoming record. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/More-heap-usage-in-Solr-during-indexing-tp4124898.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: More heap usage in Solr during indexing
previously we faced OOM when we try to index 1.2M records at the same time. Now we divided that into two chunks and indexing twice. So now we are not getting OOM but heap usage is more. So we are analyzing and trying to find the cause to make sure we shouldn't get OOM again. -- View this message in context: http://lucene.472066.n3.nabble.com/More-heap-usage-in-Solr-during-indexing-tp4124898p4124906.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: More heap usage in Solr during indexing
Yes Shawn. our data source is oracle DB. Here is the datasource section config. -- View this message in context: http://lucene.472066.n3.nabble.com/More-heap-usage-in-Solr-during-indexing-tp4124898p4124934.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: More heap usage in Solr during indexing
We are doing Autocommit for every five minutes. -- View this message in context: http://lucene.472066.n3.nabble.com/More-heap-usage-in-Solr-during-indexing-tp4124898p4125497.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Parallel queries to Solr
Thanks Shawn. When we run any solrj application , the below message is displayed org.apache.solr.client.solrj.impl.HttpClientUtil createClient INFO: Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false and while restarting solr we are getting this message. org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=1&socketTimeout=0&connTimeout=0&retry=false This is indicating the number of http connections by default? This can be overridden by adding the below one.? ModifiableSolrParams params = new ModifiableSolrParams(); params.add(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, "300"); params.add(HttpClientUtil.PROP_MAX_CONNECTIONS, "5000"); HttpClient httpClient = HttpClientUtil.createClient(params); Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Parallel-queries-to-Solr-tp4119959p4125806.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr data directory contains index backups
Hi, We have a solrcloud where we have one shard and four replicas.We know that Solr will create an index backup when something happens inside solrcloud. So data directory contains more than one index folders. like snapshot.20130920231746353 index.20130712112751477 index.2013... So after some time these extra files could be deleted?. how to identify the current/actual index directory.This delete would impact anything?. Please suggest. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-data-directory-contains-index-backups-tp4132590.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr data directory contains index backups
Thanks Greg. Is there any Solr configuration to do this periodically if any unused index copy or snapshot exists in data directory.? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-data-directory-contains-index-backups-tp4132590p4133221.html Sent from the Solr - User mailing list archive at Nabble.com.
How Solr join query works?
Hi All, Can anyone please explain how solr join query works in solr4.2. we have 2 different documents.Both are in the same index. document1 contains the columns: docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 and some more fields. document2 contains: docdate :01-11-2012 previousmonthdate :01-10-2012 price:10 and some more fields. Here we have the same value in previousmonthdate (in document1) and docdate (in document2).So we want to make a join query based on this to retrieve these in a single document. the final document should look like this. docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 price:10(this is from document2) Is is possible using Solr join query??? Or do we have any other approach?. Please help.. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-Solr-join-query-works-tp4110982.html Sent from the Solr - User mailing list archive at Nabble.com.
QParser parsing date into unix timestamp format
Hi, We are writing our own search handler. We are facing this below issue. We are passing a date(Date:(["2012-10-01T00:00:00.000Z"+TO+"2012-10-01T23:59:59.999Z"])) for date range search to QParser.getParser method but it is converting the date to unix timestamp format.(Date:([132217920 TO 132226559])). is there anyway to get the same date as we passed. QParser queryParser = QParser.getParser(q, defType, req); Query query = queryParser.getQuery(); DocList matchDocs2 = indexSearcher.getDocList(query1, null, null, 1, 10,1); Anyone please help. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/QParser-parsing-date-into-unix-timestamp-format-tp4111980.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Apache Solr.
You can have this kind of configuration in Data import handler xml file to index different type of files. Hope this helps. -- View this message in context: http://lucene.472066.n3.nabble.com/Apache-Solr-tp4114996p4115102.html Sent from the Solr - User mailing list archive at Nabble.com.
Parallel queries to Solr
Hi, We want to send parallel queries(2-3 queries) in the same request from client to Solr. How to send the parallel queries from client side(using Solrj). Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Parallel-queries-to-Solr-tp4119959.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Parallel queries to Solr
Thanks Jack. We are using DefaultHttpClient 4.2 .3 on solrj side to send parallel queries to Solr. But the connection manager associated with this (PoolingHttpClientConnectionManager) by default allows 2 concurrent connections only.How to send more than 2 parallel queries/how to establish more than 2 connections?. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Parallel-queries-to-Solr-tp4119959p4120210.html Sent from the Solr - User mailing list archive at Nabble.com.
Date field indexing in Solr
Hi, We are using 'solr.TrieDateField' type for indexing a date column in Solr. By default triedate will index date columns as UTC format. But we need the date as it is in the source(DB table) with time associated with that date. Do we need to use DateFormatTransformer to get the right date format. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Date-field-indexing-in-Solr-tp4120281.html Sent from the Solr - User mailing list archive at Nabble.com.
Separate logging for Solr updatereuesthandler
Hi , we have a Solr(solr4.2) setup with Jetty web server and the events are logged using log4j with the log level INFO.But here we would like to get more details about the update request received by UpdateRequestHandler. So is there anyway to configure debug log level kind of stuff for Update request only?. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Separate-logging-for-Solr-updatereuesthandler-tp4170224.html Sent from the Solr - User mailing list archive at Nabble.com.
How to track UpdateRequestHandler activities?
Hi, We are doing data update using solrj.First we are deleting the records that needs to be updated and inserting them again with the updated value.Sometimes this updates fails and we don't have any clue where it stuck.So we want to track the UpdateRequestHandler activities something like in a separated log file to find the root cause. Do we have any option to do that? Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-track-UpdateRequestHandler-activities-tp4174316.html Sent from the Solr - User mailing list archive at Nabble.com.
Replication in solrloud
Hi, We have 4 servers in Solrcloud with one shard. 2 of the servers are not in sync with other two.We like to force replication manually to keep all the servers in sync.Do we have a command to force replication? (other than Solr restart). Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Replication-in-solrloud-tp4183103.html Sent from the Solr - User mailing list archive at Nabble.com.
Adding virtual host in Jetty (Solr deployed)
Hi, We have installed solr which is running in jetty 9x version. We are trying to change the default solr url to required URL as given below. Default url: http://localhost:8983/solr Required URL :http://test.com/solr To achieve this we are trying to configure virtual host in jetty (solr-jetty-context.xml) with the below jetty documentation reference (https://wiki.eclipse.org/Jetty/Howto/Configure_Virtual_Hosts). But it is not working. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Adding virtual host in Jetty (Solr deployed)
We have added virtualHosts block in solr-jetty-context.xml file under /opt/solr/server/contexts and then restarted solr(jetty). After this while trying to access solr using the url http://www.host.com:8983/solr it says site can't be reached. http://www.eclipse.org/jetty/configure_9_0.dtd";> /solr-webapp/webapp /etc/webdefault.xml false 10.x.x.x test www.host.com -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Disadvantages of having Zookeeper instance and Solr instance in the same server
Hi, What is the pros and cons of having Zookeeper instance and Solr instance in the same VM/Server in production environment? Thanks. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
