Data Import Handler

Hi Folks, Can anyone suggest me how can customize dataconfig.xml file I want to provide database details like( db_url,uname,password ) from my own properties file instead of dataconfig.xaml file

RE: Data Import Handler

h.url=jdbc:url:something See http://wiki.apache.org/solr/SolrConfigXml?#System_property_substitution James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: Ramesh [mailto:ramesh.po...@vensaiinc.com] Sent: Wednesday, November 06, 2013 7:25 AM To: solr-user@lucene.apache.org Subj

RE: Data Import Handler

you. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: Ramesh [mailto:ramesh.po...@vensaiinc.com] Sent: Wednesday, November 13, 2013 9:00 AM To: solr-user@lucene.apache.org Subject: RE: Data Import Handler James can elaborate how to process driver

Solr is making Jboss server slower..

Hi, We are using the solr search, Sometimes due to this our entire system is becoming slow. We didn't got any clue.. So we tried to test load & stress test through the Jmeter. Here we have tried for many times with different testing ways. At random times in Jmeter we are getting the following m

deployee issu on solr

Unable to deploying solr 4.4 on JBoss -4.0.0 I am getting error like

RE: solr4.4 admin page show "loading"

Use Mozilla for better use even in IE it is not working properly -Original Message- From: William Bell [mailto:billnb...@gmail.com] Sent: Tuesday, September 24, 2013 12:02 PM To: solr-user@lucene.apache.org Subject: Re: solr4.4 admin page show "loading" Use Chrome. On Thu, Sep 19, 2013

Re: Auto-optimization for Solr indexes

Hi Erick, We index around several million documents/ day and we optimize everyday when the relative load is low. The reason we optimize is, we dont want the index sizes to grow too large and auto optimzie to kick in. When auto optimize kicks in, it results in unpredictable performance as it is CPU

Re: Auto-optimization for Solr indexes

> Here's a wonderful video about the process. you want the third one > down (TieredMergePolicy) as that's the default. > > > http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html > > Best, > Erick > > On Sun, Dec 20, 2015 at 8:26 PM,

Re: Pro and cons of using Solr Cloud vs standard Master Slave Replica

Please have a look at this post https://support.lucidworks.com/hc/en-us/articles/201298317-What-is-SolrCloud-And-how-does-it-compare-to-master-slave- We dont use Master slave architecture, however we use solr cloud and standalone solr for our documents. Indexing is a bit slow in cloud when compa

Understanding solr commit

We are facing some issue and we are finding it difficult to debug the problem. We wanted to understand how solr commit works. A background on our setup: We have 3 Node Solr Cluster running in version 5.3.1. Its a index heavy use case. In peak load, we index 400-500 documents/second. We also want t

Re: Understanding solr commit

We are using Solr Cloud with replication factor of 2 and no of shards as either 2 or 3. Thanks, Rahul On Mon, Jan 25, 2016 at 4:43 PM, Alessandro Benedetti wrote: > Let me answer in line : > > On 25 January 2016 at 11:02, Rahul Ramesh wrote: > > > We are facing so

Re: Understanding solr commit

ls about Solr heap parameters. Running Solr > on the edge of OOM is always risk of starting snowball effect and crashing > entire cluster. Also can you give us info about auto commit (both hard and > soft) you used when experienced OOM. > > Thanks, > Emir > > On 25.01.2016

Re: Understanding solr commit

e such tool that can give you info how > you Solr, JVM and host handle different load. One such tool can give you > enough info to tune your Solr. > > Regards, > Emir > > > On 25.01.2016 13:42, Rahul Ramesh wrote: > >> Can you give us bit more details about Solr heap

Re: Increasing Solr5 time out from 30 seconds while starting solr

Hi Debraj, I dont think increasing the timeout will help. Are you sure solr/ any other program is not running on 8789? Please check the output of lsof -i :8789 . Regards, Rahul On Tue, Dec 8, 2015 at 11:58 PM, Debraj Manna wrote: > Can someone help me on this? > On Dec 7, 2015 7:55 PM, "D" wro

Re: Moving to SolrCloud, specifying dataDir correctly

We currently moved data from magnetic drive to SSD. We run Solr in cloud mode. Only data is stored in the drive configuration is stored in ZK. We start solr using the -s option specifying the data dir Command to start solr ./bin/solr start -c -h -p -z -s We followed the following steps to migr

Re: solrcloud used a lot of memory and memory keep increasing during long time run

You should actually decrease solr heap size. Let me explain a bit. Solr requires very less heap memory for its operation and more memory for storing data in main memory. This is because solr uses mmap for storing the index files. Please check the link http://blog.thetaphi.de/2012/07/use-lucenes-mm

Querying Nested documents

ll other children* (1234-platform-ios and 1234-platform-andriod) even if i query based on image_uri_s (1234-images) although they are other children which are part of the parent document. Is it possible ? Appreciate your help ! Thanks, Ramesh

Re: Querying Nested documents

Yes you are right. So the query you are saying should be like below .. or did I misunderstood it http://localhost:8983/solr/demo/select?q= {!parent which='type:parent'}&fl=*,[child parentFilter=type:parent childFilter=(image_uri_s:somevalue) OR (-image_uri_s:*)]&indent=true If so, I am getting a

Re: Querying Nested documents

Mikhail - This worked great. http://localhost:8983/solr/demo/select?q={!parent which='type:parent'}image_uri_s:somevalue&fl=*,[child parentFilter=type:parent childFilter=-type:parent]&indent=true Thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/Querying-Nes

Solr 6.1.x Release Date ??

Hi, Any idea of Solr 6.1.X Release Date ?? I am interested in the [subquery] transformer and like to know the release date since its available only in 6.1.x Thanks & Regards Ramesh

Re: Solr 6.1.x Release Date ??

Hi, I found it working in [subquery] transformer solr-6.1.0-79 nightly builds. Regards Ramesh On Tue, Jun 7, 2016 at 11:08 AM, Ramesh Shankar wrote: > Hi, > > Any idea of Solr 6.1.X Release Date ?? > > I am interested in the [subquery] transformer and like to know the release &

Re: Solr 6.1.x Release Date ??

Hi, Yes, i used the solr-6.1.0-79 nightly builds and [subquery] transformer is working fine in, any idea of the expected release date for 6.1 ? Regards Ramesh -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-6-1-x-Release-Date-tp4280945p4282562.html Sent from the Solr

How to retain the original format of input document in search results in SOLR - Tomcat

display in the next line in the results. Can anyone please help on this. Thanks in advance. Regards, Ramesh p.y -- Ramesh P.Y pyrames...@gmail.com Mobile No:+91-9176361984

7.3.1: Query of death - all nodes ran out of memory and had to be shut down

Hi everyone, We ran into an issue yesterday where all our ec2 machines, running solr, ran out of memory and could not heal themselves. I'll try break down what happened here. *System Architecture:* - Solr Version: 7.3.1 - Replica Types: TLOG/PULL - Num Shards: 8 (default hashing mechanism) - Doc

Re: 7.3.1: Query of death - all nodes ran out of memory and had to be shut down

modern standards. > > So it's very likely (but not guaranteed) that using cursors will fix > this problem. > > Best, > Erick > > > > On Mon, Aug 20, 2018 at 8:55 PM, Ash Ramesh wrote: > > Hi everyone, > > > > We ran into an issue yesterday where

Re: 7.3.1: Query of death - all nodes ran out of memory and had to be shut down

Thank you all :) We have made the necessary changes to mitigate this issue On Wed, Aug 22, 2018 at 6:01 AM Shawn Heisey wrote: > On 8/20/2018 9:55 PM, Ash Ramesh wrote: > > We ran a bunch of deep paginated queries (offset of 1,000,000) with a > > filter query. We set the timeo

Understanding how timeAllowed works in a distributed cluster

Hi again, Specs: 7.3.1 | 8 Shards | Solr Cloud I was wondering how the timeAllowed parameter works when you architect your cluster in a sharded and distributed manner. This is the curl command and the timing Query: http://localhost:9983/solr/media/select?fq=someField:someRedactedValue&q=*:*&qq=*

Re: Understanding how timeAllowed works in a distributed cluster

, 2018 at 4:09 PM Ash Ramesh wrote: > > > Hi again, > > > > Specs: 7.3.1 | 8 Shards | Solr Cloud > > > > I was wondering how the timeAllowed parameter works when you architect > your > > cluster in a sharded and distributed manner. This is the curl command a

Solr Cloud not routing to PULL replicas

Hi again, We are currently using Solr 7.3.1 and have a 8 shard collection. All our TLOGs are in seperate machines & PULLs in others. Since not all shards are in the same machine, the request will be distributed. However, we are seeing that most of the 'distributed' parts of the requests are being

Potential bug? maxConnectionsPerHost on requestHandler configuration

Hi, I tried setting up a bespoke ShardHandlerFactory configuration for each request handler in solrconfig.xml. However when I stepped through the code in debug mode (via IntelliJ) I could see that the ShardHandler created and used in the searcher still didn't reflect the values in solrconfig (even

Questions about stored fields and updates.

Hi everyone, My company currently uses SOLR to completely hydrate client objects by storing all fields (stored=true). Therefore we have 2 types of fields: 1. indexed=true | stored=true : For fields that will be used for searching, sorting, etc. 2. indexed=false | stored=true: For fields

Re: Questions about stored fields and updates.

On Mon, Nov 5, 2018 at 11:29 AM Shawn Heisey wrote: > On 11/3/2018 9:45 PM, Ash Ramesh wrote: > > My company currently uses SOLR to completely hydrate client objects by > > storing all fields (stored=true). Therefore we have 2 types of fields: > > > > 1. index

Re: Questions about stored fields and updates.

Also thanks for the information Shawn! :) On Mon, Nov 5, 2018 at 12:09 PM Ash Ramesh wrote: > Sorry Shawn, > > I seem to have gotten my wording wrong. I meant that we wanted to move > away from atomic-updates to replacing/reindexing the document entirely > again when changes ar

Problem understanding why QPS is so low

Hi everybody, My team run a solr cluster which has very low QPS throughput. I have been going through the different configurations in our setup, and think that it's probably the way we have defined our request handlers that is causing the slowness. Details of our cluster are below the fold. *Que

Performance problems with extremely common terms in collection (Solr 7.4)

Hi everybody, We have a corpus of 50+ million documents in our collection. I've noticed that some queries with specific keywords tend to be extremely slow. E.g. the q=`photography' or q='background'. After digging into the raw documents, I could see that these two terms appear in greater than 90%

Re: Performance problems with extremely common terms in collection (Solr 7.4)

aphy' or 'background' to appear commonly in a document's keyword list. it seems unlikely we can use the common grams solution with our business case. Regards, Ash On Mon, Apr 8, 2019 at 5:01 PM Toke Eskildsen wrote: > On Mon, 2019-04-08 at 09:58 +1000, Ash Ramesh wrote: &g

Re: Solr running on Tomcat

Yes, we are running Solr 5.3.1 version in tomcat for our production system.it serves well. Thanks, Ramesh > On Feb 15, 2018, at 10:54 PM, GVK Prasad wrote: > > > I read some posts on setting up Solr to Run on Tomcat. But all these posts > are about Solr version 4.0 or e

Memory requirements for TLOGs (7.3.1)

Hi everybody, I have a quick question about what the memory requirements for TLOG machines are on 7.3.1. We currently run replication where there are 3 TLOGs with 8gb ram (2gb heap) and N PULL replicas with 32gb ram (4gb heap). We have > 10M documents (1 collection) with the index size being ~ 17g

Re: Memory requirements for TLOGs (7.3.1)

rick > > On Wed, Jul 18, 2018 at 7:46 AM, Shawn Heisey wrote: > > On 7/18/2018 12:04 AM, Ash Ramesh wrote: > >> > >> I have a quick question about what the memory requirements for TLOG > >> machines are on 7.3.1. We currently run replication where the

Best Practises around relevance tuning per query

Hi, We are in the process of applying a scoring model to our search results. In particular, we would like to add scores for documents per query and user context. For example, we want to have a score from 500 to 1 for the top 500 documents for the query “dog” for users who speak US English. We be

Re: Best Practises around relevance tuning per query

ping on this :) On Tue, Feb 18, 2020 at 11:50 AM Ashwin Ramesh wrote: > Hi, > > We are in the process of applying a scoring model to our search results. > In particular, we would like to add scores for documents per query and user > context. > > For example, we want to hav

Re: Best Practises around relevance tuning per query

solution. If you would describe the business > case in more detail without including the solution itself more people could > help. > > Eg it ie not clear why you have a scoring model and why this can address > business needs. > > > Am 18.02.2020 um 01:50 schrieb Ashwin Ramesh

Overseer & Backups - Questions

Hi everybody, Quick Specs: - Solr 7.4 Solr Cloud - 30gb index on 8 shards Tlog/Pull We run daily backups on our 30gb index and noticed that the overseer does not process other jobs on it's task list while the backup is being taken. They remain on the pending list (in ZK). Is this expected? Also

Re: Overseer & Backups - Questions

being flushed to segments though. Regards, Ash On Wed, Mar 11, 2020 at 3:18 AM Aroop Ganguly wrote: > May we know how you are invoking backups ? > > > On Mar 9, 2020, at 11:53 PM, Ashwin Ramesh > wrote: > > > > Hi everybody, > > > > Quick Specs: > >

Re: Overseer & Backups - Questions

h On Wed, Mar 11, 2020 at 9:21 AM Aroop Ganguly wrote: > Backups on hdfs ? > These should not be blocking if invoked asynchronously, are u doing them > async by passing the async flag? > > > On Mar 10, 2020, at 3:19 PM, Ashwin Ramesh > wrote: > > > > We use the

LTR - FieldValueFeature Question

Hi everybody, Do we need to have 'indexed=true' to be able to retrieve the value of a field via FieldValueFeature or is having docValue=true enough? Currently, we have some dynamic fields as [dynamicField=true, stored=false, indexed=false, docValue=true]. However when we noticing that the value e

Solr 7.4 - LTR reranker not adhering by Elevate Plugin

Hi everybody, We are running a query with both elevateIds=1,2,3 & a reranker phase using LTR plugin. We noticed that the results do not return in the expected order - per the elevateIds param. Example LTR rq param {!ltr.model=foo reRankDocs=250 efi.query=$q} When I used the standard reranker ({!

Cannot add replica during backup

Hi everybody, We are using solr 7.6 (SolrCloud). We notices that when the backup is running, we cannot add any replicas to the collection. By the looks of it, the job to add the replica is put into the Overseer queue, but it is not being processed. Is this expected? And are there any workarounds?

Re: Backups in SolrCloud using snapshots of individual cores?

I would love an answer to this too! On Fri, Aug 7, 2020 at 12:18 AM Bram Van Dam wrote: > Hey folks, > > Been reading up about the various ways of creating backups. The whole > "shared filesystem for Solrcloud backups"-thing is kind of a no-go in > our environment, so I've been looking for ways

Re: Cannot add replica during backup

e first place and have > autoAddReplicas=true for indemnity? > > > On Aug 10, 2020, at 6:32 PM, Ashwin Ramesh > wrote: > > > > Hi everybody, > > > > We are using solr 7.6 (SolrCloud). We notices that when the backup is > > running, we cannot add any re

Re: Cannot add replica during backup

next task in the > queue. > > It would also help to ensure your overseer is on a node with a role that > exempts it from any Solr index responsibilities. > > > > On Aug 10, 2020, at 6:43 PM, Ashwin Ramesh > wrote: > > > > Hey Aroop, the general process for ou

Re: Cannot add replica during backup

witch to an ebs > disk for backups on a different project because efs couldn't keep up. > > > On Aug 10, 2020, at 9:43 PM, Ashwin Ramesh > wrote: > > > > Hey Aroop, the general process for our backup is: > > - Connect all machines to an EFS drive (AWS&#x

Sort on docValue field is slow.

Hello everybody, Hoping to get advice on a specific issue - We have a collection of 50M documents. We recently added a featuredAt field defined as such - This field is sparely populated such that only a small subset (3-5 thousand currently) have been tagged with that field. We have a business

Re: Sort on docValue field is slow.

only difference. I am testing the raw URL call timing difference with and without the extra sort. Hope this helps, Regards, Ash On Mon, May 20, 2019 at 11:17 PM Shawn Heisey wrote: > On 5/20/2019 6:25 AM, Ashwin Ramesh wrote: > > Hoping to get advice on a specific issue - We have a coll

Are docValues useful for FilterQueries?

Hi everybody, I can't find concrete evidence whether docValues are indeed useful for filter queries. One example of a field: This field will have a value between 0-1 The only usecase for this field is to filter on a range / subset of values. There will be no scoring / querying on this field

Is it possible to skip scoring completely?

Hi everybody, I was wondering if there is a way we can tell solr (7.3+) to run none of it's scoring logic. We would like to simply add a set of filter queries and order on a specific docValue field. e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC" Thanks in advance, Ash

Re: Is it possible to skip scoring completely?

o these are just assumptions. > > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > On 12 Sep 2019, at 01:27, Ashwin Ramesh wrote: > > > > Hi everybody, >

Re: Is it possible to skip scoring completely?

Ah! Thanks so much! On Thu., 12 Sep. 2019, 11:56 pm Shawn Heisey, wrote: > On 9/12/2019 12:43 AM, Ashwin Ramesh wrote: > > I just tried a * query with filters with fl=id,score. I noticed that all > > scores were 1.0. Which I assume means no scoring was done. When I added a >

Best field type for boosting all documents

Hi everybody, We have a usecase where we want to push a popularity boost for each document in our collection. When a user searches for any term, we would like to arbitrarily add an additional boost by this value (which is different for each document). E.g. q=foo&boost=def(popularityBoostField,1)

Dealing with multi-word keywords and SOW=true

Hi everybody, I am using the edismax parser and have noticed a very specific behaviour with how sow=true (default) handles multiword keywords. We have a field called 'keywords', which uses the general KeywordTokenizerFactory. There are also other text fields like title and description. etc. When

Re: Dealing with multi-word keywords and SOW=true

d adding > it explicitly? As keyword:”ice cream” > > Best, > Erick > > > On Sep 30, 2019, at 5:27 AM, Ashwin Ramesh wrote: > > > > Hi everybody, > > > > I am using the edismax parser and have noticed a very specific behaviour > > with how sow=tr

Solr 7.6.0 - OOM Caused Down Replica. Cannot recover. Please advice

Hi everyone, We had an OOM event earlier this morning. This has caused one of our shards to lose all it's replicas and it's leader is still in a down state. We have restarted the Java process (solr) and it's still in a down state. Logs below: ``` Feb 25, 2021 @ 11:46:43.000 2021-02-25 00:46:43.26

SOLR IndexSearcher Opening

Regards, Ramesh babu Murari The information contained in this message is intended only for the recipient, and may be a confidential attorney-client communication or may otherwise be privileged and confidential and protected from disclosure. If the reader

Solr sorting issue : can not sort on multivalued field

document, but thats not used in the sort. And i have verified in both the solr versions 3.1 and 3.5. same error. Can some one help me out.. Cheers Ramesh vel

Re: Highlight feature

That is the default response format. If you would like to change that, you could extend the search handler or post process the XML data. Another option would be to use the javabin (if your app is java based) and build xml the way your app would need.   Best Regards, Ramesh >

Re: Invalid version (expected 2, but 60) on CentOS in production please Help!!!

I have seen similar errors before when the solr version and solrj version in the client don't match.   Best Regards, Ramesh