Re: Modify the log directory for dih

Hi, I am using: Solr: 7.4 OS: windows7 I start solr using a service on startup. Additional info: I am developing a web application that uses solr as search engine, I use DIH to index folders in solr using the FileListEntityProcessor. What I need is logging each index operation in a file that I ca

Re: CMS GC - Old Generation collection never finishes (due to GC Allocation Failure?)

We run a big cluster with 8 GB heap on the JVMs. When we used CMS, I gave 2 GB to the new generation. Solr queries make a ton of short-lived allocations. You want all of that to come from the new gen. I don’t fool around with ratios. I just set the numbers. We used these: -d64 -server -XX:+Use

Re: Metrics API via Solrj

Thanks a lot Jason and Shawn, it is quite smooth although there is no built in stuff like collection or schema request objects for metrics :) - Zeki ama calismiyor... Calissa yapar... -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: CMS GC - Old Generation collection never finishes (due to GC Allocation Failure?)

We use 4.3.0 I found that we went into gc hell as you describe with small newgen. We use CMS gc as well Using newration=2 got us out of that 3 wasn't enough...heap of 32 gig only I have not gone over 32 gig as testing showed diminishing returns over 32 gig. I only was brave enough to go to 4

Re: Migrate cores from 4.10.2 to 7.5.0

See SOLR-12281 for details around the new migration limitation that will be introduced from v8.0. You may be able to get yourself from 4.x to 7.x through clever hoops, but then if you want to upgrade to 8.x later, there is no other way than full

Re: Migrate cores from 4.10.2 to 7.5.0

On 10/3/2018 3:17 PM, Pure Host - Wolfgang Freudenberger wrote: Is there any way to migrate cores from 4.10.2 to 7.5.0? I guess not, but perhaps someone has an idea. ^^ In a word, no.  A specific major version of Solr is only guaranteed to read indexes built and managed *completely* by the pre

Re: Nutch+Solr

Bineesh, I don't use Nutch, so don't know if this is relevant, but I've had similar-sounding failures in doing and restoring backups.  The solution for me was to deactivate authentication while the backup was being done, and then activate it again afterwards.  Then everything was restored correctl

Re: Migrate cores from 4.10.2 to 7.5.0

Hi Wolfgang, I would say that your safest bet is to start from 7.5 schema, adjust it to suite your needs and reindex (better than to try adjust your existing schema to 7.5). If all your fields are stored in current collection, you might be able to use DIH to reindex: http://www.od-bits.com/2018/

Re: CMS GC - Old Generation collection never finishes (due to GC Allocation Failure?)

On 10/3/2018 8:01 AM, yasoobhaider wrote: Master and slave config: ram: 120GB cores: 16 At any point there are between 10-20 slaves in the cluster, each serving ~2k requests per minute. Each slave houses two collections of approx 10G (~2.5mil docs) and 2G(10mil docs) when optimized. I am workin

Re: Solr Cloud in recovering state & down state for long

On Tue, Oct 2, 2018 at 11:46 PM Shawn Heisey wrote: > On 10/2/2018 8:55 PM, Ganesh Sethuraman wrote: > > We are using 2 node SolrCloud 7.2.1 cluster with external 3 node ZK > > ensemble in AWS. There are about 60 collections at any point in time. We > > have per JVM max heap of 8GB. > > Let's foc

Migrate cores from 4.10.2 to 7.5.0

Hi guys, Is there any way to migrate cores from 4.10.2 to 7.5.0? I guess not, but perhaps someone has an idea. ^^ -- Mit freundlichem Gruß / kind regards Wolfgang Freudenberger Pure Host IT-Services Münsterstr. 14 48341 Altenberge GERMANY Tel.: (+49) 25 71 - 99 20 170 Fax: (+49) 25 71 - 99 20

Nutch+Solr

Hello, We use Solr 7.3.1 and Nutch 1.15 We've placed the authentication for our solr cloud setup using the basic auth plugin ( login details -> solr/SolrRocks) For the nutch to index data to solr, below properties added to nutch-sitexml file solr.auth true Whether to enable HTTP basi

CMS GC - Old Generation collection never finishes (due to GC Allocation Failure?)

Hi I'm working with a Solr cluster with master-slave architecture. Master and slave config: ram: 120GB cores: 16 At any point there are between 10-20 slaves in the cluster, each serving ~2k requests per minute. Each slave houses two collections of approx 10G (~2.5mil docs) and 2G(10mil docs) whe

Re: Restoring and upgrading a standalone index to SolrCloud

On 10/3/2018 10:45 AM, Shawn Heisey wrote: Here's one way to do this: Oh, and when you delete the data directory, delete the tlog directory too.  Don't copy tlog from the non-cloud install.  Solr will re-create it as long as the directory gives it permission to do so. Thanks, Shawn

Re: Restoring and upgrading a standalone index to SolrCloud

On 10/3/2018 9:42 AM, Jack Schlederer wrote: I've successfully upgraded the Lucene 5 index to Lucene 6, and then to Lucene 7, Upgrading through two major versions is not guaranteed to work.  Upgrading from an index fully built by major version X-1 is supported, but if X-2 or earlier has EVER

[ANNOUNCE] Luke 7.5.0 released

Hi, Luke 7.5.0 was just released. For this release, we have two editions of Luke. (Sorry for the confusion.) * Luke 7.5.0 - Swing edition Swing edition can be downloaded from here: https://github.com/DmitryKey/luke/releases/tag/luke-swing-7.5.0 This version of Luke works with JDK 8/9/10/11. Use

Restoring and upgrading a standalone index to SolrCloud

Hello, We currently run Solr 5.4 as our production search backend. We run it in a master/slave replication architecture, and we're starting an upgrade to Solr 7.5 using a SolrCloud architecture. One of our collections is around 20GB and hosts about 200M documents, and it would take around 6 hours

Re: How to do rollback from solrclient using python

Hi Chetra, In addition to what Jason explained, rollbacks do not work in Solr Cloud. Thanks, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 3 Oct 2018, at 14:45, Jason Gerlowski wrote: > > Hi Chet

Re: Metrics API via Solrj

On 10/3/2018 6:17 AM, Jason Gerlowski wrote: NamedList respNL = response.getResponse(); NamedList metrics = (NamedList)respNL.get("metrics"); NamedList jvmMetrics = (NamedList) metrics.get("solr.jvm"); Long numClassesLoaded = (Long) jvmMetrics.get("classes.load

Re: Modify the log directory for dih

On 10/2/2018 10:49 PM, lala wrote: Shawn Heisey-2 wrote With a change to the log4j configuration file, you can direct all logs created by the DIH classes to a separate file, no code changes needed. Since I'm a newbee regarding log4j, Can you please give me an example about how to change this co

Re: How to do rollback from solrclient using python

Hi Chetra, The syntax that you're looking for is "/solr/someCoreName/update?rollback=true". But I'm afraid Rollback might not be quite what you think it is. You mentioned: "but it doesn't work, whenever there is a commit the request still updates on the server". Yes, that is the expected behavi

Re: Metrics API via Solrj

Hi Deniz, I don't think there are any classes that simplify accessing the metrics API like there are for other APIs (e.g. CollectionAdminRequest, CoreAdminRequest, ..). But sending metrics requests in SolrJ is still possible; it's just a little bit more complicated. Anytime you want to make an A

Re: Clarification about Solr Cloud and Shard

Hi Rekha, In addition to what Shawn explained, the answer to your last question is yes and no: You can split shards, but cannot change number of shards without reindexing. And you can add nodes but you should make sure adding nodes will result in well balanced cluster. You can address scalabilit

Re: checksum failed (hardware problem?)

Hello All -- As it would happen, we've seen this error on version 6.6.2 very recently. This is also on an AWS instance, like Simon's report. The drive doesn't show any sign of being unhealthy, either from cursory investigation. FWIW, this occurred during a collection backup. Erick, is there some

Re: Creating CJK bigram tokens with ClassicTokenizer

Hi, Shawn Thank you for replying me. > CJKBigramFilter shouldn't care what tokenizer you're using. It should > work with any tokenizer. What problem are you seeing that you're trying > to solve? What version of Solr, what configuration, and what does it do > that you're not expecting, and what

How to do rollback from solrclient using python

Hi Solr team, Current I am creating a python application that accesses to solr server. I have to handle updating document and need a rollback function. I want to send a rollback request whenever exception occurs. first I try sth like this from curl command : curl http://localhost:8983/solr/mysolr/u

RE: Opinions on index optimization...

There are a few bugs for which you require to merge the index, see SOLR-8807 and related bugs. https://issues.apache.org/jira/browse/SOLR-8807 -Original message- > From:Erick Erickson > Sent: Wednesday 3rd October 2018 4:50 > To: solr-user > Subject: Re: Opinions on index optimization.

Re: Solr 6.6 LanguageDetector

Here is my schema configuration: On Wed, Oct 3, 2018 at 10:50 AM Furkan KAMACI wrote: > Hi, > > I use Solr 6.6 and try to test automatic language detection. I've added > these configuration into my solrconfig.xml. > > > class="org.apache.solr.update.processor.LangD

Solr 6.6 LanguageDetector

Hi, I use Solr 6.6 and try to test automatic language detection. I've added these configuration into my solrconfig.xml. content en,tr language_code other true true

Metrics API via Solrj

Are there anyway to get the metrics via solrj ? all of the examples seem like using plain curl or http reqs with json response. I have found org.apache.solr.client.solrj.io.stream.metrics package, but couldnt figure out how to send the requests via solrj... could anyone help me to figure out how