Hello,
We are experiencing some performance issues on a simple SolrCloud cluster of 3
replicas (1 core) but what we found during our analysis seems a bit odd, so we
thought the community could have relevant ideas on this.
Load: between 30 and 40 queries per second, constant over time of analysi
Jourdan-Weil
mailto:gael.jourdan-w...@kelkoogroup.com>>
wrote:
Hello,
We are experiencing some performance issues on a simple SolrCloud cluster of 3
replicas (1 core) but what we found during our analysis seems a bit odd, so we
thought the community could have relevant ideas on this.
Load: b
archer
that has to read much of the index off disk when commits
happen. The "smoking gun" here would be if the spikes
correlate to your commits (soft or hard-with-opensearcher-true).
Best,
Erick
On Fri, Jan 11, 2019 at 1:23 AM Gael Jourdan-Weil
wrote:
>
> Interesting indeed, we did
on't get too lost in the page to
start, just hit the "http://localhost:8983/solr/admin/metrics"; endpoint
and look for "warmupTime", then refine on how to get _only_
the warmup stats ;).
Best,
Erick
On Mon, Jan 14, 2019 at 5:08 AM Gael Jourdan-Weil
wrote:
>
> I had a
to put
a profiler on the system when you see this kind of thing and
find out where the hotspots are, otherwise it's guesswork and
I'm out of ideas.
Best,
Erick
On Tue, Jan 15, 2019 at 12:06 AM Gael Jourdan-Weil
wrote:
>
> Hi Erick,
>
f you use "date math" to round to, say, a minute, if you run
Solr long enough you'll still fill up with useless fq clauses.
Best,
Erick
On Tue, Jan 15, 2019 at 9:33 AM Gael Jourdan-Weil
wrote:
>
> @Erick:
>
>
> We will try to lower the autowarm and run some tests
Quick update just in case someone comes on this thread someday: we did lower
the autowarm but it didn't have effect on the performance issues we are seeing.
We are still investigating...
Regards,
Gaël
De : Gael Jourdan-Weil
Envoyé : mardi 15 janvier 2019
Hello,
I come again to the community for some ideas regarding a performance issue we
are having.
We have a SolrCloud cluster of 3 servers.
Each server hosts 1 replica of 2 collections.
There is no sharding, every server hosts the whole collection.
Requests are evenly distributed by a Varnish sy
On Mon, Mar 4, 2019 at 3:18 PM Gael Jourdan-Weil <
gael.jourdan-w...@kelkoogroup.com> wrote:
> Hello,
>
> I come again to the community for some ideas regarding a performance issue
> we are having.
>
> We have a SolrCloud cluster of 3 servers.
> Each server hosts
solution, but would help isolate
the issue.
First, let’s see if the not node is always the Overseer...
Best,
Erick
> On Mar 4, 2019, at 6:51 AM, Gael Jourdan-Weil
> wrote:
>
> Hello Furkan,
>
> Yes the 3 servers have exact same configuration.
>
> Varnish load balanc
___
De : Erick Erickson
Envoyé : lundi 4 mars 2019 18:53
À : Gael Jourdan-Weil
Objet : Re: SolrCloud one server with high load
Yes, Overseer is different than leader.
There is one leader is _per shard_, and it’s job is to coordinate updates to
the index for that shard.
The Overseer coordinat
Hello,
We are trying to upgrade from Solr 6.6 to Solr 7.2.1 and we are using Solr
Cloud.
Doing some tests with 2 replicas, ZooKeeper doesn't know which one to elect as
a leader:
ERROR org.apache.solr.cloud.ZkController:getLeader:1206 - Error getting leader
from zk
org.apache.solr.common.Solr
erties host2:
#Written by CorePropertiesLocator
#Wed Apr 04 13:41:10 UTC 2018
name=fr_green
shard=shard1
dataDir=/opt/kookel/data/searchSolrNode/solrindex/fr_green
coreNodeName=core_node2
=> coreNodeName for host2 is correct
On 03/04/18 15:46, Gael Jourdan-Weil wrote:
Hello everyone,
We are running a SolrCloud cluster with ZooKeeper.
This SolrCloud cluster is down most of the time (backup environment) but the
ZooKeeper instances are always up so that we can easily update configuration.
This has been working fine for a long time with Solr 6.4.0 then 6.6.0, but
. At least on startup, we
should refresh the configuration from ZK without looking at local config
versions. Can you please open a Jira issue?
On Wed, May 23, 2018 at 5:35 PM, Gael Jourdan-Weil <
gael.jourdan-w...@kelkoogroup.com> wrote:
> Hello everyone,
>
> We are running a SolrClo
Hello,
If I understand well, you want to share a SolrClient for streaming code and
another piece of non streaming code?
I think you can have a look to the SolrClientCache class.
Instantiate a SolrClientCache once (as a Bean in your case I guess).
Then you can use it for both:
* getting a u
Hello,
I was wondering if we can expect significant disk usage reduction (index size)
if we move from fields defined as "docValues=true + stored=true" to
"docValues=true + stored=false" (with useDocValuesAsStored=true as default in
both cases)?
Considering the use case we are targeting is only
sorted and deduplicated. So input like “a” “z” “h” “a” will
return “a” “h” “z”.
Best,
Erick
> On Jul 15, 2020, at 1:35 PM, Gael Jourdan-Weil
> wrote:
>
> Hello,
>
> I was wondering if we can expect significant disk usage reduction (index
> size) if we move from fields
Hello,
I'm facing a situation where a transaction log file keeps growing and is never
deleted.
The setup is as follow:
- Solr 8.4.1
- SolrCloud with 2 nodes
- 1 collection, 1 shard
On one of the node I can see the tlog files having the expected behavior, that
is new tlog files being created an
en, the
answers to the above might give a clue where to look next.
Best,
Erick
> On Jul 22, 2020, at 3:39 PM, Gael Jourdan-Weil
> wrote:
>
> Hello,
>
> I'm facing a situation where a transaction log file keeps growing and is
> never deleted.
>
> The setup is
clue.
>
> Your solr log in the one with 20G tlogs should show commits, is there
> anything that points up?
>
> It’s also a bit weird that the numbers are so very different. While not
> lock-step, I’d expect that they were reasonably close. When you restart the
> server, do
> Note that for my previous e-mail you’d have to wait 15 minutes after you
> started indexing to see a new tlog and also wait until at least 1,000 new
> document after _that_ before the large tlog went away. I don't think that’s
> your issue though.
Indeed I did wait 15 minutes but not sure 1000
I think I've come down to the root cause of this mess in our case.
Everything is confirming that the TLOG state is "BUFFERING" rather than
"ACTIVE".
1/ This can be seen with the metrics API as well where we observe:
"TLOG.replay.remaining.bytes":48997506,
"TLOG.replay.remaining.logs":1,
"TLOG.sta
Hi,
Which version of Solr are you talking about?
You should have a log4j.xml or log4j2.xml file in Solr resources that you can
customize to your needs.
At least that's the way we use to write JSON logs. Maybe there are other
options that I don't know.
Gaël,
De : fidiv...@gmail.com
Envoyé : j
Hello,
I am trying to use a Streaming Expression to query only a subset of the shards
of a collection.
I expected to be able to use the "shards" parameter like on a regular query on
"/select" for instance but this appear to not work or I don't know how to do it.
Is this somehow a feature/restri
Thanks Joel.
I will try it in the future if I still need it (for now I went for another
solution that fits my needs).
Gaël
Hi,
We are trying to split a single shard into two but we are encountering some
issues we don't understand.
Our current setup:
- 1 collection "col"
- 1 shard "shard1"
- 2 nodes, each having the whole collection (SolrCloud)
- 1 core on each node "col_core"
What we would like to have is:
- 1 coll
8
À : solr-user@lucene.apache.org
Objet : Re: How to split a shard?
On 9/26/2019 8:50 AM, Gael Jourdan-Weil wrote:
> We are trying to split a single shard into two but we are encountering some
> issues we don't understand.
> A) Create a new core "col_core2", then run
t the name of our cores is
manually chosen with core.properties file and AFAIK It's never mentionned
anywhere that cores should have a particular name.
Regards,
Gaël
____
De : Gael Jourdan-Weil
Envoyé : jeudi 26 septembre 2019 17:57
À : solr-user@lucene.apache.or
Hello,
We are experiencing some performance issues on Solr that seems related to
requests querying multiple pages of results for the same keyword at the same
time.
For instance, querying 10 pages of results (with 50 or 100 results per page) in
the same second for a given keyword, and doing that
There are two ways of making this less work for Solr, both depend on returning
only docValues="true” fields.
1> return only docValues fields. See useDocValuesAsStored.
2> use the /export handler.
Best,
Erick
> On Jan 13, 2020, at 5:31 AM, Gael Jourdan-Weil
> wrote:
>
>
Hello,
If you are talking about "physical memory" as the bar displayed in Solr UI,
that is the actual RAM your host have.
If you need more, you need more RAM, it's not related to Solr.
Gaël
De : rhys J
Envoyé : lundi 13 janvier 2020 20:46
À : solr-user@lucene.ap
Ok I understand better.
Solr does not "read" the 1 to 900 docs to retrieve 901 to 1000 but it still
needs to compute some stuff (docset intersection or something like that,
right?) and sort, which is costly, and then "read" the docs.
> Are those 10 requests happening simultaneously, or consecuti
Indeed, with a max of 1K doc to be manipulated, I don't expect issues.
We are looking at other avenues to understand our issues.
Regards,
Gaël
s of K (or, I’ve
> seen millions) it’s _very_ noticeable.
>
> With the example you’ve talked about, I doubt this is really a problem.
>
> FWIW,
> Erick
>
> > On Jan 14, 2020, at 1:40 PM, Gael Jourdan-Weil
> > wrote:
> >
> > Ok I understand better.
&
Hello,
I was wondering if someone ever had the need to send compressed (gzip) update
requests (adding/deleting documents), especially using SolrJ.
Somehow I expected it to be done by default, but didn't find any documentation
about it and when looking at the code it seems there is no option to
Answering to myself on this one.
Solr uses Jetty 9.x which does not support compressed requests by itself
meaning, the application behind Jetty (that is Solr) has to decompress by
itself which is not the case for now.
Thus even without using SolrJ, sending XML compressed in GZIP to Solr (with
c
atches to do so by default to solr) but I
don't know about the handling for solrj.
IME compression helps a little, sometimes a lot, and never hurts.
Even the admin interface benefits a lot from regular old http gzip
On Thu, Jan 7, 2021 at 8:03 AM Gael Jourdan-Weil
wrote:
>
> Answe
Hello Steven,
I believe what you are looking for cannot be accessed using SolrJ (I didn't
really check though).
But you can easily access it either via the Collections APIs and/or the Metrics
API depending on what you need exactly.
See https://lucene.apache.org/solr/guide/8_4/cluster-node-manag
Hi Walter,
>From a strict feasibility point of view, 2 totally separate Solr instances can
>perfectly run on the same server: this would be 2 distinct JVM process, I
>don't foresee any issue.
Nothing specific to look for except different ports, directories...
Do you mean using the same indexes?
Hello,
You can just use "cat" or "tail", even though the tlog is not a text file, its
content can mostly be read using these commands.
You will have one document per line and should be able to see the fields
content.
I don't know is there is a Solr command which would give better display though
41 matches
Mail list logo
