0.0
Thanks,
Wei
Thanks Jack and Shawn. I checked these Jira tickets, but I am not sure if
the slowness of MatchAllDocsQuery is also caused by the removal of
fieldcache. Can someone please explain a little bit?
Thanks,
Wei
On Fri, Nov 6, 2015 at 7:15 AM, Shawn Heisey wrote:
> On 11/5/2015 10:25 PM, J
On Fri, Nov 6, 2015 at 3:01 PM, wei wrote:
>
> > Thanks Jack and Shawn. I checked these Jira tickets, but I am not sure if
> > the slowness of MatchAllDocsQuery is also caused by the removal of
> > fieldcache. Can someone please explain a little bit?
> >
> > Thank
the explain part are different in solr4.7 and solr 5.3.1. In solr 4.7,
there is only one line
1.0 = (MATCH) MatchAllDocsQuery, product of:
1.0 = queryNorm
1.0 = (MATCH) MatchAllDocsQuery, product of:
1.0 = queryNorm
in solr 5.3.1, there is actually a boost, and
query keyword) and is slower in
the other test(without query keyword).
Thanks,
Wei
On Fri, Nov 6, 2015 at 1:41 PM, Shawn Heisey wrote:
> On 11/6/2015 1:01 PM, wei wrote:
> > Thanks Jack and Shawn. I checked these Jira tickets, but I am not sure if
> > the slowness of MatchAllDoc
Good point! I tried that, on solr5 the query time is around 100-110ms, and
on solr4 it is around 60-63ms(very consistent). Solr5 is slower.
Thanks,
Wei
On Fri, Nov 6, 2015 at 6:46 PM, Yonik Seeley wrote:
> On Fri, Nov 6, 2015 at 9:30 PM, wei wrote:
> > in solr 5.3.1, there is actuall
Thanks Yonik.
A JIRA bug is opened:
https://issues.apache.org/jira/browse/SOLR-8251
Wei
On Fri, Nov 6, 2015 at 7:10 PM, Yonik Seeley wrote:
> On Fri, Nov 6, 2015 at 9:56 PM, wei wrote:
> > Good point! I tried that, on solr5 the query time is around 100-110ms,
> and
> > on
I noticed the solr query latency spike on slave node when replicating index
from master. Especially when master just finished optimization, the slave
node will copy the whole index, and the latency is really bad.
Is there some way to fix it?
Thanks,
Wei
index, if both indexes(old and new) need to
be in the memory. The memory is not big enough to hold both(old index+new
index+jvm).
Thanks,
Wei
On Fri, Apr 3, 2015 at 3:35 PM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:
> In Solr 5.0 you can throttle the replication and limit
Hi,
In our set up there are two solr clouds:
Cloud A: production cloud serves both writes and reads
Cloud B: back up cloud serves only writes
Cloud A and B have the same shard configuration.
Write requests are sent to both cloud A and B. In certain circumstances
when Cloud A's update lags be
o is _only_ have one replica for each shard
> on Cloud A active and fetch to _that_ replica. I'd also delete the
> data dir on all the other replicas for the shard on Cloud A. Then as
> you bring the additional replicas up they'll do a full synch from the
> leader.
>
> FWIW,
e times the lag is over 10
minutes. My understanding is that as update requests goes to leader first,
the timer on the leaders would start earlier than the replicas. Am I
missing something here?
Thanks,
Wei
out could
help to improve query performance and what are the consequences.
Thanks,
Wei
ffersizeMB? Is there any concern to
increase rambuffersizeMB to a large value?
3. Can transaction logs be disabled in solr cloud? Will
functionalities(replication, peer sync) break without transaction logs?
Thanks,
Wei
On Fri, Sep 29, 2017 at 8:33 AM, Erick Erickson
wrote:
> More than you
ooks the unauthorized error is because core-admin-edit requires admin
access. How can I config authentication credentials for solr cloud's
internal request? Appreciate your help!
Thanks,
Wei
handler
stats is much higher: handler reports ~100k requests but in the access log
there are only 5k update requests. What could be the possible cause?
Thanks,
Wei
Thanks Amrit. Can you explain a bit more what kind of requests won't be
logged? Is that something configurable for solr?
Best,
Wei
On Thu, Nov 9, 2017 at 3:12 AM, Amrit Sarkar wrote:
> Wei,
>
> Are the requests coming through to collection has multiple shards and
> replic
Hi,
I noticed that lucene has introduced a new two-phase iteration API since 5,
but could not get a good understanding of how it works. Are there any
detail documentation or examples? Does the two-phase iteration result in
better query performance? Appreciate your help.
Thanks,
Wei
Hello Mikhail,
Thank you so much for the info. Trying to digest it first.Can you
elaborate more on what has changed? Any pointer is greatly appreciated.
Regards,
Wei
On Mon, Jan 1, 2018 at 10:04 AM, Mikhail Khludnev wrote:
> Hello, Wei.
> Some first details have been discusse
per host, and have multiple cores(shards) in the
same solr instance.
Which would be better performance wise? For the first option I think JVM
size for each solr instance can be smaller, but deployment is more
complicated? Are there any differences for cpu utilization?
Thanks,
Wei
will be
able to get better CPU utilization on multi-core server?
Thanks,
Wei
On Sun, Aug 26, 2018 at 4:37 AM Shawn Heisey wrote:
> On 8/26/2018 12:00 AM, Wei wrote:
> > I have a question about the deployment configuration in solr cloud. When
> > we need to increase the number of
Thanks Bernd. Do you have preferLocalShards=true in both cases? Do you
notice CPU/memory utilization difference between the two deployments? How
many servers did you use in total? I am curious what's the bottleneck for
the one instance and 3 cores configuration.
Thanks,
Wei
On Mon, A
as you want.
>
> The node placement rules are primarily intended for automated or very large
> setups. Manually placing replicas is simpler for limited numbers.
>
> Best,
> Erick
> On Sun, Aug 26, 2018 at 8:10 PM Wei wrote:
> >
> > Thanks Shawn. When using multiple Sol
for defining the physical host?
Thanks,
Wei
#x27;? I cannot find more relevant
documentation on how to configure and customize 'snitch'.
Thanks,
Wei
On Sun, Sep 2, 2018 at 9:30 PM Erick Erickson
wrote:
> You need to provide a "snitch" and define a rule appropriately. This
> is a variant of "rack awareness"
ach server
will only host 2 of the 5 shards( 2 JVMs per server, each JVM have one
replica from different shards). Is it useful to set preferLocalShards=true
in this case?
Thanks,
Wei
maxMergeAtOnceExplicit because the default 30
could be too low:
100
But it doesn't seem to help. Any suggestions?
Thanks,
Wei
Thanks everyone! I checked the system metrics during the optimization
process. CPU usage is quite low, there is no I/O wait, and memory usage is
not much different from before the docValues change. So I wonder what
could be the bottleneck.
Thanks,
Wei
On Sat, Nov 3, 2018 at 1:38 PM Erick
Solr
retrieve id from docValues instead of stored field? if fl= id, title,
score, both id and title are single value field:
Do I need to have all fields stored="false" docValues="true" to make solr
retrieve from docValues only? I am using Solr 6.6.
Thanks,
Wei
the uniqueKey field need to be
always docValues? Since it is used in the first phase of distributed
search.
Thanks,
Wei
On Tue, Nov 6, 2018 at 8:30 AM Erick Erickson
wrote:
> 2. "it depends". Solr will try to do the most efficient thing
> possible. If _all_ the fields are
I see there is also a docValuesFormat option, what's the default for this
setting? Performance wise is it good to set docValuesFormat="Memory" ?
Best,
Wei
On Tue, Nov 6, 2018 at 11:55 AM Erick Erickson
wrote:
> Yes, "the most efficient possible" is associated wit
Also I notice this issue is still open:
https://issues.apache.org/jira/browse/SOLR-10816
Does that mean we still need to have stored=true for uniqueKey?
On Tue, Nov 6, 2018 at 2:14 PM Wei wrote:
> I see there is also a docValuesFormat option, what's the default for this
> setting?
there
are more than 1 segments. Is the optimize command async? What is the best
approach to validate that optimize is truly completed?
Thanks,
Wei
dvance for you input.
Thanks,
Wei
Thanks Thomas. You mentioned "Also there is no need for the
FlattenGraphFilter", that's quite interesting because the Solr
documentation says it's mandatory for indexing:
https://lucene.apache.org/solr/guide/7_6/filter-descriptions.html. Is there
any more explanation for this
bump..
On Mon, Jan 7, 2019 at 11:53 AM Wei wrote:
> Thanks Thomas. You mentioned "Also there is no need for the
> FlattenGraphFilter", that's quite interesting because the Solr
> documentation says it's mandatory for indexing:
> https://lucene.apache.org/solr/gu
finished optimization to a single segment, however all the leader replicas
still have multiple segments. Previously inn the all NRT replica cloud, I
see optimization is triggered on all nodes. Is the optimization process
different with Tlog/Pull replicas?
Best,
Wei
10
20480
But in the end I see multiple segments much smaller than the 20GB limit.
In 7.6 is it required to explicitly set the number of segments to 1? e.g
shall I use
/update?optimize=true&waitSearcher=false&maxSegments=1
Best,
Wei
On Fri, Mar 8, 2019 at 12:29 PM Erick Erickson
A side question, for heavy bulk indexing, what's the recommended setting
for auto commit? As there is no query needed during the bulking indexing
process, I have auto soft commit disabled. Is there any side effect if I
also disable auto commit?
On Sun, Mar 10, 2019 at 10:22 PM Wei
segment just to add a 1G so
> having multiple segments < 20G is perfectly normal.
>
> Best,
> Erick
>
> > On Mar 10, 2019, at 10:36 PM, Wei wrote:
> >
> > A side question, for heavy bulk indexing, what's the recommended setting
> > for auto commit? As th
completely separate query and updates, I think that I might need to have
the load-balancer set up to include only the PULL replicas. Is there any
other option?
Thanks,
Wei
rows=500 is much smaller. Am I missing something here?
Thanks,
Wei
ested yet. Please let us know if you catch anything.
Thanks,
Wei
On Mon, Jan 22, 2018 at 3:15 PM, Chris Hostetter
wrote:
>
> : Inside convertLuceneDocToSolrDoc():
> :
> :
> : https://github.com/apache/lucene-solr/blob/
> df874432b9a17b547acb24a01d3491
> : 839e6a6b69/solr
t;UIF",
- appliedMethod: "FC",
- inputDocSetSize: 33487,
- field: "color",
- numBuckets: 238
}
]
}
I also see that in standalone mode fieldValueCache is used with UIF
applied, but in cloud mode fieldValueCache is always empty. Are there any
other parameters I need to apply UIF faceting in solr cloud?
Thanks,
Wei
- "domainSize":6951
},
8. {
- "processor":"FacetFieldProcessorByArrayUIF",
- "elapse":104,
- "field":"color",
- "limit":1000,
- "numBuckets":19,
- &
o wonder the
method chosen become FC. So one myth solved; but the new myth is why the
facet.mincount is override to 0 in solr req?
Cheers,
Wei
On Thu, Feb 1, 2018 at 2:01 AM, Alessandro Benedetti
wrote:
> " Looks like when using the json facet api,
> SimpleFacets i
Adding facet.distrib.mco=true did the trick. Thanks Toke and Alessandro!
Cheers,
Wei
On Thu, Feb 8, 2018 at 1:23 AM, Toke Eskildsen wrote:
> On Fri, 2018-02-02 at 17:40 -0800, Wei wrote:
> > I tried to debug a bit and see that when executing on a cloud solr
> > server,
Thanks all! It's really great learning. A bit off the topic, after I
enabled facet.method = uif in solr cloud, the faceting performance is
actually much worse than the original fc( ~1000 ms with uif vs ~200 ms
with fc). My cloud has 8 shards with 6 replicas in each shard. I do see
that fieldV
and uif will have to pay the upfront cost again after each commit?
On Wed, Feb 14, 2018 at 11:51 AM, Yonik Seeley wrote:
> On Wed, Feb 14, 2018 at 2:28 PM, Wei wrote:
> > Thanks all! It's really great learning. A bit off the topic, after I
> > enabled facet.method = uif
score from stand alone and cloud are different. How could this happen?
With the same data, software version and query, should solr score be
exactly same regardless of cloud mode or not?
Thanks,
Wei
this
case?
Thanks,
Wei
On Thu, Jun 7, 2018 at 12:18 PM, Erick Erickson
wrote:
> Short form:
>
> As docs are updated, they're marked as deleted until the segment is
> merged. This affects things like term frequency and doc frequency
> which in turn influences the score.
>
>
B C
A D
B D
A B C
A B D
A C D
B C D
A B C D
Thanks,
Wei
nd then *:* -(V:(A AND B) AND numVals:2) -(V:(A OR
> B) AND numVals:1)
>
>
> On Tue, Jun 19, 2018 at 9:20 AM Wei wrote:
>
> > Hi,
> >
> > I have a multi-value field, and there is a limited set of values for the
> > field: A, B, C, D.
> > Is there a
ult
fq= my_text_field:Jurassic park the movie returns
thousands of results
Which one is the correct syntax? I am confused why the first query doesn't
have any match at all. I also thought 2 and 3 are the same, but turns out
quite different.
Thanks,
Wei
default field is a catch-all field)
>
> Sorry for typos I'm using my mobile
>
> Andrea
>
> Il mer 11 lug 2018, 17:54 Wei ha scritto:
>
> > Hi,
> >
> > I am running filter query on a field of text_general type and see
> > completely differe
btw, is there any difference if the fq field is a string field vs test
field?
On Wed, Jul 11, 2018 at 11:59 AM, Wei wrote:
> Thanks Erick and Andrea! If my default operator is OR, fq=
> my_text_field:(Jurassic park the movie) is equivalent to
> my_text_field:(Jurassic
> OR pa
Expansion and Document
collection" . Does that mean Solr will not abort the request if
timeAllowed is exceeded during the scoring process? What are the components
(query, facet, stats, debug etc) this metric is effectively used?
Thanks,
Wei
Thanks Mikhail! Is traditional facet subject to timeAllowed?
On Mon, Aug 6, 2018 at 3:46 AM, Mikhail Khludnev wrote:
> One note: enum facets might be stopped by timeAllowed.
>
> On Mon, Aug 6, 2018 at 1:45 PM Mikhail Khludnev wrote:
>
> > Hello, Wei.
> >
> > "
idle. There
is no change in shard handler configuration:
3
3
500
What could cause the unbalanced internal distributed request?
Thanks in advance.
Wei
:TLOG
Nothing seems to cause the strange behavior. Any suggestions how to
debug this?
-Wei
On Mon, Apr 27, 2020 at 5:42 PM Erick Erickson
wrote:
> Wei:
>
> How are you measuring utilization here? The number of incoming requests or
> CPU?
>
> The leader for each shard are certai
terms of filtering, sorting or faceting, how would query
performance compare between the two?
Thanks,
Wei
for shard requests are the first node in each shard
returned from the CLUSTERSTATUS api. Seems something wrong with shuffling
equally compared nodes when shards.preference is set. Will report back if
I find more.
On Mon, Apr 27, 2020 at 5:59 PM Wei wrote:
> Hi Eric,
>
> I am meas
l Gibney
wrote:
> FYI: https://issues.apache.org/jira/browse/SOLR-14471
> Wei, assuming you have only TLOG replicas, your "last place" matches
> (to which the random fallback ordering would not be applied -- see
> above issue) would be the same as the "first place" matches selec
Hi Phill,
What is the RAM config you are referring to, JVM size? How is that related
to the load balancing, if each node has the same configuration?
Thanks,
Wei
On Mon, May 18, 2020 at 3:07 PM Phill Campbell
wrote:
> In my previous report I was configured to use as much RAM as possi
Hi Michael,
I also verified the patch in SOLR-14471 with 8.4.1 and it fixed the issue
with shards.preference=replica.location:local,replica.type:TLOG in my
setting. Thanks!
Wei
On Thu, May 21, 2020 at 12:09 PM Phill Campbell
wrote:
> Yes, JVM heap settings.
>
> > On May 19, 2020,
) does
not work. Is it possible to stop solr from putting facet.query into filter
cache?
Thanks,
Wei
inter is
appreciated.
Best,
Wei
Thanks Dominique. I'll start with the -XX:+UseNUMA option.
Best,
Wei
On Fri, Sep 25, 2020 at 7:04 AM Dominique Bejean
wrote:
> Hi,
>
> This would be a Java VM option, not something Solr itself can know about.
> Take a look at this article in comments. May be it wil
tml,
seems Java 14 is not officially supported for Solr 8.
Best,
Wei
On Fri, Sep 25, 2020 at 5:50 PM Shawn Heisey wrote:
> On 9/23/2020 7:42 PM, Wei wrote:
> > Recently we deployed solr 8.4.1 on a batch of new servers with 2 NUMAs. I
> > noticed that query latency almost d
My understanding is that current means whether there is data pending to be
committed.
Best,
Wei
On Sat, Sep 26, 2020 at 5:09 PM Kayak28 wrote:
> Hello, Solr community:
>
>
>
> I would like to ask a question about the current icon on the core-overview
>
> under stat
Thanks Shawn. Looks like Java 11 is the way to go with -XX:+UseNUMA. Do you
see any backward compatibility issue for Solr 8 with Java 11? Can we run
Solr 8 built with JDK 8 in Java 11 JRE, or need to rebuild solr with Java
11 JDK?
Best,
Wei
On Sat, Sep 26, 2020 at 6:44 PM Shawn Heisey wrote
-XX:G1MaxNewSizePercent=20
-XX:MaxGCPauseMillis=150
-XX:+DisableExplicitGC
-XX:+DoEscapeAnalysis
-XX:+ParallelRefProcEnabled
-XX:+UnlockDiagnosticVMOptions
-XX:+UnlockExperimentalVMOptions
Compared to previous Java 8 + CMS on 2 NUMA servers, P99 latency has
improved over 20%.
Thanks,
Wei
On Mon
, docValues=false
2) indexed=true, stored=false, docValues=true
3) indexed=false, stored=false, docValues=true
What would be the performance implications for these options?
Best,
Wei
Thanks Erick. As indexed is not necessary, and docValues is more efficient
than stored fields for function queries, so we shall go with the
following:
3) indexed=false, stored=false, docValues=true.
Is my understanding correct?
Best,
Wei
On Wed, Nov 4, 2020 at 5:24 AM Erick Erickson
And in the case of both stored=true and docValues=true, Solr 8.x shall be
choosing the optimal approach by itself?
On Wed, Nov 4, 2020 at 9:15 AM Wei wrote:
> Thanks Erick. As indexed is not necessary, and docValues is more
> efficient than stored fields for function queries, so we sh
ream bar” and “vanilla ice cream” , but does not match
for “ice cold cream”.
The results seem neither exact match nor phrase match. What's the expected
behavior for fq on text fields? I have tried to look into the solr docs
but there is no clear explanation.
Thanks,
Wei
e the values for relate parameters such as
ps?
Thanks,
Wei
On Mon, Jun 24, 2019 at 4:51 PM Shawn Heisey wrote:
> On 6/24/2019 5:37 PM, Wei wrote:
> > stored="true"/>
>
> I'm assuming that the asterisks here are for emphasis, that they are not
> actually pre
Thanks Erick for the clarification. How does the ps work for fq? I
configured ps=4 for q, it doesn't apply to fq though. For phrase queries in
fq seems ps=0 is used. Is there a way to config it for fq also?
Best,
Wei
On Tue, Jun 25, 2019 at 9:51 AM Erick Erickson
wrote:
> q a
operates on all values of the
field? Given color S in query, how to calculate the similarities between S
and C1/C2/C3 and find which one is the closest?
I checked https://lucene.apache.org/solr/guide/6_6/function-queries.html but
didn't see an example.
Thanks,
Wei
Any suggestion?
On Thu, Jul 11, 2019 at 3:03 PM Wei wrote:
> Hi,
>
> I have a question regarding function query that operates on multi-value
> fields. For the following field:
>
> multivalued="true"/>
>
> Each value is a hex string representation of
We are using solr 7.6.2 with a 10
shard cloud set up.
Is there a way to block certain solr queries based on url pattern? i.e.
ignore the stats.calcdistinct request in this case.
Thanks,
Wei
Hi Mikhail,
Yes I have the timeAllowed parameter configured, still is this case it
doesn't seem to prevent the stats request from blocking other normal
queries. Is it possible to drop the request before solr executes it? maybe
at the jetty request filter?
Thanks,
Wei
On Mon, Oct 7, 2019
On Wed, Oct 9, 2019 at 9:59 AM Wei wrote:
> Thanks all. I debugged a bit and see timeAllowed does not limit stats
> call. Also I think it would be useful for solr to support a white list or
> black list of operations as Toke suggested. Will create jira for it.
> Currently seems the
requests to the cloud are blocked? Does leader need
to wait for response from each replica to inform client that update is
successful?
Best,
Wei
g for the time
out?
Also the bad tlog replica is not reachable at the time, so we did a
DELETEREPLICA command with collections API to remove it from the cloud.
Thanks,
Wei
On Tue, Nov 19, 2019 at 5:52 AM Erick Erickson
wrote:
> How long are updates blocked and how did the tlog replica on
Hi,
I see this lucene optimization to disable hit counts for better query
performance:
https://issues.apache.org/jira/browse/LUCENE-8060
Is the feature available in Solr 8.3?
Thanks,
Wei
Thanks! Looking forward to have this feature in Solr.
On Wed, Nov 20, 2019 at 5:30 PM Tomás Fernández Löbbe
wrote:
> Not yet:
> https://issues.apache.org/jira/browse/SOLR-13289
>
> On Wed, Nov 20, 2019 at 4:57 PM Wei wrote:
>
> > Hi,
> >
> > I see this lucene o
wondering if
the commits could be caused by the leader initialed recovery process. Will
the Tlog leader do extra commits for the replica to sync up in recovery
process?
Best,
Wei
On Tue, Nov 19, 2019 at 1:22 PM Wei wrote:
> Hi Erick,
>
> I observed that the update request rate dropped fr
til
https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/java/org/noggit/JSONUtil.java
but seems it is not able to convert parts of the query response such as
facet. Are there any other options available?
Thanks,
Wei
on on this. Any
pointer is greatly appreciated.
Best,
Wei
uce fewer counts.
>
> On Thu, Jan 23, 2020 at 2:11 AM Wei wrote:
>
> > Hi,
> >
> > I am excited to see Lucene 8 introduced BlockMax WAND as a major speed
> > improvement https://issues.apache.org/jira/browse/LUCENE-8135. My
> > question
> > is, how does it
I think it just means they won't officially support deploying war to tomcat
or other container. make sense to me if I was in charge of solr, I would
just support jetty, predictable with a single configuration. I wouldn't
want to spent countless hrs supporting various configurations. Instead use
Hi, guys:
I met two questions about solr and lucene, wish people to help out.
use payload query but can NOT with numerical field type. for example:
I implemented my own requesthandler, refer to
http://hnagtech.wordpress.com/2013/04/19/using-payloads-with-solr-4-x/
Hi, guys:
I met two questions about solr and lucene, wish people to help out.
use payload query but can NOT with numerical field type. for example:
I implemented my own requesthandler, refer to
http://hnagtech.wordpress.com/2013/04/19/using-payloads-with-solr-4-x/
We are migrating from Solr 3.5 to Solr 4.2.
After some performance testing, we found 4.2's memory usage is a lot higher
than 3.5. Our 12GM max heap process used to handle the test pretty well with
3.5. while, with 4.2, the same test runs into serious GC half way (20
minutes) into the test.
Anyone
No, exactly the same JVM of Java6
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-4-memory-usage-increase-tp4064066p4064108.html
Sent from the Solr - User mailing list archive at Nabble.com.
Here is the JVM info:
$ java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-4-memory-usage-increase-tp4064066p4064271.h
We have master/slave setup. We disabled autocommits/autosoftcommits. So the
slave only replicates from master and serve query. Master does all the
indexing and commit every 5 minutes. Slave polls master every 2.5 minutes
and does replication.
Both tests with Solr 3.5 and 4.2 was run with the same
Hi All,
I am a new user of Solr.
We are now trying to enable searching on Digg dataset.
It has story_id as the primary key and comment_id are the comment id
which commented story_id, so story_id and comment_id is one-to-many
relationship.
These comment_ids can be replied by some repliers, so
1 - 100 of 151 matches
Mail list logo
