Hi Mark, Thanks for confirming Dwane's advice from your own experience. I
will shift to a streaming expressions implementation.
Best
Goutham
On Fri, Sep 25, 2020 at 7:03 PM Mark H. Wood wrote:
> On Fri, Sep 25, 2020 at 11:49:22AM +0530, Goutham Tholpadi wrote:
> > I have around 30M documents in
Thanks a ton, Dwane. I went through the article and the documentation link.
This corresponds exactly to my use case.
Best
Goutham
On Fri, Sep 25, 2020 at 2:59 PM Dwane Hall wrote:
> Goutham I suggest you read Hossman's excellent article on deep paging and
> why returning rows=(some large number
On Fri, Sep 25, 2020 at 11:49:22AM +0530, Goutham Tholpadi wrote:
> I have around 30M documents in Solr, and I am doing repeated *:* queries
> with rows=1, and changing start to 0, 1, 2, and so on, in a
> loop in my script (using pysolr).
>
> At the start of the iteration, the calls to
Goutham I suggest you read Hossman's excellent article on deep paging and why
returning rows=(some large number) is a bad idea. It provides an thorough
overview of the concept and will explain it better than I ever could
(https://lucidworks.com/post/coming-soon-to-solr-efficient-cursor-based-ite
Thanks Guys.
i will try two level document routing in case of file_collection.
i really don't understand why index size is high for file_collection as
same file is available in main_collection.
(each file indexed as one document with all commands in main collection
and same file is indexed as n
If you can find/know which fields (or combination) in your document divides
/ groups the data together would be the fields for custom routing. Solr
supports up to two level.
E.g. if you have field with say documentType or country or etc. would
help. See the document routing at
https://cwiki.apac
Usually I just let the compositeId do its thing and only go for custom
routing when the default proves inadequate.
Note: your 480M documents may very well be too many for three shards!
You really have to test
Erick
On Mon, Mar 14, 2016 at 10:04 AM, Anil wrote:
> Hi Erick,
> In b/w, Do you
Hi Erick,
In b/w, Do you recommend any effective shard distribution method ?
Regards,
Anil
On 14 March 2016 at 22:30, Erick Erickson wrote:
> Try shards.info=true, but pinging the shard directly is the most certain.
>
>
> Best,
> Erick
>
> On Mon, Mar 14, 2016 at 9:48 AM, Anil wrote:
> > HI Er
thanks Eric. i will try that. Some how i am not able to run a query on the
shard directly because of kerberos. i even tried curl --negotiate.
Regards,
Anil
On 14 March 2016 at 22:30, Erick Erickson wrote:
> Try shards.info=true, but pinging the shard directly is the most certain.
>
>
> Best,
>
Try shards.info=true, but pinging the shard directly is the most certain.
Best,
Erick
On Mon, Mar 14, 2016 at 9:48 AM, Anil wrote:
> HI Erik,
>
> we have used document routing to balance the shards load and for
> expand/collapse. it is mainly used for main_collection which holds one to
> many r
HI Erik,
we have used document routing to balance the shards load and for
expand/collapse. it is mainly used for main_collection which holds one to
many relationship records. In file_collection, it is only for load
distribution.
25GB for entire solr service. each machine will act as shard for som
Hi Shusheel,
we have enabled kerberos. so solr is accessed using Hue only. i will check
if I can get the similar information using Hue. Thanks.
Regards,
Anil
On 14 March 2016 at 19:34, Susheel Kumar wrote:
> Hello Anil,
>
> Can you go to Solr Admin Panel -> Dashboard and share all 4 memory
> p
bq: The slowness is happening for file_collection. though it has 3 shards,
documents are available in 2 shards. shard1 - 150M docs and shard2 has 330M
docs , shard3 is empty.
Well, this collection terribly balanced. Putting 330M docs on a single shard is
pushing the limits, the only time I've seen
For each of the solr machines/shards you have. Thanks.
On Mon, Mar 14, 2016 at 10:04 AM, Susheel Kumar
wrote:
> Hello Anil,
>
> Can you go to Solr Admin Panel -> Dashboard and share all 4 memory
> parameters under System / share the snapshot. ?
>
> Thanks,
> Susheel
>
> On Mon, Mar 14, 2016 at
Hello Anil,
Can you go to Solr Admin Panel -> Dashboard and share all 4 memory
parameters under System / share the snapshot. ?
Thanks,
Susheel
On Mon, Mar 14, 2016 at 5:36 AM, Anil wrote:
> HI Toke and Jack,
>
> Please find the details below.
>
> * How large are your 3 shards in bytes? (total
HI Toke and Jack,
Please find the details below.
* How large are your 3 shards in bytes? (total index across replicas)
-- *146G. i am using CDH (cloudera), not sure how to check the
index size of each collection on each shard*
* What storage system do you use (local SSD, local spinning
HI Shawn, Jack and Eric,
Thank you very much.
Regards,
Anil
On 14 March 2016 at 02:55, Shawn Heisey wrote:
> On 3/13/2016 9:36 AM, Jack Krupansky wrote:
> > (We should have a wiki/doc page for the "usual list of suspects" when
> > queries are/appear slow, rather than need to repeat the same m
On 3/13/2016 9:36 AM, Jack Krupansky wrote:
> (We should have a wiki/doc page for the "usual list of suspects" when
> queries are/appear slow, rather than need to repeat the same mantra(s) for
> every inquiry on this topic.)
There's this page, with the disclaimer that I wrote almost all of it:
ht
Yeah, there's some good material there, but probably still too inaccessible
for the average "help, my queries are slow" inquiry we get so frequently on
this list.
Another useful page is:
https://wiki.apache.org/solr/SolrPerformanceProblems
-- Jack Krupansky
On Sun, Mar 13, 2016 at 2:58 PM, Eric
Jack:
https://wiki.apache.org/solr/SolrPerformanceFactors
and
http://wiki.apache.org/lucene-java/ImproveSearchingSpeed
are already there, we can add to them
Best,
Erick
On Sun, Mar 13, 2016 at 9:18 AM, Anil wrote:
> Thanks Toke and Jack.
>
> Jack,
>
> Yes. it is 480 million :)
>
> I will sh
Thanks Toke and Jack.
Jack,
Yes. it is 480 million :)
I will share the additional details soon. thanks.
Regards,
Anil
On 13 March 2016 at 21:06, Jack Krupansky wrote:
> (We should have a wiki/doc page for the "usual list of suspects" when
> queries are/appear slow, rather than need to r
(We should have a wiki/doc page for the "usual list of suspects" when
queries are/appear slow, rather than need to repeat the same mantra(s) for
every inquiry on this topic.)
-- Jack Krupansky
On Sun, Mar 13, 2016 at 11:29 AM, Toke Eskildsen
wrote:
> Anil wrote:
> > i have indexed a data (com
Anil wrote:
> i have indexed a data (commands from files) with 10 fields and 3 of them is
> text fields. collection is created with 3 shards and 2 replicas. I have
> used document routing as well.
> Currently collection holds 47,80,01,405 records.
...480 million, right? Funny digit grouping in I
Hi Abhijeet,
On Mon, Aug 22, 2011 at 3:09 PM, abhijit bashetti wrote:
>
> 1. Can I update a specific field while re-indexing?
>
Solr doesn't support updating specific fields. You must always create a
complete document with values for all fields while indexing. If you keep the
same value for the
> Hi,
> Suppose i have a content field of
> type text.
> an example on content field is as shown below:
> "After frustrated waiting period to get my credit card from
> the ICICI Bank,
> today I decided to write them a online petition stating my
> problem... Below
> is the unedited version of l
Hi,
Sorry i forgot to mention that comment field is a text field.
Regards,
Raakhi
On Thu, Nov 12, 2009 at 8:05 PM, Grant Ingersoll wrote:
>
> On Nov 12, 2009, at 8:55 AM, Rakhi Khatwani wrote:
>
> > Hi,
> > I am using solr 1.3 and i hv inserted some data in my comment
> > field.
> > for
On Nov 12, 2009, at 8:55 AM, Rakhi Khatwani wrote:
> Hi,
> I am using solr 1.3 and i hv inserted some data in my comment
> field.
> for example:
>
> for document1:
>
> The iPhone 3GS finally adds common cell phone features like multimedia
> messaging, video recording, and voice dialing.
27 matches
Mail list logo
