Solr JSON facet range out of memory exception
Dear all Solr users/developeres, Hi, I am going to use Solr JSON facet range on a date filed which is stored as long milis. Unfortunately I got java heap space exception no matter how much memory assigned to Solr Java heap! I already test that with 2g heap space for Solr core with 50k documents!! Is there any performance concern regarding the JSON facet range on long field? Sincerely, -- A.Nazemian
Solr document duplicated during pagination
HI, i am loading solr recrods for a particular query to application cache. Lets say total number of eligible records (numFound) are 501. my solr queries would be page 1 : q=*:*&start=0&rows=100 page 2 : q=*:*&start=100&rows=100 page 3 : q=*:*&start=200&rows=100 page 4 : q=*:*&start=300&rows=100 page 5 : q=*:*&start=400&rows=100 page 6 : q=*:*&start=500&rows=100 i see page 1 & 2 has common documents and similarly in other pages as well. Is this correct behavior ? Please correct. Thanks, Anil
Re: Solr JSON facet range out of memory exception
On Sun, Apr 10, 2016 at 3:47 AM, Ali Nazemian wrote: > Dear all Solr users/developeres, > Hi, > I am going to use Solr JSON facet range on a date filed which is stored as > long milis. Unfortunately I got java heap space exception no matter how > much memory assigned to Solr Java heap! I already test that with 2g heap > space for Solr core with 50k documents!! You mean the entire index is 50K documents? Or do you mean the subset of documents to be faceted? If you're getting an OOM with the former (with a 2G heap), it sounds like you've hit some sort of bug. What does your faceting command look like? -Yonik
Re: Set Config API user properties with SolrJ
Addendum: Apparently the code works fine with HttpSolrClient, but not with
EmbeddedSolrServer (used in our tests).The most recent version I tested
this was 5.5.0
Georg Sorst schrieb am So., 10. Apr. 2016 um
01:49 Uhr:
> Hi,
>
> how can you set Config API values from SolrJ? Does anyone have an example
> for this?
>
> Here's what I'm currently trying:
>
> /* Build the structure for the request */
> Map parameters = new HashMap() {{
> put("key", "value");
> }};
> final NamedList requestParameters = new NamedList<>();
> requestParameters.add("set-user-property", parameters);
>
> /* Build the JSON */
> CharArr json = new CharArr();
> new SchemaRequestJSONWriter(json).write(requestParameters);
> ContentStreamBase.StringStream stringStream = new
> ContentStreamBase.StringStream(json.toString());
> Collection contentStreams = Collections.
> singletonList(stringStream);
>
> /* Send the request */
> GenericSolrRequest request = new
> GenericSolrRequest(SolrRequest.METHOD.POST, "/config/overlay", null);
> request.setContentStreams(contentStreams);
> SimpleSolrResponse response = request.process(new HttpSolrClient("
> http://localhost:8983/solr/test";));
>
> The JSON is looking good, but it's doing... nothing. The response just
> contains the default config-overlay contents (znodeVersion). Any idea why?
>
> Thanks!
> Georg
> --
> *Georg M. Sorst I CTO*
> FINDOLOGIC GmbH
>
>
>
> Jakob-Haringer-Str. 5a | 5020 Salzburg I T.: +43 662 456708
> E.: g.so...@findologic.com
> www.findologic.com Folgen Sie uns auf: XING
> facebook
> Twitter
>
>
> Wir sehen uns auf dem *Shopware Community Day in Ahaus am 20.05.2016!*
> Hier Termin
> vereinbaren!
> Wir sehen uns auf der* dmexco in Köln am 14.09. und 15.09.2016!* Hier
> Termin
> vereinbaren!
>
--
*Georg M. Sorst I CTO*
FINDOLOGIC GmbH
Jakob-Haringer-Str. 5a | 5020 Salzburg I T.: +43 662 456708
E.: g.so...@findologic.com
www.findologic.com Folgen Sie uns auf: XING
facebook
Twitter
Wir sehen uns auf dem *Shopware Community Day in Ahaus am 20.05.2016!* Hier
Termin
vereinbaren!
Wir sehen uns auf der* dmexco in Köln am 14.09. und 15.09.2016!* Hier
Termin
vereinbaren!
Re: Solr document duplicated during pagination
It will not happen but you must: 1. Have Unique ID for each document 2. Make sure you define this field in the schema.xml YOUR_DOC_UQ_ID_FIELD_NAME 3. If you are using multiple shards query and not using solr cloud then you have to make sure you are not inserting the same document into two different shards. The uniqueness I mentioned in sections 1,2 is only for a specific shard/core. There is no way that one solr core will enforce uniqueness on other shards/cores unless you use solr cloud. On Sun, Apr 10, 2016 at 2:53 PM, Anil wrote: > HI, > > i am loading solr recrods for a particular query to application cache. > > Lets say total number of eligible records (numFound) are 501. > > my solr queries would be > > page 1 : q=*:*&start=0&rows=100 > page 2 : q=*:*&start=100&rows=100 > page 3 : q=*:*&start=200&rows=100 > page 4 : q=*:*&start=300&rows=100 > page 5 : q=*:*&start=400&rows=100 > page 6 : q=*:*&start=500&rows=100 > > i see page 1 & 2 has common documents and similarly in other pages as well. > Is this correct behavior ? Please correct. > > > Thanks, > Anil >
Error executing query
Hello All, When running my search method I had this exception QueryResponse res = server.query(params); ex = (org.apache.solr.client.solrj.SolrServerException) org.apache.solr.client.solrj.SolrServerException: Error executing query Regards, Adel
Re: Error executing query
You have to provide a lot more context before anyone can say anything meaningful. You might want to review: http://wiki.apache.org/solr/UsingMailingLists Best, Erick On Sun, Apr 10, 2016 at 6:30 AM, Adel Mohamed Khalifa wrote: > Hello All, > When running my search method I had this exception > QueryResponse res = server.query(params); > > ex = (org.apache.solr.client.solrj.SolrServerException) > org.apache.solr.client.solrj.SolrServerException: Error executing query > > Regards, > Adel > >
Re: Solr document duplicated during pagination
If the index is being updated while indexing, this can happen. But what do you mean by "i see page 1 & 2 has common documents and similarly in other pages as well"? Is it the _same_ id ( as Lior mentions)? Docs are "the same" to Solr if and only if they have the same Best, Erick On Sun, Apr 10, 2016 at 6:13 AM, Lior Sapir wrote: > It will not happen but you must: > 1. Have Unique ID for each document > 2. Make sure you define this field in the schema.xml > YOUR_DOC_UQ_ID_FIELD_NAME > 3. If you are using multiple shards query and not using solr cloud then > you have to make sure you are not inserting the same document into two > different shards. The uniqueness I mentioned in sections 1,2 is only for a > specific shard/core. There is no way that one solr core will enforce > uniqueness on other shards/cores unless you use solr cloud. > > > On Sun, Apr 10, 2016 at 2:53 PM, Anil wrote: > >> HI, >> >> i am loading solr recrods for a particular query to application cache. >> >> Lets say total number of eligible records (numFound) are 501. >> >> my solr queries would be >> >> page 1 : q=*:*&start=0&rows=100 >> page 2 : q=*:*&start=100&rows=100 >> page 3 : q=*:*&start=200&rows=100 >> page 4 : q=*:*&start=300&rows=100 >> page 5 : q=*:*&start=400&rows=100 >> page 6 : q=*:*&start=500&rows=100 >> >> i see page 1 & 2 has common documents and similarly in other pages as well. >> Is this correct behavior ? Please correct. >> >> >> Thanks, >> Anil >>
Limiting regex queries
Hey all,
I am using lucene and solr version 4.2, and was wondering what would
be the best way to not allow regex queries with very large numbers.
Something like blah{1234567} or blah{1234, 123445678}
Re: Limiting regex queries
OK, why is this a problem? This smells like an XY problem,
you want to take some specific action, but it's not at all
clear what the problem is. There might be other ways
of doing this.
If you're allowing regexes on numeric fields, using real
number fields (trie) and using range queries is a much
better way to go.
Best,
Erick
On Sun, Apr 10, 2016 at 9:28 AM, Michael Harkins wrote:
> Hey all,
>
> I am using lucene and solr version 4.2, and was wondering what would
> be the best way to not allow regex queries with very large numbers.
> Something like blah{1234567} or blah{1234, 123445678}
Re: Limiting regex queries
Well the originally architecture is out of my hands , but when someone
sends in a query like that, if the range is a large number , my system
basically shuts down and the cpu spikes with a large increase in
memory usage. The queries are for strings. The query itself was an
accident but I want to be able to prevent an accident from bringing
down the index.
> On Apr 10, 2016, at 12:34 PM, Erick Erickson wrote:
>
> OK, why is this a problem? This smells like an XY problem,
> you want to take some specific action, but it's not at all
> clear what the problem is. There might be other ways
> of doing this.
>
> If you're allowing regexes on numeric fields, using real
> number fields (trie) and using range queries is a much
> better way to go.
>
> Best,
> Erick
>
>> On Sun, Apr 10, 2016 at 9:28 AM, Michael Harkins wrote:
>> Hey all,
>>
>> I am using lucene and solr version 4.2, and was wondering what would
>> be the best way to not allow regex queries with very large numbers.
>> Something like blah{1234567} or blah{1234, 123445678}
Re: Saving Solr filter query.
Thanks so much Erick and John for your inputs. This was surely helpful input for me. I tried using RDBMS and stored user specific queries into RDBMS. It is working for me. Thanks & Regards, -- *Pritam Kute* On Thu, Apr 7, 2016 at 3:01 AM, John Bickerstaff wrote: > Right... You can store that anywhere - but at least consider not storing > it in your existing SOLR collection just because it's there... It's not > really the same kind of data -- it's application meta-data and/or > user-specific data... > > Getting it out later will be more difficult than if you store it > separately... > > A relational data store is ideal for this kind of thing - especially if > it's associated with a user and you're already storing user data in an > RDBMS. It's just a string... > > Alternatively, a "user" document or a "saved search" document in another > Solr collection could hold the data... > > On Wed, Apr 6, 2016 at 3:22 PM, Erick Erickson > wrote: > > > That's more of an app-level feature, there's nothing in Solr that does > > this for you. > > > > Some people have used a different Solr collection to store the queries > > as strings for display, but that's again something you build on top of > > Solr, not a core feature. > > > > Best, > > Erick > > > > On Wed, Apr 6, 2016 at 2:32 AM, Pritam Kute > > wrote: > > > Hi, > > > > > > I have designed one web page on which user can search and filter his > data > > > based on some term facets. I am using Apache Solr 5.3.1 for the same. > It > > is > > > working perfectly fine. > > > > > > Now my requirement is to save the query which I have executed on Solr, > > so, > > > in future, if I need to search the same results, I have to just extract > > the > > > saved query and make a query to Solr server (I mean the feature like > > saving > > > the favorite filters). > > > > > > Any help would be useful. Thanks in advance. > > > > > > Thanks & Regards, > > > -- > > > *Pritam Kute* > > >
