Re: Solr restricting time-consuming/heavy processing queries

[Mark Robinson]

Hello,
Could someone share their thoughts please or point to some link that helps
understand my above queries?
In the Solr documentation I came across a few lines on timeAllowed and
shardHandler, but if there was an example scenario for both it would help
understand them more thoroughly.
Also curious to know different ways if any n SOLR to restrict/ limit a time
consuming query from processing for a long time.

Thanks!
Mark

On Fri, Aug 9, 2019 at 2:15 PM Mark Robinson 
wrote:

>
> Hello,
> I have the following questions please:-
>
> In solrconfig.xml I created a new "/selecttimeout" handler copying
> "/select" handler and added the following to my new "/selecttimeout":-
>   
> 10
> 20
>   
>
> 1.
> Does the above mean that if I dont get a request once in 10ms on the
> socket handling the /selecttimeout handler, that socket will be closed?
>
> 2.
> Same with  connTimeOut? ie the connection  object remains live only if at
> least a connection request comes once in every 20 mS; if not the object
> gets closed?
>
> Suppose a time consumeing query (say with lots of facets etc...), is fired
> against SOLR. How can I prevent Solr processing it for not more than 1s?
>
> 3.
> Is this achieved by setting timeAllowed=1000?  Or are there any other ways
> to do this in Solr?
>
> 4
> For the same purpose to prevent heavy queries overloading SOLR, does the
>  above help in anyway or is it that shardHandler has nothing
> to restrict a query once fired against Solr?
>
>
> Could someone pls share your views?
>
> Thanks!
> Mark
>

Re: Solr restricting time-consuming/heavy processing queries

[Jan Høydahl]

What is the root use case you are trying to solve? What kind of solr install is 
this and do you not have control over the clients or what is the reason that 
users overload your servers?

Normally you would scale the cluster to handle normal expected load instead of 
trying to give users timeout exceptions. What kind of query times do you 
experience that are above 1s and are these not important enough to invest extra 
HW? Trying to understand the real reason behind your questions.

Jan Høydahl

> 11. aug. 2019 kl. 11:43 skrev Mark Robinson :
> 
> Hello,
> Could someone share their thoughts please or point to some link that helps
> understand my above queries?
> In the Solr documentation I came across a few lines on timeAllowed and
> shardHandler, but if there was an example scenario for both it would help
> understand them more thoroughly.
> Also curious to know different ways if any n SOLR to restrict/ limit a time
> consuming query from processing for a long time.
> 
> Thanks!
> Mark
> 
> On Fri, Aug 9, 2019 at 2:15 PM Mark Robinson 
> wrote:
> 
>> 
>> Hello,
>> I have the following questions please:-
>> 
>> In solrconfig.xml I created a new "/selecttimeout" handler copying
>> "/select" handler and added the following to my new "/selecttimeout":-
>>  
>>10
>>20
>>  
>> 
>> 1.
>> Does the above mean that if I dont get a request once in 10ms on the
>> socket handling the /selecttimeout handler, that socket will be closed?
>> 
>> 2.
>> Same with  connTimeOut? ie the connection  object remains live only if at
>> least a connection request comes once in every 20 mS; if not the object
>> gets closed?
>> 
>> Suppose a time consumeing query (say with lots of facets etc...), is fired
>> against SOLR. How can I prevent Solr processing it for not more than 1s?
>> 
>> 3.
>> Is this achieved by setting timeAllowed=1000?  Or are there any other ways
>> to do this in Solr?
>> 
>> 4
>> For the same purpose to prevent heavy queries overloading SOLR, does the
>>  above help in anyway or is it that shardHandler has nothing
>> to restrict a query once fired against Solr?
>> 
>> 
>> Could someone pls share your views?
>> 
>> Thanks!
>> Mark
>> 

java.nio.file.accessdeniedexception

[Rathor, Piyush]

HI Team



We are facing following issue in data update on solr:  
java.nio.file.accessdeniedexception



[cid:image003.jpg@01D55032.D808F430]



Thanks & Regards

Piyush



This message (including any attachments) contains confidential information 
intended for a specific individual and purpose, and is protected by law. If you 
are not the intended recipient, you should delete this message and any 
disclosure, copying, or distribution of this message, or the taking of any 
action based on it, by you is strictly prohibited.

Deloitte refers to a Deloitte member firm, one of its related entities, or 
Deloitte Touche Tohmatsu Limited ("DTTL"). Each Deloitte member firm is a 
separate legal entity and a member of DTTL. DTTL does not provide services to 
clients. Please see www.deloitte.com/about to learn more.

v.E.1

Solr is very slow with term vectors

[Vignan Malyala]

Hi guys,

I made my custom qparser plugin in Solr for scoring. The plugin only does
cosine similarity of vectors for each record. I use term vectors here.
Results are fine!

BUT, Solr response is very slow with term vectors. It takes around 55
seconds for each request for 100 records.
How do I make it faster to get my results in ms ?
Please respond soon as its lil urgent.

Note: All my values are stored and indexed. I am not using Solr Cloud.

Solr is very slow with term vectors

[Vignan Malyala]

Hi

I made by custom qparser plugin in Solr for scoring. The plugin only does
cosine similarity of vectors. for each record. Results are fine!

*BUT, Solr response is very slow. It takes around 55 seconds for each
request.*
*How do I make it faster to get my results in ms ?*
*Please respond soon as its lil urgent.*

Note: All my values are stored and indexed. I am not using Solr Cloud.

Regards,
Sai Vignan Malyala

Re: Solr is very slow with term vectors

[Doug Turnbull]

Hi Vignan,

We need to see more details / code of what your query parser plugin does
exactly with term vectors, we can't really help you without more details.
Is it open source? Can you share a minimal example that recreates the
problem?

On Sun, Aug 11, 2019 at 1:19 PM Vignan Malyala  wrote:

> Hi guys,
>
> I made my custom qparser plugin in Solr for scoring. The plugin only does
> cosine similarity of vectors for each record. I use term vectors here.
> Results are fine!
>
> BUT, Solr response is very slow with term vectors. It takes around 55
> seconds for each request for 100 records.
> How do I make it faster to get my results in ms ?
> Please respond soon as its lil urgent.
>
> Note: All my values are stored and indexed. I am not using Solr Cloud.
>


-- 
*Doug Turnbull **| CTO* | OpenSource Connections
, LLC | 240.476.9983
Author: Relevant Search 
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.

Solr join

[Iniyan]

Hi,

I was trying to do join between 2 collections. For that I have followed the
tutorial how to create colocating collection .

Created one collection with one shard and 2 replication factors.

Created another collection and added the query parameter
withCollection  = first collection name.

Seems like always this second collection is not created in first collection
cores.

Because of that join throws an error no active replicas found in 

I am using solr 5.x

Could anyone please help me?

Thanks
Iniyan P
-- 
Regards,
Iniyan P

Re: Solr is very slow with term vectors

[Walter Underwood]

tf.idf was invented because cosine similarity is too much computation. tf.idf 
gives similar results much, much faster than cosine distance.

I would expect cosine similarity to be slow. I would also expect retrieving 1 
million records to be slow. Doing both of those in one minute is pretty good.

As Kernighan and Paugher said in 1978, "Don’t diddle code to make it 
faster—find a better algorithm.”

https://en.wikipedia.org/wiki/The_Elements_of_Programming_Style

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Aug 11, 2019, at 10:40 AM, Doug Turnbull 
>  wrote:
> 
> Hi Vignan,
> 
> We need to see more details / code of what your query parser plugin does
> exactly with term vectors, we can't really help you without more details.
> Is it open source? Can you share a minimal example that recreates the
> problem?
> 
> On Sun, Aug 11, 2019 at 1:19 PM Vignan Malyala  wrote:
> 
>> Hi guys,
>> 
>> I made my custom qparser plugin in Solr for scoring. The plugin only does
>> cosine similarity of vectors for each record. I use term vectors here.
>> Results are fine!
>> 
>> BUT, Solr response is very slow with term vectors. It takes around 55
>> seconds for each request for 100 records.
>> How do I make it faster to get my results in ms ?
>> Please respond soon as its lil urgent.
>> 
>> Note: All my values are stored and indexed. I am not using Solr Cloud.
>> 
> 
> 
> -- 
> *Doug Turnbull **| CTO* | OpenSource Connections
> , LLC | 240.476.9983
> Author: Relevant Search 
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.

Solr De-Duplication Confusion

[Ievgen Rezvaniuk]

Hi all!
I'm a bit confused with the de-duplication logic and setup of solr.
In configuration options I may set:

   - enabled (true - default |false - to disable deduplication)
   - overwriteDups (" If true, the default, when a document exists that
   already matches this signature, it will be overwritten.")

I havent dig enough, but if enabled=true, and overwriteDups=false - nothing
happens.

Additionally I'd expect discarding of adding a duplicate document, as in my
case, previously saved document may be populated with some additional data,
which will be lost with overwrite. How is it possible to achieve this?

Thanks, in advance,

-- 
Jake

Re: java.nio.file.accessdeniedexception

[Shawn Heisey]

On 8/11/2019 8:52 AM, Rathor, Piyush wrote:
We are facing following issue in data update on solr: 
  java.nio.file.accessdeniedexception


Attachments almost never make it to the list. We cannot see the picture 
you sent.


https://www.dropbox.com/s/6axm2g1if4cwffs/solr-user-message-view.png?dl=0

You will need to find another way to provide the information.

Thanks,
Shawn

Solr mailing list

[Margo Breäs | INDI]

Unsubscribe


Met vriendelijke groet / Kind regards,
Margo Breäs
​Categoriespecialist
​T. +31 88 0666 000
​E. margo.br...@indi.nl
​W. www.indi.nl