Solr 5.x bug with Service installation script?
Hi, I have installed Solr 5.3.1 using the Service Installation Script. I was able to successfully start and stop Solr using service solr start/stop commands and Solr loads up just fine. However, when I stop Solr service and copy an index of a core from one server to another with same exact version of Solr and its corresponding conf and restart the service, it complains about write.lock file when none exists under the path that it specifies in the log. To validate whether the issue is with the data that is being copied or the service script itself, I copied the collection directory with new index into example-DIH directory and restarted Solr manually bin/solr start -e dih -m 2g, it worked without any error. So, atleast this validates that collection data is just fine and service script is creating a lock everytime a new index is copied from another server though it has the same exact Solr version. Did anyone experience the same? Any thoughts if this is a bug? Thanks! AL
Re: Solr 5.x bug with Service installation script?
yes, I always shutdown both source and destination Solr before copying the index over from one to another. Somehow the write.lock only happens when Solr restarts from service script. If loads just fine when started manually. On Mon, May 9, 2016 at 1:20 PM, Abdel Belkasri wrote: > Did you copy the core while solr is running? if yes, first shuown source > and destination solr, copy intex to the other solr, then restat solr nodes. > Lock files get written to the core while solr is running and doing indexing > or searching, etc. > > On Mon, May 9, 2016 at 12:38 PM, A Laxmi wrote: > > > Hi, > > > > I have installed Solr 5.3.1 using the Service Installation Script. I was > > able to successfully start and stop Solr using service solr start/stop > > commands and Solr loads up just fine. > > > > However, when I stop Solr service and copy an index of a core from one > > server to another with same exact version of Solr and its corresponding > > conf and restart the service, it complains about write.lock file when > none > > exists under the path that it specifies in the log. > > > > To validate whether the issue is with the data that is being copied or > the > > service script itself, I copied the collection directory with new index > > into example-DIH directory and restarted Solr manually bin/solr start -e > > dih -m 2g, it worked without any error. So, atleast this validates that > > collection data is just fine and service script is creating a lock > > everytime a new index is copied from another server though it has the > same > > exact Solr version. > > > > Did anyone experience the same? Any thoughts if this is a bug? > > > > Thanks! > > AL > > > > > > -- > Abdel K. Belkasri, PhD >
Re: Solr 5.x bug with Service installation script?
Hi Erick - I used "sudo service solr stop" to shut it down. On Tue, May 10, 2016 at 12:26 AM, Erick Erickson wrote: > How do you shut down your Solrs? Any kind of un-graceful > stopping (kill -9 is a favorite) may leave the lock file around. > > It can't be coming from nowhere, so my guess is that > it's present in the source or destination before > you do your copy... > > Best, > Erick > > On Mon, May 9, 2016 at 10:30 AM, A Laxmi wrote: > > yes, I always shutdown both source and destination Solr before copying > the > > index over from one to another. Somehow the write.lock only happens when > > Solr restarts from service script. If loads just fine when started > manually. > > > > On Mon, May 9, 2016 at 1:20 PM, Abdel Belkasri > wrote: > > > >> Did you copy the core while solr is running? if yes, first shuown source > >> and destination solr, copy intex to the other solr, then restat solr > nodes. > >> Lock files get written to the core while solr is running and doing > indexing > >> or searching, etc. > >> > >> On Mon, May 9, 2016 at 12:38 PM, A Laxmi > wrote: > >> > >> > Hi, > >> > > >> > I have installed Solr 5.3.1 using the Service Installation Script. I > was > >> > able to successfully start and stop Solr using service solr start/stop > >> > commands and Solr loads up just fine. > >> > > >> > However, when I stop Solr service and copy an index of a core from one > >> > server to another with same exact version of Solr and its > corresponding > >> > conf and restart the service, it complains about write.lock file when > >> none > >> > exists under the path that it specifies in the log. > >> > > >> > To validate whether the issue is with the data that is being copied or > >> the > >> > service script itself, I copied the collection directory with new > index > >> > into example-DIH directory and restarted Solr manually bin/solr start > -e > >> > dih -m 2g, it worked without any error. So, atleast this validates > that > >> > collection data is just fine and service script is creating a lock > >> > everytime a new index is copied from another server though it has the > >> same > >> > exact Solr version. > >> > > >> > Did anyone experience the same? Any thoughts if this is a bug? > >> > > >> > Thanks! > >> > AL > >> > > >> > >> > >> > >> -- > >> Abdel K. Belkasri, PhD > >> >
Re: Solr 5.x bug with Service installation script?
Hi Shawn - You brought up a good point. This might be a possible reason. I'll test it out. Thanks! My index (4.5g) usually takes about 15-20 secs to load. One other observation - even though it says write.lock file in a specific data directory path, when I look up the directory, I don't see any write.lock file in there. It is really confusing. AL On Tue, May 10, 2016 at 10:37 AM, Shawn Heisey wrote: > On 5/9/2016 11:30 AM, A Laxmi wrote: > > yes, I always shutdown both source and destination Solr before copying > the > > index over from one to another. Somehow the write.lock only happens when > > Solr restarts from service script. If loads just fine when started > manually. > > One possible problem: > > The bin/solr script (which is used by the init script) only waits for 5 > seconds for Solr to stop gracefully before killingit forcibly. This can > leave write.lock files behind. > > I thought it had increased to 30 seconds in a recent version and that it > was possibly even configurable in solr.in.sh, but I just checked the > 6.0.0 download. It's still only 5 seconds, and the value is hard-coded > in the script. This is only enough time if you have a very small number > of very small indexes. > > Thanks, > Shawn > >
Complexity of a document?
Hi, Is it possible to determine how complex a document is using Solr? Complexity in terms of whether document is readable by a 7th grade vs. PHD Grad? Thanks! AL
Re: Complexity of a document?
Yes, length of the words would be one way but was wondering if there are any ways to identify the complexity. On Wed, May 11, 2016 at 3:36 PM, Joel Bernstein wrote: > I'm wondering if the size of the vocabulary used would be enough for this? > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Wed, May 11, 2016 at 3:32 PM, A Laxmi wrote: > > > Hi, > > > > Is it possible to determine how complex a document is using Solr? > > Complexity in terms of whether document is readable by a 7th grade vs. > PHD > > Grad? > > > > Thanks! > > AL > > >
Re: Complexity of a document?
Yes, length of the words would be one way but was wondering if there are any other ways to identify the complexity. On Wed, May 11, 2016 at 3:46 PM, A Laxmi wrote: > Yes, length of the words would be one way but was wondering if there are > any ways to identify the complexity. > > On Wed, May 11, 2016 at 3:36 PM, Joel Bernstein > wrote: > >> I'm wondering if the size of the vocabulary used would be enough for this? >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> On Wed, May 11, 2016 at 3:32 PM, A Laxmi wrote: >> >> > Hi, >> > >> > Is it possible to determine how complex a document is using Solr? >> > Complexity in terms of whether document is readable by a 7th grade vs. >> PHD >> > Grad? >> > >> > Thanks! >> > AL >> > >> > >
Re: Complexity of a document?
> * What I mean is that a technical paper will have a different type of complexity from let's say a Shakespearean play, because the former will have technical jargon, while the latter will have really high level* * vocabulary.*Good point. But, I am thinking a 7th grade might find both of them complex to understand - one because of technical jargon and other because of high level vocabulary? If it helps, I am looking at a set of user manuals of various products. I am trying to determine which of those user manuals are easier to read and which are more complex in comparison. On Wed, May 11, 2016 at 3:58 PM, Binoy Dalal wrote: > Please correct me if I'm wrong, but I think what Joel means is the variety > of words in a document. > > One more aspect that will come into play here, I think, is the different > types of complexity. > What I mean is that a technical paper will have a different type of > complexity from let's say a Shakespearean play, because the former will > have technical jargon, while the latter will have really high level > vocabulary. > > On Thu, 12 May 2016, 01:17 A Laxmi, wrote: > > > Yes, length of the words would be one way but was wondering if there are > > any other ways to identify the complexity. > > > > On Wed, May 11, 2016 at 3:46 PM, A Laxmi wrote: > > > > > Yes, length of the words would be one way but was wondering if there > are > > > any ways to identify the complexity. > > > > > > On Wed, May 11, 2016 at 3:36 PM, Joel Bernstein > > > wrote: > > > > > >> I'm wondering if the size of the vocabulary used would be enough for > > this? > > >> > > >> Joel Bernstein > > >> http://joelsolr.blogspot.com/ > > >> > > >> On Wed, May 11, 2016 at 3:32 PM, A Laxmi > > wrote: > > >> > > >> > Hi, > > >> > > > >> > Is it possible to determine how complex a document is using Solr? > > >> > Complexity in terms of whether document is readable by a 7th grade > vs. > > >> PHD > > >> > Grad? > > >> > > > >> > Thanks! > > >> > AL > > >> > > > >> > > > > > > > > > -- > Regards, > Binoy Dalal >
Boost Search results
Hi, When I started to compare the search results with the two options below, I see a lot of difference in the search results esp. the* urls that show up on the top *(*Relevancy *perspective). (1) Nutch 2.2.1 (with *Solr 4.0*) (2) Bing custom search set-up I wonder how should I tweak the boost parameters to get the best results on the top like how Bing, Google does. Please suggest why I see a difference and what parameters are best to configure in Solr to achieve what I see from Bing, or Google search relevancy. Here is what i got in solrconfig.xml: edismax text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4 *:* 10 *,score Thanks
Re: Boost Search results
Hi Markus, Yes, you are right. I passed the qf from my front-end framework
(PHP which uses SolrClient). This is how I got it set-up:
$this->solr->set_param('defType','edismax');
$this->solr->set_param('qf','title^10 content^5 url^5');
where you can see qf = title^10 content^5 url^5
On Fri, Apr 18, 2014 at 4:02 PM, Markus Jelsma
wrote:
> Hi, replicating full features search engine behaviour is not going to work
> with nutch and solr out of the box. You are missing a thousand features
> such as proper main content extraction, deduplication, classification of
> content and hub or link pages, and much more. These things are possible to
> implement but you may want to start with having you solr request handler
> better configured, to begin with, your qf parameter does not have nutchs
> default title and content field selected.
>
>
> A Laxmi schreef:Hi,
>
>
> When I started to compare the search results with the two options below, I
> see a lot of difference in the search results esp. the* urls that show up
> on the top *(*Relevancy *perspective).
>
> (1) Nutch 2.2.1 (with *Solr 4.0*)
> (2) Bing custom search set-up
>
> I wonder how should I tweak the boost parameters to get the best results on
> the top like how Bing, Google does.
>
> Please suggest why I see a difference and what parameters are best to
> configure in Solr to achieve what I see from Bing, or Google search
> relevancy.
>
> Here is what i got in solrconfig.xml:
>
> edismax
>
> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
>
>*:*
>10
>*,score
>
>
> Thanks
>
Re: Boost Search results
Markus, like I mentioned in my last email, I have got the qf with title,
content and url. That doesn't help a whole lot. Could you please advise if
there are any other parameters that I should consider for solr request
handler config or the numbers I have got for title, content, url in qf
parameter have to be modified?
Thanks for your help..
On Fri, Apr 18, 2014 at 4:08 PM, A Laxmi wrote:
> Hi Markus, Yes, you are right. I passed the qf from my front-end framework
> (PHP which uses SolrClient). This is how I got it set-up:
>
> $this->solr->set_param('defType','edismax');
> $this->solr->set_param('qf','title^10 content^5 url^5');
>
> where you can see qf = title^10 content^5 url^5
>
>
>
>
>
>
> On Fri, Apr 18, 2014 at 4:02 PM, Markus Jelsma > wrote:
>
>> Hi, replicating full features search engine behaviour is not going to
>> work with nutch and solr out of the box. You are missing a thousand
>> features such as proper main content extraction, deduplication,
>> classification of content and hub or link pages, and much more. These
>> things are possible to implement but you may want to start with having you
>> solr request handler better configured, to begin with, your qf parameter
>> does not have nutchs default title and content field selected.
>>
>>
>> A Laxmi schreef:Hi,
>>
>>
>> When I started to compare the search results with the two options below, I
>> see a lot of difference in the search results esp. the* urls that show up
>> on the top *(*Relevancy *perspective).
>>
>> (1) Nutch 2.2.1 (with *Solr 4.0*)
>> (2) Bing custom search set-up
>>
>> I wonder how should I tweak the boost parameters to get the best results
>> on
>> the top like how Bing, Google does.
>>
>> Please suggest why I see a difference and what parameters are best to
>> configure in Solr to achieve what I see from Bing, or Google search
>> relevancy.
>>
>> Here is what i got in solrconfig.xml:
>>
>> edismax
>>
>> text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
>>
>>*:*
>>10
>>*,score
>>
>>
>> Thanks
>>
>
>
Re: is there any way to post images and attachments to this mailing list?
Just upload them in Google Drive and share the link with this group. On Fri, Apr 18, 2014 at 9:15 PM, Candygram For Mongo < candygram.for.mo...@gmail.com> wrote: > >
