can searcher.getReader().getFieldNames() return only stored fields?

2010-08-12 Thread Gerald 

Collection myFL =
searcher.getReader().getFieldNames(IndexReader.FieldOption.ALL);

will return all fields in the schema (i.e. index, stored, and
indexed+stored).

Collection myFL =
searcher.getReader().getFieldNames(IndexReader.FieldOption.INDEXED );

likely returns all fields that are indexed (I havent tried).

however, both of these can/will return fields that are not stored.  is there
a parameter that I can use to only return fields that are stored?

there does not seem to be a IndexReader.FieldOption.STORED and cant tell if
any of the others might work

any info helpful. thx
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/can-searcher-getReader-getFieldNames-return-only-stored-fields-tp1124178p1124178.html
Sent from the Solr - User mailing list archive at Nabble.com.

how do I create custom function that uses multiple ValuesSources?

2010-09-01 Thread Gerald 

using the NvlValueSourceParser example, I was able to create a custom
function that has two parameters; a valuesource (a solr field) and a string
literal.  i.e.: myfunc(mysolrfield, "test")

it works well but is a pretty simple function.

what is the the best way to implement a (more complex) custom function that
contains two (or more) ValuesSources as parameters??  i.e.:  

myfunc2(myValuesSource1, myValuesSources2, "test") or
myfunc2(div(myValuesSource1,3), sum(myValuesSources2, 2), "test") or
myfunc2(myValuesSource1, myValuesSource2, myValuesSource3)

I dont have a concrete example right now but will likely get some
application ideas once I figure this out

any thoughts/examples on something like this?
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/how-do-I-create-custom-function-that-uses-multiple-ValuesSources-tp1402645p1402645.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how do I create custom function that uses multiple ValuesSources?

2010-09-01 Thread Gerald 

Figured this out about ten minutes after I posted the message, and much
simpler than I thought it would be.

I used the SumFloatFunction (which extends MultiFloatFunction) as a starting
point and was able to achieve what I was going for for my test; a simple
string length function that returns the length of the specified fields after
concatenation

very nice being able to create custom functions.

now on to creating custom functions that handle multiValued data types
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/how-do-I-create-custom-function-that-uses-multiple-ValuesSources-tp1402645p1403070.html
Sent from the Solr - User mailing list archive at Nabble.com.

how/why would I use LiteralValueSource and can I create a custom string function?

2010-09-02 Thread Gerald 

while investigating custom functions in solr, I noticed LiteralValueSource

according to the one line documentation will "Pass a the field value through
as a String, no matter the type"

how would I use such a value source?  if it is a value source, I should be
able to use it in functionqueries for string related tests.

the reason I was wondering was that I wanted to see if I could create a
custom function that returned a string value source (after some
manipulation) to the query, possibly something like this:

http://mysolrserver:8080/solr/select?q=*:* AND
_val_:("mycustomstrfunction(mysolrfield):'somestringvalue'")&sort=score asc

is this type of functionality feasible in solr?
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/how-why-would-I-use-LiteralValueSource-and-can-I-create-a-custom-string-function-tp1408444p1408444.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how/why would I use LiteralValueSource and can I create a custom string function?

2010-09-02 Thread Gerald 

Thanks Grant

Am looking forward to the day when I can create a SOLR URL that looks
something like this:

http://mysolrserver:8080/solr/select?q=*:* AND
mycustomstrfunction(mysolrstrfield):'somestringvalue' AND
mycustomintfunction(mysolrintfield):[1 TO 100]
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/how-why-would-I-use-LiteralValueSource-and-can-I-create-a-custom-string-function-tp1408444p1409400.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: how/why would I use LiteralValueSource and can I create a custom string function?

2010-09-03 Thread Gerald 

I dont really have any specific use case in mind; I was just wondering what I
could (or couldn't) do with custom functions

possible reasons for allowing that type of syntax include:

1. in general, to simplify queries, and make them more readable, by
eliminating the need for the _val_ hack (which is useful but syntactically
inelegant) 
2. allow for more complex queries than we can do currently with _val_
3. if we treat custom functions exactly the same as we treat value sources
(like literals and solr fields), that may make some other solr features
(like sorting by function) cleaner/easier to implement/maintain

do you feel that I should I open this up as a JIRA suggestion?
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/how-why-would-I-use-LiteralValueSource-and-can-I-create-a-custom-string-function-tp1408444p1412903.html
Sent from the Solr - User mailing list archive at Nabble.com.

problem with RegexTransformer and delimited data

2010-04-12 Thread Gerald 

I have some delimited data that I would like to import but am having issues
getting the regex patterns to work properly with Solr. The following is just
one example of the issues I have experienced.

The regex required for this example should be very simple (delimited data). 
I have some regex patterns that work fine with online regex test sites like
http://www.regular-expressions.info/javascriptexample.html.  However solr
doesnt seem to handle them.

I am using the feb3/2010 trunk under windows

questions:
1. what sort of regex parser/flavor is used for splitBy and regex?  I
assumed it was the Java regex but doesnt seem to be
2. is there some documentation somewhere of what is/isnt acceptable regex
for Solr?
3. any ideas on how to get this [what I thought was] simple import routine
to work?

Any and all help/suggestions are appreciated.  Thx in advance.

--
Details:


1. my data looks something like this for each record: 
dataA1|^dataA2|?dataB1|^dataB2|?dataC1|^dataC2

2. I want to split out and save dataA1, dataB1, and dataC1 to a multivalue
field  ... and ignore the rest.

3. splitBy seems to work fine but my regex split doesnt; I end up with
either no data in the field or data that is not split.  for example I get:


 dataA1|^dataA2
 dataB1|^dataB2
 dataC1|^dataC2



4. The relevant parts of my schema include:


 
 
  
  
  
 
 
  
  
 

...



...
 
...


5. my data-conf.xml contains this sort of line:


...

 
 ...



6. patterns I have tried include "(.*)(\|.*)", "(.*)\|(.*)", "(.*)\|.*",
"(.*)(\|^.*)", ...

-- 
View this message in context: 
http://n3.nabble.com/problem-with-RegexTransformer-and-delimited-data-tp713846p713846.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: problem with RegexTransformer and delimited data

2010-04-12 Thread Gerald 

forgot to mention that I DID use replaceWith="$1" in tests where the pattern
was like "(.*)(\|.*)" in order to only get the first group
-- 
View this message in context: 
http://n3.nabble.com/problem-with-RegexTransformer-and-delimited-data-tp713846p714636.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: problem with RegexTransformer and delimited data

2010-04-13 Thread Gerald 

Thanks guys. Unfortunately, neither pattern works.

I tried various combos including these:

([^|]*)\|([^|]*)  with replaceWith="$1"
(.*?)(\|.*) with replaceWith="$1"
(.*?)\|.* with and without replaceWith="$1"
(.*?)\|  with and without replaceWith="$1"

As previously mentioned, I have tried many other ways without success.

I did notice that if I dont do the stripBy that it removes the everything
from the last "|^" onwards leaving me something like
"dataA1|^dataA2|?dataB1|^dataB2|?dataC1".

to me this doesnt look like a regex pattern issue; instead looks more like a
solr/lucene issue with regex.

any other suggestions welcome.  otherwise, will have to create custom
transformer
-- 
View this message in context: 
http://n3.nabble.com/problem-with-RegexTransformer-and-delimited-data-tp713846p716203.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: problem with RegexTransformer and delimited data

2010-04-13 Thread Gerald 

Thanks guys. Unfortunately, neither pattern works. 

I tried various combos including these: 

([^|]*)\|([^|]*)  with replaceWith="$1" 
(.*?)(\|.*) with replaceWith="$1" 
(.*?)\|.* with and without replaceWith="$1" 
(.*?)\|  with and without replaceWith="$1" 

As previously mentioned, I have tried many other ways without success. 

I did notice that if I dont do the stripBy that it removes the everything
from the last "|^" onwards leaving me something like
"dataA1|^dataA2|?dataB1|^dataB2|?dataC1". 

to me this doesnt look like a regex pattern issue; instead looks more like a
solr/lucene issue with regex. 

any other suggestions welcome.  otherwise, will have to create custom
transformer
-- 
View this message in context: 
http://n3.nabble.com/problem-with-RegexTransformer-and-delimited-data-tp713846p716206.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: problem with RegexTransformer and delimited data

2010-04-13 Thread Gerald 

AWESOME.  may take me some time to understand the regex pattern but it worked

And many thanks for looking into RegexTransformer.process().  Nice to know
that splitby cant be used with regex or replacewith etc


Many thanks Steve.
-- 
View this message in context: 
http://n3.nabble.com/problem-with-RegexTransformer-and-delimited-data-tp713846p716749.html
Sent from the Solr - User mailing list archive at Nabble.com.

NPE error when extending DefaultSolrHighlighter

2010-05-31 Thread Gerald 

I was looking at solr-386 and thought I would try to create a custom
highlighter for something I was doing. 

I created a class that looks something like this: 

public class CustomOutputHighlighter extends DefaultSolrHighlighter { 
 @Override 
 public NamedList doHighlighting(DocList docs, Query query,
SolrQueryRequest req, String[] defaultFields) throws IOException { 
 NamedList highlightedValues = super.doHighlighting(docs,
query, req, defaultFields); 
  
 // do more stuff here 

 return highlightedValues 
 } 
} 

and have replaced the  line in my solrconfig xml so that it
looks something like this: 

 

and left all the existing default highlighting parameters as-is 

The code compiles with no problem, and should simply perform the normal
highlighting (since all I am doing is calling the original doHighlighting
code and returning the results).  However, when I start Solr, I get an NPE
error: 

java.lang.NullPointerException 
at
org.apache.solr.highlight.DefaultSolrHighlighter.init(DefaultSolrHighlighter.java:75)
 
at
org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:437) 
at org.apache.solr.core.SolrCore.initHighLighter(SolrCore.java:612) 
at org.apache.solr.core.SolrCore.(SolrCore.java:558) 
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:137)
 
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83) 
at
org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:99) 
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) 
at
org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:594) 
at org.mortbay.jetty.servlet.Context.startContext(Context.java:139) 
at
org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218) 
at
org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500) 
at
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448) 
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) 
at
org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) 
at
org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:161)
 
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) 
at
org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:147) 
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) 
at
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117) 
at org.mortbay.jetty.Server.doStart(Server.java:210) 
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40) 
at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:929) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
at java.lang.reflect.Method.invoke(Unknown Source) 
at org.mortbay.start.Main.invokeMain(Main.java:183) 
at org.mortbay.start.Main.start(Main.java:497) 
at org.mortbay.start.Main.main(Main.java:115) 

It doesn't seem to even call my custom highlighter (I put a breakpoint in
which did not get hit).  Any ideas re what I am doing wrong?? 

If I use the default highlighter, I don't get this error and have no
problems 

I am using a copy of 1.5.0-dev solr ($Id: CHANGES.txt 906924 2010-02-05
12:43:11Z noble $) 

thanks for any advice 
-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/NPE-error-when-extending-DefaultSolrHighlighter-tp859670p859670.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Update to solr 5 - custom phrase query implementation issue

2016-02-03 Thread Gerald Reinhart 

On 02/02/2016 03:20 PM, Erik Hatcher wrote:

On Feb 2, 2016, at 8:57 AM, Elodie Sannier  wrote:

Hello,

We are using solr 4.10.4 and we want to update to 5.4.1.

With solr 4.10.4:
- we extend PhraseQuery with a custom class in order to remove some
terms from phrase queries with phrase slop (update of add(Term term, int
position) method)
- in order to use our implementation, we extend ExtendedSolrQueryParser
with a custom class and we override the method newPhraseQuery but with
solr 5 this method does not exist anymore

How can we do this with solr 5.4.1 ?


You’ll want to override this method, it looks like:

protected Query getFieldQuery(String field, String queryText, int slop)



Hi Erik,

To change the behavior of the PhraseQuery either:
   - we change it after the natural cycle. The PhraseQuery is supposed
to be immutable and the setSlop(int s) is deprecated... we don't really
want to do this.
   - we override the code that actually build it :
org.apache.solr.search.ExtendedDismaxQParser.getFieldQuery(String field,
String val, int slop) PROTECTED
  use getAliasedQuery() PROTECTED
use getQuery() PRIVATE which use new PhraseQuery.Builder()
to create the query...

so not easy to override the behavior: we would need to
overrride/duplicate getAliasedQuery() and getQuery() methods. we don't
really want to do this either.

So we don't really know where to go.

Thanks,


Gerald (I'm working with Elodie on the subject)



Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

Re: Update to solr 5 - custom phrase query implementation issue

2016-02-03 Thread Gerald Reinhart 

Erik, here is some context :

   - migration from solr 4.10.4 to 5.4.1.
   - we have our own synonym implementation that do not use solr
synonym mechanism: at the time, we needed to manage multi token synonym
and it wasn't covered by the Lucene features. So basically we
- let' say that "playstation portable" is a synonym of
"psp". We identify "psp" and "playstation portable" as .
- at index time, on every document with "psp" we replace it
by "psp "
- at query-time:
   - By extending SearchHandler, we replace the query
"playstation portable" by  "((playstation AND portable) OR )".
   - (1) By extending ExtendedSolrQueryParser, we do
not add synonym ids in PhraseQuery. We need advice for the migration to
Solr 5.4.1
   - By extending BooleanQuery, we adjust the
coordination factor. We need advice for the migration to Solr 5.4.1 (see
other question in the mailing list)

   Hope it's clearer

Thanks

Gérald


(1)
public class MyPhraseQuery extends PhraseQuery {
   private int deltaPosition = 0;

   @Override
   public void add(Term term, int position) {
  String termText = term.text();
  if (!termText.matches(Constants.SYNONYM_PIVOT_ID_REGEX)) {
 super.add(term, position - deltaPosition);
  }else{
 deltaPosition++;
  }
   }
}


On 02/03/2016 03:05 PM, Erik Hatcher wrote:

Gerald - I don’t quite understand, sorry - perhaps best if you could post your 
code (or some test version you can work with and share here) so we can see what 
exactly you’re trying to do.Maybe there’s other ways to achieve what you 
want, maybe with somehow leveraging a StopFilter-like facility to remove phrase 
terms.   edismax has some stop word (inclusion, rather than exclusion, though) 
magic with the pf2 and pf3 and stopwords parameters - maybe worth leveraging 
something like how that works or maybe adding some options or pluggability to 
the edismax phrase/stopword facility?

  Erik




On Feb 3, 2016, at 6:05 AM, Gerald Reinhart  wrote:

On 02/02/2016 03:20 PM, Erik Hatcher wrote:

On Feb 2, 2016, at 8:57 AM, Elodie Sannier  wrote:

Hello,

We are using solr 4.10.4 and we want to update to 5.4.1.

With solr 4.10.4:
- we extend PhraseQuery with a custom class in order to remove some
terms from phrase queries with phrase slop (update of add(Term term, int
position) method)
- in order to use our implementation, we extend ExtendedSolrQueryParser
with a custom class and we override the method newPhraseQuery but with
solr 5 this method does not exist anymore

How can we do this with solr 5.4.1 ?

You’ll want to override this method, it looks like:

protected Query getFieldQuery(String field, String queryText, int slop)



Hi Erik,

To change the behavior of the PhraseQuery either:
   - we change it after the natural cycle. The PhraseQuery is supposed
to be immutable and the setSlop(int s) is deprecated... we don't really
want to do this.
   - we override the code that actually build it :
org.apache.solr.search.ExtendedDismaxQParser.getFieldQuery(String field,
String val, int slop) PROTECTED
  use getAliasedQuery() PROTECTED
use getQuery() PRIVATE which use new PhraseQuery.Builder()
to create the query...

so not easy to override the behavior: we would need to
overrride/duplicate getAliasedQuery() and getQuery() methods. we don't
really want to do this either.

So we don't really know where to go.

Thanks,


Gerald (I'm working with Elodie on the subject)



Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.



--

Kelkoo



*Gérald Reinhart *Software engineer

*E*gerald.reinh...@kelkoo.com <mailto:steve.con...@kelkoo.com>
*Y!Messenger*gerald.reinhart

*A*Rue des Méridiens 38130 Echirolles






Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

[Migration Solr4 to Solr5] Collection reload error

2016-02-29 Thread Gerald Reinhart 

Hi,

   We are facing an issue during a migration from Solr4 to Solr5.

Given
   - migration from solr 4.10.4 to 5.4.1
   - 2 collections
   - cloud with one leader and several replicas
   - in solrconfig.xml: maxWarmingSearchers=1
   - no code change

When collection reload using /admin/collections using solrj

Then

2016-02-29 13:42:49,011 [http-8080-3] INFO
org.apache.solr.core.CoreContainer:reload:848  - Reloading SolrCore
'fr_blue' using configuration from collection fr_blue
2016-02-29 13:42:45,428 [http-8080-6] INFO
org.apache.solr.search.SolrIndexSearcher::237  - Opening
Searcher@58b65fc[fr_blue] main
(...)
2016-02-29 13:42:49,077 [http-8080-3] WARN
org.apache.solr.core.SolrCore:getSearcher:1762  - [fr_blue] Error
opening new searcher. exceeded limit of maxWarmingSearchers=1, try again
later.
2016-02-29 13:42:49,091 [http-8080-3] ERROR
org.apache.solr.handler.RequestHandlerBase:log:139  -
org.apache.solr.common.SolrException: Error handling 'reload' action
at
org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:770)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:230)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:184)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:438)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Unable to reload core
[fr_blue]
at
org.apache.solr.core.CoreContainer.reload(CoreContainer.java:854)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:768)
... 20 more
Caused by: org.apache.solr.common.SolrException: Error opening new
searcher. exceeded limit of maxWarmingSearchers=1, try again later.
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1764)
at org.apache.solr.core.SolrCore.reload(SolrCore.java:474)
at
org.apache.solr.core.CoreContainer.reload(CoreContainer.java:849)
... 21 more


Thanks


Gérald and Elodie


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

[ISSUE] backup on a recovering index should fail

2016-03-01 Thread Gerald Reinhart 


Hi,

   In short: backup on a recovering index should fail.

   We are using the backup command "http:// ...
/replication?command=backup&location=/tmp" against one server of the
cluster.
   Most of the time there is no issue with this command.

   But in some particular case, the server can be in recovery mode. In
this case, the command is doing a backup on an index that is not
complete and return a http code 200. We end up with a partial index
backup ! As a workaround we will do this backup against the leader of
the cloud: the leader is never in recovery mode.

   In our opinion, the backup command on a recovering index should
return a http code 503 Service Unavailable (and not http code 200 OK).

   Shall we open a issue or it's an expected behaviour ?

   Thanks,


Gérald and Elodie


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

Re: [Migration Solr4 to Solr5] Collection reload error

2016-03-07 Thread Gerald Reinhart 


Hi,

 To give you some context, we are migrating from Solr4 and solr5,
the client code and the configuration haven't changed but now we are
facing this problem. We have already checked the commit behaviour
configuration and it seems good.

Here it is :

Server side, we have 2 collections (main and temp with blue and green
aliases) :

   solrconfig.xml:

  
  
 (...)
 
   90
   false
 

 

  

Client side, we have 2 different modes:

1 - Full recovery :

- Delete all documents from the temp collection
  solrClient.deleteByQuery("*:*")

- Add all new documents in temp collection (can be more
than 5Millions),
  solrClient.add(doc, -1) // commitWithinMs == -1

-  Commit when all documents are added
  solrClient.commit(false,false) // waitFlush == false ,
waitSearcher == false

-  Swap blue and green using "create alias" command

-  Reload the temp collection to clean the cache. This is
at this point we have the issue.

2 - Incremental :

-  Add or delete documents from the main collection
   solrClient.add(doc, 180)   // commitWithin
== 30 mn
   solrClient.deleteById(doc, 180) // commitWithin == 30 mn

Maybe you will spot something obviously wrong ?

Thanks

Gérald and Elodie



On 03/04/2016 12:41 PM, Dmitry Kan wrote:

Hi,

Check the the autoCommit and autoSoftCommit nodes in the solrconfig.xml.
Set them to reasonable values. The idea is that if you commit too often,
searchers will be warmed up and thrown away. If at any point in time you
get overlapping commits, there will be several searchers sitting on the
deck.

Dmitry

On Mon, Feb 29, 2016 at 4:20 PM, Gerald Reinhart 
wrote:
Hi,

We are facing an issue during a migration from Solr4 to Solr5.

Given
- migration from solr 4.10.4 to 5.4.1
- 2 collections
- cloud with one leader and several replicas
- in solrconfig.xml: maxWarmingSearchers=1
- no code change

When collection reload using /admin/collections using solrj

Then

2016-02-29 13:42:49,011 [http-8080-3] INFO
org.apache.solr.core.CoreContainer:reload:848  - Reloading SolrCore
'fr_blue' using configuration from collection fr_blue
2016-02-29 13:42:45,428 [http-8080-6] INFO
org.apache.solr.search.SolrIndexSearcher::237  - Opening
Searcher@58b65fc[fr_blue] main
(...)
2016-02-29 13:42:49,077 [http-8080-3] WARN
org.apache.solr.core.SolrCore:getSearcher:1762  - [fr_blue] Error
opening new searcher. exceeded limit of maxWarmingSearchers=1, try again
later.
2016-02-29 13:42:49,091 [http-8080-3] ERROR
org.apache.solr.handler.RequestHandlerBase:log:139  -
org.apache.solr.common.SolrException: Error handling 'reload' action
 at

org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:770)
 at

org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:230)
 at

org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:184)
 at

org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
 at

org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
 at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:438)
 at

org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
 at

org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
 at

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at

org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at

org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at

org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at

org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
 at

org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
 at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
 at

org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
 at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
 at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Unable to reload core
[fr_blue]
 at
org.apache.solr.core.CoreContainer.reload(CoreContainer.java:854)
 at

org.apache.solr.hand

[Solr-5-4-1] Why SolrCloud leader is putting all replicas in recovery at the same time ?

2016-10-06 Thread Gerald Reinhart 


Hello everyone,

   Our Solr Cloud  works very well for several months without any significant 
changes: the traffic to serve is stable, no major release deployed...

   But randomly, the Solr Cloud leader puts all the replicas in recovery at the 
same time for no obvious reason.

   Hence, we can not serve the queries any more and the leader is overloaded 
while replicating all the indexes on the replicas at the same time which 
eventually implies a downtime of approximately 30 minutes.

   Is there a way to prevent it ? Ideally, a configuration saying a percentage 
of replicas to be put in recovery at the same time?

Thanks,

Gérald, Elodie and Ludovic


--
[cid:part1.0508.06030105@kelkoo.com]

Gérald Reinhart Software Engineer

E   
gerald.reinh...@kelkoo.comY!Messenger gerald.reinhart
T +33 (0)4 56 09 07 41
A Parc Sud Galaxie - Le Calypso, 6 rue des Méridiens, 38130 Echirolles


[cid:part4.08030706.00010405@kelkoo.com]




Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

Re: [Solr-5-4-1] Why SolrCloud leader is putting all replicas in recovery at the same time ?

2016-10-13 Thread Gerald Reinhart 


Hi Pushkar Raste,

  Thanks for your hits.
  We will try the 3rd solution and keep you posted.

Gérald Reinhart

On 10/07/2016 02:23 AM, Pushkar Raste wrote:
A couple of questions/suggestions
- This normally happens after leader election, when new leader gets elected, it 
will force all the nodes to sync with itself.
Check logs to see when this happens, if leader was changed. If that is true 
then you will have to investigate why leader change takes place.
I suspect leader goes into long enough GC pause that makes zookeeper leader is 
no longer available and initiates leader election.

- What version of Solr you are using.  
SOLR-8586<https://issues.apache.org/jira/browse/SOLR-8586> introduced 
IndexFingerprint check, unfortunately it was broken and hence replica would always do full 
index replication. Issue is now fixed in 
SOLR-9310<https://issues.apache.org/jira/browse/SOLR-9310>, this should help replicas 
recover faster.

- You should also increase ulog log size (default threshold is 100 docs or 10 
tlogs whichever is hit first). This will again help replicas recover faster 
from tlogs (of course, there would be a threshold after which recovering from 
tlog would in fact take longer than copying over all the index files from 
leader)


On Thu, Oct 6, 2016 at 5:23 AM, Gerald Reinhart 
mailto:gerald.reinh...@kelkoo.com>> wrote:

Hello everyone,

   Our Solr Cloud  works very well for several months without any significant 
changes: the traffic to serve is stable, no major release deployed...

   But randomly, the Solr Cloud leader puts all the replicas in recovery at the 
same time for no obvious reason.

   Hence, we can not serve the queries any more and the leader is overloaded 
while replicating all the indexes on the replicas at the same time which 
eventually implies a downtime of approximately 30 minutes.

   Is there a way to prevent it ? Ideally, a configuration saying a percentage 
of replicas to be put in recovery at the same time?

Thanks,

Gérald, Elodie and Ludovic


--
[cid:part1.0508.06030105@kelkoo.com]

Gérald Reinhart Software Engineer

E <mailto:gerald.reinh...@kelkoo.com> <mailto:gerald.reinh...@kelkoo.com> 
gerald.reinh...@kelkoo.com<mailto:gerald.reinh...@kelkoo.com>Y!Messenger gerald.reinhart
T +33 (0)4 56 09 07 41
A Parc Sud Galaxie - Le Calypso, 6 rue des Méridiens, 38130 Echirolles


[cid:part4.08030706.00010405@kelkoo.com]




Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.



--


--
[cid:part9.0705.09050102@kelkoo.com]

Gérald Reinhart Software Engineer

E <mailto:gerald.reinh...@kelkoo.com> <mailto:gerald.reinh...@kelkoo.com> 
gerald.reinh...@kelkoo.com<mailto:gerald.reinh...@kelkoo.com>Y!Messenger gerald.reinhart
T +33 (0)4 56 09 07 41
A Parc Sud Galaxie - Le Calypso, 6 rue des Méridiens, 38130 Echirolles


[cid:part12.05000204.08070006@kelkoo.com]




Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

[Benchmark SOLR] JETTY VS TOMCAT

2017-01-27 Thread Gerald Reinhart 

Hello,

   We are migrating our platform
   from
- Solr 5.4.1 hosted by a Tomcat
   to
- Solr 5.4.1 standalone (hosted by Jetty)

=> Jetty is 15% slower than Tomcat in the same conditions.


  Here are details about the benchmarks :

  Context :
   - Index with 9 000 000 documents
   - Gatling launch queries extracted from the real traffic
   - Server :  R410 with 16 virtual CPU and 96G mem

  Results with 20 clients in // during 10 minutes:
   For Tomcat :
   - 165 Queries per seconds
   - 120ms mean response time

   For Jetty :
   - 139 Queries per seconds
   - 142ms mean response time

We have checked :
  - the load of the server => same
  - the io wait => same
  - the memory used in the JVM => same
  - JVM GC settings => same

 For us, it's a blocker for the migration.

 Is it a known issue ? (I found that :
http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)

 How can we improve the performance of Jetty ? (We have already
followed
http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html
recommendation)

Many thanks,


Gérald Reinhart


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

[Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

2017-01-30 Thread Gerald Reinhart 


Hello,

In addition to the following settings, we have tried to :
- force Jetty to use more threads
- put the same GC options as our Tomcat
- change nb of acceptors and selectors

   and every time Jetty is slower than Tomcat.

   Any advice is welcome

Thanks,

Gérald Reinhart




On 01/27/2017 11:22 AM, Gerald Reinhart wrote:

Hello,

 We are migrating our platform
 from
  - Solr 5.4.1 hosted by a Tomcat
 to
  - Solr 5.4.1 standalone (hosted by Jetty)

=> Jetty is 15% slower than Tomcat in the same conditions.


Here are details about the benchmarks :

Context :
 - Index with 9 000 000 documents
 - Gatling launch queries extracted from the real traffic
 - Server :  R410 with 16 virtual CPU and 96G mem

Results with 20 clients in // during 10 minutes:
 For Tomcat :
 - 165 Queries per seconds
 - 120ms mean response time

 For Jetty :
 - 139 Queries per seconds
 - 142ms mean response time

We have checked :
- the load of the server => same
- the io wait => same
- the memory used in the JVM => same
- JVM GC settings => same

   For us, it's a blocker for the migration.

   Is it a known issue ? (I found that :
http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)

   How can we improve the performance of Jetty ? (We have already
followed
http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html
recommendation)

  Many thanks,


Gérald Reinhart


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.



--

Gérald Reinhart
Software Engineer

<http://www.kelkoo.com/>

*E*gerald.reinh...@kelkoo.com
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

Re: [Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

2017-02-01 Thread Gerald Reinhart 



We have done some profiling with Visualvm, but nothing obvious appeared.

Thank Rick for the advice.

Gérald Reinhart


On 02/01/2017 11:17 AM, Rick Leir wrote:

There is a profiling tool in Eclipse that can show you a tree of method
calls, with timing information. I have found this useful in the past to
investigate a performance problem. But it might not help if the problem
only occurs at 165 queries per second (is that true?).

cheers -- Rick


On 2017-01-30 04:02 AM, Gerald Reinhart wrote:

Hello,

 In addition to the following settings, we have tried to :
 - force Jetty to use more threads
 - put the same GC options as our Tomcat
 - change nb of acceptors and selectors

and every time Jetty is slower than Tomcat.

Any advice is welcome

Thanks,

Gérald Reinhart




On 01/27/2017 11:22 AM, Gerald Reinhart wrote:

Hello,

  We are migrating our platform
  from
   - Solr 5.4.1 hosted by a Tomcat
  to
   - Solr 5.4.1 standalone (hosted by Jetty)

=> Jetty is 15% slower than Tomcat in the same conditions.


 Here are details about the benchmarks :

 Context :
  - Index with 9 000 000 documents
  - Gatling launch queries extracted from the real traffic
  - Server :  R410 with 16 virtual CPU and 96G mem

 Results with 20 clients in // during 10 minutes:
  For Tomcat :
  - 165 Queries per seconds
  - 120ms mean response time

  For Jetty :
  - 139 Queries per seconds
  - 142ms mean response time

We have checked :
 - the load of the server => same
 - the io wait => same
 - the memory used in the JVM => same
 - JVM GC settings => same

For us, it's a blocker for the migration.

Is it a known issue ? (I found that :
http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)

How can we improve the performance of Jetty ? (We have already
followed
http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html

recommendation)

   Many thanks,


Gérald Reinhart


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à
l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
destinataire de ce message, merci de le détruire et d'en avertir
l'expéditeur.


--

Gérald Reinhart
Software Engineer

<http://www.kelkoo.com/>

*E*gerald.reinh...@kelkoo.com
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à
l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
destinataire de ce message, merci de le détruire et d'en avertir
l'expéditeur.



--

Gérald Reinhart
Software Engineer

<http://www.kelkoo.com/>

*E*gerald.reinh...@kelkoo.com
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

Re: [Migration Solr5 to Solr6] Unwanted deleted files references

2017-03-14 Thread Gerald Reinhart 


Hi,

   The custom code we have is something like this :

public class MySearchHandlerextends SearchHandler {


@Override public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse 
rsp)throws Exception {

SolrIndexSearcher  searcher =req.getSearcher();

try{

 // Do stuff with the searcher


}finally {
req.close();
}



}

}

 Despite the fact that we always close the request each time we get
a SolrIndexSearcher from the request, the number of SolrIndexSearcher
instances is increasing. Each time a new commit is done on the index, a
new Searcher is created (this is normal) but the old one remains. Is
there something wrong with this custom code ? Shall we try something
explained there :
http://stackoverflow.com/questions/20515493/solr-huge-number-of-open-searchers
? Thanks, Gérald Reinhart (working with Elodie on the subject) On
03/07/2017 05:45 PM, Elodie Sannier wrote:

Thank you Erick for your answer.

The files are deleted even without JVM restart but they are still seen
as DELETED by the kernel.

We have a custom code and for the migration to Solr 6.4.0 we have added
a new code with req.getSearcher() but without "close".
We will decrement the reference count on a resource for the Searcher
(prevent the Searcher remains open after a commit) and see if it fixes
the problem.

Elodie

On 03/07/2017 03:55 PM, Erick Erickson wrote:

Just as a sanity check, if you restart the Solr JVM, do the files
disappear from disk?

Do you have any custom code anywhere in this chain? If so, do you open
any searchers but
fail to close them? Although why 6.4 would manifest the problem but
other code wouldn't
is a mystery, just another sanity check.

Best,
Erick

On Tue, Mar 7, 2017 at 6:44 AM, Elodie Sannier  wrote:

Hello,

We have migrated from Solr 5.4.1 to Solr 6.4.0 and the disk usage has
increased.
We found hundreds of references to deleted index files being held by solr.
Before the migration, we had 15-30% of disk space used, after the migration
we have 60-90% of disk space used.

We are using Solr Cloud with 2 collections.

The commands applied on the collections are:
- for incremental indexation mode: add, deleteById with commitWithin of 30
minutes
- for full indexation mode: add, deleteById, commit
- for switch between incremental and full mode: deleteByQuery, createAlias,
reload
- there is also an autocommit every 15 minutes

We have seen the email "Solr leaking references to deleted files"
2016-05-31 which describe the same problem but the mentioned bugs are fixed.

We manually tried to force a commit, a reload and an optimize on the
collections without effect.

Is a problem of configuration (merge / delete policy) or a possible
regression in the Solr code ?

Thank you


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce
message, merci de le détruire et d'en avertir l'expéditeur.


--

Elodie Sannier
Software engineer



*E*elodie.sann...@kelkoo.fr*Skype*kelkooelodies
*T*+33 (0)4 56 09 07 55
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

--

Gérald ReinhartSoftware Engineer



*E*gerald.reinh...@kelkoo.com*A*Parc Sud Galaxie, 6, rue des Méridiens,
38130 Echirolles, FR


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

SolrIndexSearcher accumulation

2017-04-07 Thread Gerald Reinhart 


Hi,

   We have some custom code that extends SearchHandler to be able to :
- do an extra request
- merge/combine the original request and the extra request results

   On Solr 5.x, our code was working very well, now with Solr 6.x we
have the following issue:  the number of SolrIndexSearcher are
increasing (we can see them in the admin view > Plugins/ Stats > Core ).
As SolrIndexSearcher are accumulating, we have the following issues :
   - the memory used by Solr is increasing => OOM after a long
period of time in production
   - some files in the index has been deleted from the system but
the Solr JVM still hold them => ("fake") Full disk after a long period
of time in production

   We are wondering,
  - what has changed between Solr 5.x and Solr 6.x in the
management of the SolrIndexSearcher ?
  - what would be the best way, in a Solr plugin, to perform 2
queries and merge the results to a single SolrQueryResponse ?

   Thanks a lot.

Gérald, Elodie, Ludo and André



Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.

CurrencyField querying in Solr 4

2013-02-25 Thread Gerald Blanck 
We are attempting to leverage the CurrecyField type.  We have defined the
currency field type as:




And defined a field as:




When querying the field with something like:

my_money:[* TO *]

The result is ALL documents (even though only 1 document actually has this
field populated.


When querying the field with something like:

+my_money:[* TO *] -my_money:0

We get the single document back.


So, it looks like there is a default of "0" for documents that don't have
the money value (even though required=false).  Is it possible to eliminate
this default?  Would we have better luck if we upgraded to Solr 4.1?

Thanks in advance.
-- 

*Gerald Blanck*

baro*m*eter*IT*

1331 Tyler Street NE, Suite 100
Minneapolis, MN 55413


612.208.2802

gerald.bla...@barometerit.com

Nested Join Queries

2012-11-02 Thread Gerald Blanck 
At a high level, I have a need to be able to execute a query that joins
across cores, and that query during its joining may join back to the
originating core.

Example:
Find all Books written by an Author who has written a best selling Book.

In Solr query syntax
A) against the book core - bestseller:true
B) against the author core - {!join fromIndex=book from=id
to=bookid}bestseller:true
C) against the book core - {!join fromIndex=author from=id
to=authorid}{!join fromIndex=book from=id to=bookid}bestseller:true

A - returns results
B - returns results
C - does not return results

Given that A and C use the same core, I started looking for join code that
compares the originating core to the fromIndex and found this
in JoinQParserPlugin (line #159).

if (info.getReq().getCore() == fromCore) {

  // if this is the same core, use the searcher passed in...
otherwise we could be warming and

  // get an older searcher from the core.

  fromSearcher = searcher;

} else {

  // This could block if there is a static warming query with a
join in it, and if useColdSearcher is true.

  // Deadlock could result if two cores both had useColdSearcher
and had joins that used eachother.

  // This would be very predictable though (should happen every
time if misconfigured)

  fromRef = fromCore.getSearcher(false, true, null);


  // be careful not to do anything with this searcher that requires
the thread local

  // SolrRequestInfo in a manner that requires the core in the
request to match

  fromSearcher = fromRef.get();

}

I found that if I were to modify the above code so that it always follows
the logic in the else block, I get the results I expect.

Can someone explain to me why the code is written as it is?  And if we were
to run with only the else block being executed, what type of adverse
impacts we might have?

Does anyone have other ideas on how to solve this issue?

Thanks in advance.
-Gerald

Re: Nested Join Queries

2012-11-07 Thread Gerald Blanck 
Thank you Erick for your reply.  I understand that search is not an RDBMS.
 Yes, we do have a huge combinatorial explosion if we de-normalize and
duplicate data.  In fact, I believe our use case is exactly what the Solr
developers were trying to solve with the addition of the Join query.  And
while the example I gave illustrates the problem we are solving with the
Join functionality, it is simplistic in nature compared to what we have in
actuality.

Am still looking for an answer here if someone can shed some light.  Thanks.


On Sat, Nov 3, 2012 at 9:38 PM, Erick Erickson wrote:

> I'm going to go a bit sideways on you, partly because I can't answer the
> question ...
>
> But, every time I see someone doing what looks like substituting "core" for
> "table" and
> then trying to use Solr like a DB, I get on my soap-box and preach..
>
> In this case, consider de-normalizing your DB so you can ask the query in
> terms
> of search rather than joins. e.g.
>
> Make each document a combination of the author and the book, with an
> additional
> field "author_has_written_a_bestseller". Now your query becomes a really
> simple
> search, "author:name AND author_has_written_a_bestseller:true". True, this
> kind
> of approach isn't as flexible as an RDBMS, but it's a _search_ rather than
> a query.
> Yes, it replicates data, but unless you have a huge combinatorial
> explosion, that's
> not a problem.
>
> And the join functionality isn't called "pseudo" for nothing. It was
> written for a specific
> use-case. It is often expensive, especially when the field being joined has
> many unique
> values.
>
> FWIW,
> Erick
>
>
> On Fri, Nov 2, 2012 at 11:32 AM, Gerald Blanck <
> gerald.bla...@barometerit.com> wrote:
>
> > At a high level, I have a need to be able to execute a query that joins
> > across cores, and that query during its joining may join back to the
> > originating core.
> >
> > Example:
> > Find all Books written by an Author who has written a best selling Book.
> >
> > In Solr query syntax
> > A) against the book core - bestseller:true
> > B) against the author core - {!join fromIndex=book from=id
> > to=bookid}bestseller:true
> > C) against the book core - {!join fromIndex=author from=id
> > to=authorid}{!join fromIndex=book from=id to=bookid}bestseller:true
> >
> > A - returns results
> > B - returns results
> > C - does not return results
> >
> > Given that A and C use the same core, I started looking for join code
> that
> > compares the originating core to the fromIndex and found this
> > in JoinQParserPlugin (line #159).
> >
> > if (info.getReq().getCore() == fromCore) {
> >
> >   // if this is the same core, use the searcher passed in...
> > otherwise we could be warming and
> >
> >   // get an older searcher from the core.
> >
> >   fromSearcher = searcher;
> >
> > } else {
> >
> >   // This could block if there is a static warming query with a
> > join in it, and if useColdSearcher is true.
> >
> >   // Deadlock could result if two cores both had useColdSearcher
> > and had joins that used eachother.
> >
> >   // This would be very predictable though (should happen every
> > time if misconfigured)
> >
> >   fromRef = fromCore.getSearcher(false, true, null);
> >
> >
> >   // be careful not to do anything with this searcher that
> requires
> > the thread local
> >
> >   // SolrRequestInfo in a manner that requires the core in the
> > request to match
> >
> >   fromSearcher = fromRef.get();
> >
> > }
> >
> > I found that if I were to modify the above code so that it always follows
> > the logic in the else block, I get the results I expect.
> >
> > Can someone explain to me why the code is written as it is?  And if we
> were
> > to run with only the else block being executed, what type of adverse
> > impacts we might have?
> >
> > Does anyone have other ideas on how to solve this issue?
> >
> > Thanks in advance.
> > -Gerald
> >
>



-- 

*Gerald Blanck*

baro*m*eter*IT*

1331 Tyler Street NE, Suite 100
Minneapolis, MN 55413


612.208.2802

gerald.bla...@barometerit.com

Re: Nested Join Queries

2012-11-13 Thread Gerald Blanck 
Thank you Mikhail.  Unfortunately BlockJoinQuery is not an option we can
leverage.

- We have modeled our document types as different indexes/cores.
- Our relationships which we are attempting to join across are not
single-parent to many-children relationships.  They are in fact many to
many.
- Additionally, memory usage is a concern.

FYI.  After making the code change I mentioned in my original post, we have
completed a full test cycle and did not experience any adverse impacts to
the change.  And our join query functionality returns the results we
wanted.  I would still be interested in hearing an explanation as to why
the code is written as it is in v4.0.0.

Thanks.




On Tue, Nov 13, 2012 at 8:31 AM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> Please find reference materials
>
>
> http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
> http://blog.griddynamics.com/2012/08/block-join-query-performs.html
>
>
>
>
> On Tue, Nov 13, 2012 at 3:25 PM, Gerald Blanck <
> gerald.bla...@barometerit.com> wrote:
>
>> Thank you.  I've not heard of BlockJoin.  I will look into it today.
>>  Thanks.
>>
>>
>> On Tue, Nov 13, 2012 at 5:05 AM, Mikhail Khludnev <
>> mkhlud...@griddynamics.com> wrote:
>>
>>> Replied. pls check maillist.
>>>
>>>
>>>
>>> On Tue, Nov 13, 2012 at 11:44 AM, Mikhail Khludnev <
>>> mkhlud...@griddynamics.com> wrote:
>>>
>>>> Gerald,
>>>>
>>>> I wonder if you tried to approach BlockJoin for your problem? Can you
>>>> afford less frequent updates?
>>>>
>>>>
>>>> On Wed, Nov 7, 2012 at 5:40 PM, Gerald Blanck <
>>>> gerald.bla...@barometerit.com> wrote:
>>>>
>>>>> Thank you Erick for your reply.  I understand that search is not an
>>>>> RDBMS.
>>>>>  Yes, we do have a huge combinatorial explosion if we de-normalize and
>>>>> duplicate data.  In fact, I believe our use case is exactly what the
>>>>> Solr
>>>>> developers were trying to solve with the addition of the Join query.
>>>>>  And
>>>>> while the example I gave illustrates the problem we are solving with
>>>>> the
>>>>> Join functionality, it is simplistic in nature compared to what we
>>>>> have in
>>>>> actuality.
>>>>>
>>>>> Am still looking for an answer here if someone can shed some light.
>>>>>  Thanks.
>>>>>
>>>>>
>>>>> On Sat, Nov 3, 2012 at 9:38 PM, Erick Erickson <
>>>>> erickerick...@gmail.com>wrote:
>>>>>
>>>>> > I'm going to go a bit sideways on you, partly because I can't answer
>>>>> the
>>>>> > question ...
>>>>> >
>>>>> > But, every time I see someone doing what looks like substituting
>>>>> "core" for
>>>>> > "table" and
>>>>> > then trying to use Solr like a DB, I get on my soap-box and
>>>>> preach..
>>>>> >
>>>>> > In this case, consider de-normalizing your DB so you can ask the
>>>>> query in
>>>>> > terms
>>>>> > of search rather than joins. e.g.
>>>>> >
>>>>> > Make each document a combination of the author and the book, with an
>>>>> > additional
>>>>> > field "author_has_written_a_bestseller". Now your query becomes a
>>>>> really
>>>>> > simple
>>>>> > search, "author:name AND author_has_written_a_bestseller:true".
>>>>> True, this
>>>>> > kind
>>>>> > of approach isn't as flexible as an RDBMS, but it's a _search_
>>>>> rather than
>>>>> > a query.
>>>>> > Yes, it replicates data, but unless you have a huge combinatorial
>>>>> > explosion, that's
>>>>> > not a problem.
>>>>> >
>>>>> > And the join functionality isn't called "pseudo" for nothing. It was
>>>>> > written for a specific
>>>>> > use-case. It is often expensive, especially when the field being
>>>>> joined has
>>>>> > many unique
>>>>> > values.
>>>>> >
>>>>> > FWIW,
>>>>&g

Re: Nested Join Queries

2012-11-14 Thread Gerald Blanck 
Mikhail-

Let me know how to contribute a test case and I will put it on my to do
list.

When your many-to-many BlockJoin solution matures I would love to see it.

Thanks.
-Gerald


On Tue, Nov 13, 2012 at 11:52 PM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:

> Gerald,
> Nice to hear the the your problem is solved. Can you contribute a test
> case to reproduce this issue?
>
> FWIW, my team successfully deals with Many-to-Many in BlockJoin. It works,
> but solution is a little bit immature yet.
>
>
>
> On Wed, Nov 14, 2012 at 5:59 AM, Gerald Blanck <
> gerald.bla...@barometerit.com> wrote:
>
>> Thank you Mikhail.  Unfortunately BlockJoinQuery is not an option we can
>> leverage.
>>
>> - We have modeled our document types as different indexes/cores.
>> - Our relationships which we are attempting to join across are not
>> single-parent to many-children relationships.  They are in fact many to
>> many.
>> - Additionally, memory usage is a concern.
>>
>> FYI.  After making the code change I mentioned in my original post, we
>> have completed a full test cycle and did not experience any adverse impacts
>> to the change.  And our join query functionality returns the results we
>> wanted.  I would still be interested in hearing an explanation as to why
>> the code is written as it is in v4.0.0.
>>
>> Thanks.
>>
>>
>>
>>
>> On Tue, Nov 13, 2012 at 8:31 AM, Mikhail Khludnev <
>> mkhlud...@griddynamics.com> wrote:
>>
>>> Please find reference materials
>>>
>>>
>>> http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
>>> http://blog.griddynamics.com/2012/08/block-join-query-performs.html
>>>
>>>
>>>
>>>
>>> On Tue, Nov 13, 2012 at 3:25 PM, Gerald Blanck <
>>> gerald.bla...@barometerit.com> wrote:
>>>
>>>> Thank you.  I've not heard of BlockJoin.  I will look into it today.
>>>>  Thanks.
>>>>
>>>>
>>>> On Tue, Nov 13, 2012 at 5:05 AM, Mikhail Khludnev <
>>>> mkhlud...@griddynamics.com> wrote:
>>>>
>>>>> Replied. pls check maillist.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 13, 2012 at 11:44 AM, Mikhail Khludnev <
>>>>> mkhlud...@griddynamics.com> wrote:
>>>>>
>>>>>> Gerald,
>>>>>>
>>>>>> I wonder if you tried to approach BlockJoin for your problem? Can you
>>>>>> afford less frequent updates?
>>>>>>
>>>>>>
>>>>>> On Wed, Nov 7, 2012 at 5:40 PM, Gerald Blanck <
>>>>>> gerald.bla...@barometerit.com> wrote:
>>>>>>
>>>>>>> Thank you Erick for your reply.  I understand that search is not an
>>>>>>> RDBMS.
>>>>>>>  Yes, we do have a huge combinatorial explosion if we de-normalize
>>>>>>> and
>>>>>>> duplicate data.  In fact, I believe our use case is exactly what the
>>>>>>> Solr
>>>>>>> developers were trying to solve with the addition of the Join query.
>>>>>>>  And
>>>>>>> while the example I gave illustrates the problem we are solving with
>>>>>>> the
>>>>>>> Join functionality, it is simplistic in nature compared to what we
>>>>>>> have in
>>>>>>> actuality.
>>>>>>>
>>>>>>> Am still looking for an answer here if someone can shed some light.
>>>>>>>  Thanks.
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Nov 3, 2012 at 9:38 PM, Erick Erickson <
>>>>>>> erickerick...@gmail.com>wrote:
>>>>>>>
>>>>>>> > I'm going to go a bit sideways on you, partly because I can't
>>>>>>> answer the
>>>>>>> > question ...
>>>>>>> >
>>>>>>> > But, every time I see someone doing what looks like substituting
>>>>>>> "core" for
>>>>>>> > "table" and
>>>>>>> > then trying to use Solr like a DB, I get on my soap-box and
>>>>>>> preach..
>>>>>>> >
>>>>>>> > In this case, consider de-normalizing your DB so you can ask the
>>>>&g

FACET_SORT_INDEX descending?

2009-09-25 Thread Gerald Snyder 
Is there any value for the "f.my_year_facet.facet.sort"  parameter that 
will return the facet values in descending order?   So far I only see 
"index" and "count" as the choices. 


http://lucene.apache.org/solr/api/org/apache/solr/common/params/FacetParams.html#FACET_SORT_INDEX

Thanks.
Gerald Snyder 
Florida Center for Library Automation

Re: FACET_SORT_INDEX descending?

2009-10-06 Thread Gerald Snyder 
Reverse alphabetical ordering.   The option "index" provides 
alphabetical ordering. 

I have a year_facet field, that I would like to display in reverse order 
(most recent years first).  Perhaps there is some other way to 
accomplish this.


Thanks.
--Gerald


Chris Hostetter wrote:

: Is there any value for the "f.my_year_facet.facet.sort"  parameter that will
: return the facet values in descending order?   So far I only see "index" and
: "count" as the choices. 


descending what?

(count is descending order by count)



-Hoss

Re: FACET_SORT_INDEX descending?

2009-10-14 Thread Gerald Snyder 

Thanks for the answer and the alternative idea.--Gerald


Chris Hostetter wrote:

: Reverse alphabetical ordering.   The option "index" provides alphabetical
: ordering. 

be careful: "index" doesn't mean "alphabetical" -- it means the natural 
ordering of terms as they exist in the index. for non ascii characters 
this is not neccessarily something that could be considered alphabetical 
(or sensical in terms of the locale).


The short answer is: no, there is no way to get "reverse" index order at 
the moment.


: I have a year_facet field, that I would like to display in reverse order (most
: recent years first).  Perhaps there is some other way to accomplish this.

the simplest way is to encode the year in some format thta will cause it 
to naturally sort in the order you want - so instead of indexing "1976" 
and "2007" you could index "8024:1976" and "7993:2007" and then only 
display the part that comes after the ":"




-Hoss

Re: "begins with" searches

2009-10-26 Thread Gerald Snyder 
Are you using the field name suffixes like Blacklight?xxx_text, 
_xxx_facet, xxx_string?   With the xxx_string field you can request 
"begins with" search, but you may need some different search term 
normalization than with a _text search. 



Gerald Snyder
Florida Center for Library Automation


Bernadette Houghton wrote:

We need to offer "begins with" type searches, e.g. a search for "surname, f" will retrieve "surname, 
firstname", "surname, f", "surname fm" etc.

Ideally, the user would be able to enter something like "surname f*".

However, wildcards don't work on phrase searches, nor do range searches.

Any suggestions as to how best to search for "begins with" phrases; or, how to 
best configure solr to support such searches?

TIA
Bernadette Houghton, Library Business Applications Developer
Deakin University Geelong Victoria 3217 Australia.
Phone: 03 5227 8230 International: +61 3 5227 8230
Fax: 03 5227 8000 International: +61 3 5227 8000
MSN: bern_hough...@hotmail.com
Email: 
bernadette.hough...@deakin.edu.au<mailto:bernadette.hough...@deakin.edu.au>
Website: http://www.deakin.edu.au
<http://www.deakin.edu.au/>Deakin University CRICOS Provider Code 00113B (Vic)

Important Notice: The contents of this email are intended solely for the named 
addressee and are confidential; any unauthorised use, reproduction or storage 
of the contents is expressly prohibited. If you have received this email in 
error, please delete it and any attachments immediately and advise the sender 
by return email or telephone.
Deakin University does not warrant that this email and any attachments are 
error or virus free

ClassCastException in SolrJ 7.6+

2019-03-11 Thread Gerald Bonfiglio 
I'm seeing the following Exception using JSON Facet API in SolrJ 7.6, 7.7, 
7.7.1:

Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to 
java.lang.Integer
  at 
org.apache.solr.client.solrj.response.json.NestableJsonFacet.(NestableJsonFacet.java:52)
  at 
org.apache.solr.client.solrj.response.QueryResponse.extractJsonFacetingInfo(QueryResponse.java:200)
  at 
org.apache.solr.client.solrj.response.QueryResponse.getJsonFacetingResponse(QueryResponse.java:571)





[Nastel  Technologies]

The information contained in this e-mail and in any attachment is confidential 
and
is intended solely for the use of the individual or entity to which it is 
addressed.
Access, copying, disclosure or use of such information by anyone else is 
unauthorized.
If you are not the intended recipient, please delete the e-mail and refrain 
from use of such information.

RE: ClassCastException in SolrJ 7.6+

2019-03-11 Thread Gerald Bonfiglio 
public static void main(String[] args) {
try {
SolrQuery solrQuery = new SolrQuery("*:*");
solrQuery.setRows(0);
solrQuery.setParam("json.facet", 
"{grp_0:{field:evntnm,limit:-1,type:terms,mincount:1,sort:{index:asc}}}");

List solrHosts = new ArrayList(1);
solrHosts.add("http://localhost:8983/solr";);
CloudSolrClient solrServer = new 
CloudSolrClient.Builder(solrHosts).build();
solrServer.setIdField("_uniqueKey");

QueryResponse solrResult = solrServer.query("events", solrQuery);
NestableJsonFacet jsonFacets = solrResult.getJsonFacetingResponse();
System.out.println(jsonFacets.toString());
}
catch (Throwable e) {
e.printStackTrace();
}
}

-Original Message-
From: Jason Gerlowski [mailto:gerlowsk...@gmail.com]
Sent: Monday, March 11, 2019 1:24 PM
To: solr-user@lucene.apache.org
Subject: Re: ClassCastException in SolrJ 7.6+

Hi Gerald,

That looks like it might be a bug in SolrJ's JSON faceting support.
Do you have a small code snippet that reproduces the problem?  That'll
help us confirm it's a bug, and get us started on fixing it.

Best,

Jason

On Mon, Mar 11, 2019 at 10:29 AM Gerald Bonfiglio  wrote:
>
> I'm seeing the following Exception using JSON Facet API in SolrJ 7.6, 7.7, 
> 7.7.1:
>
> Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to 
> java.lang.Integer
>   at 
> org.apache.solr.client.solrj.response.json.NestableJsonFacet.(NestableJsonFacet.java:52)
>   at 
> org.apache.solr.client.solrj.response.QueryResponse.extractJsonFacetingInfo(QueryResponse.java:200)
>   at 
> org.apache.solr.client.solrj.response.QueryResponse.getJsonFacetingResponse(QueryResponse.java:571)
>
>
>
>
>
> [Nastel  Technologies]<http://www.nastel.com/>
>
> The information contained in this e-mail and in any attachment is 
> confidential and
> is intended solely for the use of the individual or entity to which it is 
> addressed.
> Access, copying, disclosure or use of such information by anyone else is 
> unauthorized.
> If you are not the intended recipient, please delete the e-mail and refrain 
> from use of such information.




[Nastel  Technologies]<http://www.nastel.com/>

The information contained in this e-mail and in any attachment is confidential 
and
is intended solely for the use of the individual or entity to which it is 
addressed.
Access, copying, disclosure or use of such information by anyone else is 
unauthorized.
If you are not the intended recipient, please delete the e-mail and refrain 
from use of such information.

NPE deleting expired docs (SOLR-13281)

2019-03-12 Thread Gerald Bonfiglio 
Has anyone else observed NPEs attempting to have expired docs removed?  I'm 
seeing the following exceptions:

2019-02-28 04:06:34.849 ERROR (autoExpireDocs-30-thread-1) [ ] 
o.a.s.u.p.DocExpirationUpdateProcessorFactory Runtime error in periodic 
deletion of expired docs: null
java.lang.NullPointerException: null
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.handleReplicationFactor(DistributedUpdateProcessor.java:992)
 ~[solr-core-7.7.0.jar:7.7.0 8c831daf4eb41153c25ddb152501ab5bae3ea3d5 - jimczi 
- 2019-02-04 23:23:46]
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:960)
 ~[solr-core-7.7.0.jar:7.7.0 8c831daf4eb41153c25ddb152501ab5bae3ea3d5 - jimczi 
- 2019-02-04 23:23:46]

Seems all that's required to reproduce it is to include 
DocExpirationUpdateProcessorFactory in an updateRequestProcessorChain.

More details can be found at: 
https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13281





[Nastel  Technologies]

The information contained in this e-mail and in any attachment is confidential 
and
is intended solely for the use of the individual or entity to which it is 
addressed.
Access, copying, disclosure or use of such information by anyone else is 
unauthorized.
If you are not the intended recipient, please delete the e-mail and refrain 
from use of such information.

RE: Duplicate values in Multi Value Fields

2019-03-14 Thread Gerald Bonfiglio 
I've used this before, by specifying the chain as the default processor chain 
by putting the following directly under the  entry:

  

  uniq-fields

  

Not sure if this is the best way, but since our app is the only one using Solr, 
we want every update to use the chain across all our collections.

-Original Message-
From: Alexis Aravena Silva [mailto:aarav...@itsofteg.com]
Sent: Thursday, March 14, 2019 11:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Duplicate values in Multi Value Fields

Does anyone know how to config this in solrconfig?, the idea is that solr uses 
it when I execute the data import:





  _nombreArea_


  




  uniq-fields

  






From: Alexis Aravena Silva
Sent: Thursday, March 14, 2019 11:26:07 AM
To: solr-user@lucene.apache.org
Subject: Re: Duplicate values in Multi Value Fields


I've tried with the following, but it doesn't work, it seems like solr doesn't 
take the configuration:





  _nombreArea_


  




  uniq-fields

  




From: MUNENDRA S.N 
Sent: Thursday, March 14, 2019 11:17:40 AM
To: solr-user@lucene.apache.org
Subject: Re: Duplicate values in Multi Value Fields

Probably you could add-distinct operation for unique values in multivalued
fields

https://lucene.apache.org/solr/guide/7_3/updating-parts-of-documents.html

On Thu, Mar 14, 2019, 7:40 PM Jörn Franke  wrote:

> With an update request processor
>
> https://lucene.apache.org/solr/7_4_0//solr-core/org/apache/solr/update/processor/UniqFieldsUpdateProcessorFactory.html
>
> > Am 14.03.2019 um 15:01 schrieb Alexis Aravena Silva <
> aarav...@itsofteg.com>:
> >
> > Hello,
> >
> >
> > I'm indexing data into some MultiValueFields, but  I have duplicates,
> how can I remove the duplicate values at indexing time?
> >
> >
> > I'm using Solr 7.
> >
> >
> > sample:
> >
> >
> > _nombreArea_":["MICROBIOLOGÍA", "QUÍMICA", "MICROBIOLOGÍA",
> "MICROBIOLOGÍA", "MICROBIOLOGÍA", "QUÍMICA", "QUÍMICA", "MICROBIOLOGÍA",
> "MICROBIOLOGÍA", "QUÍMICA", "MICROBIOLOGÍA", "QUÍMICA"],
> >
> >
> > Regards,
> >
> > Alexis Aravena S.
> >
> >
>




[Nastel  Technologies]

The information contained in this e-mail and in any attachment is confidential 
and
is intended solely for the use of the individual or entity to which it is 
addressed.
Access, copying, disclosure or use of such information by anyone else is 
unauthorized.
If you are not the intended recipient, please delete the e-mail and refrain 
from use of such information.

newbie question on SOLR distributed searches with many "shards"

2008-11-25 Thread Gerald De Conto 
I wasn't able to find examples/anything via google so thought I'd ask:

 

Say I want to implement a solution using distributed searches with many
"shards" in SOLR 1.3.0. Also, say there are too many shards to pass in
via the URL (dozens, hundreds, whatever)

 

Is there a way to specify in solrconfig.xml (or elsewhere) a list of the
shard URLs to use?

 

I saw references to a shards.txt but no info on it.  I also saw bits of
info that suggested that there MIGHT be another way to do this.

 

Any info appreciated on doing this sort of distributed search.

 

thx

RE: newbie question on SOLR distributed searches with many "shards"

2008-11-26 Thread Gerald De Conto 
Thx Paul

How would I do so? i.e. would you be able to provide a simple example of 
putting shard info into the SearchHandlers defaults or invariants section?

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Noble Paul 
??? ??
Sent: Tuesday, November 25, 2008 8:29 PM
To: solr-user@lucene.apache.org
Subject: Re: newbie question on SOLR distributed searches with many "shards"

anything that is passed as a request parameter can be put into the
SearchHandlers defaults or invariants section .


This is equivalent to passing the shard url in the request

However this expects that you may need to setup a loadbalancer if a
shard hhos more than one host


On Wed, Nov 26, 2008 at 12:25 AM, Gerald De Conto
<[EMAIL PROTECTED]> wrote:
> I wasn't able to find examples/anything via google so thought I'd ask:
>
>
>
> Say I want to implement a solution using distributed searches with many
> "shards" in SOLR 1.3.0. Also, say there are too many shards to pass in
> via the URL (dozens, hundreds, whatever)
>
>
>
> Is there a way to specify in solrconfig.xml (or elsewhere) a list of the
> shard URLs to use?
>
>
>
> I saw references to a shards.txt but no info on it.  I also saw bits of
> info that suggested that there MIGHT be another way to do this.
>
>
>
> Any info appreciated on doing this sort of distributed search.
>
>
>
> thx
>
>



-- 
--Noble Paul