How can I convert xml message for updating a Solr index to a javabin file
Hi all, Is there a way I can covert a xml Solr update message file to javabin file? If so, How? How can I use curl to update Solr by javabin message file? Thank you very much.
RE: How can I convert xml message for updating a Solr index to a javabin file
I want to measure xml vs javabin update message indexing performance. -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Thursday, April 24, 2014 2:04 PM To: solr-user@lucene.apache.org Subject: Re: How can I convert xml message for updating a Solr index to a javabin file Why would you want to do this? Javabin is used by SolrJ to communicate with Solr. XML is good enough for communicating from the command line/curl, as is JSON. Attempting to use javabin just seems to add an unnecessary complication. Upayavifra On Thu, Apr 24, 2014, at 10:20 AM, Elran Dvir wrote: > Hi all, > Is there a way I can covert a xml Solr update message file to javabin > file? If so, How? > How can I use curl to update Solr by javabin message file? > > Thank you very much. Email secured by Check Point
RE: How can I convert xml message for updating a Solr index to a javabin file
Does anyone know a way to do this? Thanks. -Original Message- From: Elran Dvir Sent: Thursday, April 24, 2014 4:11 PM To: solr-user@lucene.apache.org Subject: RE: How can I convert xml message for updating a Solr index to a javabin file I want to measure xml vs javabin update message indexing performance. -Original Message- From: Upayavira [mailto:u...@odoko.co.uk] Sent: Thursday, April 24, 2014 2:04 PM To: solr-user@lucene.apache.org Subject: Re: How can I convert xml message for updating a Solr index to a javabin file Why would you want to do this? Javabin is used by SolrJ to communicate with Solr. XML is good enough for communicating from the command line/curl, as is JSON. Attempting to use javabin just seems to add an unnecessary complication. Upayavifra On Thu, Apr 24, 2014, at 10:20 AM, Elran Dvir wrote: > Hi all, > Is there a way I can covert a xml Solr update message file to javabin > file? If so, How? > How can I use curl to update Solr by javabin message file? > > Thank you very much. Email secured by Check Point
Is there a way to change transientCacheSize dynamically without restarting Solr
Hi All, Is there an API in Solr to change transientCacheSize dynamically without the need to restart Solr? Is there other Solr configuration parameters that can be changed dynamically? Thanks.
Reloading core with param vs unloading and creating with params
Hi all, I created a new patch https://issues.apache.org/jira/browse/SOLR-6063 , enabling changes in core properties without the need to unload and create it. Considering the change in patch, is reloading a core with transient=true and loadOnStartup=false equivalent in memory footprint to unloading the core and creating it with the same parameters? Thanks.
RE: Is there a way to change transientCacheSize dynamically without restarting Solr
Does anyone have an answer for this? Thanks. From: Elran Dvir Sent: Thursday, May 15, 2014 12:32 PM To: solr-user@lucene.apache.org Subject: Is there a way to change transientCacheSize dynamically without restarting Solr Hi All, Is there an API in Solr to change transientCacheSize dynamically without the need to restart Solr? Are there other Solr configuration parameters that can be changed dynamically? Thanks.
RE: Reloading core with param vs unloading and creating with params
Hi, Did anyone have a chance to take a look at this issue? Thanks. From: Elran Dvir Sent: Thursday, May 15, 2014 12:22 PM To: solr-user@lucene.apache.org Subject: Reloading core with param vs unloading and creating with params Hi all, I created a new patch https://issues.apache.org/jira/browse/SOLR-6063 , enabling changes in core properties without the need to unload and create it. Considering the change in patch, is reloading a core with transient=true and loadOnStartup=false equivalent in memory footprint to unloading the core and creating it with the same parameters? Thanks.
RE: Is there a way to change transientCacheSize dynamically without restarting Solr
Thank you, Shalin. -Original Message- From: Shalin Shekhar Mangar [mailto:shalinman...@gmail.com] Sent: Monday, May 19, 2014 10:18 AM To: solr-user@lucene.apache.org Subject: Re: Is there a way to change transientCacheSize dynamically without restarting Solr Hi Elran, No, I don't think there is way to do that right now. On Mon, May 19, 2014 at 12:18 PM, Elran Dvir wrote: > Does anyone have an answer for this? > Thanks. > > From: Elran Dvir > Sent: Thursday, May 15, 2014 12:32 PM > To: solr-user@lucene.apache.org > Subject: Is there a way to change transientCacheSize dynamically > without restarting Solr > > Hi All, > > Is there an API in Solr to change transientCacheSize dynamically > without the need to restart Solr? > Are there other Solr configuration parameters that can be changed > dynamically? > > Thanks. > -- Regards, Shalin Shekhar Mangar. Email secured by Check Point
wildcard matches in EnumField - what do I need to change in code to enable wildcard matches?
Hi all, In my index, I have an EnumField called severity. This is its configuration in enumsConfig.xml: Not Available Low Medium High Critical My index contains documents with these values. When I search for severity:High, I get results. But when I search for severity:H* , I get no results. What do I need to change in Solr code to enable wildcard matches in EnumField (or any other field)? Thanks.
change in EnumField configuration - what do you think?
Hi all, I am the one that contributed EnumField code to Solr. There was a long discussion how the integer values of an enum field should be indicated in the configuration. It was decided that the integer value wouldn't be written explicitly, but would be implicitly determined by the value order. For example: Not Available Low Medium High Critical "Not Available" will get the value 0, "Low" will get 1 and so on. I have a use case where this configuration does not meet my needs. I have an external system that indexes the data. One field has a closed set of values, the values are sent as integer but should be represented as String. EnumField is perfect for it. The problem is that the values are predefined and not order sequentially (the values were "spaced" for future growth). It looks like this: So I think we have to be able to indicate the integer values of an EnumField in the configuration. What do you think? Thanks.
RE: change in EnumField configuration - what do you think?
Thank you for your quick responses.
The numeric values are determined by external system which I don't have any
control of. There is a gap between values to reserve holes that may be filled
in later in the future.
The values are sent as integers. In Solr, there is no need to retrieve the
numeric values or query them. However, The numeric values determine the sorting
order of values.
In order to support both forms and for backward compatibility, let's introduce
a new attribute called explicit, which by default is false. This attribute will
be used to know which form to expect.
For example:
With explicit="false" ("explicit" can be omitted because it's the default):
Not Available
Low
Medium
High
Critical
With explicit="true":
Do you agree?
Thanks.
-Original Message-
From: Jack Krupansky [mailto:j...@basetechnology.com]
Sent: Monday, June 02, 2014 6:17 PM
To: solr-user@lucene.apache.org
Subject: Re: change in EnumField configuration - what do you think?
Do these numeric values have any significance to the application, or are they
merely to reserve holes that will be later filled in without reindexing
existing documents? I mean, there is no API to retrieve the numeric values or
query them, right? IOW, they are not like stored values or docvalues, right?
-- Jack Krupansky
-Original Message-
From: Erick Erickson
Sent: Monday, June 2, 2014 10:46 AM
To: solr-user@lucene.apache.org
Subject: Re: change in EnumField configuration - what do you think?
Would both then be supported? I see where it would be easily detectable.
And I also assume that this wouldn't break back-compat?
Best
Erick
On Mon, Jun 2, 2014 at 6:22 AM, Elran Dvir wrote:
> Hi all,
>
> I am the one that contributed EnumField code to Solr.
> There was a long discussion how the integer values of an enum field
> should be indicated in the configuration.
> It was decided that the integer value wouldn't be written explicitly,
> but would be implicitly determined by the value order.
> For example:
>
> Not Available
> Low
> Medium
> High
> Critical
>
> "Not Available" will get the value 0, "Low" will get 1 and so on.
>
> I have a use case where this configuration does not meet my needs.
> I have an external system that indexes the data.
> One field has a closed set of values, the values are sent as integer
> but should be represented as String. EnumField is perfect for it.
> The problem is that the values are predefined and not order
> sequentially (the values were "spaced" for future growth). It looks like this:
>
>
>
>
>
>
>
>
> So I think we have to be able to indicate the integer values of an
> EnumField in the configuration.
> What do you think?
>
> Thanks.
>
Email secured by Check Point
copy EnumField to text field
Hi all, I have an enumField called severity. these are its relevant definitions in schema.xml: And in enumsConfig.xml: Not Available Low Medium High Critical The default field for free text search is text. An enum field can be sent with its integer value or with its string value, and the value will stored and indexed as integer and displayed as string. When severity is sent with "Not Available", there will be matches for the free text search of "Not Available". When severity is sent with "0" (the integer equivalent of " Not Available"), there will be no matches for the free text search of "Not Available". In order to enable it, the following change should be made in DocumentBuilder: Instead of: // Perhaps trim the length of a copy field Object val = v; The code will be: // Perhaps trim the length of a copy field Object val = sfield.getType().toExternal(sfield.createField(v, 1.0f)); Am I right? It seems to work. I think this change is suitable for all field types. What do you think? But when no value is sent with severity, and the default of 0 is used, the fix doesn't seem to work. How can I make it work also for default values? Thanks.
RE: copy EnumField to text field
Are you saying that default values are for query and not for indexing? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Monday, July 28, 2014 9:38 AM To: solr-user Subject: Re: copy EnumField to text field On Mon, Jul 28, 2014 at 1:31 PM, Elran Dvir wrote: > But when no value is sent with severity, and the default of 0 is used, the > fix doesn't seem to work. I guess the default in this case is figured out at the query time because there is no empty value as such. So that would be too late for copyField. If I am right, then you could probably use UpdateRequestProcessor and set the default value explicitly (DefaultValueUpdateProcessorFactory). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 Email secured by Check Point
RE: copy EnumField to text field
So if I have a document without severity, I can't see severity has its default value (0) in the stage of copy fields (in class DocumentBuilder)? Thanks. -Original Message- From: Jack Krupansky [mailto:j...@basetechnology.com] Sent: Monday, July 28, 2014 2:39 PM To: solr-user@lucene.apache.org Subject: Re: copy EnumField to text field There is a distinction between the original source input value for the indexing process and what value is actually indexed. Query searching will see whatever is actually indexed, not the original source input value. An URP could explicitly set the source input value to the default value if it is missing, but you would have to specify an explicit value for the URP to use. -- Jack Krupansky -Original Message- From: Elran Dvir Sent: Monday, July 28, 2014 4:12 AM To: solr-user@lucene.apache.org Subject: RE: copy EnumField to text field Are you saying that default values are for query and not for indexing? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Monday, July 28, 2014 9:38 AM To: solr-user Subject: Re: copy EnumField to text field On Mon, Jul 28, 2014 at 1:31 PM, Elran Dvir wrote: > But when no value is sent with severity, and the default of 0 is used, > the fix doesn't seem to work. I guess the default in this case is figured out at the query time because there is no empty value as such. So that would be too late for copyField. If I am right, then you could probably use UpdateRequestProcessor and set the default value explicitly (DefaultValueUpdateProcessorFactory). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 Email secured by Check Point Email secured by Check Point
RE: copy EnumField to text field
Do you think that I that the change I suggested In DocumentBuilder is right or should we leave it as it? The change: Instead of: // Perhaps trim the length of a copy field Object val = v; The code will be: // Perhaps trim the length of a copy field Object val = sfield.getType().toExternal(sfield.createField(v, 1.0f)); Thanks. -Original Message- From: Jack Krupansky [mailto:j...@basetechnology.com] Sent: Monday, July 28, 2014 3:24 PM To: solr-user@lucene.apache.org Subject: Re: copy EnumField to text field Correct - copy field copies the raw, original, source input value, before the actual field type has had a chance to process it in any way. -- Jack Krupansky -Original Message- From: Elran Dvir Sent: Monday, July 28, 2014 8:08 AM To: solr-user@lucene.apache.org Subject: RE: copy EnumField to text field So if I have a document without severity, I can't see severity has its default value (0) in the stage of copy fields (in class DocumentBuilder)? Thanks. -Original Message- From: Jack Krupansky [mailto:j...@basetechnology.com] Sent: Monday, July 28, 2014 2:39 PM To: solr-user@lucene.apache.org Subject: Re: copy EnumField to text field There is a distinction between the original source input value for the indexing process and what value is actually indexed. Query searching will see whatever is actually indexed, not the original source input value. An URP could explicitly set the source input value to the default value if it is missing, but you would have to specify an explicit value for the URP to use. -- Jack Krupansky -Original Message- From: Elran Dvir Sent: Monday, July 28, 2014 4:12 AM To: solr-user@lucene.apache.org Subject: RE: copy EnumField to text field Are you saying that default values are for query and not for indexing? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Monday, July 28, 2014 9:38 AM To: solr-user Subject: Re: copy EnumField to text field On Mon, Jul 28, 2014 at 1:31 PM, Elran Dvir wrote: > But when no value is sent with severity, and the default of 0 is used, > the fix doesn't seem to work. I guess the default in this case is figured out at the query time because there is no empty value as such. So that would be too late for copyField. If I am right, then you could probably use UpdateRequestProcessor and set the default value explicitly (DefaultValueUpdateProcessorFactory). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 Email secured by Check Point Email secured by Check Point Email secured by Check Point
RE: copy EnumField to text field
Hi all, I got an answer about default value. But what about the code change I suggested? Do you think it's good? For your convenience, I am rewriting my original message: I have an enumField called severity. these are its relevant definitions in schema.xml: And in enumsConfig.xml: Not Available Low Medium High Critical The default field for free text search is text. An enum field can be sent with its integer value or with its string value, and the value will stored and indexed as integer and displayed as string. When severity is sent with "Not Available", there will be matches for the free text search of "Not Available". When severity is sent with "0" (the integer equivalent of " Not Available"), there will be no matches for the free text search of "Not Available". In order to enable it, the following change should be made in DocumentBuilder: Instead of: // Perhaps trim the length of a copy field Object val = v; The code will be: // Perhaps trim the length of a copy field Object val = sfield.getType().toExternal(sfield.createField(v, 1.0f)); Am I right? It seems to work. I think this change is suitable for all field types. What do you think? But when no value is sent with severity, and the default of 0 is used, the fix doesn't seem to work. How can I make it work also for default values? Thanks. -Original Message- From: Elran Dvir Sent: Monday, July 28, 2014 3:49 PM To: solr-user@lucene.apache.org Subject: RE: copy EnumField to text field Do you think that I that the change I suggested In DocumentBuilder is right or should we leave it as it? The change: Instead of: // Perhaps trim the length of a copy field Object val = v; The code will be: // Perhaps trim the length of a copy field Object val = sfield.getType().toExternal(sfield.createField(v, 1.0f)); Thanks. -Original Message- From: Jack Krupansky [mailto:j...@basetechnology.com] Sent: Monday, July 28, 2014 3:24 PM To: solr-user@lucene.apache.org Subject: Re: copy EnumField to text field Correct - copy field copies the raw, original, source input value, before the actual field type has had a chance to process it in any way. -- Jack Krupansky -Original Message- From: Elran Dvir Sent: Monday, July 28, 2014 8:08 AM To: solr-user@lucene.apache.org Subject: RE: copy EnumField to text field So if I have a document without severity, I can't see severity has its default value (0) in the stage of copy fields (in class DocumentBuilder)? Thanks. -Original Message- From: Jack Krupansky [mailto:j...@basetechnology.com] Sent: Monday, July 28, 2014 2:39 PM To: solr-user@lucene.apache.org Subject: Re: copy EnumField to text field There is a distinction between the original source input value for the indexing process and what value is actually indexed. Query searching will see whatever is actually indexed, not the original source input value. An URP could explicitly set the source input value to the default value if it is missing, but you would have to specify an explicit value for the URP to use. -- Jack Krupansky -Original Message- From: Elran Dvir Sent: Monday, July 28, 2014 4:12 AM To: solr-user@lucene.apache.org Subject: RE: copy EnumField to text field Are you saying that default values are for query and not for indexing? Thanks. -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Monday, July 28, 2014 9:38 AM To: solr-user Subject: Re: copy EnumField to text field On Mon, Jul 28, 2014 at 1:31 PM, Elran Dvir wrote: > But when no value is sent with severity, and the default of 0 is used, > the fix doesn't seem to work. I guess the default in this case is figured out at the query time because there is no empty value as such. So that would be too late for copyField. If I am right, then you could probably use UpdateRequestProcessor and set the default value explicitly (DefaultValueUpdateProcessorFactory). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 Email secured by Check Point Email secured by Check Point Email secured by Check Point
committer attention of SOLR-5972
Hi all, I created a patch for new statistics facet capabilities to StatsComponent facet - limit, sort and missing. This can be found here: https://issues.apache.org/jira/browse/SOLR-5972 Can it have committer attention? Thanks.
distributed search is significantly slower than direct search
Hi all, I am using Solr 4.4 with multi cores. One core (called template) is my "routing" core. When I run http://127.0.0.1:8983/solr/template/select?rows=5000&q=*:*&shards=127.0.0.1:8983/solr/core1, it consistently takes about 7s. When I run http://127.0.0.1:8983/solr/core1/select?rows=5000&q=*:*, it consistently takes about 40ms. I profiled the distributed query. This is the distributed query process (I hope the terms are accurate): When solr identifies a distributed query, it sends the query to the shard and get matched shard docs. Then it sends another query to the shard to get the Solr documents. Most time is spent in the last stage in the function "process" of "QueryComponent" in: for (int i=0; i
RE: distributed search is significantly slower than direct search
Erick, Thanks for your response. We are upgrading our system using Solr. We need to preserve old functionality. Our client displays 5K document and groups them. Is there a way to refactor code in order to improve distributed documents fetching? Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Wednesday, October 30, 2013 3:17 AM To: solr-user@lucene.apache.org Subject: Re: distributed search is significantly slower than direct search You can't. There will inevitably be some overhead in the distributed case. That said, 7 seconds is quite long. 5,000 rows is excessive, and probably where your issue is. You're having to go out and fetch the docs across the wire. Perhaps there is some batching that could be done there, I don't know whether this is one document per request or not. Why 5K docs? Best, Erick On Tue, Oct 29, 2013 at 2:54 AM, Elran Dvir wrote: > Hi all, > > I am using Solr 4.4 with multi cores. One core (called template) is my > "routing" core. > > When I run > http://127.0.0.1:8983/solr/template/select?rows=5000&q=*:*&shards=127. > 0.0.1:8983/solr/core1, > it consistently takes about 7s. > When I run http://127.0.0.1:8983/solr/core1/select?rows=5000&q=*:*, it > consistently takes about 40ms. > > I profiled the distributed query. > This is the distributed query process (I hope the terms are accurate): > When solr identifies a distributed query, it sends the query to the > shard and get matched shard docs. > Then it sends another query to the shard to get the Solr documents. > Most time is spent in the last stage in the function "process" of > "QueryComponent" in: > > for (int i=0; i int id = req.getSearcher().getFirstMatch( > new Term(idField.getName(), > idField.getType().toInternal(idArr.get(i; > > How can I make my distributed query as fast as the direct one? > > Thanks. > Email secured by Check Point
RE: distributed search is significantly slower than direct search
he local and remote case, but that's a stab in the dark. > > Not much help I know, > Erick > > > > On Wed, Nov 13, 2013 at 2:52 AM, Elran Dvir wrote: > > > Erick, Thanks for your response. > > > > We are upgrading our system using Solr. > > We need to preserve old functionality. Our client displays 5K > > document and groups them. > > > > Is there a way to refactor code in order to improve distributed > > documents fetching? > > > > Thanks. > > > > -Original Message- > > From: Erick Erickson [mailto:erickerick...@gmail.com] > > Sent: Wednesday, October 30, 2013 3:17 AM > > To: solr-user@lucene.apache.org > > Subject: Re: distributed search is significantly slower than direct > search > > > > You can't. There will inevitably be some overhead in the distributed > case. > > That said, 7 seconds is quite long. > > > > 5,000 rows is excessive, and probably where your issue is. You're > > having to go out and fetch the docs across the wire. Perhaps there > > is some batching that could be done there, I don't know whether this > > is one document per request or not. > > > > Why 5K docs? > > > > Best, > > Erick > > > > > > On Tue, Oct 29, 2013 at 2:54 AM, Elran Dvir > wrote: > > > > > Hi all, > > > > > > I am using Solr 4.4 with multi cores. One core (called template) > > > is my "routing" core. > > > > > > When I run > > > http://127.0.0.1:8983/solr/template/select?rows=5000&q=*:*&shards=127. > > > 0.0.1:8983/solr/core1, > > > it consistently takes about 7s. > > > When I run > > > http://127.0.0.1:8983/solr/core1/select?rows=5000&q=*:*, it consistently > > > takes about 40ms. > > > > > > I profiled the distributed query. > > > This is the distributed query process (I hope the terms are accurate): > > > When solr identifies a distributed query, it sends the query to > > > the shard and get matched shard docs. > > > Then it sends another query to the shard to get the Solr documents. > > > Most time is spent in the last stage in the function "process" of > > > "QueryComponent" in: > > > > > > for (int i=0; i > > int id = req.getSearcher().getFirstMatch( > > > new Term(idField.getName(), > > > idField.getType().toInternal(idArr.get(i; > > > > > > How can I make my distributed query as fast as the direct one? > > > > > > Thanks. > > > > > > > > > Email secured by Check Point > > > Email secured by Check Point
suggestion for new custom atomic update
Hi all,
This is my use case:
I have a stored field, field_a, which is atomic updated (let's say by "inc").
field_a is stored but not indexed due to the large number of distinct values it
can have.
I need to index field_b (I need facet and stats on it) which is not in the
document but its value is based on a calculation of the recent (e.g. summed)
value of field_a.
There is no way to do it nowadays.
So I thought of a new method: custom atomic update.
There will be a new interface in Solr:
public interface CustomAtomicUpdater {
public void update(SolrInputDocument oldDoc, String fieldName, Object
fieldVal) ;
}
There will be a new attribute for fields in schema.xml called
"customAtomicUpdateClass" (and all support in code, of course).
The value is a class which is an implementation of CustomAtomicUpdater.
In our example it will be defined for field_a.
In method "getUpdatedDocument" in DistributedUpdateProcessor.java, we will add
handling of "custom" case:
} else if ("custom".equals(key)) {
updateField = true;
SchemaField sf = schema.getField(sif.getName());
String customAtomicUpdaterClassName =
sf.getCustomAtomicUpdaterClass();
if (customAtomicUpdaterClassName == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, "There is no
customAtomicUpdaterClass defined for " + sif + ".");
}
CustomAtomicUpdater updater = schema.getResourceLoader()
.newInstance(customAtomicUpdaterClassName,
CustomAtomicUpdater.class);
if (updater == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, "Was unable to
create instance of " + customAtomicUpdaterClassName + ".");
}
updater.update(oldDoc, sif.getName(), fieldVal);
}
In my implementation I will sum field_a (oldvalue + newvalue) and update
field_b according to my logic.
Example of use:
128
What do say about my suggestion?
Thanks.
RE: suggestion for new custom atomic update
Hi all,
Did anyone have a chance to review my idea?
Thanks.
-Original Message-
From: Elran Dvir
Sent: Monday, October 20, 2014 12:42 PM
To: solr-user
Subject: suggestion for new custom atomic update
Hi all,
This is my use case:
I have a stored field, field_a, which is atomic updated (let's say by "inc").
field_a is stored but not indexed due to the large number of distinct values it
can have.
I need to index field_b (I need facet and stats on it) which is not in the
document but its value is based on a calculation of the recent (e.g. summed)
value of field_a.
There is no way to do it nowadays.
So I thought of a new method: custom atomic update.
There will be a new interface in Solr:
public interface CustomAtomicUpdater {
public void update(SolrInputDocument oldDoc, String fieldName, Object
fieldVal) ; }
There will be a new attribute for fields in schema.xml called
"customAtomicUpdateClass" (and all support in code, of course).
The value is a class which is an implementation of CustomAtomicUpdater.
In our example it will be defined for field_a.
In method "getUpdatedDocument" in DistributedUpdateProcessor.java, we will add
handling of "custom" case:
} else if ("custom".equals(key)) {
updateField = true;
SchemaField sf = schema.getField(sif.getName());
String customAtomicUpdaterClassName =
sf.getCustomAtomicUpdaterClass();
if (customAtomicUpdaterClassName == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, "There is no
customAtomicUpdaterClass defined for " + sif + ".");
}
CustomAtomicUpdater updater = schema.getResourceLoader()
.newInstance(customAtomicUpdaterClassName,
CustomAtomicUpdater.class);
if (updater == null) {
throw new SolrException(ErrorCode.BAD_REQUEST, "Was unable to
create instance of " + customAtomicUpdaterClassName + ".");
}
updater.update(oldDoc, sif.getName(), fieldVal);
}
In my implementation I will sum field_a (oldvalue + newvalue) and update
field_b according to my logic.
Example of use:
128
What do say about my suggestion?
Thanks.
RE: suggestion for new custom atomic update
Thanks for your response.
If the calculation is based on the most recent summed value of field_a and the
value of field_a in the update, how can I?
Thanks.
-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
Sent: Sunday, October 26, 2014 2:11 PM
To: solr-user
Subject: RE: suggestion for new custom atomic update
Can't you do the calculation in custom UpdateRequestProcessor?
Regards,
Alex
On 26/10/2014 4:17 am, "Elran Dvir" wrote:
> Hi all,
>
> Did anyone have a chance to review my idea?
>
> Thanks.
>
> -Original Message-
> From: Elran Dvir
> Sent: Monday, October 20, 2014 12:42 PM
> To: solr-user
> Subject: suggestion for new custom atomic update
>
> Hi all,
>
> This is my use case:
> I have a stored field, field_a, which is atomic updated (let's say by
> "inc"). field_a is stored but not indexed due to the large number of
> distinct values it can have.
> I need to index field_b (I need facet and stats on it) which is not in
> the document but its value is based on a calculation of the recent (e.g.
> summed) value of field_a.
> There is no way to do it nowadays.
> So I thought of a new method: custom atomic update.
>
> There will be a new interface in Solr:
>
> public interface CustomAtomicUpdater {
> public void update(SolrInputDocument oldDoc, String fieldName,
> Object
> fieldVal) ; }
>
> There will be a new attribute for fields in schema.xml called
> "customAtomicUpdateClass" (and all support in code, of course).
> The value is a class which is an implementation of CustomAtomicUpdater.
> In our example it will be defined for field_a.
>
> In method "getUpdatedDocument" in DistributedUpdateProcessor.java, we
> will add handling of "custom" case:
>
>} else if ("custom".equals(key)) {
> updateField = true;
> SchemaField sf = schema.getField(sif.getName());
> String customAtomicUpdaterClassName =
> sf.getCustomAtomicUpdaterClass();
> if (customAtomicUpdaterClassName == null) {
> throw new SolrException(ErrorCode.BAD_REQUEST, "There is
> no customAtomicUpdaterClass defined for " + sif + ".");
> }
> CustomAtomicUpdater updater = schema.getResourceLoader()
> .newInstance(customAtomicUpdaterClassName,
> CustomAtomicUpdater.class);
> if (updater == null) {
> throw new SolrException(ErrorCode.BAD_REQUEST, "Was
> unable to create instance of " + customAtomicUpdaterClassName + ".");
> }
> updater.update(oldDoc, sif.getName(), fieldVal);
>
> }
>
> In my implementation I will sum field_a (oldvalue + newvalue) and
> update field_b according to my logic.
>
> Example of use:
>
>
> 128
>
>
>
> What do say about my suggestion?
>
> Thanks.
>
Email secured by Check Point
RE: suggestion for new custom atomic update
I will explain with an example.
Let's say field_a is sent in the update with the value of 5. field_a is already
stored in the document with the value 8.
After the update field_a should have the value 13 (sum).
The value of field_b will be based on the value of 13 and not 5.
Is there a way in URP to know what is the value which is already stored in
field_a?
Thank you very much.
-Original Message-
From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
Sent: Sunday, October 26, 2014 6:07 PM
To: solr-user
Subject: Re: suggestion for new custom atomic update
I am not sure what the problem is. URP catches all operations. So, you can
modify the source document to add the calculation when the field_a is either
new or updated.
Or are you trying to calculate things across multiple documents? In that case,
neither mine nor your solution will work, I think.
Regards,
Alex.
Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and
newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers
community: https://www.linkedin.com/groups?gid=6713853
On 26 October 2014 12:00, Elran Dvir wrote:
> Thanks for your response.
>
> If the calculation is based on the most recent summed value of field_a and
> the value of field_a in the update, how can I?
>
> Thanks.
>
> -Original Message-
> From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
> Sent: Sunday, October 26, 2014 2:11 PM
> To: solr-user
> Subject: RE: suggestion for new custom atomic update
>
> Can't you do the calculation in custom UpdateRequestProcessor?
>
> Regards,
> Alex
> On 26/10/2014 4:17 am, "Elran Dvir" wrote:
>
>> Hi all,
>>
>> Did anyone have a chance to review my idea?
>>
>> Thanks.
>>
>> -Original Message-
>> From: Elran Dvir
>> Sent: Monday, October 20, 2014 12:42 PM
>> To: solr-user
>> Subject: suggestion for new custom atomic update
>>
>> Hi all,
>>
>> This is my use case:
>> I have a stored field, field_a, which is atomic updated (let's say by
>> "inc"). field_a is stored but not indexed due to the large number of
>> distinct values it can have.
>> I need to index field_b (I need facet and stats on it) which is not
>> in the document but its value is based on a calculation of the recent (e.g.
>> summed) value of field_a.
>> There is no way to do it nowadays.
>> So I thought of a new method: custom atomic update.
>>
>> There will be a new interface in Solr:
>>
>> public interface CustomAtomicUpdater {
>> public void update(SolrInputDocument oldDoc, String fieldName,
>> Object
>> fieldVal) ; }
>>
>> There will be a new attribute for fields in schema.xml called
>> "customAtomicUpdateClass" (and all support in code, of course).
>> The value is a class which is an implementation of CustomAtomicUpdater.
>> In our example it will be defined for field_a.
>>
>> In method "getUpdatedDocument" in DistributedUpdateProcessor.java, we
>> will add handling of "custom" case:
>>
>>} else if ("custom".equals(key)) {
>> updateField = true;
>> SchemaField sf = schema.getField(sif.getName());
>> String customAtomicUpdaterClassName =
>> sf.getCustomAtomicUpdaterClass();
>> if (customAtomicUpdaterClassName == null) {
>> throw new SolrException(ErrorCode.BAD_REQUEST, "There
>> is no customAtomicUpdaterClass defined for " + sif + ".");
>> }
>> CustomAtomicUpdater updater = schema.getResourceLoader()
>> .newInstance(customAtomicUpdaterClassName,
>> CustomAtomicUpdater.class);
>> if (updater == null) {
>> throw new SolrException(ErrorCode.BAD_REQUEST, "Was
>> unable to create instance of " + customAtomicUpdaterClassName + ".");
>> }
>> updater.update(oldDoc, sif.getName(), fieldVal);
>>
>> }
>>
>> In my implementation I will sum field_a (oldvalue + newvalue) and
>> update field_b according to my logic.
>>
>> Example of use:
>>
>>
>> 128
>>
>>
>>
>> What do say about my suggestion?
>>
>> Thanks.
>>
>
>
> Email secured by Check Point
Email secured by Check Point
RE: suggestion for new custom atomic update
Thank you very much for your suggestion.
I created an update processor factory with my logic.
I changed the update processor chain to be:
But nothing seems to happen.
When I move my class to be the first in the chain, the logic is running (not as
I want, of course. It's calculated based on the update value rather than the
stored value) .
How can I define a custom update processor factory that will run after
DistributedUpdateProcessorFactory?
Thank you very much.
-Original Message-
From: Matthew Nigl [mailto:matthew.n...@gmail.com]
Sent: Monday, October 27, 2014 12:10 PM
To: solr-user@lucene.apache.org
Subject: Re: suggestion for new custom atomic update
You can get the summed value, 13, if you add a processor after
DistributedUpdateProcessorFactory in the URP chain. Then one possibility would
be to clone this value to another field, such as field_b, and run other
processors on that field.
Or for something more customized, you can use the
StatelessScriptUpdateProcessorFactory, and retrieve the value of field_a with:
var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument
var field_a = doc.getFieldValue("field_a");
Note if you try to get the value of field_a before
DistributedUpdateProcessorFactory, then using your example with an atomic
update, the value would be 5 (the value of the increment from the input
document).
On 27 October 2014 18:03, Elran Dvir wrote:
> I will explain with an example.
> Let's say field_a is sent in the update with the value of 5. field_a
> is already stored in the document with the value 8.
> After the update field_a should have the value 13 (sum).
> The value of field_b will be based on the value of 13 and not 5.
> Is there a way in URP to know what is the value which is already
> stored in field_a?
>
> Thank you very much.
>
> -Original Message-
> From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
> Sent: Sunday, October 26, 2014 6:07 PM
> To: solr-user
> Subject: Re: suggestion for new custom atomic update
>
> I am not sure what the problem is. URP catches all operations. So, you
> can modify the source document to add the calculation when the field_a
> is either new or updated.
>
> Or are you trying to calculate things across multiple documents? In
> that case, neither mine nor your solution will work, I think.
>
> Regards,
>Alex.
> Personal: http://www.outerthoughts.com/ and @arafalov Solr resources
> and
> newsletter: http://www.solr-start.com/ and @solrstart Solr
> popularizers
> community: https://www.linkedin.com/groups?gid=6713853
>
>
> On 26 October 2014 12:00, Elran Dvir wrote:
> > Thanks for your response.
> >
> > If the calculation is based on the most recent summed value of
> > field_a
> and the value of field_a in the update, how can I?
> >
> > Thanks.
> >
> > -Original Message-
> > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
> > Sent: Sunday, October 26, 2014 2:11 PM
> > To: solr-user
> > Subject: RE: suggestion for new custom atomic update
> >
> > Can't you do the calculation in custom UpdateRequestProcessor?
> >
> > Regards,
> > Alex
> > On 26/10/2014 4:17 am, "Elran Dvir" wrote:
> >
> >> Hi all,
> >>
> >> Did anyone have a chance to review my idea?
> >>
> >> Thanks.
> >>
> >> -Original Message-
> >> From: Elran Dvir
> >> Sent: Monday, October 20, 2014 12:42 PM
> >> To: solr-user
> >> Subject: suggestion for new custom atomic update
> >>
> >> Hi all,
> >>
> >> This is my use case:
> >> I have a stored field, field_a, which is atomic updated (let's say
> >> by "inc"). field_a is stored but not indexed due to the large
> >> number of distinct values it can have.
> >> I need to index field_b (I need facet and stats on it) which is not
> >> in the document but its value is based on a calculation of the
> >> recent
> (e.g.
> >> summed) value of field_a.
> >> There is no way to do it nowadays.
> >> So I thought of a new method: custom atomic update.
> >>
> >> There will be a new interface in Solr:
> >>
> >> public interface CustomAtomicUpdater {
> >> public void update(SolrInputDocument oldDoc, String fieldName,
> >> Object
> >> fieldVal) ; }
> >>
> >> There will be a new attribute for fields in schema.xml called
> >> "customAtomicUpdateClass" (and all support in code, of course).
> >> The value is a class which is an implementation of CustomAtomicUpdater.
> >
RE: suggestion for new custom atomic update
Shalin and Matthew,
Thank you very much.
-Original Message-
From: Matthew Nigl [mailto:matthew.n...@gmail.com]
Sent: Monday, October 27, 2014 7:24 PM
To: solr-user@lucene.apache.org
Subject: Re: suggestion for new custom atomic update
No problem Elran. As Shalin mentioned, you will need to do it like this:
On 28 October 2014 03:22, Shalin Shekhar Mangar
wrote:
> Hi Elran,
>
> You need to explicitly specify the DistributedUpdateProcessorFactory
> in the chain and then add your custom processor after it.
>
> On Mon, Oct 27, 2014 at 9:26 PM, Elran Dvir wrote:
>
> > Thank you very much for your suggestion.
> >
> > I created an update processor factory with my logic.
> > I changed the update processor chain to be:
> > > class="solr.RunUpdateProcessorFactory" /> > class="mycode.solr_plugins.FieldManipulationProcessorFactory"
> />
> >
> > But nothing seems to happen.
> > When I move my class to be the first in the chain, the logic is
> > running (not as I want, of course. It's calculated based on the
> > update value
> rather
> > than the stored value) .
> > How can I define a custom update processor factory that will run
> > after DistributedUpdateProcessorFactory?
> >
> > Thank you very much.
> >
> >
> > -Original Message-
> > From: Matthew Nigl [mailto:matthew.n...@gmail.com]
> > Sent: Monday, October 27, 2014 12:10 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: suggestion for new custom atomic update
> >
> > You can get the summed value, 13, if you add a processor after
> > DistributedUpdateProcessorFactory in the URP chain. Then one
> > possibility would be to clone this value to another field, such as
> > field_b, and run other processors on that field.
> >
> > Or for something more customized, you can use the
> > StatelessScriptUpdateProcessorFactory, and retrieve the value of
> > field_a
> > with:
> >
> > var doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument
> > var field_a = doc.getFieldValue("field_a");
> >
> > Note if you try to get the value of field_a before
> > DistributedUpdateProcessorFactory, then using your example with an
> > atomic update, the value would be 5 (the value of the increment from
> > the input document).
> >
> >
> > On 27 October 2014 18:03, Elran Dvir wrote:
> >
> > > I will explain with an example.
> > > Let's say field_a is sent in the update with the value of 5.
> > > field_a is already stored in the document with the value 8.
> > > After the update field_a should have the value 13 (sum).
> > > The value of field_b will be based on the value of 13 and not 5.
> > > Is there a way in URP to know what is the value which is already
> > > stored in field_a?
> > >
> > > Thank you very much.
> > >
> > > -Original Message-
> > > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
> > > Sent: Sunday, October 26, 2014 6:07 PM
> > > To: solr-user
> > > Subject: Re: suggestion for new custom atomic update
> > >
> > > I am not sure what the problem is. URP catches all operations. So,
> > > you can modify the source document to add the calculation when the
> > > field_a is either new or updated.
> > >
> > > Or are you trying to calculate things across multiple documents?
> > > In that case, neither mine nor your solution will work, I think.
> > >
> > > Regards,
> > >Alex.
> > > Personal: http://www.outerthoughts.com/ and @arafalov Solr
> > > resources and
> > > newsletter: http://www.solr-start.com/ and @solrstart Solr
> > > popularizers
> > > community: https://www.linkedin.com/groups?gid=6713853
> > >
> > >
> > > On 26 October 2014 12:00, Elran Dvir wrote:
> > > > Thanks for your response.
> > > >
> > > > If the calculation is based on the most recent summed value of
> > > > field_a
> > > and the value of field_a in the update, how can I?
> > > >
> > > > Thanks.
> > > >
> > > > -Original Message-
> > > > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com]
> > > > Sent: Sunday, October 26, 2014 2:11 PM
> > > > To: solr-user
> > > > Subject: RE: suggestion for new custom atomic update
> > > >
> > > > Can't you do the calculation in custom UpdateRequestProcessor?
> > >
How to apply SOLR-6024 to Solr 4.8
Hi all, I am trying to apply SOLR-6024 patch to Solr 4.8. I have some compilation errors with it (detailed in Jira: https://issues.apache.org/jira/i#browse/SOLR-6024). How can I change the patch to be applied to 4.8? Thanks.
Stats calculation of existInDoc on multivalue fields which are doc valued
Hi all, I uploaded a patch (https://issues.apache.org/jira/browse/SOLR-5972) that contains a new statistics result for a field - existInDoc. It returns the number of documents in which the field has a value (not missing). This patch is bason on Solr 4.4. For multivalue fields there is a calculation of existInDoc inside the class UnInvertedField. Since Solr 4.10 there was a fix for a stats calculation of multi valued field which is doc valued. The class handling it is DocValuesStats. I want to support existInDoc calculation also for multi valued - doc valued field. How Should I change DocValuesStats to support this? Thanks.
StatsComponent doesn't work if field's type is TextField - can I change field's type to String
Hi all,
StatsComponent doesn't work if field's type is TextField.
I get the following message:
"Field type
textstring{class=org.apache.solr.schema.TextField,analyzer=org.apache.solr.analysis.TokenizerChain,args={positionIncrementGap=100,
sortMissingLast=true}} is not currently supported".
My field configuration is:
So, the reason my field is of type TextField is that in the document indexed
there may be multiple values in the field separated by new lines.
The tokenizer is splitting it to multiple values and the field is indexed as
multi-valued field.
Is there a way I can define the field as regular String field? Or a way to make
StatsComponent work with TextField?
Thank you very much.
RE: StatsComponent doesn't work if field's type is TextField - can I change field's type to String
Erick, thanks for the response.
I think the stats component works with strings.
In StatsValuesFactory, I see the following code:
public static StatsValues createStatsValues(SchemaField sf) {
...
else if (StrField.class.isInstance(fieldType)) {
return new StringStatsValues(sf);
}
}
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Wednesday, June 26, 2013 5:30 PM
To: solr-user@lucene.apache.org
Subject: Re: StatsComponent doesn't work if field's type is TextField - can I
change field's type to String
>From the stats component page:
"The stats component returns simple statistics for indexed numeric fields
within the DocSet"
So string, text, anything non-numeric won't work. You can declare it
multiValued but then you have to add multiple values for the field when you
send the doc to Solr or implement a custom update component to break them up.
At least there's no filter that I know of that takes a delimited set of numbers
and transforms them.
FWIW,
Erick
On Wed, Jun 26, 2013 at 4:14 AM, Elran Dvir wrote:
> Hi all,
>
> StatsComponent doesn't work if field's type is TextField.
> I get the following message:
> "Field type
> textstring{class=org.apache.solr.schema.TextField,analyzer=org.apache.
> solr.analysis.TokenizerChain,args={positionIncrementGap=100,
> sortMissingLast=true}} is not currently supported".
>
> My field configuration is:
>
> sortMissingLast="true">
>
> />
>
>
>
> multiValued="true"/>
>
> So, the reason my field is of type TextField is that in the document indexed
> there may be multiple values in the field separated by new lines.
> The tokenizer is splitting it to multiple values and the field is indexed as
> multi-valued field.
>
> Is there a way I can define the field as regular String field? Or a way to
> make StatsComponent work with TextField?
>
> Thank you very much.
Email secured by Check Point
new field type - enum field
Hi All, We have encountered a use case in our system where we have a few fields (Severity. Risk etc) with a closed set of values, where the sort order for these values is pre-determined but not lexicographic (Critical is higher than High). Generically this is very close to how enums work. To implement, I have prototyped a new type of field: EnumField where the inputs are a closed predefined set of strings in a special configuration file (similar to currency.xml). I'm thinking of contributing the code back since I think it's a nice little feature that other people may like to use and would like to get some feedback from the community. If there's positive feedback, I'll open an issue with a patch for the functionality. Thanks.
RE: new field type - enum field
Hi, I have implemented like Chris described it: The field is indexed as numeric, but displayed as string, according to configuration. It applies to facet, pivot, group and query. How do we proceed? How do I contribute it? Thanks. -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: Thursday, July 25, 2013 4:40 AM To: solr-user@lucene.apache.org Subject: Re: new field type - enum field : Doable at Lucene level by any chance? Given how well the Trie fields compress (ByteField and ShortField have been deprecated in favor of TrieIntField for this reason) it probably just makes sense to treat it as a numeric at the Lucene level. : > If there's positive feedback, I'll open an issue with a patch for the functionality. I've typically dealt with this sort of thing at the client layer using a simple numeric field in Solr, or used an UpdateProcessor to convert the String->numeric mapping when indexing & used clinet logic of a DocTransformer to handle the stored value at query time -- but having a built in FieldType that handles that for you automatically (and helps ensure the indexed values conform to the enum) would certainly be cool if you'd like to contribute it. -Hoss Email secured by Check Point
RE: new field type - enum field
Hi, I have created an issue: https://issues.apache.org/jira/browse/SOLR-5084 I tried to attach my patch, but it failed: " Cannot attach file Solr-5084.patch: Unable to communicate with JIRA." What am I doing wrong? Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Thursday, July 25, 2013 3:25 PM To: solr-user@lucene.apache.org Subject: Re: new field type - enum field Start here: http://wiki.apache.org/solr/HowToContribute Then, when your patch is ready submit a JIRA and attach your patch. Then nudge (gently) if none of the committers picks it up and applies it NOTE: It is _not_ necessary that the first version of your patch is completely polished. I often put up partial/incomplete patches (comments with //nocommit are explicitly caught by the "ant precommit" target for instance) to see if anyone has any comments before polishing. Best Erick On Thu, Jul 25, 2013 at 5:04 AM, Elran Dvir wrote: > Hi, > > I have implemented like Chris described it: > The field is indexed as numeric, but displayed as string, according to > configuration. > It applies to facet, pivot, group and query. > > How do we proceed? How do I contribute it? > > Thanks. > > -Original Message- > From: Chris Hostetter [mailto:hossman_luc...@fucit.org] > Sent: Thursday, July 25, 2013 4:40 AM > To: solr-user@lucene.apache.org > Subject: Re: new field type - enum field > > > : Doable at Lucene level by any chance? > > Given how well the Trie fields compress (ByteField and ShortField have been > deprecated in favor of TrieIntField for this reason) it probably just makes > sense to treat it as a numeric at the Lucene level. > > : > If there's positive feedback, I'll open an issue with a patch for the > functionality. > > I've typically dealt with this sort of thing at the client layer using > a simple numeric field in Solr, or used an UpdateProcessor to convert > the > String->numeric mapping when indexing & used clinet logic of a > DocTransformer to handle the stored value at query time -- but having a built > in FieldType that handles that for you automatically (and helps ensure the > indexed values conform to the enum) would certainly be cool if you'd like to > contribute it. > > > -Hoss > > Email secured by Check Point Email secured by Check Point
RE: new field type - enum field
Thanks, Erick. I have tried it four times. It keeps failing. The problem reoccurred today. Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Monday, July 29, 2013 2:44 AM To: solr-user@lucene.apache.org Subject: Re: new field type - enum field You should be able to attach a patch, wonder if there was some temporary glitch in the JIRA. Is this persisting. Let us know if this continues... Erick On Sun, Jul 28, 2013 at 12:11 PM, Elran Dvir wrote: > Hi, > > I have created an issue: > https://issues.apache.org/jira/browse/SOLR-5084 > I tried to attach my patch, but it failed: " Cannot attach file > Solr-5084.patch: Unable to communicate with JIRA." > What am I doing wrong? > > Thanks. > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Thursday, July 25, 2013 3:25 PM > To: solr-user@lucene.apache.org > Subject: Re: new field type - enum field > > Start here: http://wiki.apache.org/solr/HowToContribute > > Then, when your patch is ready submit a JIRA and attach your patch. Then > nudge (gently) if none of the committers picks it up and applies it > > NOTE: It is _not_ necessary that the first version of your patch is > completely polished. I often put up partial/incomplete patches (comments with > //nocommit are explicitly caught by the "ant precommit" target for instance) > to see if anyone has any comments before polishing. > > Best > Erick > > On Thu, Jul 25, 2013 at 5:04 AM, Elran Dvir wrote: >> Hi, >> >> I have implemented like Chris described it: >> The field is indexed as numeric, but displayed as string, according to >> configuration. >> It applies to facet, pivot, group and query. >> >> How do we proceed? How do I contribute it? >> >> Thanks. >> >> -Original Message- >> From: Chris Hostetter [mailto:hossman_luc...@fucit.org] >> Sent: Thursday, July 25, 2013 4:40 AM >> To: solr-user@lucene.apache.org >> Subject: Re: new field type - enum field >> >> >> : Doable at Lucene level by any chance? >> >> Given how well the Trie fields compress (ByteField and ShortField have been >> deprecated in favor of TrieIntField for this reason) it probably just makes >> sense to treat it as a numeric at the Lucene level. >> >> : > If there's positive feedback, I'll open an issue with a patch for the >> functionality. >> >> I've typically dealt with this sort of thing at the client layer >> using a simple numeric field in Solr, or used an UpdateProcessor to >> convert the >> String->numeric mapping when indexing & used clinet logic of a >> DocTransformer to handle the stored value at query time -- but having a >> built in FieldType that handles that for you automatically (and helps ensure >> the indexed values conform to the enum) would certainly be cool if you'd >> like to contribute it. >> >> >> -Hoss >> >> Email secured by Check Point > > Email secured by Check Point Email secured by Check Point
RE: new field type - enum field
Hi, I have managed to attach the patch in Jira. Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Monday, July 29, 2013 2:15 PM To: solr-user@lucene.apache.org Subject: Re: new field type - enum field OK, if you can attach it to an e-mail, I'll attach it. Just to check, though, make sure you're logged in. I've been fooled once or twice by being automatically signed out... Erick On Mon, Jul 29, 2013 at 3:17 AM, Elran Dvir wrote: > Thanks, Erick. > > I have tried it four times. It keeps failing. > The problem reoccurred today. > > Thanks. > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Monday, July 29, 2013 2:44 AM > To: solr-user@lucene.apache.org > Subject: Re: new field type - enum field > > You should be able to attach a patch, wonder if there was some temporary > glitch in the JIRA. Is this persisting. > > Let us know if this continues... > > Erick > > On Sun, Jul 28, 2013 at 12:11 PM, Elran Dvir wrote: >> Hi, >> >> I have created an issue: >> https://issues.apache.org/jira/browse/SOLR-5084 >> I tried to attach my patch, but it failed: " Cannot attach file >> Solr-5084.patch: Unable to communicate with JIRA." >> What am I doing wrong? >> >> Thanks. >> >> -Original Message- >> From: Erick Erickson [mailto:erickerick...@gmail.com] >> Sent: Thursday, July 25, 2013 3:25 PM >> To: solr-user@lucene.apache.org >> Subject: Re: new field type - enum field >> >> Start here: http://wiki.apache.org/solr/HowToContribute >> >> Then, when your patch is ready submit a JIRA and attach your patch. Then >> nudge (gently) if none of the committers picks it up and applies it >> >> NOTE: It is _not_ necessary that the first version of your patch is >> completely polished. I often put up partial/incomplete patches (comments >> with //nocommit are explicitly caught by the "ant precommit" target for >> instance) to see if anyone has any comments before polishing. >> >> Best >> Erick >> >> On Thu, Jul 25, 2013 at 5:04 AM, Elran Dvir wrote: >>> Hi, >>> >>> I have implemented like Chris described it: >>> The field is indexed as numeric, but displayed as string, according to >>> configuration. >>> It applies to facet, pivot, group and query. >>> >>> How do we proceed? How do I contribute it? >>> >>> Thanks. >>> >>> -Original Message- >>> From: Chris Hostetter [mailto:hossman_luc...@fucit.org] >>> Sent: Thursday, July 25, 2013 4:40 AM >>> To: solr-user@lucene.apache.org >>> Subject: Re: new field type - enum field >>> >>> >>> : Doable at Lucene level by any chance? >>> >>> Given how well the Trie fields compress (ByteField and ShortField have been >>> deprecated in favor of TrieIntField for this reason) it probably just makes >>> sense to treat it as a numeric at the Lucene level. >>> >>> : > If there's positive feedback, I'll open an issue with a patch for the >>> functionality. >>> >>> I've typically dealt with this sort of thing at the client layer >>> using a simple numeric field in Solr, or used an UpdateProcessor to >>> convert the >>> String->numeric mapping when indexing & used clinet logic of a >>> DocTransformer to handle the stored value at query time -- but having a >>> built in FieldType that handles that for you automatically (and helps >>> ensure the indexed values conform to the enum) would certainly be cool if >>> you'd like to contribute it. >>> >>> >>> -Hoss >>> >>> Email secured by Check Point >> >> Email secured by Check Point > > Email secured by Check Point Email secured by Check Point
RE: new field type - enum field
Hi all, Did anyone have a chance to look at the code? It's attached here: https://issues.apache.org/jira/browse/SOLR-5084 Thank you very much. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Monday, July 29, 2013 2:15 PM To: solr-user@lucene.apache.org Subject: Re: new field type - enum field OK, if you can attach it to an e-mail, I'll attach it. Just to check, though, make sure you're logged in. I've been fooled once or twice by being automatically signed out... Erick On Mon, Jul 29, 2013 at 3:17 AM, Elran Dvir wrote: > Thanks, Erick. > > I have tried it four times. It keeps failing. > The problem reoccurred today. > > Thanks. > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Monday, July 29, 2013 2:44 AM > To: solr-user@lucene.apache.org > Subject: Re: new field type - enum field > > You should be able to attach a patch, wonder if there was some temporary > glitch in the JIRA. Is this persisting. > > Let us know if this continues... > > Erick > > On Sun, Jul 28, 2013 at 12:11 PM, Elran Dvir wrote: >> Hi, >> >> I have created an issue: >> https://issues.apache.org/jira/browse/SOLR-5084 >> I tried to attach my patch, but it failed: " Cannot attach file >> Solr-5084.patch: Unable to communicate with JIRA." >> What am I doing wrong? >> >> Thanks. >> >> -Original Message- >> From: Erick Erickson [mailto:erickerick...@gmail.com] >> Sent: Thursday, July 25, 2013 3:25 PM >> To: solr-user@lucene.apache.org >> Subject: Re: new field type - enum field >> >> Start here: http://wiki.apache.org/solr/HowToContribute >> >> Then, when your patch is ready submit a JIRA and attach your patch. Then >> nudge (gently) if none of the committers picks it up and applies it >> >> NOTE: It is _not_ necessary that the first version of your patch is >> completely polished. I often put up partial/incomplete patches (comments >> with //nocommit are explicitly caught by the "ant precommit" target for >> instance) to see if anyone has any comments before polishing. >> >> Best >> Erick >> >> On Thu, Jul 25, 2013 at 5:04 AM, Elran Dvir wrote: >>> Hi, >>> >>> I have implemented like Chris described it: >>> The field is indexed as numeric, but displayed as string, according to >>> configuration. >>> It applies to facet, pivot, group and query. >>> >>> How do we proceed? How do I contribute it? >>> >>> Thanks. >>> >>> -Original Message- >>> From: Chris Hostetter [mailto:hossman_luc...@fucit.org] >>> Sent: Thursday, July 25, 2013 4:40 AM >>> To: solr-user@lucene.apache.org >>> Subject: Re: new field type - enum field >>> >>> >>> : Doable at Lucene level by any chance? >>> >>> Given how well the Trie fields compress (ByteField and ShortField have been >>> deprecated in favor of TrieIntField for this reason) it probably just makes >>> sense to treat it as a numeric at the Lucene level. >>> >>> : > If there's positive feedback, I'll open an issue with a patch for the >>> functionality. >>> >>> I've typically dealt with this sort of thing at the client layer >>> using a simple numeric field in Solr, or used an UpdateProcessor to >>> convert the >>> String->numeric mapping when indexing & used clinet logic of a >>> DocTransformer to handle the stored value at query time -- but having a >>> built in FieldType that handles that for you automatically (and helps >>> ensure the indexed values conform to the enum) would certainly be cool if >>> you'd like to contribute it. >>> >>> >>> -Hoss >>> >>> Email secured by Check Point >> >> Email secured by Check Point > > Email secured by Check Point Email secured by Check Point
Upgrade Solr index from 4.0 to 4.2.1
Hi all, I have a 4.0 Solr (sharded/cored) index. I upgraded Solr to 4.2.1 and tried to load the existing index with it. I got the following exception: May 21, 2013 12:03:42 PM org.apache.solr.common.SolrException log SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: other_2013-05-04 at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1672) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1057) at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634) at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) at java.util.concurrent.FutureTask.run(FutureTask.java:177) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:482) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) at java.util.concurrent.FutureTask.run(FutureTask.java:177) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) at java.lang.Thread.run(Thread.java:779) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.(SolrCore.java:822) at org.apache.solr.core.SolrCore.(SolrCore.java:618) at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1021) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051) ... 10 more Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1435) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1547) at org.apache.solr.core.SolrCore.(SolrCore.java:797) ... 13 more Caused by: org.apache.solr.common.SolrException: Error opening Reader at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172) at org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:183) at org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:179) at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1411) ... 15 more Caused by: org.apache.lucene.index.CorruptIndexException: codec mismatch: actual codec=Lucene40StoredFieldsIndex vs expected codec=Lucene41StoredFieldsIndex (resource: MMapIndexInput(path="/var/solr/multicore_solr/other_2013-05-04/data/index/_3gfk.fdx")) at org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:140) at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:130) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:102) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:113) at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:147) at org.apache.lucene.index.SegmentReader.(SegmentReader.java:56) at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783) at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52) at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88) at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34) at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169) ... 18 more It seems the problem is with my stored fields. This is the configuration of all sored fields: Field "id" is my unique key. So, I looked for a way to upgrade the version of the index. I tried Lucene's IndexUpgrader as described here: http://lucene.apache.org/core/4_2_1/core/org/apache/lucene/index/IndexUpgrader.html I issued this command: java -cp lucene-core-4.2.1.jar org.apache.lucene.index.IndexUpgrader -verbose ./other_2013-05-04/data/index I got similar exception: Exception in thread "main" java.io.IOException: background merge hit exception: _3ea8(5.0):C170283/17098 _3g2k(5.0):C18785/1858 _3gfe(5.0):C1858/10 _3gfj(5.0):C3 _3gfg(5.0):C3 _3gfi(5.0):C2 _3gfh(5.0):C1 _3gff(5.0):C1 _3gfk(5.0):C1 into _3gfm [maxNumSegments=1] at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1674) at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.j
RE: Upgrade Solr index from 4.0 to 4.2.1
Why LUCENE_42?Why not LUCENE_41? Do I still need to run IndexUpgrader or just loading will be enough? Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, May 21, 2013 2:52 PM To: solr-user@lucene.apache.org Subject: Re: Upgrade Solr index from 4.0 to 4.2.1 This is always something that gives me a headache, but what happens if you change in solrconfig.xml to LUCENE_40? I'm assuming it's LUCENE_42... Best Erick On Tue, May 21, 2013 at 5:48 AM, Elran Dvir wrote: > Hi all, > > I have a 4.0 Solr (sharded/cored) index. > I upgraded Solr to 4.2.1 and tried to load the existing index with it. I got > the following exception: > > May 21, 2013 12:03:42 PM org.apache.solr.common.SolrException log > SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: > other_2013-05-04 > at > org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1672) > at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:1057) > at > org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634) > at > org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) > at java.util.concurrent.FutureTask.run(FutureTask.java:177) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:482) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) > at java.util.concurrent.FutureTask.run(FutureTask.java:177) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) > at java.lang.Thread.run(Thread.java:779) > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.(SolrCore.java:822) > at org.apache.solr.core.SolrCore.(SolrCore.java:618) > at > org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1021) > at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051) > ... 10 more > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at > org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1435) > at > org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1547) > at org.apache.solr.core.SolrCore.(SolrCore.java:797) > ... 13 more > Caused by: org.apache.solr.common.SolrException: Error opening Reader > at > org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172) > at > org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:183) > at > org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:179) > at > org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1411) > ... 15 more > Caused by: org.apache.lucene.index.CorruptIndexException: codec mismatch: > actual codec=Lucene40StoredFieldsIndex vs expected > codec=Lucene41StoredFieldsIndex (resource: > MMapIndexInput(path="/var/solr/multicore_solr/other_2013-05-04/data/index/_3gfk.fdx")) > at > org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:140) > at > org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:130) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:102) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:113) > at > org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:147) > at > org.apache.lucene.index.SegmentReader.(SegmentReader.java:56) > at > org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62) > at > org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783) > at > org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52) > at > org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88) > at > org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34) > at > org.apache.solr.search.SolrIndexSearcher.getRe
RE: Upgrade Solr index from 4.0 to 4.2.1
My index is originally of version 4.0. My methods failed with this configuration. So, I changed solrconfig.xml in my index to both versions: LUCENE_42 and LUCENE_41. For each version in each method (loading and IndexUpgrader), I see the same errors as before. Thanks. -Original Message- From: Elran Dvir Sent: Tuesday, May 21, 2013 6:48 PM To: solr-user@lucene.apache.org Subject: RE: Upgrade Solr index from 4.0 to 4.2.1 Why LUCENE_42?Why not LUCENE_41? Do I still need to run IndexUpgrader or just loading will be enough? Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, May 21, 2013 2:52 PM To: solr-user@lucene.apache.org Subject: Re: Upgrade Solr index from 4.0 to 4.2.1 This is always something that gives me a headache, but what happens if you change in solrconfig.xml to LUCENE_40? I'm assuming it's LUCENE_42... Best Erick On Tue, May 21, 2013 at 5:48 AM, Elran Dvir wrote: > Hi all, > > I have a 4.0 Solr (sharded/cored) index. > I upgraded Solr to 4.2.1 and tried to load the existing index with it. I got > the following exception: > > May 21, 2013 12:03:42 PM org.apache.solr.common.SolrException log > SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: > other_2013-05-04 > at > org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1672) > at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:1057) > at > org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634) > at > org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) > at java.util.concurrent.FutureTask.run(FutureTask.java:177) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:482) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) > at java.util.concurrent.FutureTask.run(FutureTask.java:177) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) > at java.lang.Thread.run(Thread.java:779) > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.(SolrCore.java:822) > at org.apache.solr.core.SolrCore.(SolrCore.java:618) > at > org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1021) > at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051) > ... 10 more > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at > org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1435) > at > org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1547) > at org.apache.solr.core.SolrCore.(SolrCore.java:797) > ... 13 more > Caused by: org.apache.solr.common.SolrException: Error opening Reader > at > org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172) > at > org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:183) > at > org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:179) > at > org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1411) > ... 15 more > Caused by: org.apache.lucene.index.CorruptIndexException: codec mismatch: > actual codec=Lucene40StoredFieldsIndex vs expected > codec=Lucene41StoredFieldsIndex (resource: > MMapIndexInput(path="/var/solr/multicore_solr/other_2013-05-04/data/index/_3gfk.fdx")) > at > org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:140) > at > org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:130) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:102) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:113) > at > org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:147) > at > org.apache.lucene.index.SegmentReader.(SegmentReader.java:56) > at > org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62) > at > org.apache.lucene.index.SegmentInfos$F
RE: Upgrade Solr index from 4.0 to 4.2.1
So having tried all combinations of LUCENE_40, 41 and 42 we're still having no success in getting our indexes to load with Solr 4.2.1... Any direction we can look into ? in our system the underlying data is very slow to re-index and would take an unreasonable amount of time at a customer site to wait for information to become available after an upgrade, so we're very hopeful there can be a way to upgrade a Lucene index "properly". Thanks, Elran -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Wednesday, May 22, 2013 2:25 PM To: solr-user@lucene.apache.org Subject: Re: Upgrade Solr index from 4.0 to 4.2.1 LUCENE_40 since your original index was built with 4.0. As for the other, I'll defer to people who actually know what they're talking about. Best Erick On Wed, May 22, 2013 at 5:19 AM, Elran Dvir wrote: > My index is originally of version 4.0. My methods failed with this > configuration. > So, I changed solrconfig.xml in my index to both versions: LUCENE_42 and > LUCENE_41. > For each version in each method (loading and IndexUpgrader), I see the same > errors as before. > > Thanks. > > -Original Message- > From: Elran Dvir > Sent: Tuesday, May 21, 2013 6:48 PM > To: solr-user@lucene.apache.org > Subject: RE: Upgrade Solr index from 4.0 to 4.2.1 > > Why LUCENE_42?Why not LUCENE_41? > Do I still need to run IndexUpgrader or just loading will be enough? > > Thanks. > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, May 21, 2013 2:52 PM > To: solr-user@lucene.apache.org > Subject: Re: Upgrade Solr index from 4.0 to 4.2.1 > > This is always something that gives me a headache, but what happens if you > change in solrconfig.xml to LUCENE_40? I'm assuming it's > LUCENE_42... > > Best > Erick > > On Tue, May 21, 2013 at 5:48 AM, Elran Dvir wrote: >> Hi all, >> >> I have a 4.0 Solr (sharded/cored) index. >> I upgraded Solr to 4.2.1 and tried to load the existing index with it. I got >> the following exception: >> >> May 21, 2013 12:03:42 PM org.apache.solr.common.SolrException log >> SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: >> other_2013-05-04 >> at >> org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1672) >> at >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:1057) >> at >> org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634) >> at >> org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629) >> at >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) >> at java.util.concurrent.FutureTask.run(FutureTask.java:177) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:482) >> at >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:345) >> at java.util.concurrent.FutureTask.run(FutureTask.java:177) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) >> at java.lang.Thread.run(Thread.java:779) >> Caused by: org.apache.solr.common.SolrException: Error opening new searcher >> at org.apache.solr.core.SolrCore.(SolrCore.java:822) >> at org.apache.solr.core.SolrCore.(SolrCore.java:618) >> at >> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1021) >> at >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051) >> ... 10 more >> Caused by: org.apache.solr.common.SolrException: Error opening new searcher >> at >> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1435) >> at >> org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1547) >> at org.apache.solr.core.SolrCore.(SolrCore.java:797) >> ... 13 more >> Caused by: org.apache.solr.common.SolrException: Error opening Reader >> at >> org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172) >> at >> org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:183) >> at >> org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:179) >> at >>
