Re: ZooKeeper issues with AWS

[Björn Häuser]

Hello,

> On 31. Aug 2018, at 21:53, Shawn Heisey  wrote:
> 
> 
> As Walter hinted, ZooKeeper 3.4.x is not capable of dynamically 
> adding/removing servers to/from the ensemble.  To do this successfully, all 
> ZK servers and all ZK clients must be upgraded to 3.5.x.  Solr is a ZK client 
> when running in cloud mode.  The 3.5.x version of ZK is currently in beta.  
> When a stable version is released, Solr will have its dependency upgraded in 
> the next release.  We do not know if you can successfully replace the ZK jar 
> in Solr with a 3.5.x version without making changes to the code.

as far as I can see the required fix for this is finally in 3.4.13: 

- https://github.com/apache/zookeeper/pull/451 

- https://zookeeper.apache.org/doc/r3.4.13/releasenotes.html 

- https://issues.apache.org/jira/browse/ZOOKEEPER-2184 


Would be great to have this in the next solr update.

Also we are solving this with using a kuberrnetes service, which does not 
change the ip-address when the zk nodes are restarted. This worked pretty will 
with solr 6.6.x, but we are having problems with solr 7.3.x and 7.4.x. There we 
occasionally get a “zk client disconnected”. Next step will be to upgrade our 
zk clusters from 3.4.10 to 3.4.13.

Thank you
Björn

Re: Need to connect solr with solrj from AWS lambda

[Mikhail Khludnev]

Hello, Srini.
Are you sure that security group let to reach Solr host?

On Sat, Sep 1, 2018 at 1:08 AM nalsrini  wrote:

> Hi,
> I need to connect solr with solrj from AWS java lambda. I use solr 5.3.
>
> I get the client object like this:
> SolrClient client = new
> HttpSolrClient(System.getenv(SysEnvConstants.SOLR_HOST));
>
> I neither get an error nor a response when I call these(for example) from
> the lambda:
>
> SolrDocument sorld = client.getById(id);
>
> OR
>
> UpdateResponse ur = client.deleteByQuery(sb.toString());
>
> thanks
> Srini
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


-- 
Sincerely yours
Mikhail Khludnev

facet values from stored values

[nirsh]

Hi,

is there a way to return from the stored value of facets and not the indexed
value?
we have normalization we perform, such as lower case, punctuation etc..
and need to display the stored value of it.

Thanks,
 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: ZooKeeper issues with AWS

[Shawn Heisey]

On 9/1/2018 3:42 AM, Björn Häuser wrote:

as far as I can see the required fix for this is finally in 3.4.13:

- https://issues.apache.org/jira/browse/ZOOKEEPER-2184 


Would be great to have this in the next solr update.


Issue created.

https://issues.apache.org/jira/browse/SOLR-12727

Note that you can actually do this upgrade yourself on your Solr 
install.  In server/solr-webapp/webapp/WEB-INF/lib, just delete the 
current zookeeper jar, copy the 3.4.13 jar into the directory, then 
restart Solr.  If you're on Windows, you'll need to stop Solr before you 
can do that.  Windows doesn't allow deleting a file that is open.


I expect that if you do this upgrade yourself, Solr should work without 
problems.  Typically in the past when a new ZK version is included, no 
code changes are required.


Thanks,
Shawn

Re: facet values from stored values

[Erick Erickson]

In a word, no. Performance would be horrible. I'd do the transformations in
your indexing code to a docValuea field and facet on that field.

On Sat, Sep 1, 2018, 07:17 nirsh  wrote:

> Hi,
>
> is there a way to return from the stored value of facets and not the
> indexed
> value?
> we have normalization we perform, such as lower case, punctuation etc..
> and need to display the stored value of it.
>
> Thanks,
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: facet values from stored values

[Walter Underwood]

I wrote some code to call an analysis chain in an update request processor
then add those values to fields. I started with some work by Eric Hatcher then
updated it for recent releases of Java.

Look at this discussion to see if it is the kind of thing you want.

http://lucene.472066.n3.nabble.com/Running-an-analyzer-chain-in-an-update-request-processor-td4384207.html

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Sep 1, 2018, at 1:02 PM, Erick Erickson  wrote:
> 
> In a word, no. Performance would be horrible. I'd do the transformations in
> your indexing code to a docValuea field and facet on that field.
> 
> On Sat, Sep 1, 2018, 07:17 nirsh  wrote:
> 
>> Hi,
>> 
>> is there a way to return from the stored value of facets and not the
>> indexed
>> value?
>> we have normalization we perform, such as lower case, punctuation etc..
>> and need to display the stored value of it.
>> 
>> Thanks,
>> 
>> --
>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>> 

Re: Sorting by custom order

[Shawn Heisey]

On 9/1/2018 12:36 AM, Salvo Bonanno wrote:

I need to sort a results set in a particolar way... the documents
looks like this:


There are two problems with what you want and your setup.


I need to sort them by profile_txt value (it's a multiValue field but
actually it contains just a single value), but since the possible
values are just 5, I'd like to tokenize them for deciding in wich
order the should came.


You can't sort on a multivalued field.  The fact that your documents 
only contain one value in that field makes no difference.  Solr will 
*refuse* to sort on the field.



the order should follows a simple schema:

1. profile_txt = "Gold"
2. profile_txt = "Super"
3. profile_txt = "Smart"
4. profile_txt = "Base"
5. profile_txt = "Essential"


This is the other problem.  Solr only sorts on two criteria -- numeric, 
or alphanumeric.  The sort order you want doesn't fit either of these.


Since you can't change the schema, I'm not entirely sure what you can 
do.  I think it's probably possible to provide a custom sort plugin for 
Solr, but I have absolutely no idea how to do it.


I don't know if it's possible to devise a function query that would 
result in this ordering or not.  Clever usage of the function query 
feature in Solr can do a lot of things.  This function, ascending, MIGHT 
do it:


if(exists(profile_txt),if(eq(profile_txt,Gold),5,if(eq(profile_txt,Super),10,if(eq(profile_txt,Smart),15,if(eq(profile_txt,Base),20,if(eq(profile_txt,Essential),25,1000))

I do not know if the string values need to be wrapped in quotes for that 
to work.  I'm also not entirely sure that I've placed the parentheses 
correctly.


If you do end up being able to change the schema, then you can add a 
numeric field and have your indexing program populate that field with a 
number based on the value of profile_txt, then you could sort on that 
field. Solr would not be able to do this for you unless you wrote a 
custom UpdateProcessor and activated it in your solrconfig.xml file.


Thanks,
Shawn

Re: Sorting by custom order

[Walter Underwood]

Use a enum field. Those are designed to map arbitrary values to a sort order.
The standard example is to apply an order to something that isn’t alpha like:
Error, Warning, Info.

https://lucene.apache.org/solr/guide/6_6/working-with-enum-fields.html

Enums convert an ordered set into a numeric sort order.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Sep 1, 2018, at 2:10 PM, Shawn Heisey  wrote:
> 
> On 9/1/2018 12:36 AM, Salvo Bonanno wrote:
>> I need to sort a results set in a particolar way... the documents
>> looks like this:
> 
> There are two problems with what you want and your setup.
> 
>> I need to sort them by profile_txt value (it's a multiValue field but
>> actually it contains just a single value), but since the possible
>> values are just 5, I'd like to tokenize them for deciding in wich
>> order the should came.
> 
> You can't sort on a multivalued field.  The fact that your documents only 
> contain one value in that field makes no difference.  Solr will *refuse* to 
> sort on the field.
> 
>> the order should follows a simple schema:
>> 
>> 1. profile_txt = "Gold"
>> 2. profile_txt = "Super"
>> 3. profile_txt = "Smart"
>> 4. profile_txt = "Base"
>> 5. profile_txt = "Essential"
> 
> This is the other problem.  Solr only sorts on two criteria -- numeric, or 
> alphanumeric.  The sort order you want doesn't fit either of these.
> 
> Since you can't change the schema, I'm not entirely sure what you can do.  I 
> think it's probably possible to provide a custom sort plugin for Solr, but I 
> have absolutely no idea how to do it.
> 
> I don't know if it's possible to devise a function query that would result in 
> this ordering or not.  Clever usage of the function query feature in Solr can 
> do a lot of things.  This function, ascending, MIGHT do it:
> 
> if(exists(profile_txt),if(eq(profile_txt,Gold),5,if(eq(profile_txt,Super),10,if(eq(profile_txt,Smart),15,if(eq(profile_txt,Base),20,if(eq(profile_txt,Essential),25,1000))
> 
> I do not know if the string values need to be wrapped in quotes for that to 
> work.  I'm also not entirely sure that I've placed the parentheses correctly.
> 
> If you do end up being able to change the schema, then you can add a numeric 
> field and have your indexing program populate that field with a number based 
> on the value of profile_txt, then you could sort on that field. Solr would 
> not be able to do this for you unless you wrote a custom UpdateProcessor and 
> activated it in your solrconfig.xml file.
> 
> Thanks,
> Shawn
> 

Re: BUMP: Atomic updates and POST command?

[Shawn Heisey]

On 8/31/2018 7:18 PM, Scott Prentice wrote:

Yup. That does the trick! Here's my command line ..

    $ ./bin/post -c core01 -format solr /home/xtech/solrtest/test1b.json

I saw that "-format solr" option, but it wasn't clear what it did. 
It's still not clear to me how that changes the endpoint to allow for 
updates. But nice to see that it works! 


I think the assumption with JSON-style updates and the post tool is that 
you are sending "generic" json documents, not Solr-formatted json 
commands.  So the post tool sends to the /update/json/docs handler, 
which can handle those easily.  I believe that telling it that the 
format is "solr" means that the JSON input is instructions to Solr, not 
just document content.  It very likely sends it to /update/json or 
/update when that's the case.


I don't know if you know this, but the bin/post command calls something 
in Solr that is named SimplePostTool.  It is, as that name suggests, a 
very simple tool.  Although you CAN use it in production, a large 
percentage of users find that they outgrow its capabilities and must 
move to writing their own indexing system.


Thanks,
Shawn