Re: SolrCloud 5.2.1 - collection creation error
Seems like I'm missing something simple here. SolrCloud installs and seems
to work fine and the examples work as they should on each individual
machine. What I'm trying to do is setup 4 separate Solr machines (Ubuntu
14.04) with 3 external zookeepers (the ones currently being used by our
production Solr 5.10 boxes).
I took Ansible out of the equation and installed Java 9 PPA and Solr 5.21
manually on fresh machines. I change the solr.in.sh file to point at my 3
external zookeepers and make sure everything else is set there according to
the "Taking Solr to Production" steps. I have uploaded the example
data_driven_schema_configs config files to zookeeper. I tried both the API
and command line to create a collection with the uploaded configs and get
the same error that I posted previously.
What additional steps might I be overlooking in setting up 4 separate Solr
machines for Solr Cloud?
On Fri, Jul 17, 2015 at 4:52 PM, Aaron Gibbons <
agibb...@synergydatasystems.com> wrote:
> I started from scratch with fresh Ubuntu machines and just wiped them and
> tried again. I run my Ansible playbook (below) to install Java 8 (Tried
> Oracle this time and even tried installing it manually) and SolrCloud 5.2.1
> as described previously. Solr cloud appears to be working fine but I still
> get the same error creating a collection. There is nothing else on these
> machines so I'm not sure where the conflict would come from. I'm using all
> the standard locations and settings just adding our external zookeepers. I
> can't see where Ansible would be causing a conflict here either it's just
> running the commands from the tutorial across the 4 machines.
>
> ...
>
> roles:
>
> - role: 'williamyeh.oracle-java'
>
> tasks:
>
> - name: Download Solr.
>
> get_url:
>
> url: "http://archive.apache.org/dist/lucene/solr/{{ solr_version
> }}/{{ solr_filename }}.tgz"
>
> dest: "{{ solr_workspace }}/{{ solr_filename }}.tgz"
>
> force: no
>
> - name: Extract the installation script.
>
> command: >
>
> tar xzf {{ solr_workspace }}/{{ solr_filename }}.tgz {{
> solr_filename }}/bin/install_solr_service.sh --strip-components=2
>
> - name: Run installation Script.
>
> command: "sudo bash ./install_solr_service.sh {{ solr_filename }}.tgz"
>
> - name: Stop solr.
>
> service: name=solr state=stopped
>
> - name: Copy Template init Config file into bin and restart.
>
> template:
>
> src: "solr-init-5.x.j2"
>
> dest: /var/solr/solr.in.sh
>
> - name: Replace Log4j.properties file for production logging settings.
>
> copy:
>
> src: "log4j.properties"
>
> dest: "/var/solr/log4j.properties"
> ...
>
> - name: Add sqljdbc.jar file to solr dist files
>
> copy:
>
> src: "sqljdbc4.jar"
>
> dest: "/opt/solr/dist/sqljdbc4.jar"
>
> - name: Start solr with new config.
>
> service: name=solr state=restarted
>
>
> On Thu, Jul 16, 2015 at 8:48 PM, Erick Erickson
> wrote:
>
>> It looks at a glance like you're in "Jar hell" and have one or more jar
>> files from "somewhere else" in your classpath, possibly a jar file from
>> an older Solr or one of the libraries.
>>
>> Best,
>> Erick
>>
>> On Thu, Jul 16, 2015 at 6:17 AM, Aaron Gibbons
>> wrote:
>> > I'm installing SolrCloud 5.2.1 on 4 Ubuntu 14.04 machines with 3
>> external
>> > zookeepers. I've installed the solr machines using Ansible following
>> the
>> > "Taking Solr to Production" steps.
>> >
>> >1. Download 5.2.1
>> >2. Extract installation script
>> >3. Run installation script
>> >
>> > Then I stop solr and make my configuration changes to the solr.in.sh
>> file
>>
>> > (adding zookeepers) and log4j.properties (recommended changes). Restart
>> > solr and everything looks good.
>> >
>> > The problem I have is that I can't create a collection. I create the
>> > collection folder in /var/solr/data and tried both the bin script and
>> API
>> > but get the error below. I've tried 5.2.0 also and both Java 7 and 8
>> with
>> > the same result.
>> >
>> > 50047java.io.InvalidClassException:
>> > org.apache.solr.client.solrj.SolrResponse; local class incompatible:
>> stream
>> > classdesc serialVersionUID = 3123208377723774018, local class
>> > serialVersionUI
Re: SolrCloud 5.2.1 - collection creation error
I've mainly used Oracle Java 8, but tested 7 also. Typically I'll wipe the
machines and start from scratch before installing a different version. The
latest attempt followed these steps exactly on each machine:
- sudo apt-get install python-software-properties
- sudo add-apt-repository ppa:webupd8team/java
- sudo apt-get update
- sudo apt-get install oracle-java8-installer
- cd ~
- wget http://apache.mirrors.ionfish.org/lucene/solr/5.2.1/solr-5.2.1.tgz
- tar xzf solr-5.2.1.tgz solr-5.2.1/bin/install_solr_service.sh
--strip-components=2
- sudo bash ./install_solr_service.sh solr-5.2.1.tgz
- sudo service solr stop
I then modify the solr.in.sh file in /var/solr/ and add my zookeepers,
hostname, etc according to the steps in Taking Solr to Production.
- sudo service solr start
Verify the interface is working and cloud tab is present. Then try to add
a core (config is already in zookeeper).
...collections?action=CREATE&name=test&collection.configname=test&numShards=2&replicationFactor=2
Or
/opt/solr/bin/solr create -c test -n test -shards 2 -replicationFactor 2
Nothing fancy at all...
How many versions of Java do you have? I bet the bin/solr script is
using SolrJ (i.e. Java) to communicate with Solr itself (also Java), but
for some weird reason they are using incompatible Java versions.
SolrJ uses Java serialization to communicate with Solr, therefore it
must use the same Jars and the same Java version in order to succeed at
that. Something weird is going on around that area, it seems.
Make sure you are using the same Java to start Solr as you are using to
run the bin/solr create script, and make sure you are using the same
version of Solr, too.
Upayavira
On Wed, Jul 22, 2015, at 08:42 PM, Aaron Gibbons wrote:
> Seems like I'm missing something simple here. SolrCloud installs and
> seems
> to work fine and the examples work as they should on each individual
> machine. What I'm trying to do is setup 4 separate Solr machines (Ubuntu
> 14.04) with 3 external zookeepers (the ones currently being used by our
> production Solr 5.10 boxes).
>
> I took Ansible out of the equation and installed Java 9 PPA and Solr 5.21
> manually on fresh machines. I change the solr.in.sh file to point at my
> 3
> external zookeepers and make sure everything else is set there according
> to
> the "Taking Solr to Production" steps. I have uploaded the example
> data_driven_schema_configs config files to zookeeper. I tried both the
> API
> and command line to create a collection with the uploaded configs and get
> the same error that I posted previously.
>
> What additional steps might I be overlooking in setting up 4 separate
> Solr
> machines for Solr Cloud?
>
>
> On Fri, Jul 17, 2015 at 4:52 PM, Aaron Gibbons <
> agibb...@synergydatasystems.com> wrote:
>
> > I started from scratch with fresh Ubuntu machines and just wiped them
and
> > tried again. I run my Ansible playbook (below) to install Java 8 (Tried
> > Oracle this time and even tried installing it manually) and SolrCloud
5.2.1
> > as described previously. Solr cloud appears to be working fine but I
still
> > get the same error creating a collection. There is nothing else on these
> > machines so I'm not sure where the conflict would come from. I'm using
all
> > the standard locations and settings just adding our external
zookeepers. I
> > can't see where Ansible would be causing a conflict here either it's
just
> > running the commands from the tutorial across the 4 machines.
> >
> > ...
> >
> > roles:
> >
> > - role: 'williamyeh.oracle-java'
> >
> > tasks:
> >
> > - name: Download Solr.
> >
> > get_url:
> >
> > url: "http://archive.apache.org/dist/lucene/solr/{{ solr_version
> > }}/{{ solr_filename }}.tgz"
> >
> > dest: "{{ solr_workspace }}/{{ solr_filename }}.tgz"
> >
> > force: no
> >
> > - name: Extract the installation script.
> >
> > command: >
> >
> > tar xzf {{ solr_workspace }}/{{ solr_filename }}.tgz {{
> > solr_filename }}/bin/install_solr_service.sh --strip-components=2
> >
> > - name: Run installation Script.
> >
> > command: "sudo bash ./install_solr_service.sh {{ solr_filename
}}.tgz"
> >
> > - name: Stop solr.
> >
> > service: name=solr state=stopped
> >
> > - name: Copy Template init Config file into bin and restart.
> >
> > template:
> >
> > src: "solr-init-5.x.j2"
> >
> > dest: /var/solr/solr.in.sh
> >
> > - name: Re
Re: SolrCloud 5.2.1 - collection creation error
*When you run bin/solr you are doing that on the instance itself? * Yes *You show a collections API URL below. Does that fail the same way?* Error from API: 50042java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755org.apache.solr.common.SolrException: java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 at org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:62)... Error from bin script: null:org.apache.solr.common.SolrException: java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 at org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:62)... *Is the core created correctly, even though bin/solr cannot handle the response back to you?* No. The Solr admin and the init script both have the java home set to: /usr/lib/jvm/java-8-oracle/jre On Thu, Jul 23, 2015 at 10:05 AM, Upayavira wrote: > I'd still like to just confirm that you're using the same Java for > running Solr and for running bin/solr. > > When you run bin/solr you are doing that on the instance itself? > > You show a collections API URL below. Does that fail the same way? > Basically, the exception you showed was a SolrJ error. Using the > collections API directly over HTTP should remove SolrJ from the picture > and thus remove your exception. Also, it seems to be an exception > parsing the SolrResponse. That would suggest that the action has > occurred already. Is the core created correctly, even though bin/solr > cannot handle the response back to you? > > Upayavira > > On Thu, Jul 23, 2015, at 01:50 PM, Aaron Gibbons wrote: > > I've mainly used Oracle Java 8, but tested 7 also. Typically I'll wipe > > the > > machines and start from scratch before installing a different version. > > The > > latest attempt followed these steps exactly on each machine: > > > >- sudo apt-get install python-software-properties > > > > > >- sudo add-apt-repository ppa:webupd8team/java > > > > > >- sudo apt-get update > > > > > >- sudo apt-get install oracle-java8-installer > > > > > >- cd ~ > >- wget > >http://apache.mirrors.ionfish.org/lucene/solr/5.2.1/solr-5.2.1.tgz > > > > > >- tar xzf solr-5.2.1.tgz solr-5.2.1/bin/install_solr_service.sh > >--strip-components=2 > > > > > >- sudo bash ./install_solr_service.sh solr-5.2.1.tgz > > > > > >- sudo service solr stop > > > > I then modify the solr.in.sh file in /var/solr/ and add my zookeepers, > > hostname, etc according to the steps in Taking Solr to Production. > > > >- sudo service solr start > > > > Verify the interface is working and cloud tab is present. Then try to > > add > > a core (config is already in zookeeper). > > > > > ...collections?action=CREATE&name=test&collection.configname=test&numShards=2&replicationFactor=2 > > > > Or > > > > /opt/solr/bin/solr create -c test -n test -shards 2 -replicationFactor 2 > > > > Nothing fancy at all... > > > > > > How many versions of Java do you have? I bet the bin/solr script is > > using SolrJ (i.e. Java) to communicate with Solr itself (also Java), but > > for some weird reason they are using incompatible Java versions. > > > > SolrJ uses Java serialization to communicate with Solr, therefore it > > must use the same Jars and the same Java version in order to succeed at > > that. Something weird is going on around that area, it seems. > > > > Make sure you are using the same Java to start Solr as you are using to > > run the bin/solr create script, and make sure you are using the same > > version of Solr, too. > > > > Upayavira > > > > On Wed, Jul 22, 2015, at 08:42 PM, Aaron Gibbons wrote: > > > Seems like I'm missing something simple here. SolrCloud installs and > > > seems > > > to work fine and the examples work as they should on each individual > > > machine. What I'm trying to do is setup 4 separate Solr machines > (Ubuntu > > > 14.04) with 3 external zookeepers (the ones currently being used by our > > > production Solr 5.10 boxes). > > > > > >
Re: SolrCloud 5.2.1 - collection creation error
I originally started using Ansible playbooks which did install the JDK (with the same error), but have been doing manual installs to take Ansible completely out of the equation. Safari wasn't giving showing the XML response so I ran this in Chrome.. http://XX.X.XX.XXX:8983/solr/admin/collections?action=CREATE&name=test&collection.configname=test <http://XX.X.XX.XXX:8983/solr/admin/collections?action=CREATE&name=mega&collection.configname=mega> 500 29 java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 org.apache.solr.common.SolrException: java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 at org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:62) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:228) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:168) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:621) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:60) ... 27 more 500 On Thu, Jul 23, 2015 at 11:26 AM, Upayavira wrote: > Have you tried it with a JDK? I tend to use JDK rather than JRE, but > don't recall whether this is a specific requirement for Solr. > > Can you show the URL you use for the API, and the JSON/XML response you > get? I wouldn't expect to see mention of solrj in the API because it > isn't used. Just for the record, by API I mean HTTP API, so calling the > solr instance from a browser, for example. > > Upayavira > > On Thu, Jul 23, 2015, at 04:07 PM, Aaron Gibbons wrote: > > *When you run bin/solr you are doing that on the instance itself? * > > Yes > > > > *You show a collections API URL below. Does that fail the same way?* > > Error from API: > > 50042java.io.InvalidClassException: > > org.apache.solr.client.solrj
Re: SolrCloud 5.2.1 - collection creation error
Ah, now we're on to something! Solr 4.10.0 is also using the same zookeepers, and both are using Oracle Java 8 JRE. Did some research and uploaded a new config to zookeeper using chroot to isolate them. Changed the init script to have ZK_Host="zk1,zk2,zk3/DevConfigs". I did see that you should bootstrap the chroot configs (znode tree) to Solr home as well. I'm now able to run the commands and create a collection! Thank you for all the help! On Thu, Jul 23, 2015 at 1:24 PM, Upayavira wrote: > Hmmm, what other Solr nodes do you have connected to Zookeeper? Are any > of them running a different Java or Solr version? > > It looks like you have another node connected to your Zookeeper that has > taken the role of "overseer" and it is sending back serialized java > objects that your own node cannot handle. > > Upayavira > > On Thu, Jul 23, 2015, at 04:52 PM, Aaron Gibbons wrote: > > I originally started using Ansible playbooks which did install the JDK > > (with the same error), but have been doing manual installs to take > > Ansible > > completely out of the equation. > > > > Safari wasn't giving showing the XML response so I ran this in Chrome.. > > > http://XX.X.XX.XXX:8983/solr/admin/collections?action=CREATE&name=test&collection.configname=test > > < > http://XX.X.XX.XXX:8983/solr/admin/collections?action=CREATE&name=mega&collection.configname=mega > > > > > > > > 500 > > 29 > > > > > > > > java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; > > local class incompatible: stream classdesc serialVersionUID = > > 3123208377723774018, local class serialVersionUID = 3945300637328478755 > > > > > > org.apache.solr.common.SolrException: java.io.InvalidClassException: > > org.apache.solr.client.solrj.SolrResponse; local class incompatible: > > stream > > classdesc serialVersionUID = 3123208377723774018, local class > > serialVersionUID = 3945300637328478755 at > > > org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:62) > > at > > > org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:228) > > at > > > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:168) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) > > at > > > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660) > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:431) at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) > > at > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > at > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) > > at > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > > at org.eclipse.jetty.server.Server.handle(Server.java:497) at > > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) > > at > > > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > > at > > > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:
Solr MLT with stream.body returns different results on each shard
I have a fresh install of Solr 5.2.1 with about 3 million docs freshly indexed (I can also reproduce this issue on 4.10.0). When I use the Solr MorelikeThisHandler with content stream I'm getting different results per shard. I also looked at using a standard MLT query, but I need to be able to stream in a fairly large block of text for comparison that is not in the index (different type of document). A standard MLT query http://testsolr2:8983/solr/mega/select?q=electronics&mlt.flt=text&mlt.mintf=0&fl=id,score appears to return consistent results between shards. Any reason why the content stream query would be different between shards? Thank you for your help! Aaron *Content Stream Example:* http://testsolr1:8983/solr/mega/mlt?stream.body=electronics&mlt.flt=text&mlt.mintf=0&fl=id,score *Returns: * 0 3 http://testsolr2:8983/solr/mega/mlt?stream.body=electronics&mlt.flt=text&mlt.mintf=0&fl=id,score *Returns: * 0 1
Solr Keyword query on a specific field.
Is there any way to have a single field search use the same keyword search logic as the default query? I define q.op as AND in my query which gets applied to any main keywords but any keywords I'm trying to use within a field do not get the same logic applied. Example: q=(title:(Test Keywords)) the space is treated as OR regardless of q.op q=(Test Keywords) the space is defined by q.op which is AND Using the correct operators (AND OR * - +...) it works great as I have it defined. There's just this one little caveat when you use spaces between keywords expecting the q.op operator to be applied. Thanks, Aaron
Re: Solr Keyword query on a specific field.
The input for the title field is user based so a wide range of things can
be entered there. Quoting the title is not what I'm looking for. I also
checked and q.op is AND and MM is 100%. In addition to the Title field the
user can also use general keywords so setting local params (df) to
something else would not work either to my knowledge.
To give you a better idea of what I'm trying to accomplish: I have a form
to allow users to search on Title, Keywords and add a location. The correct
operators are applied between each of these and also for the main keywords
themselves. The only issue is with the default operator being applied
within the Title sections's keywords. My goal is to have the Title keywords
work the same as the general keywords but only be applied to the title
field vs the default text field.
On Fri, Oct 30, 2015 at 6:35 PM, davidphilip cherian <
davidphilipcher...@gmail.com> wrote:
> >> "Is there any way to have a single field search use the same keyword
> search logic as the default query?"
> Do a phrase search, with double quotes surrounding the multiple keywords,
> it should work.
>
> Try q=title:("Test Keywords")
>
> You could possibly try adding this q.op as local param to query as shown
> below.
>
> https://cwiki.apache.org/confluence/display/solr/Local+Parameters+in+Queries
>
> If you are using edismax query parser, check for what is mm pram
> set. q.op=AND => mm=100%; q.op=OR => mm=0%)
>
> https://wiki.apache.org/solr/ExtendedDisMax#mm_.28Minimum_.27Should.27_Match.29
>
>
> On Fri, Oct 30, 2015 at 3:27 PM, Aaron Gibbons <
> agibb...@synergydatasystems.com> wrote:
>
> > Is there any way to have a single field search use the same keyword
> search
> > logic as the default query? I define q.op as AND in my query which gets
> > applied to any main keywords but any keywords I'm trying to use within a
> > field do not get the same logic applied.
> > Example:
> > q=(title:(Test Keywords)) the space is treated as OR regardless of q.op
> > q=(Test Keywords) the space is defined by q.op which is AND
> >
> > Using the correct operators (AND OR * - +...) it works great as I have it
> > defined. There's just this one little caveat when you use spaces between
> > keywords expecting the q.op operator to be applied.
> > Thanks,
> > Aaron
> >
>
Re: Solr Keyword query on a specific field.
I apologize for the long delay in response. I was able to get it to work
tho! Thank you!!
The local parameters were confusing to me at first. I'm using SolrNet to
build the search which has LocalParams that I am already specifying, but
those are not applied to the title portion. What I ended up doing is
applying the local params like the link you provided suggested, basically
wrapping just the Title portion of the query.
So in SolrNet I did:
solrQueries.Add(new SolrQuery("{!q.op=AND mm=100% df=current_position_title
v='" + Title Keywords + "'}"));
And the Solr query ends up being:
{!q.op=AND+mm=100%+df=current_position_title+v='Title Keywords'}
Now users can enter their keywords or boolean string and it's working just
as a Solr keyword search does. Exactly what I wanted!
I just have to figure out how to do the same thing using Sunspot tho.
On Sun, Nov 8, 2015 at 7:01 PM, davidphilip cherian <
davidphilipcher...@gmail.com> wrote:
> Nested queries might help.
>
> http://www.slideshare.net/erikhatcher/solr-query-parsing-tips-and-tricks
>
> On Mon, Nov 2, 2015 at 10:20 AM, Aaron Gibbons <
> agibb...@synergydatasystems.com> wrote:
>
> > The input for the title field is user based so a wide range of things can
> > be entered there. Quoting the title is not what I'm looking for. I also
> > checked and q.op is AND and MM is 100%. In addition to the Title field
> the
> > user can also use general keywords so setting local params (df) to
> > something else would not work either to my knowledge.
> >
> > To give you a better idea of what I'm trying to accomplish: I have a form
> > to allow users to search on Title, Keywords and add a location. The
> correct
> > operators are applied between each of these and also for the main
> keywords
> > themselves. The only issue is with the default operator being applied
> > within the Title sections's keywords. My goal is to have the Title
> keywords
> > work the same as the general keywords but only be applied to the title
> > field vs the default text field.
> >
> > On Fri, Oct 30, 2015 at 6:35 PM, davidphilip cherian <
> > davidphilipcher...@gmail.com> wrote:
> >
> > > >> "Is there any way to have a single field search use the same keyword
> > > search logic as the default query?"
> > > Do a phrase search, with double quotes surrounding the multiple
> keywords,
> > > it should work.
> > >
> > > Try q=title:("Test Keywords")
> > >
> > > You could possibly try adding this q.op as local param to query as
> shown
> > > below.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/solr/Local+Parameters+in+Queries
> > >
> > > If you are using edismax query parser, check for what is mm pram
> > > set. q.op=AND => mm=100%; q.op=OR => mm=0%)
> > >
> > >
> >
> https://wiki.apache.org/solr/ExtendedDisMax#mm_.28Minimum_.27Should.27_Match.29
> > >
> > >
> > > On Fri, Oct 30, 2015 at 3:27 PM, Aaron Gibbons <
> > > agibb...@synergydatasystems.com> wrote:
> > >
> > > > Is there any way to have a single field search use the same keyword
> > > search
> > > > logic as the default query? I define q.op as AND in my query which
> gets
> > > > applied to any main keywords but any keywords I'm trying to use
> within
> > a
> > > > field do not get the same logic applied.
> > > > Example:
> > > > q=(title:(Test Keywords)) the space is treated as OR regardless of
> q.op
> > > > q=(Test Keywords) the space is defined by q.op which is AND
> > > >
> > > > Using the correct operators (AND OR * - +...) it works great as I
> have
> > it
> > > > defined. There's just this one little caveat when you use spaces
> > between
> > > > keywords expecting the q.op operator to be applied.
> > > > Thanks,
> > > > Aaron
> > > >
> > >
> >
>
SolrCloud 5.2.1 - collection creation error
I'm installing SolrCloud 5.2.1 on 4 Ubuntu 14.04 machines with 3 external zookeepers. I've installed the solr machines using Ansible following the "Taking Solr to Production" steps. 1. Download 5.2.1 2. Extract installation script 3. Run installation script Then I stop solr and make my configuration changes to the solr.in.sh file (adding zookeepers) and log4j.properties (recommended changes). Restart solr and everything looks good. The problem I have is that I can't create a collection. I create the collection folder in /var/solr/data and tried both the bin script and API but get the error below. I've tried 5.2.0 also and both Java 7 and 8 with the same result. 50047java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755org.apache.solr.common.SolrException: java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 at org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:62) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:228) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:168) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:646) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:417) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse; local class incompatible: stream classdesc serialVersionUID = 3123208377723774018, local class serialVersionUID = 3945300637328478755 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) at org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:60) ... 27 more 500
Re: SolrCloud 5.2.1 - collection creation error
I started from scratch with fresh Ubuntu machines and just wiped them and
tried again. I run my Ansible playbook (below) to install Java 8 (Tried
Oracle this time and even tried installing it manually) and SolrCloud 5.2.1
as described previously. Solr cloud appears to be working fine but I still
get the same error creating a collection. There is nothing else on these
machines so I'm not sure where the conflict would come from. I'm using all
the standard locations and settings just adding our external zookeepers. I
can't see where Ansible would be causing a conflict here either it's just
running the commands from the tutorial across the 4 machines.
...
roles:
- role: 'williamyeh.oracle-java'
tasks:
- name: Download Solr.
get_url:
url: "http://archive.apache.org/dist/lucene/solr/{{ solr_version
}}/{{ solr_filename }}.tgz"
dest: "{{ solr_workspace }}/{{ solr_filename }}.tgz"
force: no
- name: Extract the installation script.
command: >
tar xzf {{ solr_workspace }}/{{ solr_filename }}.tgz {{ solr_filename
}}/bin/install_solr_service.sh --strip-components=2
- name: Run installation Script.
command: "sudo bash ./install_solr_service.sh {{ solr_filename }}.tgz"
- name: Stop solr.
service: name=solr state=stopped
- name: Copy Template init Config file into bin and restart.
template:
src: "solr-init-5.x.j2"
dest: /var/solr/solr.in.sh
- name: Replace Log4j.properties file for production logging settings.
copy:
src: "log4j.properties"
dest: "/var/solr/log4j.properties"
...
- name: Add sqljdbc.jar file to solr dist files
copy:
src: "sqljdbc4.jar"
dest: "/opt/solr/dist/sqljdbc4.jar"
- name: Start solr with new config.
service: name=solr state=restarted
On Thu, Jul 16, 2015 at 8:48 PM, Erick Erickson
wrote:
> It looks at a glance like you're in "Jar hell" and have one or more jar
> files from "somewhere else" in your classpath, possibly a jar file from
> an older Solr or one of the libraries.
>
> Best,
> Erick
>
> On Thu, Jul 16, 2015 at 6:17 AM, Aaron Gibbons
> wrote:
> > I'm installing SolrCloud 5.2.1 on 4 Ubuntu 14.04 machines with 3 external
> > zookeepers. I've installed the solr machines using Ansible following the
> > "Taking Solr to Production" steps.
> >
> >1. Download 5.2.1
> >2. Extract installation script
> >3. Run installation script
> >
> > Then I stop solr and make my configuration changes to the solr.in.sh
> file
> > (adding zookeepers) and log4j.properties (recommended changes). Restart
> > solr and everything looks good.
> >
> > The problem I have is that I can't create a collection. I create the
> > collection folder in /var/solr/data and tried both the bin script and API
> > but get the error below. I've tried 5.2.0 also and both Java 7 and 8 with
> > the same result.
> >
> > 50047java.io.InvalidClassException:
> > org.apache.solr.client.solrj.SolrResponse; local class incompatible:
> stream
> > classdesc serialVersionUID = 3123208377723774018, local class
> > serialVersionUID =
> 3945300637328478755org.apache.solr.common.SolrException:
> > java.io.InvalidClassException: org.apache.solr.client.solrj.SolrResponse;
> > local class incompatible: stream classdesc serialVersionUID =
> > 3123208377723774018, local class serialVersionUID = 3945300637328478755
> at
> >
> org.apache.solr.client.solrj.SolrResponse.deserialize(SolrResponse.java:62)
> > at
> >
> org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:228)
> > at
> >
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:168)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> > at
> >
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:646)
> > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:417) at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> > at
> >
> org.eclipse.jetty.security.SecurityHa
Solr Keyword search across docs?
I'm trying to produce a search that would optionally join the contents of 2
documents then allow a keyword search on them as if it were a single doc.
For example I'd have Person index and a Note index. I want to search the
person document combined with the notes for keywords A AND B AND C. A and
B are in the Person Document but C is in at Person's Notes.
What is the best way to accomplish this and still preserve searching on
just Person or Note individually?
I tried a join but it seems to keyword search only the Person document and
then filters those down by those with Notes.
Simplified query:
q={!join+from=person_note_id+to=id}A+B+C&fq=type:Person
The only other option I can see is to index all of the Notes contents on
the Person record. But this seems like a lot when I also have the
requirement to have them in their own separate index to search on their own.
Thank you for any help!
Group only top 50 results not All results.
I'm trying to figure out how I can query solr for the top X results THEN group and count only those top 50 by their owner. I can run a query to get the top 50 results that I want. solr/select?q=(current_position_title%3a(TEST))&rows=50 I've tried Faceting but I get all results faceted not just the top 50: solr/select?q=(current_position_title%3a(TEST))&start=0&rows=50&facet=true&facet.field=recruiterkeyid&facet.limit=-1&facet.mincount=1&facet.sort=true I've tried Grouping and get all results again grouped not just the top 50. solr/select?q=(current_position_title%3a(TEST))&rows=50&group=true&group.field=recruiterkeyid&group.limit=1&group.format=grouped&version=2.2 I could also run one search to get the top X record Id's then run a second Grouped query on those but I was hoping there was a less expensive way run the search. So what I need to get back are the distinct recruiterkeyid's from the top X query and the count of how many there are only in the top X results. I'll ultimately want to query the results for each of individual recruiterkeyid as well. I'm using SolrNet to build the query. Thank you for your help, Aaron
Solr changing the search when given many qf fields?
I found a tipping point where the search being built changes with the number of qf fields being passed in. Example search: "foo bar" solr 7.2.1 select?q.op=AND&defType=edismax&q=foo bar Debugging the query you can see it results in: "parsedquery_toString":"+(+(text:foo) +(text:bar))" Adding more qf values you get: "text name_text" "parsedquery_toString":"+(+(name_text:foo | text:foo) +(name_text:bar | text:bar))" "text name_text city_text" "parsedquery_toString":"+(+(city_text:foo | name_text:foo | text:foo) +(city_text:bar | name_text:bar | text:bar))" The search continues to build this way until I get to a certain amount of qf values. Large number of qf values: "34 values.." "parsedquery_toString":"+(+((+comments_text:foo +comments_text:bar) | (+zip_text:foo +zip_text:bar) | (+city_text:foo +city_text:bar) | (+street_address_text:foo +street_address_text:bar) | (+street_address_two_text:foo +street_address_two_text:bar) | (+state_text:foo +state_text:bar)..." Now the search is requiring both foo and bar to be in each qf field in the search, not foo to be in any qf field and bar to be in any qf field. I had to cut the number of qf values down to 15 to get it back to the correct search. Why is the search changing? Is there any way around this or a better way we should be doing the search? I realize we could copy all of the fields to the default text field. However, most of the fields are searchable individually as well as keyword searchable so specifying the fields vs using the default text field makes sense in that respect. Thank you, Aaron
Re: Solr changing the search when given many qf fields?
Thank you Emir! This was certainly the case. One field was processed with EdgeNGramTokenizerFactory and should not have been. Fixed. On Thu, Aug 16, 2018 at 4:46 AM, Emir Arnautović < emir.arnauto...@sematext.com> wrote: > Hi Aaron, > It is probably not about number of fields but related to different > analysis of different fields. As long as all your fields analyzers produce > the same tokens you should get “term centric” query. Once any of your > analyzers produce different token, it’ll become “field centric”. It is > likely that one of your fields (tipping point) is string and produces “foo > bar” token. > Here is blog post explaining this part of edismax in a context of multi > term synonyms: https://opensourceconnections.com/blog/2018/02/20/edismax- > and-multiterm-synonyms-oddities/ <https://opensourceconnections.com/ > blog/2018/02/20/edismax-and-multiterm-synonyms-oddities/> > > HTH, > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > On 15 Aug 2018, at 17:23, Aaron Gibbons > wrote: > > > > I found a tipping point where the search being built changes with the > > number of qf fields being passed in. > > > > Example search: "foo bar" > > solr 7.2.1 > > select?q.op=AND&defType=edismax&q=foo bar > > > > Debugging the query you can see it results in: > > "parsedquery_toString":"+(+(text:foo) +(text:bar))" > > > > Adding more qf values you get: > > "text name_text" > > "parsedquery_toString":"+(+(name_text:foo | text:foo) +(name_text:bar | > > text:bar))" > > "text name_text city_text" > > "parsedquery_toString":"+(+(city_text:foo | name_text:foo | text:foo) > > +(city_text:bar | name_text:bar | text:bar))" > > > > The search continues to build this way until I get to a certain amount of > > qf values. > > > > Large number of qf values: > > "34 values.." > > "parsedquery_toString":"+(+((+comments_text:foo +comments_text:bar) | > > (+zip_text:foo +zip_text:bar) | (+city_text:foo +city_text:bar) | > > (+street_address_text:foo +street_address_text:bar) | > > (+street_address_two_text:foo +street_address_two_text:bar) | > > (+state_text:foo +state_text:bar)..." > > Now the search is requiring both foo and bar to be in each qf field in > the > > search, not foo to be in any qf field and bar to be in any qf field. I > had > > to cut the number of qf values down to 15 to get it back to the correct > > search. > > > > Why is the search changing? Is there any way around this or a better way > we > > should be doing the search? > > I realize we could copy all of the fields to the default text field. > However, > > most of the fields are searchable individually as well as keyword > > searchable so specifying the fields vs using the default text field makes > > sense in that respect. > > > > Thank you, > > Aaron > >
