Solr search speed very low
Dear ladies and gentlemen. I'm newbie with Solr, I didn't find an aswer in wiki, so I'm writing here. I'm analysing Solr performance and have 1 problem. *Search time is about 7-10 seconds per query.* I have a *.csv 5Gb-database with about 15 fields and 1 key field (record number). I uploaded it to Solr without any problem using curl. This database contains information about books and I'm intrested in keyword search using one of the fields (not a key field). I mean that if I search, for example, for word "Hello", I expect response with sentences containing "Hello": "Hello all" "Hello World" "I say Hello to all" etc. I tested it from console using time command and curl: /usr/bin/time -o test_results/time_solr -a curl "http://localhost:8983/solr/select/?q=itemname:*$query*&version=2.2&start=0&rows=10&indent=on"; -6 2>&1 >> test_results/response_solr So, my query is *itemname:*$query**. 'Itemname' - is the name of field. $query - is a bash variable containing only 1 word. All works fine. *But unfortunately, search time is about 7-10 seconds per query.* For example, Sphinx spent only about 0.3 second per query. If I use only $query, without stars (*), I receive answer pretty fast, but only exact matches. And I want to see any sentence containing my $query in the response. Thats why I'm using stars. NOW THE QUESTION. Is my query syntax correct (*field:*word**) for keyword search)? Why response time is so big? Can I reduce search time? Thank You in advance, Kind Regards, Andrey Sapegin, Software Developer, Unister GmbH Barfußgässchen 11 | 04109 Leipzig andrey.sape...@unister-gmbh.de <mailto:%20andreas.b...@unister-gmbh.de> www.unister.de <http://www.unister.de>
Problems with SolrCloud
Dear ladies and gentlemen. Can I ask you to help me with SolrCloud 1) I try to setup a SolrCloud on 2 computers with 3 Zookepers, but it fails:( I need to set Zookeper port to 8001, so I change clientPort=8001 in solr/zoo.cfg. When I try the command from the example C, to run shard1, it works: java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar start.jar But if I change it to and try to run shard1: java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:8001,localhost:8004 -jar start.jar it fails with the following message: SEVERE: java.lang.IllegalArgumentException: solr/zoo_data/myid file is missing 2) to solve it I tried to set *-Dsolr.solr.home=/data/a.sapegin/SolrCloud/shard1* (without any slashes in the end) But then I receive another exception: "Caused by: org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /data/a.sapegin/SolrCloud/shard1//zoo.cfg" I think this "//" is a bug. Could you please help? Thank You in advance, Kind Regards, -- Andrey Sapegin, Software Developer, Unister GmbH Dittrichring 18-20 | 04109 Leipzig +49 (0)341 492885069, +4915778339304, andrey.sape...@unister-gmbh.de www.unister.de
Problems with SolrCloud
Dear ladies and gentlemen. Can I ask you to help me with SolrCloud 1) I try to setup a SolrCloud on 2 computers with 3 Zookepers, but it fails:( I need to set Zookeper port to 8001, so I change clientPort=8001 in solr/zoo.cfg. When I try the command from the example C, to run shard1, it works: java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar start.jar But if I change it to and try to run shard1: java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:8001,localhost:8004 -jar start.jar it fails with the following message: SEVERE: java.lang.IllegalArgumentException: solr/zoo_data/myid file is missing 2) to solve it I tried to set *-Dsolr.solr.home=/data/a.sapegin/SolrCloud/shard1* (without any slashes in the end) But then I receive another exception: "Caused by: org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing /data/a.sapegin/SolrCloud/shard1//zoo.cfg" I think this "//" is a bug. Could you please help? Thank You in advance, Kind Regards, -- Andrey Sapegin, Software Developer, Unister GmbH Dittrichring 18-20 | 04109 Leipzig +49 (0)341 492885069, +4915778339304, andrey.sape...@unister-gmbh.de www.unister.de
Re: Problems with SolrCloud
I have solved this problem by creating myid file with Zookeper server id in solr/zoo_data folder. However, it's still not working. Now when I try to run it, I have the following in the console output: ... Jul 6, 2011 4:23:11 PM org.apache.zookeeper.ClientCnxn$SendThread startConnect INFO: Opening socket connection to server localhost/127.0.0.1:8022 Jul 6, 2011 4:23:11 PM org.apache.zookeeper.ClientCnxn$SendThread primeConnection INFO: Socket connection established to localhost/127.0.0.1:8022, initiating session Jul 6, 2011 4:23:11 PM org.apache.zookeeper.ClientCnxn$SendThread run INFO: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect Jul 6, 2011 4:23:11 PM org.apache.zookeeper.server.NIOServerCnxn$Factory run INFO: Accepted socket connection from /127.0.0.1:44432 Jul 6, 2011 4:23:11 PM org.apache.zookeeper.server.NIOServerCnxn doIO WARNING: Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running Jul 6, 2011 4:23:11 PM org.apache.zookeeper.server.NIOServerCnxn closeSock INFO: Closed socket connection for client /127.0.0.1:44432 (no session established for client) ... And http://localhost:8983/solr/admin/zookeeper.jsp is not available:( My configuration (SolrCloud with 2 shards (and 1 replica for each shard) with 4 Zookeepers): 1) shard1 + Zookeper without -Djetty.port it will use standard port 8983 solr/zoo.cfg changed to use port 8011: clientPort=8011 (shard 1, replica 1) the file myid in the default Zookeeper data folder contains: 11 (shard 1, replica 1) $ screen -S shard1 java -server -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf -DzkRun -DzkHost=localhost:8011,localhost:8012,localhost:8021,localhost:8022 -jar start.jar 2) shard1-replica + Zookeper solr/zoo.cfg should changed to use port 8012: clientPort=8012 (shard 1, replica 2) the file myid in the default Zookeeper data folder contains: 12 (shard 1, replica 2) $ screen -S shard1_replica java -server -Djetty.port=8984 -DhostPort=8984 -DzkRun -DzkHost=localhost:8011,localhost:8012,localhost:8021,localhost:8022 -jar start.jar 3) shard2 + Zookeper solr/zoo.cfg should changed to use port 8021: clientPort=8021 (shard 2, replica 1) the file myid in the default Zookeeper data folder contains: 21 (shard 2, replica 1) $ screen -S shard2 java -server -Djetty.port=8985 -DhostPort=8985 -DzkRun -DzkHost=localhost:8011,localhost:8012,localhost:8021,localhost:8022 -jar start.jar 4) shard2-replica + Zookeper: solr/zoo.cfg should changed to use port 8022: clientPort=8022 (shard 2, replica 2) the file myid in the default Zookeeper data folder contains: 22 (shard 2, replica 2) $ screen -S shard2_replica java -server -Djetty.port=8986 -DhostPort=8986 -DzkRun -DzkHost=localhost:8011,localhost:8012,localhost:8021,localhost:8022 -jar start.jar On 06/30/2011 05:04 PM, Andrey Sapegin wrote: > Dear ladies and gentlemen. > > Can I ask you to help me with SolrCloud > > 1) I try to setup a SolrCloud on 2 computers with 3 Zookepers, but it > fails:( > > I need to set Zookeper port to 8001, so I change clientPort=8001 in > solr/zoo.cfg. > > When I try the command from the example C, to run shard1, it works: > java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf > -DzkRun -DzkHost=localhost:9983,localhost:8574,localhost:9900 -jar > start.jar > > But if I change it to and try to run shard1: > java -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf > -DzkRun -DzkHost=localhost:8001,localhost:8004 -jar start.jar > > it fails with the following message: > SEVERE: java.lang.IllegalArgumentException: solr/zoo_data/myid file is > missing > > 2) to solve it I tried to set > *-Dsolr.solr.home=/data/a.sapegin/SolrCloud/shard1* > (without any slashes in the end) > > But then I receive another exception: > "Caused by: > org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: > Error processing /data/a.sapegin/SolrCloud/shard1//zoo.cfg" > > I think this "//" is a bug. > > > Could you please help? > Thank You in advance, > Kind Regards, > > -- Andrey Sapegin, Software Developer, Unister GmbH Dittrichring 18-20 | 04109 Leipzig +49 (0)341 492885069, +4915778339304, andrey.sape...@unister-gmbh.de www.unister.de
