Is there a document that tells how to create multiple threads? Search
returns many hits which orbit this idea, but I haven't spotted one
which tells how.
Thanks
Jack
On Fri, Mar 15, 2013 at 1:01 PM, Mark Miller <markrmil...@gmail.com> wrote:
> You def have to use multiple threads with it for it to be fast, but 3 or 4
> docs a second still sounds absurdly slow.
>
> - Mark
>
> On Mar 15, 2013, at 2:58 PM, Luis Cappa Banda <luisca...@gmail.com> wrote:
>
>> And up! :-)
>>
>> I´ve been wondering if using CloudSolrServer has something to do here. Does
>> it have a bad performance when a CloudSolrServer singletong receives
>> multiple queries? Is it recommended to have a CloudSolrServer instances
>> list and select one of them with a Round Robin criteria?
>>
>>
>>
>> 2013/3/14 Luis Cappa Banda <luisca...@gmail.com>
>>
>>> Hello!
>>>
>>> Thanks a lot, Erick! I've attached some stack traces during a normal
>>> 'engine' running.
>>>
>>> Cheers,
>>>
>>> - Luis Cappa
>>>
>>>
>>> 2013/3/13 Erick Erickson <erickerick...@gmail.com>
>>>
>>>> Stack traces..
>>>>
>>>> First,
>>>> jps -l
>>>>
>>>> that will give you a the process IDs of your running Java processes. Then:
>>>>
>>>> jstack <pid from above>
>>>>
>>>> Usually I pipe the output from jstack into a text file...
>>>>
>>>> Best
>>>> Erick
>>>>
>>>>
>>>> On Wed, Mar 13, 2013 at 1:48 PM, Luis Cappa Banda <luisca...@gmail.com
>>>>> wrote:
>>>>
>>>>> Uhm, how can I do that... 'cleanly'? I know that with JConsole it´s
>>>> posible
>>>>> to output this traces, but with a .war application built on top of
>>>> Spring I
>>>>> don´t know how can I do that. In any case, here is my CloudSolrServer
>>>>> wrapper that is used by other classes. There is no sync method or piece
>>>> of
>>>>> code:
>>>>>
>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>> - -
>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>
>>>>> *public class BinaryLBHttpSolrServer extends LBHttpSolrServer {*
>>>>>
>>>>> private static final long serialVersionUID = 3905956120804659445L;
>>>>> public BinaryLBHttpSolrServer(String[] endpoints) throws
>>>>> MalformedURLException {
>>>>> super(endpoints);
>>>>> }
>>>>>
>>>>> @Override
>>>>> protected HttpSolrServer makeServer(String server) throws
>>>>> MalformedURLException {
>>>>> HttpSolrServer solrServer = super.makeServer(server);
>>>>> solrServer.setRequestWriter(new BinaryRequestWriter());
>>>>> return solrServer;
>>>>> }
>>>>> }
>>>>>
>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>> - -
>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>>>
>>>>> *public class CloudSolrHttpServerImpl implements CloudSolrHttpServer {*
>>>>> private CloudSolrServer cloudSolrServer;
>>>>>
>>>>> private Logger log = Logger.getLogger(CloudSolrHttpServerImpl.class);
>>>>>
>>>>> public CloudSolrHttpServerImpl(String zookeeperEndpoints, String[]
>>>>> endpoints, int clientTimeout,
>>>>> int connectTimeout, String cloudCollection) {
>>>>> try {
>>>>> BinaryLBHttpSolrServer lbSolrServer = new *BinaryLBHttpSolrServer*
>>>>> (endpoints);
>>>>> this.cloudSolrServer = new CloudSolrServer(zookeeperEndpoints,
>>>>> lbSolrServer);
>>>>> this.cloudSolrServer.setZkConnectTimeout(connectTimeout);
>>>>> this.cloudSolrServer.setZkClientTimeout(clientTimeout);
>>>>> this.cloudSolrServer.setDefaultCollection(cloudCollection);
>>>>> } catch (MalformedURLException e) {
>>>>> log.error(e);
>>>>> }
>>>>> }
>>>>>
>>>>> @Override
>>>>> public QueryResponse *search*(SolrQuery query) throws
>>>> SolrServerException {
>>>>> return cloudSolrServer.query(query, METHOD.POST);
>>>>> }
>>>>>
>>>>> @Override
>>>>> public boolean *index*(DocumentBean user) {
>>>>> boolean indexed = false;
>>>>> int retries = 0;
>>>>> do {
>>>>> indexed = addBean(user);
>>>>> retries++;
>>>>> } while(!indexed && retries<4);
>>>>> return indexed;
>>>>> }
>>>>> @Override
>>>>> public boolean *update*(SolrInputDocument updateDoc) {
>>>>> boolean update = false;
>>>>> int retries = 0;
>>>>>
>>>>> do {
>>>>> update = addSolrInputDocument(updateDoc);
>>>>> retries++;
>>>>> } while(!update && retries<4);
>>>>> return update;
>>>>> }
>>>>> @Override
>>>>> public void commit() {
>>>>> try {
>>>>> cloudSolrServer.commit();
>>>>> } catch (SolrServerException e) {
>>>>> log.error(e);
>>>>> } catch (IOException e) {
>>>>> log.error(e);
>>>>> }
>>>>> }
>>>>>
>>>>> @Override
>>>>> public boolean *delete*(String ... ids) {
>>>>> boolean deleted = false;
>>>>> List<String> idList = Arrays.asList(ids);
>>>>> try {
>>>>> this.cloudSolrServer.deleteById(idList);
>>>>> this.cloudSolrServer.commit(true, true);
>>>>> deleted = true;
>>>>>
>>>>> } catch (SolrServerException e) {
>>>>> log.error(e);
>>>>>
>>>>> } catch (IOException e) {
>>>>> log.error(e);
>>>>> }
>>>>> return deleted;
>>>>> }
>>>>>
>>>>> @Override
>>>>> public void *optimize*() {
>>>>> try {
>>>>> this.cloudSolrServer.optimize();
>>>>> } catch (SolrServerException e) {
>>>>> log.error(e);
>>>>> } catch (IOException e) {
>>>>> log.error(e);
>>>>> }
>>>>> }
>>>>> /*
>>>>> * ********************
>>>>> * Getters & setters *
>>>>> * ********************
>>>>> * */
>>>>> public CloudSolrServer getSolrServer() {
>>>>> return cloudSolrServer;
>>>>> }
>>>>>
>>>>> public void setSolrServer(CloudSolrServer solrServer) {
>>>>> this.cloudSolrServer = solrServer;
>>>>> }
>>>>>
>>>>> private boolean addBean(DocumentBean user) {
>>>>> boolean added = false;
>>>>> try {
>>>>> this.cloudSolrServer.addBean(user, 100);
>>>>> this.commit();
>>>>>
>>>>> } catch (IOException e) {
>>>>> log.error(e);
>>>>>
>>>>> } catch (SolrServerException e) {
>>>>> log.error(e);
>>>>> }catch(SolrException e) {
>>>>> log.error(e);
>>>>> }
>>>>> return added;
>>>>> }
>>>>> private boolean addSolrInputDocument(SolrInputDocument updateDoc) {
>>>>> boolean added = false;
>>>>> try {
>>>>> this.cloudSolrServer.add(updateDoc, 100);
>>>>> this.commit();
>>>>> added = true;
>>>>> } catch (IOException e) {
>>>>> log.error(e);
>>>>>
>>>>> } catch (SolrServerException e) {
>>>>> log.error(e);
>>>>> }catch(SolrException e) {
>>>>> log.error(e);
>>>>> }
>>>>> return added;
>>>>> }
>>>>> }
>>>>>
>>>>> Thank you very much, Mark.
>>>>>
>>>>>
>>>>> - Luis Cappa
>>>>>
>>>>>
>>>>>
>>>>> And
>>>>> 2013/3/13 Mark Miller <markrmil...@gmail.com>
>>>>>
>>>>>>
>>>>>> Could you capture some thread stack traces in the 'engine' and see if
>>>>>> there are any blocking methods?
>>>>>>
>>>>>> - Mark
>>>>>>
>>>>>> On Mar 13, 2013, at 1:34 PM, Luis Cappa Banda <luisca...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>>> Just one correction:
>>>>>>>
>>>>>>> When I said:
>>>>>>>
>>>>>>> - I´ve checked SolrCloud via Solr Admin interface and it´s OK:
>>>>>>> everything is green, and I cant execute queries directly into
>>>> Solr.
>>>>>>>
>>>>>>> I mean:
>>>>>>>
>>>>>>>
>>>>>>> - I´ve checked SolrCloud via Solr Admin interface and it´s OK:
>>>>>>> everything is green, and *I can* execute queries directly into
>>>> Solr.
>>>>>>>
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> - Luis Cappa
>>>>>>>
>>>>>>>
>>>>>>> 2013/3/13 Luis Cappa Banda <luisca...@gmail.com>
>>>>>>>
>>>>>>>> Hello, guys!
>>>>>>>>
>>>>>>>> I´ve been experiencing some annoying behavior with my current
>>>>> production
>>>>>>>> scenario. Here is the snapshot:
>>>>>>>>
>>>>>>>>
>>>>>>>> - SolrCloud: 2 shards
>>>>>>>> - Zookeeper ensemble: 3 nodes in *different machines *(most of
>>>> the
>>>>>>>> tutorials installs 3 Zookeeper nodes in the same machine).
>>>>>>>> - This is the zoo.cfg from every
>>>>>>>>
>>>>>>>> tickTime=2000 // I´ve also tried with 60000
>>>>>>>>
>>>>>>>> initLimit=10
>>>>>>>>
>>>>>>>> syncLimit=5
>>>>>>>>
>>>>>>>> dataDir=/var/lib/zookeeper
>>>>>>>>
>>>>>>>> clientPort=9000
>>>>>>>>
>>>>>>>> server.1=zoohost1:2888:3888
>>>>>>>>
>>>>>>>> server.2=zoohost1:2888:3888
>>>>>>>>
>>>>>>>> server.3=zoohost1:2888:3888
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> - I´ve developed a Java Application with a REST API (let´s call
>>>> it *
>>>>>>>> engine*) that dispatches queries into SolrCloud. It´s a wrapper
>>>>> around
>>>>>>>> CloudSolrServer, so it´s mandatory to specify some Zookeeper
>>>>>> configuration
>>>>>>>> params too. They are loaded dynamically when the application is
>>>>>> deployed in
>>>>>>>> a Tomcat server, but the current values that I´m using are as
>>>>> follows:
>>>>>>>>
>>>>>>>> cloudSolrServer.*setZkConnectTimeout(60000)*
>>>>>>>>
>>>>>>>> cloudSolrServer.*setZkClientTimeout(60000)*
>>>>>>>> *
>>>>>>>> *
>>>>>>>> *
>>>>>>>> *
>>>>>>>>
>>>>>>>> *THE PROBLEM*
>>>>>>>> *
>>>>>>>> *
>>>>>>>> Everything goes OK, but after two days more or less (yes, I´ve
>>>> checked
>>>>>>>> that this behavior occurrs periodically, more or less) the *engine
>>>>>> blocks
>>>>>>>> * and cannot dispatch any query to SolrCloud.
>>>>>>>>
>>>>>>>> - The *engine *log only outputs "updating Zookeeper..." one last
>>>>> time,
>>>>>>>> but never updates.
>>>>>>>> - I´ve checked SolrCloud via Solr Admin interface and it´s OK:
>>>>>>>> everything is green, and I cant execute queries directly into
>>>> Solr.
>>>>>>>> - So then Solr appears to be OK, so the next step is to restart
>>>>>> *engine
>>>>>>>> but *it again appears "updating Zookeeper...". Unfortunately
>>>> switch
>>>>>>>> off + switch on doesn´t work here, :-(
>>>>>>>> - I´ve checked too Zookeeper logs and it appears some connection
>>>> log
>>>>>>>> outs, but the ensemble appears to be OK too.
>>>>>>>> - *The end: *If I restart Zookeeper one by one, and I restart
>>>>>>>> SolrCloud, plus I restart the engine, the problem is solved. I´m
>>>>> using
>>>>>>>> Amazon AWS as hostage, so I discard connection problems between
>>>>>> instances.
>>>>>>>>
>>>>>>>>
>>>>>>>> Does anyone experienced something similar? Can anybody shed some
>>>> light
>>>>>> on
>>>>>>>> this problem?
>>>>>>>>
>>>>>>>> Thank you very much.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>>
>>>>>>>> - Luis Cappa
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>
