Me neither. Please, Mark, can you tell us how?
2013/3/15 Jack Park <jackp...@topicquests.org>
> Is there a document that tells how to create multiple threads? Search
> returns many hits which orbit this idea, but I haven't spotted one
> which tells how.
>
> Thanks
> Jack
>
> On Fri, Mar 15, 2013 at 1:01 PM, Mark Miller <markrmil...@gmail.com>
> wrote:
> > You def have to use multiple threads with it for it to be fast, but 3 or
> 4 docs a second still sounds absurdly slow.
> >
> > - Mark
> >
> > On Mar 15, 2013, at 2:58 PM, Luis Cappa Banda <luisca...@gmail.com>
> wrote:
> >
> >> And up! :-)
> >>
> >> I´ve been wondering if using CloudSolrServer has something to do here.
> Does
> >> it have a bad performance when a CloudSolrServer singletong receives
> >> multiple queries? Is it recommended to have a CloudSolrServer instances
> >> list and select one of them with a Round Robin criteria?
> >>
> >>
> >>
> >> 2013/3/14 Luis Cappa Banda <luisca...@gmail.com>
> >>
> >>> Hello!
> >>>
> >>> Thanks a lot, Erick! I've attached some stack traces during a normal
> >>> 'engine' running.
> >>>
> >>> Cheers,
> >>>
> >>> - Luis Cappa
> >>>
> >>>
> >>> 2013/3/13 Erick Erickson <erickerick...@gmail.com>
> >>>
> >>>> Stack traces..
> >>>>
> >>>> First,
> >>>> jps -l
> >>>>
> >>>> that will give you a the process IDs of your running Java processes.
> Then:
> >>>>
> >>>> jstack <pid from above>
> >>>>
> >>>> Usually I pipe the output from jstack into a text file...
> >>>>
> >>>> Best
> >>>> Erick
> >>>>
> >>>>
> >>>> On Wed, Mar 13, 2013 at 1:48 PM, Luis Cappa Banda <
> luisca...@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Uhm, how can I do that... 'cleanly'? I know that with JConsole it´s
> >>>> posible
> >>>>> to output this traces, but with a .war application built on top of
> >>>> Spring I
> >>>>> don´t know how can I do that. In any case, here is my CloudSolrServer
> >>>>> wrapper that is used by other classes. There is no sync method or
> piece
> >>>> of
> >>>>> code:
> >>>>>
> >>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >>>> - -
> >>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >>>>>
> >>>>> *public class BinaryLBHttpSolrServer extends LBHttpSolrServer {*
> >>>>>
> >>>>> private static final long serialVersionUID = 3905956120804659445L;
> >>>>> public BinaryLBHttpSolrServer(String[] endpoints) throws
> >>>>> MalformedURLException {
> >>>>> super(endpoints);
> >>>>> }
> >>>>>
> >>>>> @Override
> >>>>> protected HttpSolrServer makeServer(String server) throws
> >>>>> MalformedURLException {
> >>>>> HttpSolrServer solrServer = super.makeServer(server);
> >>>>> solrServer.setRequestWriter(new BinaryRequestWriter());
> >>>>> return solrServer;
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >>>> - -
> >>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >>>>>
> >>>>> *public class CloudSolrHttpServerImpl implements CloudSolrHttpServer
> {*
> >>>>> private CloudSolrServer cloudSolrServer;
> >>>>>
> >>>>> private Logger log = Logger.getLogger(CloudSolrHttpServerImpl.class);
> >>>>>
> >>>>> public CloudSolrHttpServerImpl(String zookeeperEndpoints, String[]
> >>>>> endpoints, int clientTimeout,
> >>>>> int connectTimeout, String cloudCollection) {
> >>>>> try {
> >>>>> BinaryLBHttpSolrServer lbSolrServer = new *BinaryLBHttpSolrServer*
> >>>>> (endpoints);
> >>>>> this.cloudSolrServer = new CloudSolrServer(zookeeperEndpoints,
> >>>>> lbSolrServer);
> >>>>> this.cloudSolrServer.setZkConnectTimeout(connectTimeout);
> >>>>> this.cloudSolrServer.setZkClientTimeout(clientTimeout);
> >>>>> this.cloudSolrServer.setDefaultCollection(cloudCollection);
> >>>>> } catch (MalformedURLException e) {
> >>>>> log.error(e);
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> @Override
> >>>>> public QueryResponse *search*(SolrQuery query) throws
> >>>> SolrServerException {
> >>>>> return cloudSolrServer.query(query, METHOD.POST);
> >>>>> }
> >>>>>
> >>>>> @Override
> >>>>> public boolean *index*(DocumentBean user) {
> >>>>> boolean indexed = false;
> >>>>> int retries = 0;
> >>>>> do {
> >>>>> indexed = addBean(user);
> >>>>> retries++;
> >>>>> } while(!indexed && retries<4);
> >>>>> return indexed;
> >>>>> }
> >>>>> @Override
> >>>>> public boolean *update*(SolrInputDocument updateDoc) {
> >>>>> boolean update = false;
> >>>>> int retries = 0;
> >>>>>
> >>>>> do {
> >>>>> update = addSolrInputDocument(updateDoc);
> >>>>> retries++;
> >>>>> } while(!update && retries<4);
> >>>>> return update;
> >>>>> }
> >>>>> @Override
> >>>>> public void commit() {
> >>>>> try {
> >>>>> cloudSolrServer.commit();
> >>>>> } catch (SolrServerException e) {
> >>>>> log.error(e);
> >>>>> } catch (IOException e) {
> >>>>> log.error(e);
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> @Override
> >>>>> public boolean *delete*(String ... ids) {
> >>>>> boolean deleted = false;
> >>>>> List<String> idList = Arrays.asList(ids);
> >>>>> try {
> >>>>> this.cloudSolrServer.deleteById(idList);
> >>>>> this.cloudSolrServer.commit(true, true);
> >>>>> deleted = true;
> >>>>>
> >>>>> } catch (SolrServerException e) {
> >>>>> log.error(e);
> >>>>>
> >>>>> } catch (IOException e) {
> >>>>> log.error(e);
> >>>>> }
> >>>>> return deleted;
> >>>>> }
> >>>>>
> >>>>> @Override
> >>>>> public void *optimize*() {
> >>>>> try {
> >>>>> this.cloudSolrServer.optimize();
> >>>>> } catch (SolrServerException e) {
> >>>>> log.error(e);
> >>>>> } catch (IOException e) {
> >>>>> log.error(e);
> >>>>> }
> >>>>> }
> >>>>> /*
> >>>>> * ********************
> >>>>> * Getters & setters *
> >>>>> * ********************
> >>>>> * */
> >>>>> public CloudSolrServer getSolrServer() {
> >>>>> return cloudSolrServer;
> >>>>> }
> >>>>>
> >>>>> public void setSolrServer(CloudSolrServer solrServer) {
> >>>>> this.cloudSolrServer = solrServer;
> >>>>> }
> >>>>>
> >>>>> private boolean addBean(DocumentBean user) {
> >>>>> boolean added = false;
> >>>>> try {
> >>>>> this.cloudSolrServer.addBean(user, 100);
> >>>>> this.commit();
> >>>>>
> >>>>> } catch (IOException e) {
> >>>>> log.error(e);
> >>>>>
> >>>>> } catch (SolrServerException e) {
> >>>>> log.error(e);
> >>>>> }catch(SolrException e) {
> >>>>> log.error(e);
> >>>>> }
> >>>>> return added;
> >>>>> }
> >>>>> private boolean addSolrInputDocument(SolrInputDocument updateDoc) {
> >>>>> boolean added = false;
> >>>>> try {
> >>>>> this.cloudSolrServer.add(updateDoc, 100);
> >>>>> this.commit();
> >>>>> added = true;
> >>>>> } catch (IOException e) {
> >>>>> log.error(e);
> >>>>>
> >>>>> } catch (SolrServerException e) {
> >>>>> log.error(e);
> >>>>> }catch(SolrException e) {
> >>>>> log.error(e);
> >>>>> }
> >>>>> return added;
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> Thank you very much, Mark.
> >>>>>
> >>>>>
> >>>>> - Luis Cappa
> >>>>>
> >>>>>
> >>>>>
> >>>>> And
> >>>>> 2013/3/13 Mark Miller <markrmil...@gmail.com>
> >>>>>
> >>>>>>
> >>>>>> Could you capture some thread stack traces in the 'engine' and see
> if
> >>>>>> there are any blocking methods?
> >>>>>>
> >>>>>> - Mark
> >>>>>>
> >>>>>> On Mar 13, 2013, at 1:34 PM, Luis Cappa Banda <luisca...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Just one correction:
> >>>>>>>
> >>>>>>> When I said:
> >>>>>>>
> >>>>>>> - I´ve checked SolrCloud via Solr Admin interface and it´s OK:
> >>>>>>> everything is green, and I cant execute queries directly into
> >>>> Solr.
> >>>>>>>
> >>>>>>> I mean:
> >>>>>>>
> >>>>>>>
> >>>>>>> - I´ve checked SolrCloud via Solr Admin interface and it´s OK:
> >>>>>>> everything is green, and *I can* execute queries directly into
> >>>> Solr.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>>
> >>>>>>> - Luis Cappa
> >>>>>>>
> >>>>>>>
> >>>>>>> 2013/3/13 Luis Cappa Banda <luisca...@gmail.com>
> >>>>>>>
> >>>>>>>> Hello, guys!
> >>>>>>>>
> >>>>>>>> I´ve been experiencing some annoying behavior with my current
> >>>>> production
> >>>>>>>> scenario. Here is the snapshot:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> - SolrCloud: 2 shards
> >>>>>>>> - Zookeeper ensemble: 3 nodes in *different machines *(most of
> >>>> the
> >>>>>>>> tutorials installs 3 Zookeeper nodes in the same machine).
> >>>>>>>> - This is the zoo.cfg from every
> >>>>>>>>
> >>>>>>>> tickTime=2000 // I´ve also tried with 60000
> >>>>>>>>
> >>>>>>>> initLimit=10
> >>>>>>>>
> >>>>>>>> syncLimit=5
> >>>>>>>>
> >>>>>>>> dataDir=/var/lib/zookeeper
> >>>>>>>>
> >>>>>>>> clientPort=9000
> >>>>>>>>
> >>>>>>>> server.1=zoohost1:2888:3888
> >>>>>>>>
> >>>>>>>> server.2=zoohost1:2888:3888
> >>>>>>>>
> >>>>>>>> server.3=zoohost1:2888:3888
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> - I´ve developed a Java Application with a REST API (let´s call
> >>>> it *
> >>>>>>>> engine*) that dispatches queries into SolrCloud. It´s a wrapper
> >>>>> around
> >>>>>>>> CloudSolrServer, so it´s mandatory to specify some Zookeeper
> >>>>>> configuration
> >>>>>>>> params too. They are loaded dynamically when the application is
> >>>>>> deployed in
> >>>>>>>> a Tomcat server, but the current values that I´m using are as
> >>>>> follows:
> >>>>>>>>
> >>>>>>>> cloudSolrServer.*setZkConnectTimeout(60000)*
> >>>>>>>>
> >>>>>>>> cloudSolrServer.*setZkClientTimeout(60000)*
> >>>>>>>> *
> >>>>>>>> *
> >>>>>>>> *
> >>>>>>>> *
> >>>>>>>>
> >>>>>>>> *THE PROBLEM*
> >>>>>>>> *
> >>>>>>>> *
> >>>>>>>> Everything goes OK, but after two days more or less (yes, I´ve
> >>>> checked
> >>>>>>>> that this behavior occurrs periodically, more or less) the *engine
> >>>>>> blocks
> >>>>>>>> * and cannot dispatch any query to SolrCloud.
> >>>>>>>>
> >>>>>>>> - The *engine *log only outputs "updating Zookeeper..." one last
> >>>>> time,
> >>>>>>>> but never updates.
> >>>>>>>> - I´ve checked SolrCloud via Solr Admin interface and it´s OK:
> >>>>>>>> everything is green, and I cant execute queries directly into
> >>>> Solr.
> >>>>>>>> - So then Solr appears to be OK, so the next step is to restart
> >>>>>> *engine
> >>>>>>>> but *it again appears "updating Zookeeper...". Unfortunately
> >>>> switch
> >>>>>>>> off + switch on doesn´t work here, :-(
> >>>>>>>> - I´ve checked too Zookeeper logs and it appears some connection
> >>>> log
> >>>>>>>> outs, but the ensemble appears to be OK too.
> >>>>>>>> - *The end: *If I restart Zookeeper one by one, and I restart
> >>>>>>>> SolrCloud, plus I restart the engine, the problem is solved. I´m
> >>>>> using
> >>>>>>>> Amazon AWS as hostage, so I discard connection problems between
> >>>>>> instances.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Does anyone experienced something similar? Can anybody shed some
> >>>> light
> >>>>>> on
> >>>>>>>> this problem?
> >>>>>>>>
> >>>>>>>> Thank you very much.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> - Luis Cappa
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >
>
