RE: Connect to SolrCloud using proxy in SolrJ
HI All,
Pinging this again. Could someone please advise.
Thanks and Regards,
Preeti Bhat
From: Preeti Bhat
Sent: Wednesday, September 28, 2016 7:14 PM
To: solr-user@lucene.apache.org
Subject: Connect to SolrCloud using proxy in SolrJ
Hi All,
I am trying to connect to the Solrcloud using the zookeeper host string in my
java application(CloudSolrClient). I am able to connect to the solrcloud when
there are no proxy settings needed, but when trying to connect to the code
using proxy settings, I am getting the below error.
1) Without Proxy
System.setProperty("javax.net.ssl.keyStore", keyStore);
System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
System.setProperty("javax.net.ssl.trustStore", trustStore);
System.setProperty("javax.net.ssl.trustStorePassword", trustStorePsswd);
HttpClientBuilder builder = HttpClientBuilder.create();
builder.useSystemProperties();
CloseableHttpClient httpclient = builder.build();
cloud_client = new CloudSolrClient(zkHost, httpclient);
@SuppressWarnings("rawtypes")
SolrRequest req = new QueryRequest();
req.setBasicAuthCredentials(UserName, Password);
cloud_client.request(req, collectionName);
cloud_client.setDefaultCollection(collectionName);
2) With Proxy code
System.setProperty("javax.net.ssl.keyStore", keyStore);
System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
System.setProperty("javax.net.ssl.trustStore", trustStore);
System.setProperty("javax.net.ssl.trustStorePassword", trustStorePsswd);
System.setProperty("http.proxyHost", ProxyHost);
System.setProperty("http.proxyPort", ProxyPort);
HttpHost proxy = new HttpHost(ProxyHost, Integer.parseInt(ProxyPort));
RequestConfig defaultRequestConfig = RequestConfig.custom()
.setProxy(new HttpHost(ProxyHost,
Integer.parseInt(ProxyPort))).build();
HttpGet httpget = new HttpGet();
httpget.setConfig(defaultRequestConfig);
HttpClientBuilder builder = HttpClientBuilder.create();
builder.useSystemProperties();
CloseableHttpClient httpclient = builder.build();
httpclient.execute(proxy, httpget);
cloud_client = new CloudSolrClient(zkHost, httpclient);
@SuppressWarnings("rawtypes")
SolrRequest req = new QueryRequest();
req.setBasicAuthCredentials(UserName, Password);
cloud_client.request(req, collectionName);
cloud_client.setDefaultCollection(collectionName);
Please note that the Zookeepers are already whitelisted in the proxy under TCP
protocol & SOLR servers are under https protocol.
Could someone please advise on this??
Thanks and Regards,
Preeti Bhat
NOTICE TO RECIPIENTS: This communication may contain confidential and/or
privileged information. If you are not the intended recipient (or have received
this communication in error) please notify the sender and
it-supp...@shoregrp.com immediately, and destroy this communication. Any
unauthorized copying, disclosure or distribution of the material in this
communication is strictly forbidden. Any views or opinions presented in this
email are solely those of the author and do not necessarily represent those of
the company. Finally, the recipient should check this email and any attachments
for the presence of viruses. The company accepts no liability for any damage
caused by any virus transmitted by this email.
Re: Connect to SolrCloud using proxy in SolrJ
Hi,
not sure, have you tried to add proxy configuration for https ?
System.setProperty("https.proxyHost", ProxyHost);
System.setProperty("https.proxyPort", ProxyPort);
Bests,
Vincenzo
On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
wrote:
> HI All,
>
> Pinging this again. Could someone please advise.
>
>
> Thanks and Regards,
> Preeti Bhat
>
> From: Preeti Bhat
> Sent: Wednesday, September 28, 2016 7:14 PM
> To: solr-user@lucene.apache.org
> Subject: Connect to SolrCloud using proxy in SolrJ
>
> Hi All,
>
> I am trying to connect to the Solrcloud using the zookeeper host string in
> my java application(CloudSolrClient). I am able to connect to the solrcloud
> when there are no proxy settings needed, but when trying to connect to the
> code using proxy settings, I am getting the below error.
>
>
> 1) Without Proxy
>
> System.setProperty("javax.net.ssl.keyStore", keyStore);
>
> System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
>
> System.setProperty("javax.net.ssl.trustStore", trustStore);
>
> System.setProperty("javax.net.ssl.trustStorePassword", trustStorePsswd);
>
>
> HttpClientBuilder builder = HttpClientBuilder.create();
>builder.useSystemProperties();
>CloseableHttpClient httpclient = builder.build();
> cloud_client = new CloudSolrClient(zkHost, httpclient);
> @SuppressWarnings("rawtypes")
> SolrRequest req = new QueryRequest();
> req.setBasicAuthCredentials(UserName, Password);
>cloud_client.request(req, collectionName);
>
> cloud_client.setDefaultCollection(collectionName);
>
> 2) With Proxy code
>
> System.setProperty("javax.net.ssl.keyStore", keyStore);
>
> System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
>
> System.setProperty("javax.net.ssl.trustStore", trustStore);
>
> System.setProperty("javax.net.ssl.trustStorePassword", trustStorePsswd);
> System.setProperty("http.proxyHost", ProxyHost);
> System.setProperty("http.proxyPort", ProxyPort);
> HttpHost proxy = new HttpHost(ProxyHost, Integer.parseInt(ProxyPort));
> RequestConfig defaultRequestConfig = RequestConfig.custom()
> .setProxy(new HttpHost(ProxyHost,
> Integer.parseInt(ProxyPort))).build();
>
> HttpGet httpget = new HttpGet();
> httpget.setConfig(defaultRequestConfig);
>
> HttpClientBuilder builder = HttpClientBuilder.create();
> builder.useSystemProperties();
> CloseableHttpClient httpclient = builder.build();
> httpclient.execute(proxy, httpget);
> cloud_client = new CloudSolrClient(zkHost, httpclient);
> @SuppressWarnings("rawtypes")
> SolrRequest req = new QueryRequest();
> req.setBasicAuthCredentials(UserName, Password);
> cloud_client.request(req, collectionName);
> cloud_client.setDefaultCollection(collectionName);
>
>
> Please note that the Zookeepers are already whitelisted in the proxy under
> TCP protocol & SOLR servers are under https protocol.
> Could someone please advise on this??
>
> Thanks and Regards,
> Preeti Bhat
>
>
>
> NOTICE TO RECIPIENTS: This communication may contain confidential and/or
> privileged information. If you are not the intended recipient (or have
> received this communication in error) please notify the sender and
> it-supp...@shoregrp.com immediately, and destroy this communication. Any
> unauthorized copying, disclosure or distribution of the material in this
> communication is strictly forbidden. Any views or opinions presented in
> this email are solely those of the author and do not necessarily represent
> those of the company. Finally, the recipient should check this email and
> any attachments for the presence of viruses. The company accepts no
> liability for any damage caused by any virus transmitted by this email.
>
>
>
--
Vincenzo D'Amore
email: v.dam...@gmail.com
skype: free.dev
mobile: +39 349 8513251
solr deployment help
halo solr apache, introduce me, my name is kenfan and i work in IT company, indonesian. i want to deploy solr 6.2.0 to openshift online. in openshift online i used jbossews-2.0. after that i upload a solr-webapps to webapps openshift online. i just followed tutorial around on net.. i still stuck.. any help? Regards, *Ken Fan* *PT. Ifabula Digital Kreasi* Office : Rukan Kencana Niaga Blok D1 No. 1E Meruya Jakarta Barat - Indonesia
RE: Connect to SolrCloud using proxy in SolrJ
Hi Vincenzo,
Yes, I have tried using the https protocol. We are not able to connect to
Zookeeper's.
I am getting the below error message.
Could not connect to ZooKeeper zkHost within 1 ms
Thanks and Regards,
Preeti Bhat
-Original Message-
From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
Sent: Thursday, September 29, 2016 1:57 PM
To: solr-user@lucene.apache.org
Subject: Re: Connect to SolrCloud using proxy in SolrJ
Hi,
not sure, have you tried to add proxy configuration for https ?
System.setProperty("https.proxyHost", ProxyHost);
System.setProperty("https.proxyPort", ProxyPort);
Bests,
Vincenzo
On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
wrote:
> HI All,
>
> Pinging this again. Could someone please advise.
>
>
> Thanks and Regards,
> Preeti Bhat
>
> From: Preeti Bhat
> Sent: Wednesday, September 28, 2016 7:14 PM
> To: solr-user@lucene.apache.org
> Subject: Connect to SolrCloud using proxy in SolrJ
>
> Hi All,
>
> I am trying to connect to the Solrcloud using the zookeeper host
> string in my java application(CloudSolrClient). I am able to connect
> to the solrcloud when there are no proxy settings needed, but when
> trying to connect to the code using proxy settings, I am getting the below
> error.
>
>
> 1) Without Proxy
>
> System.setProperty("javax.net.ssl.keyStore", keyStore);
>
> System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
>
> System.setProperty("javax.net.ssl.trustStore", trustStore);
>
> System.setProperty("javax.net.ssl.trustStorePassword",
> trustStorePsswd);
>
>
> HttpClientBuilder builder = HttpClientBuilder.create();
>builder.useSystemProperties();
>CloseableHttpClient httpclient = builder.build(); cloud_client
> = new CloudSolrClient(zkHost, httpclient);
> @SuppressWarnings("rawtypes")
> SolrRequest req = new QueryRequest();
> req.setBasicAuthCredentials(UserName, Password);
>cloud_client.request(req, collectionName);
>
> cloud_client.setDefaultCollection(collectionName);
>
> 2) With Proxy code
>
> System.setProperty("javax.net.ssl.keyStore", keyStore);
>
> System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
>
> System.setProperty("javax.net.ssl.trustStore", trustStore);
>
> System.setProperty("javax.net.ssl.trustStorePassword",
> trustStorePsswd); System.setProperty("http.proxyHost", ProxyHost);
> System.setProperty("http.proxyPort", ProxyPort); HttpHost proxy = new
> HttpHost(ProxyHost, Integer.parseInt(ProxyPort)); RequestConfig
> defaultRequestConfig = RequestConfig.custom()
> .setProxy(new
> HttpHost(ProxyHost, Integer.parseInt(ProxyPort))).build();
>
> HttpGet httpget = new HttpGet();
> httpget.setConfig(defaultRequestConfig);
>
> HttpClientBuilder builder = HttpClientBuilder.create();
> builder.useSystemProperties(); CloseableHttpClient httpclient =
> builder.build(); httpclient.execute(proxy, httpget); cloud_client =
> new CloudSolrClient(zkHost, httpclient);
> @SuppressWarnings("rawtypes")
> SolrRequest req = new QueryRequest();
> req.setBasicAuthCredentials(UserName, Password);
> cloud_client.request(req, collectionName);
> cloud_client.setDefaultCollection(collectionName);
>
>
> Please note that the Zookeepers are already whitelisted in the proxy
> under TCP protocol & SOLR servers are under https protocol.
> Could someone please advise on this??
>
> Thanks and Regards,
> Preeti Bhat
>
>
>
> NOTICE TO RECIPIENTS: This communication may contain confidential
> and/or privileged information. If you are not the intended recipient
> (or have received this communication in error) please notify the
> sender and it-supp...@shoregrp.com immediately, and destroy this
> communication. Any unauthorized copying, disclosure or distribution of
> the material in this communication is strictly forbidden. Any views or
> opinions presented in this email are solely those of the author and do
> not necessarily represent those of the company. Finally, the recipient
> should check this email and any attachments for the presence of
> viruses. The company accepts no liability for any damage caused by any virus
> transmitted by this email.
>
>
>
--
Vincenzo D'Amore
email: v.dam...@gmail.com
skype: free.dev
mobile: +39 349 8513251
NOTICE TO RECIPIENTS: This communication may contain confidential and/or
privileged information. If you are not the intended recipient (or have received
this communication in error) please notify the sender and
it-supp...@shoregrp.com immediately, and destroy this communication. Any
unauthorized copying, disclosure or distribution of the material in this
communication is strictly forbidden. Any views or opinions presented in this
email are solely those of the author and do not necessarily represent those of
the company. Finally, the recipient should check this email and any attachments
for the presence of viruses. The company accepts no liability for any damage
caused by any virus transmitted by this email.
Re: Connect to SolrCloud using proxy in SolrJ
Zookeeper clients connect on tcp not http. Perhaps SOCKS proxy might help,
but I don't know exactly.
On Thu, Sep 29, 2016 at 11:55 AM, Preeti Bhat
wrote:
> Hi Vincenzo,
>
> Yes, I have tried using the https protocol. We are not able to connect to
> Zookeeper's.
>
> I am getting the below error message.
>
> Could not connect to ZooKeeper zkHost within 1 ms
>
> Thanks and Regards,
> Preeti Bhat
>
> -Original Message-
> From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> Sent: Thursday, September 29, 2016 1:57 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Connect to SolrCloud using proxy in SolrJ
>
> Hi,
>
> not sure, have you tried to add proxy configuration for https ?
>
> System.setProperty("https.proxyHost", ProxyHost);
> System.setProperty("https.proxyPort", ProxyPort);
>
>
> Bests,
> Vincenzo
>
> On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
> wrote:
>
> > HI All,
> >
> > Pinging this again. Could someone please advise.
> >
> >
> > Thanks and Regards,
> > Preeti Bhat
> >
> > From: Preeti Bhat
> > Sent: Wednesday, September 28, 2016 7:14 PM
> > To: solr-user@lucene.apache.org
> > Subject: Connect to SolrCloud using proxy in SolrJ
> >
> > Hi All,
> >
> > I am trying to connect to the Solrcloud using the zookeeper host
> > string in my java application(CloudSolrClient). I am able to connect
> > to the solrcloud when there are no proxy settings needed, but when
> > trying to connect to the code using proxy settings, I am getting the
> below error.
> >
> >
> > 1) Without Proxy
> >
> > System.setProperty("javax.net.ssl.keyStore", keyStore);
> >
> > System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
> >
> > System.setProperty("javax.net.ssl.trustStore", trustStore);
> >
> > System.setProperty("javax.net.ssl.trustStorePassword",
> > trustStorePsswd);
> >
> >
> > HttpClientBuilder builder = HttpClientBuilder.create();
> >builder.useSystemProperties();
> >CloseableHttpClient httpclient = builder.build(); cloud_client
> > = new CloudSolrClient(zkHost, httpclient);
> > @SuppressWarnings("rawtypes")
> > SolrRequest req = new QueryRequest();
> > req.setBasicAuthCredentials(UserName, Password);
> >cloud_client.request(req, collectionName);
> >
> > cloud_client.setDefaultCollection(collectionName);
> >
> > 2) With Proxy code
> >
> > System.setProperty("javax.net.ssl.keyStore", keyStore);
> >
> > System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
> >
> > System.setProperty("javax.net.ssl.trustStore", trustStore);
> >
> > System.setProperty("javax.net.ssl.trustStorePassword",
> > trustStorePsswd); System.setProperty("http.proxyHost", ProxyHost);
> > System.setProperty("http.proxyPort", ProxyPort); HttpHost proxy = new
> > HttpHost(ProxyHost, Integer.parseInt(ProxyPort)); RequestConfig
> > defaultRequestConfig = RequestConfig.custom()
> > .setProxy(new
> > HttpHost(ProxyHost, Integer.parseInt(ProxyPort))).build();
> >
> > HttpGet httpget = new HttpGet();
> > httpget.setConfig(defaultRequestConfig);
> >
> > HttpClientBuilder builder = HttpClientBuilder.create();
> > builder.useSystemProperties(); CloseableHttpClient httpclient =
> > builder.build(); httpclient.execute(proxy, httpget); cloud_client =
> > new CloudSolrClient(zkHost, httpclient);
> > @SuppressWarnings("rawtypes")
> > SolrRequest req = new QueryRequest();
> > req.setBasicAuthCredentials(UserName, Password);
> > cloud_client.request(req, collectionName);
> > cloud_client.setDefaultCollection(collectionName);
> >
> >
> > Please note that the Zookeepers are already whitelisted in the proxy
> > under TCP protocol & SOLR servers are under https protocol.
> > Could someone please advise on this??
> >
> > Thanks and Regards,
> > Preeti Bhat
> >
> >
> >
> > NOTICE TO RECIPIENTS: This communication may contain confidential
> > and/or privileged information. If you are not the intended recipient
> > (or have received this communication in error) please notify the
> > sender and it-supp...@shoregrp.com immediately, and destroy this
> > communication. Any unauthorized copying, disclosure or distribution of
> > the material in this communication is strictly forbidden. Any views or
> > opinions presented in this email are solely those of the author and do
> > not necessarily represent those of the company. Finally, the recipient
> > should check this email and any attachments for the presence of
> > viruses. The company accepts no liability for any damage caused by any
> virus transmitted by this email.
> >
> >
> >
>
>
> --
> Vincenzo D'Amore
> email: v.dam...@gmail.com
> skype: free.dev
> mobile: +39 349 8513251
>
> NOTICE TO RECIPIENTS: This communication may contain confidential and/or
> privileged information. If you are not the intended recipient (or have
> received this communication in error) please notify the sender and
> it-supp...@shoregrp.com immediately, and destroy this communication. Any
> unauthorized copying, disclosure or di
Re: solr hierarchical search hyponyms hypernyms
Hi Shawn, Thanks for your reply! Until now, I was using Solr 5.4.1, and there were a lot of difference between schemaless mode and with schema mode. Reading your answer, it seems that now there is a better and simpler management about this feature, right? I'm going to become familiar with Solr 6.2.1 version. Thanks. Best, Francesco -- View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298571.html Sent from the Solr - User mailing list archive at Nabble.com.
Permission error using install_solr_service script.sh
Hi, I'm having troubles to run the install_solr_service in Centos 7.2. I have this errors: -bash: /opt/usr/solr/bin/solr: Permission denied -bash: /opt/usr/solr/bin/solr: Permission denied the problematic line is the line 315 on install_solr_service.sh find "$SOLR_VAR_DIR" -type f -print0 | xargs -0 chmod 0640 This line should set execution permissions to executable files, the permissions should be something like 750 to allow the user and the user group to run the scripts. - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/Permission-error-using-install-solr-service-script-sh-tp4298580.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Connect to SolrCloud using proxy in SolrJ
Well, looking around I found at this issue
https://issues.apache.org/jira/browse/ZOOKEEPER-2250
As far as I know zookeeper doesn't support socks proxy (may be better ask
in the zookeeper forum).
Anyway in your email you wrote that zookeeper is "whitelisted in the proxy
under TCP protocol", so I suppose your proxy is able to transparently
bridge tcp connections.
Given that make sure your proxy configuration for zookeeper is not working
on http.
Just to understand, how have you configured your zookeeper connection?
On Thu, Sep 29, 2016 at 11:06 AM, Mikhail Khludnev wrote:
> Zookeeper clients connect on tcp not http. Perhaps SOCKS proxy might help,
> but I don't know exactly.
>
> On Thu, Sep 29, 2016 at 11:55 AM, Preeti Bhat
> wrote:
>
> > Hi Vincenzo,
> >
> > Yes, I have tried using the https protocol. We are not able to connect
> to
> > Zookeeper's.
> >
> > I am getting the below error message.
> >
> > Could not connect to ZooKeeper zkHost within 1 ms
> >
> > Thanks and Regards,
> > Preeti Bhat
> >
> > -Original Message-
> > From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> > Sent: Thursday, September 29, 2016 1:57 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Connect to SolrCloud using proxy in SolrJ
> >
> > Hi,
> >
> > not sure, have you tried to add proxy configuration for https ?
> >
> > System.setProperty("https.proxyHost", ProxyHost);
> > System.setProperty("https.proxyPort", ProxyPort);
> >
> >
> > Bests,
> > Vincenzo
> >
> > On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
> > wrote:
> >
> > > HI All,
> > >
> > > Pinging this again. Could someone please advise.
> > >
> > >
> > > Thanks and Regards,
> > > Preeti Bhat
> > >
> > > From: Preeti Bhat
> > > Sent: Wednesday, September 28, 2016 7:14 PM
> > > To: solr-user@lucene.apache.org
> > > Subject: Connect to SolrCloud using proxy in SolrJ
> > >
> > > Hi All,
> > >
> > > I am trying to connect to the Solrcloud using the zookeeper host
> > > string in my java application(CloudSolrClient). I am able to connect
> > > to the solrcloud when there are no proxy settings needed, but when
> > > trying to connect to the code using proxy settings, I am getting the
> > below error.
> > >
> > >
> > > 1) Without Proxy
> > >
> > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > >
> > > System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
> > >
> > > System.setProperty("javax.net.ssl.trustStore", trustStore);
> > >
> > > System.setProperty("javax.net.ssl.trustStorePassword",
> > > trustStorePsswd);
> > >
> > >
> > > HttpClientBuilder builder = HttpClientBuilder.create();
> > >builder.useSystemProperties();
> > >CloseableHttpClient httpclient = builder.build(); cloud_client
> > > = new CloudSolrClient(zkHost, httpclient);
> > > @SuppressWarnings("rawtypes")
> > > SolrRequest req = new QueryRequest();
> > > req.setBasicAuthCredentials(UserName, Password);
> > >cloud_client.request(req, collectionName);
> > >
> > > cloud_client.setDefaultCollection(collectionName);
> > >
> > > 2) With Proxy code
> > >
> > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > >
> > > System.setProperty("javax.net.ssl.keyStorePassword", keyStorePsswd);
> > >
> > > System.setProperty("javax.net.ssl.trustStore", trustStore);
> > >
> > > System.setProperty("javax.net.ssl.trustStorePassword",
> > > trustStorePsswd); System.setProperty("http.proxyHost", ProxyHost);
> > > System.setProperty("http.proxyPort", ProxyPort); HttpHost proxy = new
> > > HttpHost(ProxyHost, Integer.parseInt(ProxyPort)); RequestConfig
> > > defaultRequestConfig = RequestConfig.custom()
> > > .setProxy(new
> > > HttpHost(ProxyHost, Integer.parseInt(ProxyPort))).build();
> > >
> > > HttpGet httpget = new HttpGet();
> > > httpget.setConfig(defaultRequestConfig);
> > >
> > > HttpClientBuilder builder = HttpClientBuilder.create();
> > > builder.useSystemProperties(); CloseableHttpClient httpclient =
> > > builder.build(); httpclient.execute(proxy, httpget); cloud_client =
> > > new CloudSolrClient(zkHost, httpclient);
> > > @SuppressWarnings("rawtypes")
> > > SolrRequest req = new QueryRequest();
> > > req.setBasicAuthCredentials(UserName, Password);
> > > cloud_client.request(req, collectionName);
> > > cloud_client.setDefaultCollection(collectionName);
> > >
> > >
> > > Please note that the Zookeepers are already whitelisted in the proxy
> > > under TCP protocol & SOLR servers are under https protocol.
> > > Could someone please advise on this??
> > >
> > > Thanks and Regards,
> > > Preeti Bhat
> > >
> > >
> > >
> > > NOTICE TO RECIPIENTS: This communication may contain confidential
> > > and/or privileged information. If you are not the intended recipient
> > > (or have received this communication in error) please notify the
> > > sender and it-supp...@shoregrp.com immediately, and destroy this
> > > communication. Any unauthorized copying, disclosure or distribution of
> > > the
RE: Connect to SolrCloud using proxy in SolrJ
Hi Vincenzo,
We are verified that the zookeeper is not working on http already. We are
getting the error message stating that there is no response from server for
both proxy and non proxy enabled browsers.
I have setup the Zookeeper in AWS, SOLR is connecting to Zookeeper using
zkCli.sh from SOLR. I have not made any changes in the zookeeper settings other
than specifying the data directory and servers names in quorum.
Also, I tried applying the SOCKS proxy in my application and got the error
stating "Malformed reply from SOCKS server". I am currently working with
Network team to see if we have separate Socks proxy settings.
Thanks and Regards,
Preeti Bhat
-Original Message-
From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
Sent: Thursday, September 29, 2016 5:02 PM
To: solr-user@lucene.apache.org
Subject: Re: Connect to SolrCloud using proxy in SolrJ
Well, looking around I found at this issue
https://issues.apache.org/jira/browse/ZOOKEEPER-2250
As far as I know zookeeper doesn't support socks proxy (may be better ask in
the zookeeper forum).
Anyway in your email you wrote that zookeeper is "whitelisted in the proxy
under TCP protocol", so I suppose your proxy is able to transparently bridge
tcp connections.
Given that make sure your proxy configuration for zookeeper is not working on
http.
Just to understand, how have you configured your zookeeper connection?
On Thu, Sep 29, 2016 at 11:06 AM, Mikhail Khludnev wrote:
> Zookeeper clients connect on tcp not http. Perhaps SOCKS proxy might
> help, but I don't know exactly.
>
> On Thu, Sep 29, 2016 at 11:55 AM, Preeti Bhat
>
> wrote:
>
> > Hi Vincenzo,
> >
> > Yes, I have tried using the https protocol. We are not able to
> > connect
> to
> > Zookeeper's.
> >
> > I am getting the below error message.
> >
> > Could not connect to ZooKeeper zkHost within 1 ms
> >
> > Thanks and Regards,
> > Preeti Bhat
> >
> > -Original Message-
> > From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> > Sent: Thursday, September 29, 2016 1:57 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Connect to SolrCloud using proxy in SolrJ
> >
> > Hi,
> >
> > not sure, have you tried to add proxy configuration for https ?
> >
> > System.setProperty("https.proxyHost", ProxyHost);
> > System.setProperty("https.proxyPort", ProxyPort);
> >
> >
> > Bests,
> > Vincenzo
> >
> > On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
> >
> > wrote:
> >
> > > HI All,
> > >
> > > Pinging this again. Could someone please advise.
> > >
> > >
> > > Thanks and Regards,
> > > Preeti Bhat
> > >
> > > From: Preeti Bhat
> > > Sent: Wednesday, September 28, 2016 7:14 PM
> > > To: solr-user@lucene.apache.org
> > > Subject: Connect to SolrCloud using proxy in SolrJ
> > >
> > > Hi All,
> > >
> > > I am trying to connect to the Solrcloud using the zookeeper host
> > > string in my java application(CloudSolrClient). I am able to
> > > connect to the solrcloud when there are no proxy settings needed,
> > > but when trying to connect to the code using proxy settings, I am
> > > getting the
> > below error.
> > >
> > >
> > > 1) Without Proxy
> > >
> > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > >
> > > System.setProperty("javax.net.ssl.keyStorePassword",
> > > keyStorePsswd);
> > >
> > > System.setProperty("javax.net.ssl.trustStore", trustStore);
> > >
> > > System.setProperty("javax.net.ssl.trustStorePassword",
> > > trustStorePsswd);
> > >
> > >
> > > HttpClientBuilder builder = HttpClientBuilder.create();
> > >builder.useSystemProperties();
> > >CloseableHttpClient httpclient = builder.build();
> > > cloud_client = new CloudSolrClient(zkHost, httpclient);
> > > @SuppressWarnings("rawtypes")
> > > SolrRequest req = new QueryRequest();
> > > req.setBasicAuthCredentials(UserName, Password);
> > >cloud_client.request(req, collectionName);
> > >
> > > cloud_client.setDefaultCollection(collectionName);
> > >
> > > 2) With Proxy code
> > >
> > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > >
> > > System.setProperty("javax.net.ssl.keyStorePassword",
> > > keyStorePsswd);
> > >
> > > System.setProperty("javax.net.ssl.trustStore", trustStore);
> > >
> > > System.setProperty("javax.net.ssl.trustStorePassword",
> > > trustStorePsswd); System.setProperty("http.proxyHost", ProxyHost);
> > > System.setProperty("http.proxyPort", ProxyPort); HttpHost proxy =
> > > new HttpHost(ProxyHost, Integer.parseInt(ProxyPort));
> > > RequestConfig defaultRequestConfig = RequestConfig.custom()
> > > .setProxy(new
> > > HttpHost(ProxyHost, Integer.parseInt(ProxyPort))).build();
> > >
> > > HttpGet httpget = new HttpGet();
> > > httpget.setConfig(defaultRequestConfig);
> > >
> > > HttpClientBuilder builder = HttpClientBuilder.create();
> > > builder.useSystemProperties(); CloseableHttpClient httpclient =
> > > builder.build(); httpclient.execute(proxy, httpget); cloud_client
> > > = new Clo
Re: Issue with Solr & Kerberos
Anyone has any idea ? Thanks, Loïc Loïc CHANEL System Big Data engineer MS&T - WASABI - Worldline (Villeurbanne, France) 2016-09-27 10:42 GMT+02:00 Loïc Chanel : > Hi Davide, > > Thanks for your quick answer. But I just updated all JCE packages and > restarted solr : still get the same error. > Another idea ? How did you solve it ? > > Regards, > > > Loïc > > Loïc CHANEL > System Big Data engineer > MS&T - WASABI - Worldline (Villeurbanne, France) > > 2016-09-27 10:31 GMT+02:00 Davide Isoardi : > >> Hi, >> I had the same issue. >> You should try to download the latest versione of JCE and of JCE policy >> files. >> >> Davide Isoardi >> eCube S.r.l. >> isoa...@ecubecenter.it >> http://www.ecubecenter.it >> Tel. +390113999301 >> Mobile +393288204915 >> Fax. +390113999309 >> >> >> Informativa ai sensi del Decr.Lgs Privacy n.196/2003 >> ECUBE tratta i dati personali secondo quanto specificato nella pagina >> “Privacy Policy” disponibile su http://www.ecubecenter.it/privacy.pdf. >> Le informazioni contenute nel presente messaggio sono destinate >> esclusivamente al/ai destinatario/i indicato/i. Qualora riceviate il >> presente messaggio per errore, vi preghiamo di voler cortesemente darcene >> notizia via e-mail (i...@ecubecenter.it) e di provvedere ad eliminare il >> messaggio ricevuto erroneamente, essendo illegittimo ed illecito ogni >> diverso utilizzo. >> >> >> -Messaggio originale- >> Da: Loïc Chanel [mailto:loic.cha...@telecomnancy.net.INVALID] >> Inviato: martedì 27 settembre 2016 09:45 >> A: solr-user@lucene.apache.org >> Oggetto: Issue with Solr & Kerberos >> >> Hi all, >> >> I'm trying to install Solr to use it with Apache Ranger logs, but it >> seems it can't decrypt its Kerberos key. >> Indeed, when I try a curl : "curl -v -u solr: --noproxy '*' >> http://:8983/solr/ --negotiate", I get the following error : >> Error 403 GSSException: Failure unspecified at GSS-API level (Mechanism >> level: Invalid argument (400) - Cannot find key of appropriate type to >> decrypt AP REP - AES256 CTS mode with HMAC SHA1-96) I know that AES256 keys >> require Java Cryptography Extensions, and I successfully installed it >> according to KeyLenghtDetector ( >> https://gist.github.com/jehrhardt/5167854 ), so this shouldn't be a >> problem. >> >> Is there a known issue with Solr and AES256 keys ? Does anyone has an >> idea about where I should look at ? >> Thanks for your help, >> >> >> Loïc >> >> Loïc CHANEL >> System Big Data engineer >> MS&T - WASABI - Worldline (Villeurbanne, France) >> > >
Re: Connect to SolrCloud using proxy in SolrJ
Well, how have configured *cliend-side* zookeeper connection? I mean from
the SolrJ side.
And again as simple check to see if the proxy is working correctly with
zookeeper you could use telnet.
You should use the proxy hostname and proxy port available for zookeeper
(it should be different from the port used for http proxy)
# telnet proxy-host-name proxy-port
and when the connection is established write:
ruok
> "The server will respond with imok if it is running."
https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html
On Thu, Sep 29, 2016 at 1:40 PM, Preeti Bhat
wrote:
> Hi Vincenzo,
>
> We are verified that the zookeeper is not working on http already. We are
> getting the error message stating that there is no response from server for
> both proxy and non proxy enabled browsers.
> I have setup the Zookeeper in AWS, SOLR is connecting to Zookeeper using
> zkCli.sh from SOLR. I have not made any changes in the zookeeper settings
> other than specifying the data directory and servers names in quorum.
>
> Also, I tried applying the SOCKS proxy in my application and got the error
> stating "Malformed reply from SOCKS server". I am currently working with
> Network team to see if we have separate Socks proxy settings.
>
> Thanks and Regards,
> Preeti Bhat
>
> -Original Message-
> From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> Sent: Thursday, September 29, 2016 5:02 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Connect to SolrCloud using proxy in SolrJ
>
> Well, looking around I found at this issue
> https://issues.apache.org/jira/browse/ZOOKEEPER-2250
> As far as I know zookeeper doesn't support socks proxy (may be better ask
> in the zookeeper forum).
>
> Anyway in your email you wrote that zookeeper is "whitelisted in the proxy
> under TCP protocol", so I suppose your proxy is able to transparently
> bridge tcp connections.
>
> Given that make sure your proxy configuration for zookeeper is not working
> on http.
>
> Just to understand, how have you configured your zookeeper connection?
>
>
>
> On Thu, Sep 29, 2016 at 11:06 AM, Mikhail Khludnev
> wrote:
>
> > Zookeeper clients connect on tcp not http. Perhaps SOCKS proxy might
> > help, but I don't know exactly.
> >
> > On Thu, Sep 29, 2016 at 11:55 AM, Preeti Bhat
> >
> > wrote:
> >
> > > Hi Vincenzo,
> > >
> > > Yes, I have tried using the https protocol. We are not able to
> > > connect
> > to
> > > Zookeeper's.
> > >
> > > I am getting the below error message.
> > >
> > > Could not connect to ZooKeeper zkHost within 1 ms
> > >
> > > Thanks and Regards,
> > > Preeti Bhat
> > >
> > > -Original Message-
> > > From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> > > Sent: Thursday, September 29, 2016 1:57 PM
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Connect to SolrCloud using proxy in SolrJ
> > >
> > > Hi,
> > >
> > > not sure, have you tried to add proxy configuration for https ?
> > >
> > > System.setProperty("https.proxyHost", ProxyHost);
> > > System.setProperty("https.proxyPort", ProxyPort);
> > >
> > >
> > > Bests,
> > > Vincenzo
> > >
> > > On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
> > >
> > > wrote:
> > >
> > > > HI All,
> > > >
> > > > Pinging this again. Could someone please advise.
> > > >
> > > >
> > > > Thanks and Regards,
> > > > Preeti Bhat
> > > >
> > > > From: Preeti Bhat
> > > > Sent: Wednesday, September 28, 2016 7:14 PM
> > > > To: solr-user@lucene.apache.org
> > > > Subject: Connect to SolrCloud using proxy in SolrJ
> > > >
> > > > Hi All,
> > > >
> > > > I am trying to connect to the Solrcloud using the zookeeper host
> > > > string in my java application(CloudSolrClient). I am able to
> > > > connect to the solrcloud when there are no proxy settings needed,
> > > > but when trying to connect to the code using proxy settings, I am
> > > > getting the
> > > below error.
> > > >
> > > >
> > > > 1) Without Proxy
> > > >
> > > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > > >
> > > > System.setProperty("javax.net.ssl.keyStorePassword",
> > > > keyStorePsswd);
> > > >
> > > > System.setProperty("javax.net.ssl.trustStore", trustStore);
> > > >
> > > > System.setProperty("javax.net.ssl.trustStorePassword",
> > > > trustStorePsswd);
> > > >
> > > >
> > > > HttpClientBuilder builder = HttpClientBuilder.create();
> > > >builder.useSystemProperties();
> > > >CloseableHttpClient httpclient = builder.build();
> > > > cloud_client = new CloudSolrClient(zkHost, httpclient);
> > > > @SuppressWarnings("rawtypes")
> > > > SolrRequest req = new QueryRequest();
> > > > req.setBasicAuthCredentials(UserName, Password);
> > > >cloud_client.request(req, collectionName);
> > > >
> > > > cloud_client.setDefaultCollection(collectionName);
> > > >
> > > > 2) With Proxy code
> > > >
> > > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > > >
> > > > System.setProperty("javax.net.ssl.keyStorePassword",
> > > > keyStorePsswd);
> > > >
>
Re: Connect to SolrCloud using proxy in SolrJ
As Vincenzo mentioned above you shall try to check using telnet and if
connection fails, then you should try to set http proxy on terminal/command
line using this and then give try again with telnet. As long as telnet
works, your code should also be able to connect
export http.proxy=http://
\:@:
Thanks,
Susheel
On Thu, Sep 29, 2016 at 8:09 AM, Vincenzo D'Amore
wrote:
> Well, how have configured *cliend-side* zookeeper connection? I mean from
> the SolrJ side.
>
> And again as simple check to see if the proxy is working correctly with
> zookeeper you could use telnet.
>
> You should use the proxy hostname and proxy port available for zookeeper
> (it should be different from the port used for http proxy)
>
> # telnet proxy-host-name proxy-port
>
> and when the connection is established write:
>
> ruok
>
> > "The server will respond with imok if it is running."
>
> https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html
>
>
> On Thu, Sep 29, 2016 at 1:40 PM, Preeti Bhat
> wrote:
>
> > Hi Vincenzo,
> >
> > We are verified that the zookeeper is not working on http already. We are
> > getting the error message stating that there is no response from server
> for
> > both proxy and non proxy enabled browsers.
> > I have setup the Zookeeper in AWS, SOLR is connecting to Zookeeper using
> > zkCli.sh from SOLR. I have not made any changes in the zookeeper settings
> > other than specifying the data directory and servers names in quorum.
> >
> > Also, I tried applying the SOCKS proxy in my application and got the
> error
> > stating "Malformed reply from SOCKS server". I am currently working with
> > Network team to see if we have separate Socks proxy settings.
> >
> > Thanks and Regards,
> > Preeti Bhat
> >
> > -Original Message-
> > From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> > Sent: Thursday, September 29, 2016 5:02 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Connect to SolrCloud using proxy in SolrJ
> >
> > Well, looking around I found at this issue
> > https://issues.apache.org/jira/browse/ZOOKEEPER-2250
> > As far as I know zookeeper doesn't support socks proxy (may be better ask
> > in the zookeeper forum).
> >
> > Anyway in your email you wrote that zookeeper is "whitelisted in the
> proxy
> > under TCP protocol", so I suppose your proxy is able to transparently
> > bridge tcp connections.
> >
> > Given that make sure your proxy configuration for zookeeper is not
> working
> > on http.
> >
> > Just to understand, how have you configured your zookeeper connection?
> >
> >
> >
> > On Thu, Sep 29, 2016 at 11:06 AM, Mikhail Khludnev
> > wrote:
> >
> > > Zookeeper clients connect on tcp not http. Perhaps SOCKS proxy might
> > > help, but I don't know exactly.
> > >
> > > On Thu, Sep 29, 2016 at 11:55 AM, Preeti Bhat
> > >
> > > wrote:
> > >
> > > > Hi Vincenzo,
> > > >
> > > > Yes, I have tried using the https protocol. We are not able to
> > > > connect
> > > to
> > > > Zookeeper's.
> > > >
> > > > I am getting the below error message.
> > > >
> > > > Could not connect to ZooKeeper zkHost within 1 ms
> > > >
> > > > Thanks and Regards,
> > > > Preeti Bhat
> > > >
> > > > -Original Message-
> > > > From: Vincenzo D'Amore [mailto:v.dam...@gmail.com]
> > > > Sent: Thursday, September 29, 2016 1:57 PM
> > > > To: solr-user@lucene.apache.org
> > > > Subject: Re: Connect to SolrCloud using proxy in SolrJ
> > > >
> > > > Hi,
> > > >
> > > > not sure, have you tried to add proxy configuration for https ?
> > > >
> > > > System.setProperty("https.proxyHost", ProxyHost);
> > > > System.setProperty("https.proxyPort", ProxyPort);
> > > >
> > > >
> > > > Bests,
> > > > Vincenzo
> > > >
> > > > On Thu, Sep 29, 2016 at 10:12 AM, Preeti Bhat
> > > >
> > > > wrote:
> > > >
> > > > > HI All,
> > > > >
> > > > > Pinging this again. Could someone please advise.
> > > > >
> > > > >
> > > > > Thanks and Regards,
> > > > > Preeti Bhat
> > > > >
> > > > > From: Preeti Bhat
> > > > > Sent: Wednesday, September 28, 2016 7:14 PM
> > > > > To: solr-user@lucene.apache.org
> > > > > Subject: Connect to SolrCloud using proxy in SolrJ
> > > > >
> > > > > Hi All,
> > > > >
> > > > > I am trying to connect to the Solrcloud using the zookeeper host
> > > > > string in my java application(CloudSolrClient). I am able to
> > > > > connect to the solrcloud when there are no proxy settings needed,
> > > > > but when trying to connect to the code using proxy settings, I am
> > > > > getting the
> > > > below error.
> > > > >
> > > > >
> > > > > 1) Without Proxy
> > > > >
> > > > > System.setProperty("javax.net.ssl.keyStore", keyStore);
> > > > >
> > > > > System.setProperty("javax.net.ssl.keyStorePassword",
> > > > > keyStorePsswd);
> > > > >
> > > > > System.setProperty("javax.net.ssl.trustStore", trustStore);
> > > > >
> > > > > System.setProperty("javax.net.ssl.trustStorePassword",
> > > > > trustStorePsswd);
> > > > >
> > > > >
> > > > > HttpClientBuilder builder = HttpClientBuilder.cre
Archiving documents
Hi, We would like to archive documents based on some criteria (like those that were not modified for more than an year OR are least used) in order to reduce storage requirements. I would like hear some of the best practices followed. How about having main collection and optionally an archive collection (or one or more archive collections?) to where we move documents (at regular intervals) from the main collection based on some criteria (least used or modified date etc.) and provide a flag during search whether to include archived documents in search or not? Thanks, Vasu
Re: Migrating to Solr 6.1.0 from 5.5.0
Arjun, Your input is a POLYGON -- as seen in the error message. The "Try JTS" was hopefully a clue -- on https://cwiki.apache.org/confluence/display/solr/Spatial+Search search for "JTS" and you should see how to set the spatialContextFactory to JTS, and a mention of needing JTS jar. I'll try and add a bit more info on suggesting exactly where to put it and a download link. I'll also mention a shortcut so you don't have to type out the classname -- a recent feature in 6.2. Since you said you were upgrading... presumably your spatialContextFactory attribute was already set for this to work at all in 5.5? The package reference changed for this value -- I imagine you would have seen a warning/error to this effect in Solr's logs. Do you? ~ David On Tue, Sep 27, 2016 at 10:29 AM William Bell wrote: > the documentation is not good on this. Not sure how to fix it either. > > On Tue, Sep 27, 2016 at 3:41 AM, M, Arjun (Nokia - IN/Bangalore) < > arju...@nokia.com> wrote: > > > Hi, > > > > We are getting the below errors when migrating Solr from 5.5.0 to > > 6.1.0. Could anyone help in resolving the issue, if you have come across > > this? > > > > > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: > > Error from server at http://127.0.0.1:41569/solr/collection1: Unable to > > parse shape given formats "lat,lon", "x y" or as WKT because > > java.text.ParseException: java.lang.UnsupportedOperationException: > > Unsupported shape of this SpatialContext. Try JTS or Geo3D. input: > > POLYGON((-10 30, -40 40, -10 -20, 0 0, -10 30)) > > > > Thanks in advance.. > > > > Thanks & Regards, > >Arjun M > > > > > > > > > > > -- > Bill Bell > billnb...@gmail.com > cell 720-256-8076 > -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
Re: solr hierarchical search hyponyms hypernyms
Hi Andrea, Thanks very much for your complete reply. You're right, I'm new about Solr, so I'm sorry if'm asking trivial questions, or I'not exaustive in my questions! About the scenario, I try to explain it: I have to load the thesaurus in Solr core, and the user would be able to query that thesaurus, when searching a keyword. Getting into the details: I search a keyword T, if this T has BT and/or NT, I'd like to retrive that terms, and show that. I hope this clarifies the scenario! Ciao, Francesco -- View this message in context: http://lucene.472066.n3.nabble.com/solr-hierarchical-search-hyponyms-hypernyms-tp4298385p4298569.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Suggester (AnalyzingInfix n BlendedInfix)
Mind you I have no proof that this will cure the problem in 5.2.1, but it seems like it'd be quick to test Good Luck! Erick On Wed, Sep 28, 2016 at 8:51 PM, Manohar Sripada wrote: > Sure Erick! I will try applying the patch. > > Thanks > > On Wednesday, September 28, 2016, Erick Erickson > wrote: > >> AnalyzingInfixSuggester is a mini Solr index, it's working >> as designed by returning the choices you see. I don't think >> you can persuade it to do what you want OOB. >> >> I took a quick look at SOLR-7865 and it's a very simple fix, just >> 3 lines of code change and the rest is test code. Could you >> consider applying that patch to the 5.2.1 code base? and using that >> rather than fully upgrading? >> >> Best, >> Erick >> >> On Wed, Sep 28, 2016 at 4:21 AM, Manohar Sripada > > wrote: >> > I am implementing auto suggestion on Business Name. I >> > used BlendedInfixLookupFactory which worked in all my uses until I >> > encountered into this bug (https://issues.apache.org/ >> jira/browse/SOLR-7865), >> > where suggest.count doesn't work in Solr 5.2.1. But, I can't upgrade >> > anytime soon. :( >> > >> > I tried using AnalyzingInfixSuggester, but, I encountered couple of >> issues >> > with this. Can someone help me with these? >> > >> >1. This lookupImpl is returning duplicate business names ( >> >https://issues.apache.org/jira/browse/LUCENE-6336) in results (the >> data >> >has duplicate business names) which isn't happening >> >with BlendedInfixLookupFactory. I don't want duplicate values. >> >2. Second one is, AnalyzingInfixSuggester is searching on all input >> >keywords - For example, if am looking for "Apple Corporation", it is >> >returning, "Apple Inc", "Apple Corporation", "Oracle Corporation", >> >"Microsoft Corporation". I need the data with only "Apple >> Corporation". >> >Again, this is working fine in BlendedInfixLookupFactory. >> > >> > I don't want fuzzy searches, so, I am not using it. >> > >> > Below are the respective configurations. The fields type uses Standard >> > Tokenizer. >> > >> > >> > businessName_BIF >> > BlendedInfixLookupFactory >> > DocumentDictionaryFactory >> > business_name >> > true >> > true >> > true >> > >> > text_standard >> > true >> > true >> > true >> > suggest_test_business_name_bif >> > 0 >> > false >> > linear >> > 10 >> > >> > revenues >> > id >> > >> > >> > >> > businessName_AIF >> > AnalyzingInfixLookupFactory >> > DocumentDictionaryFactory >> > business_name >> > true >> > true >> > true >> > >> > text_standard >> > true >> > true >> > true >> > suggest_test_business_name_aif >> > 0 >> > true >> > false >> > >> > revenues >> > id >> > >>
Re: slow updates/searches
bq: The QTimes increase as the number of words in a phrase increase Well, there's more work to do as the # of words increases, and if you have large slops there's more work yet. Best, Erick On Wed, Sep 28, 2016 at 5:54 PM, Rallavagu wrote: > Thanks Erick. > > I have added queries for "firstSearcher" and "newSearcher". After startup, > pmap shows well populated mmap entries and have better QTimes than before. > > However, phrase queries (edismax with pf2) are still sluggish. The QTimes > increase as the number of words in a phrase increase. None of the mmap > "warming" seem to have any impact on this. Am I missing anything? Thanks. > > On 9/24/16 5:20 PM, Erick Erickson wrote: >> >> Hmm.. >> >> About <1>: Yep, GC is one of the "more art than science" bits of >> Java/Solr. Siiih. >> >> About <2>: that's what autowarming is about. Particularly the >> filterCache and queryResultCache. My guess is that you have the >> autowarm count on those two caches set to zero. Try setting it to some >> modest number like 16 or 32. The whole _point_ of those parameters is >> to smooth out these kinds of spikes. Additionally, the newSearcher >> event (also in solrconfig.xml) is explicitly intended to allow you to >> hard-code queries that fill the internal caches as well as the mmap OS >> memory from disk, people include facets, sorts and the like in that >> event. It's fired every time a new searcher is opened (i.e. whenever >> you commit and open a new searcher)... >> >> FirstSearcher is for restarts. The difference is that newSearcher >> presumes Solr has been running for a while and the autowarm counts >> have something to work from. OTOH, when you start Solr there's no >> history to autowarm so firstSeracher can be quite a bit more complex >> than newSearcher. Practically, most people just copy newSearcher into >> firstSearcher on the assumption that restarting Solr is pretty >> rare. >> >> about <3> MMap stuff will be controlled by the OS I think. I actually >> worked with a much more primitive system at one point that would be >> dog-slow during off-hours. Someone wrote an equivalent of a cron job >> to tickle the app upon occasion to prevent periodic slowness. >> >> for a nauseating set of details about hard and soft commits, see: >> >> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >> >> Best, >> Erick >> >> >> On Sat, Sep 24, 2016 at 11:35 AM, Rallavagu wrote: >>> >>> >>> >>> On 9/22/16 5:59 AM, Shawn Heisey wrote: On 9/22/2016 5:46 AM, Muhammad Zahid Iqbal wrote: > > > Did you find any solution to slow searches? As far as I know jetty > container default configuration is bit slow for large production > environment. This might be true for the default configuration that comes with a completely stock jetty downloaded from eclipse.org, but the jetty configuration that *Solr* ships with is adequate for just about any Solr installation. The Solr configuration may require adjustment as the query load increases, but the jetty configuration usually doesn't. Thanks, Shawn >>> >>> It turned out to be a "sequence of performance testing sessions" in order >>> to >>> locate slowness. Though I am not completely done with it, here are my >>> finding so far. We are using NRT configuration (warm up count to 0 for >>> caches and NRTCachingDirectoryFactory for index directory) >>> >>> 1. Essentially, solr searches (particularly with edismax and relevance) >>> generate lot of "garbage" that makes GC activity to kick in more often. >>> This >>> becomes even more when facets are included. This has huge impact on >>> QTimes >>> (I have 12g heap and configured 6g to NewSize). >>> >>> 2. After a fresh restart (or core reload) when searches are performed, >>> Solr >>> would initially "populate" mmap entries and this is adding to total >>> QTimes >>> (I have made sure that index files are cached at filesystem layer using >>> vmtouch - https://hoytech.com/vmtouch). When run the same test again with >>> mmap entries populated from previous tests, it shows improved QTimes >>> relative to previous test. >>> >>> 3. Seems the populated mmap entries are flushed away after certain idle >>> time >>> (not sure if it is controlled by Solr or underlying OS). This will make >>> subsequent searches to fetch from "disk" (even though the disk items are >>> cached by OS). >>> >>> So, what I am gonna try next is to tune the field(s) for facets to reduce >>> the index size if possible. Though I am not sure, if it will have impact >>> but >>> would attempt to change the "caches" even though they will be invalidated >>> after a softCommit (every 10 minutes in my case). >>> >>> Any other tips/clues/suggestions are welcome. Thanks. >>> >
Convert BytesRef to long in Solr 6.2
I have the following code inside a Solr post filter. SortedDocValues docValues = DocValues.getSortedSet(context.reader, "my_field"); long x = LegacyNumericUtils.prefixCodedToLong(docValues.lookupOrd(b)) I am in the process of upgrading from Solr 5.5 to 6.2, so I changed NumericUtils to LegacyNumericUtils. Basically, I am taking the BytesRef from a field and extracting the long from it. However, LegacyNumericUtils is the deprecated form of NumericUtils. It says that I should use the PointValues class instead. However, unless I am missing something, it does not seem to support converting BytesRef to long/int/etc. Is there a better method to do this? I would rather not deprecated code. -Kevin
Re: Convert BytesRef to long in Solr 6.2
: I am in the process of upgrading from Solr 5.5 to 6.2, so I changed NumericUtils to LegacyNumericUtils. ... : However, LegacyNumericUtils is the deprecated form of NumericUtils. It : says that I should use the PointValues class instead. However, unless I : am missing something, it does not seem to support converting BytesRef to : long/int/etc. correct -- that's a general lucene level deprecation warning encouraging existing applicatons using NumericUtils to change to use PointsField at index time -- Solr does not yet support points field, so switching your code to use LegacyNumericUtils is appropriate at this point. -Hoss http://www.lucidworks.com/
Re: Archiving documents
I'm not the expert, but I'm thinking you would need an external process to handle this. SOLR itself doesn't seem built to use it's own collection data to act on collection data (I'd love to be wrong about that). So - barring any corrections from the committers, I'm imagining you'd need to write some software that does a query against your collection for the relevant last_modified_date and then either using the returned solr document data (if you stored everything) or by re-querying the data from the original source based on an id -- you would add the document(s) to the "archive" collection. Once you were sure all was well with this process, you could issue a command to delete all the docs with a last_modified_date past a certain point (from the main collection) If there's a built-in way to accomplish this - or if others have already thought this through extensively, I'm certainly interested in hearing about it. Good luck! On Thu, Sep 29, 2016 at 6:55 AM, Vasu Y wrote: > Hi, > We would like to archive documents based on some criteria (like those that > were not modified for more than an year OR are least used) in order to > reduce storage requirements. > I would like hear some of the best practices followed. > > How about having main collection and optionally an archive collection (or > one or more archive collections?) to where we move documents (at regular > intervals) from the main collection based on some criteria (least used or > modified date etc.) and provide a flag during search whether to include > archived documents in search or not? > > Thanks, > Vasu >
RE: JSON Facet "allBuckets" behavior
Can someone help me understand?
With Thanks & Regards
Karthik Ramachandran
From: Karthik Ramachandran
Sent: Tuesday, September 27, 2016 12:21 PM
To: solr-user
Subject: JSON Facet "allBuckets" behavior
While performing json faceting with "allBuckets" and "mincount", I not sure if
I am expecting a wrong result or there is bug?
By "allBucket" definition the response, representing the union of all of the
buckets.
Schema:
Dataset:
1filename11
2filename21
3filename31
4filename41
5filename51
6filename11
7filename21
8filename31
9filename41
10filename11
11filename21
12filename31
13filename11
14filename21
15filename11
For my dataset, with request
http://localhost:8983/solr/jasonfacettest/select/?q=*:*&rows=0&json.facet=
{"sumOfDuplicates":{"type":"terms","field":"filename","mincount":2,"numBuckets":true,"allBuckets":true,"sort":"sum
desc","facet":{"sum":"sum(size)"}}}
below is the response,
"response":{"numFound":15,"start":0,"docs":[]},"facets":{"count":15,"sumOfDuplicates":{"numBuckets":4,"allBuckets":{"count":15,"sum":15.0},"buckets":[{"val":"filename1","count":5,"sum":5.0},{"val":"filename2","count":4,"sum":4.0},{"val":"filename3","count":3,"sum":3.0},{"val":"filename4","count":2,"sum":2.0}]}}}
I was wonder, why the result is not this, since I have "mincount:2"
"response":{"numFound":15,"start":0,"docs":[]},"facets":{"count":15,"sumOfDuplicates":{"numBuckets":4,"allBuckets":{"count":14,"sum":14.0},"buckets":[{"val":"filename1","count":5,"sum":5.0},{"val":"filename2","count":4,"sum":4.0},{"val":"filename3","count":3,"sum":3.0},{"val":"filename4","count":2,"sum":2.0}]}}}
Thanks for the help!!
With Thanks & Regards
Karthik Ramachandran
P Please don't print this e-mail unless you really need to
Re: how to sampling search result
If you constrain random sample to fixed number instead of percentage , reservoir sampling can be used without even calculating the total match count. this can be done on client side. you could stop sampling after a max e.g 10 million. > On Sep 28, 2016, at 10:15 AM, Pushkar Raste wrote: > > Purely of algorithmic point of view - look into reservoir sampling for > unbiased sampling. > > On Sep 28, 2016 11:00 AM, "Yongtao Liu" wrote: > > Alexandre, > > Thanks for reply. > The use case is customer want to review document based on search result. > But they do not want to review all, since it is costly. > So, they want to pick partial (from 1% to 100%) document to review. > For statistics, user also ask this function. > It is kind of common requirement > Do you know any plan to implement this feature in future? > > Post filter should work. Like collapsing query parser. > > Thanks, > Yongtao > -Original Message- > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] > Sent: Tuesday, September 27, 2016 9:25 PM > To: solr-user > Subject: Re: how to sampling search result > > I am not sure I understand what the business case is. However, you might be > able to do something with a custom post-filter. > > Regards, > Alex. > > Newsletter and resources for Solr beginners and intermediates: > http://www.solr-start.com/ > > >> On 27 September 2016 at 22:29, Yongtao Liu wrote: >> Mikhail, >> >> Thanks for your reply. >> >> Random field is based on index time. >> We want to do sampling based on search result. >> >> Like if the random field has value 1 - 100. >> And the query touched documents may all in range 90 - 100. >> So random field will not help. >> >> Is it possible we can sampling based on search result? >> >> Thanks, >> Yongtao >> -Original Message- >> From: Mikhail Khludnev [mailto:m...@apache.org] >> Sent: Tuesday, September 27, 2016 11:16 AM >> To: solr-user >> Subject: Re: how to sampling search result >> >> Perhaps, you can apply a filter on random field. >> >>> On Tue, Sep 27, 2016 at 5:57 PM, googoo wrote: >>> >>> Hi, >>> >>> Is it possible I can sampling based on "search result"? >>> Like run query first, and search result return 1 million documents. >>> With random sampling, 50% (500K) documents return for facet, and stats. >>> >>> The sampling need based on "search result". >>> >>> Thanks, >>> Yongtao >>> >>> >>> >>> -- >>> View this message in context: http://lucene.472066.n3. >>> nabble.com/how-to-sampling-search-result-tp4298269.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >> >> >> >> -- >> Sincerely yours >> Mikhail Khludnev
