AAAAIIIIIIRRRRGGGG! I feel your pain!
On Mon, Feb 14, 2011 at 3:27 PM, Jeff Schmidt <j...@535consulting.com> wrote:
> Wow, okay, it's Cassandra's fault... :)
>
> I create unit tests to use HttpClient and even HttpURLConnection, and the
> former got the non-response from the server, and the latter got unexpected
> end of file. But, if I use curl or telnet, things would work. Anyway, I
> noticed (Mac OS X 10.6.6):
>
> [imac:apache/cassandra/apache-cassandra-0.7.0] jas% netstat -an | grep 8080
> tcp4 0 0 *.8080 *.* LISTEN
> tcp46 0 0 *.8080 *.* LISTEN
> [imac:apache/cassandra/apache-cassandra-0.7.0] jas%
>
> After shutting down tomcat, the tcp4 line would still show up. Only after
> also shutting down Cassandra were there no listeners on port 8080. Starting
> tomcat and Cassandra in either order, neither failed to bind to 8080. Why my
> Java programs tried to talk to Cassandra, and telnet, Firefox, curl etc.
> managed to hook up with Solr, I don't know.
>
> I moved tomcat to port 8090 and things are good... Sigh.. What a big waste
> of time.
>
> Cheers,
>
> Jeff
>
> On Feb 14, 2011, at 2:29 PM, Jeff Schmidt wrote:
>
>> I figured instead of trying to index content, I'd simply issue a query via
>> SolrJ. This seems related to my problem below. I create a
>> CommonsHttpSolrServer instance in the manner already described and in a new
>> method:
>>
>> @Override
>> public List<String> getNodeIdsForProductId(final String productId,
>> final String partnerId) {
>>
>> final List<String> nodes = new ArrayList<String>();
>>
>> final CommonsHttpSolrServer solrServer =
>> (CommonsHttpSolrServer)getSolrServer(partnerId);
>> final SolrQuery query = new SolrQuery();
>> query.setQuery("productId:" + productId);
>> query.addField("nodeId");
>> try {
>> final QueryResponse response = solrServer.query(query);
>> final SolrDocumentList docs = response.getResults();
>> log.info(String.format("getNodeIdsForProductId - got
>> %d nodes for productId: %s",
>> docs.getNumFound(), productId));
>> for (SolrDocument doc : docs) {
>> log.info(doc);
>> }
>> } catch (SolrServerException ex) {
>> final String msg = String.format("Unable to query Solr
>> server %s, for query: %s", solrServer.getBaseURL(), query);
>> log.error(msg);
>> throw new ServiceException(msg, ex);
>> }
>>
>> return nodes;
>> }
>>
>> When issuing the query I get:
>>
>> 2011-02-14 13:13:28 INFO solr.SolrProductIndexService - getSolrServer -
>> Solr url: http://localhost:8080/solr/partner-tmo
>> 2011-02-14 13:13:28 INFO solr.SolrProductIndexService - getSolrServer -
>> construct server for url: http://localhost:8080/solr/partner-tmo
>> 2011-02-14 13:13:28 ERROR solr.SolrProductIndexService - Unable to query
>> Solr server http://localhost:8080/solr/partner-tmo, for query:
>> q=productId%3Aproduct4&fl=nodeId
>> ...
>> Caused by: org.apache.solr.client.solrj.SolrServerException:
>> org.apache.commons.httpclient.NoHttpResponseException: The server localhost
>> failed to respond
>> at
>> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:484)
>> ...
>> Caused by: org.apache.commons.httpclient.NoHttpResponseException: The server
>> localhost failed to respond
>> at
>> org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1976)
>>
>> If I run this through the proxy again, I can see the request being made as:
>>
>> GET
>> /solr/partner-tmo/select?q=productId%3Aproduct4&fl=nodeId&wt=xml&version=2.2
>> HTTP/1.1
>> User-Agent: Solr[org.apache.solr.client.solrj.impl.CommonsHttpSolrServer] 1.0
>> Host: localhost:8080
>>
>> And I get no response from Solr. If instead I use this URL in Firefox:
>>
>> http://localhost:8080/solr/partner-tmo/select?q=productId%3Aproduct4&fl=nodeId&wt=xml&version=2.2
>>
>> I get search results. What is it about SolrJ that is just not working out?
>> What basic thing am I missing? Using Firefox here, or curl below, I can talk
>> to Solr (running in Tomcat 6) just fine. But when going via SolrJ, I cannot
>> update or query. All of this stuff is running on a single system. I guess
>> I'll try a simpler app/unit test to see what happens...
>>
>> This is really a big problem for me. Any suggests are greatly appreciated.
>>
>> Thanks,
>>
>> Jeff
>>
>> On Feb 13, 2011, at 9:15 PM, Jeff Schmidt wrote:
>>
>>> Hello again:
>>>
>>> Back to the javabin iissue:
>>>
>>> On Feb 12, 2011, at 6:07 PM, Lance Norskog wrote:
>>>
>>>> --- But I'm unable to get SolrJ to work due to the 'javabin' version
>>>> mismatch. I'm using the 1.4.1 version of SolrJ, but I always get an
>>>> HTTP response code of 200, but the return entity is simply a null
>>>> byte, which does not match the version number of 1 defined in Solr
>>>> common. ---
>>>>
>>>> I've never seen this problem. At this point you are better off
>>>> starting with 3.x instead of chasing this problem down.
>>>
>>> I'm now using the latest branch_3x built Solr and SolrJ. Other places I've
>>> seen the message:
>>>
>>> Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 0)
>>> or the data in not in 'javabin' format at
>>> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:99) at
>>> org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
>>>
>>> One was told to make sure the version of Solr and SolrJ are compatible, and
>>> that the schema is valid. Unlike 1.4, I see 3.1 actually outputs the
>>> expected and received version numbers, which is helpful. You can see the
>>> invalid version of 0 is indicated which is the zero byte I receive in
>>> response.
>>>
>>> I have Solr running within Tomcat by following the wiki. I have the
>>> conf/Catalina/localhost/solr.xml file set as:
>>>
>>> <?xml version="1.0" encoding="utf-8"?>
>>> <Context
>>> docBase="/usr/local/ingenuity/isec/solr/apache-solr-3.1-SNAPSHOT.war"
>>> debug="0" crossContext="true">
>>> <Environment name="solr/home" type="java.lang.String"
>>>
>>> value="/Users/jas/535Consulting/Clients/Ingenuity/ProfServices/svn/trunk/ing/isec/src/main/solr/multicore"
>>> override="true"/>
>>> </Context>
>>>
>>> With that, I'm able to use my browser to index some content (DIH, curl
>>> etc.) and issue queries, so it seems Solr is running okay in tomcat
>>> (apache-tomcat-6.0.30). To index some Products, I have this simple method:
>>>
>>> @Override
>>> public void addProducts(final Collection<Product> products, final
>>> String indexName) {
>>>
>>> log.info(String.format("addProducts - indexing %d products to
>>> Solr core: %s",
>>> products.size(), indexName));
>>>
>>> Assert.notNull(indexName);
>>>
>>> final Collection<SolrInputDocument> docs = new
>>> ArrayList<SolrInputDocument>();
>>> for (Product product : products) {
>>>
>>> final SolrInputDocument doc =
>>> createDocumentForProduct(product);
>>> docs.add(doc);
>>> log.info("addProduct: document to index: " + doc);
>>> }
>>>
>>> final SolrServer solrServer = getSolrServer(indexName);
>>> try {
>>> solrServer.add(docs);
>>> solrServer.commit(commitWaitFlush, commitWaitSearcher);
>>> } catch (Exception ex) {
>>> final String msg = String.format("Unable to add and
>>> commit %d documents to core: %s",
>>> products.size(), indexName);
>>> log.error(msg);
>>> throw new ServiceException(msg, ex);
>>> }
>>> }
>>>
>>> And I have:
>>>
>>> protected SolrServer getSolrServer(final String indexName) {
>>>
>>> final String url = solrServerBaseUrl + indexName;
>>> log.info("getSolrServer - construct server for url: " + url);
>>> try {
>>> final CommonsHttpSolrServer solrServer = new
>>> CommonsHttpSolrServer(solrServerBaseUrl + indexName);
>>> //solrServer.setParser(new BinaryResponseParser());
>>> //solrServer.setParser(new XMLResponseParser());
>>> solrServer.setRequestWriter(new BinaryRequestWriter());
>>> return solrServer;
>>> } catch (Exception ex) {
>>> final String msg = String.format("Unable to create
>>> Solr server for url: %s", url);
>>> log.error(msg);
>>> throw new ServiceException(msg, ex);
>>> }
>>> }
>>>
>>> Note that this is code for prototyping. :)
>>>
>>> As you can see, in getSolrServer() I'm trying various settings.
>>> http://wiki.apache.org/solr/Solrj is tagged Solr 1.4, but I'm assuming it's
>>> at least very similar in 3.1. For the core in question, solrconfig.xml does
>>> have:
>>>
>>> <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
>>> <requestHandler name="/update/javabin"
>>> class="solr.BinaryUpdateRequestHandler" />
>>>
>>> I can see in the Solr log:
>>>
>>> Feb 13, 2011 3:35:14 PM org.apache.solr.core.RequestHandlers
>>> initHandlersFromConfig
>>> INFO: created /update/javabin: solr.BinaryUpdateRequestHandler
>>>
>>> Running this through the burp proxy to try to see what's going on, I can
>>> see my application making the following request to Solr via SolrJ:
>>>
>>> ------------------------------------------
>>> POST /solr/partner-tmo/update/javabin?wt=javabin&version=2 HTTP/1.1
>>> User-Agent: Solr[org.apache.solr.client.solrj.impl.CommonsHttpSolrServer]
>>> 1.0
>>> Host: localhost:8090
>>> Content-Type: application/octet-stream
>>> Content-Length: 543
>>>
>>> Äà¶msÀà'delByIdà&delByQà$docs
>>> …Áà%boostÃà$name"idà#val-ING:afa|08520åÃæ&nodeIdç'ING:afaåÃæ)productIdç%08520åÃæ0nodeSourceIdTypeç"EGåÃæ,nodeSourceIdç#672åÃæ+productTypeç≠(chemical%shRNAåÃæ-description_tç?
>>> This is the description for product
>>> 08520åÃæ'brand_sç%FLUKAåÃæ%sku_sç(A980-852å…ÁåÃæ"idç-ING:afa|08530åÃæ&nodeIdç'ING:afaåÃæ)productIdç%08530åÃæ0nodeSourceIdTypeç"EGåÃæ,nodeSourceIdç#672åÃæ+productTypeç™(chemicalåÃæ-description_tç?
>>> This is the description for product
>>> 08530åÃæ'brand_sç%FLUKAåÃæ%sku_sç(A980-853å
>>> ------------------------------------------
>>>
>>> That looks pretty binary. In response I see:
>>>
>>> ------------------------------------------
>>> HTTP/1.0 200 OK
>>> Content-type: application/octet-stream
>>> Content-length: 1
>>>
>>>
>>> ------------------------------------------
>>>
>>> Looking at the hex view, I can see the one byte of data is 0x00.
>>>
>>> My other approach was to go the XML route. So, to do this, I comment out
>>> the setting of the request writer and go with the default, which the wiki
>>> says is XML. Running this through the proxy I see:
>>>
>>> ------------------------------------------
>>> POST /solr/partner-tmo/update?wt=javabin&version=2 HTTP/1.1
>>> User-Agent: Solr[org.apache.solr.client.solrj.impl.CommonsHttpSolrServer]
>>> 1.0
>>> Host: localhost:8090
>>> Content-Type: text/xml; charset=utf-8
>>> Content-Length: 856
>>>
>>> <add><doc boost="1.0"><field name="id">ING:afa|08520</field><field
>>> name="nodeId">ING:afa</field><field name="productId">08520</field><field
>>> name="nodeSourceIdType">EG</field><field
>>> name="nodeSourceId">672</field><field
>>> name="productType">chemical</field><field
>>> name="productType">shRNA</field><field name="description_t">This is the
>>> description for product 08520</field><field
>>> name="brand_s">FLUKA</field><field name="sku_s">A980-852</field></doc><doc
>>> boost="1.0"><field name="id">ING:afa|08530</field><field
>>> name="nodeId">ING:afa</field><field name="productId">08530</field><field
>>> name="nodeSourceIdType">EG</field><field
>>> name="nodeSourceId">672</field><field
>>> name="productType">chemical</field><field name="description_t">This is the
>>> description for product 08530</field><field
>>> name="brand_s">FLUKA</field><field name="sku_s">A980-853</field></doc></add>
>>> ------------------------------------------
>>>
>>> I can see it has specified the proper update handler in the URI and XML is
>>> being uploaded. In response though, I get the exact same one as when going
>>> binary, including the application/octet-stream content type. I end up with
>>> the same javabin version mismatch stacktrace as well, even though I'm
>>> trying to talk XML. But, the presence of wt=javabin&version=2 is not very
>>> encouraging when going the default XML route.
>>>
>>> Just for grins, I added:
>>>
>>> solrServer.setParser(new XMLResponseParser());
>>>
>>> And now the exception is:
>>>
>>> Caused by:
>>> com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character (NULL,
>>> unicode 0) encountered: not valid in any content| at [row,col
>>> {unknown-source}]: [1,1]
>>> at
>>> com.ctc.wstx.sr.StreamScanner.constructNullCharException(StreamScanner.java:640)
>>>
>>> So, apparently javabin is out of the way, and now the zero byte returned is
>>> mucking up the XML parser. According to proxy, the request is similar to
>>> the previous one, but:
>>>
>>> POST /solr/partner-tmo/update?wt=xml&version=2.2 HTTP/1.1
>>>
>>> So, great we are going the XML route, but the response is the same HTTP 200
>>> and a zero byte...
>>>
>>> So, I'm not sure what's going on. I can say though that I am not seeing
>>> any log activity solr.2011-*.log once Solr has started and I attempt to
>>> issue these requests. Maybe it's tomcat? But, if I go directly to Solr
>>> outside of SolrJ and add some content to that same core, I do see log
>>> activity, and get a valid response:
>>>
>>> [imac:solr/input-data/tmo-products] jas% curl --header "Content-type:
>>> text/xml; charset=utf-8" --request POST -w "\nhttp code: %{http_code}\n" -d
>>> @nodes-products.xml "http://localhost:8080/solr/partner-tmo/update";
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <response>
>>> <lst name="responseHeader"><int name="status">0</int><int
>>> name="QTime">11</int></lst>
>>> </response>
>>>
>>> http code: 200
>>> [imac:solr/input-data/tmo-products] jas%
>>>
>>> Much better than a single null byte.
>>>
>>> Any idea of what is going on? Apologies for the long email, but I'm trying
>>> to provide all the details. This problem must reside in something I'm
>>> doing. I'm sure others are using SolrJ successfully.
>>>
>>> Thanks!
>>>
>>> Jeff
>>>
>>>>
>>>> On Sat, Feb 12, 2011 at 1:37 PM, Jeff Schmidt <j...@535consulting.com>
>>>> wrote:
>>>>> Hello:
>>>>>
>>>>> I'm working on incorporating Solr into a SaaS based life sciences
>>>>> semantic search project. This will be released in about six months. I'm
>>>>> trying to determine which version of Solr makes the most sense. When
>>>>> going to the Solr download page, there are 1.3.0, 1.4.0, and 1.4.1. I've
>>>>> been using 1.4.1 while going through some examples in my Packt book
>>>>> ("Solr 1.4 Enterprise Search Server").
>>>>>
>>>>> But, I also see that Solr 3.1 and 4.0 are in the works. According to:
>>>>>
>>>>>
>>>>> https://issues.apache.org/jira/browse/#selectedTab=com.atlassian.jira.plugin.system.project%3Aroadmap-panel
>>>>>
>>>>> there is a high degree of progress on both of those releases; including a
>>>>> slew of bug fixes, new features, performance enhancements etc. Should I
>>>>> be making use of one of the newer versions? The hierarchical faceting
>>>>> seems like it could be quite useful. Are there any guesses on when
>>>>> either 3.1 or 4.0 will be officially released?
>>>>>
>>>>> So far, 1.4.1 has been good. But I'm unable to get SolrJ to work due to
>>>>> the 'javabin' version mismatch. I'm using the 1.4.1 version of SolrJ, but
>>>>> I always get an HTTP response code of 200, but the return entity is
>>>>> simply a null byte, which does not match the version number of 1 defined
>>>>> in Solr common. Anyway, I can follow up on that issue if 1.4.1 is still
>>>>> the most appropriate version to use these days. Otherwise, I'll try again
>>>>> with whatever version you suggest.
>>>>>
>>>>> Thanks a lot!
>>>>>
>>>>> Jeff
>>>>> --
>>>>> Jeff Schmidt
>>>>> 535 Consulting
>>>>> j...@535consulting.com
>>>>> (650) 423-1068
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Lance Norskog
>>>> goks...@gmail.com
>>>
>>> --
>>> Jeff Schmidt
>>> 535 Consulting
>>> j...@535consulting.com
>>> (650) 423-1068
>>> http://www.535consulting.com
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Jeff Schmidt
>> 535 Consulting
>> j...@535consulting.com
>> (650) 423-1068
>> http://www.535consulting.com
>>
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Jeff Schmidt
> 535 Consulting
> j...@535consulting.com
> (650) 423-1068
> http://www.535consulting.com
>
>
>
>
>
>
>
>
--
Lance Norskog
goks...@gmail.com
