Re: Get list of collection
Hi Erick,
We are using the 4.7.2 version of solr and no getCollectionList() method is
present in CloudSolrServer class.
Regards,
Ankit Jain
On Tue, Dec 2, 2014 at 8:18 PM, Erick Erickson
wrote:
> I think you want CloudSolrServer.getCollectionList()
>
> Best,
> Erick
>
> On Tue, Dec 2, 2014 at 12:27 AM, Ankit Jain
> wrote:
> > Hi All,
> >
> > I have a requirement to get the list of *collection* available in Solr.
> we
> > are using solrj library.
> >
> > I am able to fetch the list of cores but not getting ways to fetch the
> list
> > of collections.
> >
> > Below is the sample example, that i am using to fetch the cores:
> >
> > CoreAdminRequest request = new CoreAdminRequest();
> > request.setAction(CoreAdminAction.STATUS);
> > CoreAdminResponse cores = request.process(server);
> >
> > // List of the cores
> > List coreList = new ArrayList();
> > for (int i = 0; i < cores.getCoreStatus().size(); i++) {
> > coreList.add(cores.getCoreStatus().getName(i));
> > }
> >
> > Please help.
> >
> > --
> > Thanks,
> > Ankit Jain
>
--
Thanks,
Ankit Jain
Problem with additional Servlet Filter (SolrRequestParsers Exception)
Hi Folks, I have a problem with an additional servlet filter defined in my web.xml (Tomcat 7.x). In Solr 4.2.1. we've successfully used a filter for processing POST request data (basically, the filter reads the POST data, collects some parameters from it and writes it back to the request, based on this example: http://www.coderanch.com/t/484631/Tomcat/configure-Tomcat-log-POST-data) To make this work, the filter has to be the first one defined in the web.xml. But now in Solr 4.8.0, if we define that filter, Solr complains that there is a filter before it and claims that we have to remove it: null:org.apache.solr.common.SolrException: Solr requires that request parameters sent using application/x-www-form-urlencoded content-type can be read through the request input stream. Unfortunately, the stream was empty / not available. This may be caused by another servlet filter calling ServletRequest.getParameter*() before SolrDispatchFilter, please remove it. at org.apache.solr.servlet.SolrRequestParsers$FormDataRequestParser.getParameterIncompatibilityException(SolrRequestParsers.java:622) Here is my web.xml: post-data-dumper-filter filters.PostDataDumperFilter post-data-dumper-filter /* SolrRequestFilter org.apache.solr.servlet.SolrDispatchFilter SolrRequestFilter /* Any idea how to solve this? Why does Solr have a problem now if there is any pre-filter defined? Thanks a lot, Stefan
RE: Reminder: FOSDEM 2015 - Open Source Search Dev Room
Hello everyone, We have extended the deadline for submissions to the FOSDEM 2015 Open Source Search Dev Room to Monday, 9 December at 23:59 CET. We are looking forward to your talk proposal! Cheers, Uwe - Uwe Schindler uschind...@apache.org Apache Lucene PMC Member / Committer Bremen, Germany http://lucene.apache.org/ > -Original Message- > From: Uwe Schindler [mailto:uschind...@apache.org] > Sent: Monday, November 24, 2014 9:33 AM > To: d...@lucene.apache.org; java-u...@lucene.apache.org; solr- > u...@lucene.apache.org; gene...@lucene.apache.org > Subject: Reminder: FOSDEM 2015 - Open Source Search Dev Room > > Hi, > > We host a Dev-Room about "Open Source Search" on this year's FOSDEM > 2015 (https://fosdem.org/2015/), taking place on January 31th and February > 1st, 2015, in Brussels, Belgium. There is still one more week to submit your > talks, so hurry up and submit your talk early! > > Here is the full CFP as posted a few weeks ago: > > Search has evolved to be much more than simply full-text search. We now > rely on “search engines” for a wide variety of functionality: > search as navigation, search as analytics and backend for data visualization > and sometimes, dare we say it, as a data store. The purpose of this dev room > is to explore the new world of open source search engines: their enhanced > functionality, new use cases, feature and architectural deep dives, and the > position of search in relation to the wider set of software tools. > > We welcome proposals from folks working with or on open source search > engines (e.g. Apache Lucene, Apache Solr, Elasticsearch, Seeks, Sphinx, etc.) > or technologies that heavily depend upon search (e.g. > NoSQL databases, Nutch, Apache Hadoop). We are particularly interested in > presentations on search algorithms, machine learning, real-world > implementation/deployment stories and explorations of the future of > search. > > Talks should be 30-60 minutes in length, including time for Q&A. > > You can submit your talks to us here: > https://docs.google.com/forms/d/11yLMj9ZlRD1EMU3Knp5y6eO3H5BRK7V3 > 8G0OxSfp84A/viewform > > Our Call for Papers will close at 23:59 CEST on Monday, December 1, 2014. We > cannot guarantee we will have the opportunity to review submissions made > after the deadline, so please submit early (and often)! > > Should you have any questions, you can contact the Dev Room > organizers: opensourcesearch-devr...@lists.fosdem.org > > Cheers, > LH on behalf of the Open Source Search Dev Room Program Committee* > > * Boaz Leskes, Isabel Drost-Fromm, Leslie Hawthorn, Ted Dunning, Torsten > Curdt, Uwe Schindler > > - > Uwe Schindler > uschind...@apache.org > Apache Lucene PMC Member / Committer > Bremen, Germany > http://lucene.apache.org/ > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org
RE: recovery process - node with stale data elected leader
FYI, I did the test with 4.7.1 and the issue is fixed. With this version, the node who was leader when the cluster went down is elected leader again even if it's not restarted first. Francois -Original Message- From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] Sent: Saturday, November 08, 2014 7:52 AM To: solr-user@lucene.apache.org Subject: Re: recovery process - node with stale data elected leader Hi, Not a direct answer to your question, sorry, but since 4.6.0 is relatively old and there have been a ton of changes around leader election, syncing, replication, etc., I'd first jump to the latest Solr and then see if this is still a problem. Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Thu, Nov 6, 2014 at 5:32 AM, wrote: > Hi all, > > Any idea on my issue below? > > Thanks > Francois > > -Original Message- > From: Grollier, Francois: IT (PRG) > Sent: Tuesday, November 04, 2014 6:19 PM > To: solr-user@lucene.apache.org > Subject: recovery process - node with stale data elected leader > > Hi, > > I'm running solrCloud 4.6.0 and I have a question/issue regarding the > recovery process. > > My cluster is made of 2 shards with 2 replicas each. Nodes A1 and B1 > are leaders, A2 and B2 followers. > > I start indexing docs and kill A2. I keep indexing for a while and > then kill A1. At this point, the cluster stops serving queries as one > shard is completely unavailable. > Then I restart A2 first, then A1. A2 gets elected leader, waits a bit > for more replicas to be up and once it sees A1 it starts the recovery process. > My understanding of the recovery process was that at this point A2 > would notice that A1 has a more up to date state and it would sync > with A1. It seems to happen like this but then I get: > > INFO - 2014-11-04 11:50:43.068; > org.apache.solr.cloud.RecoveryStrategy; > Attempting to PeerSync from http://a1:8111/solr/executions/ > core=executions - recoveringAfterStartup=false INFO - 2014-11-04 > 11:50:43.069; org.apache.solr.update.PeerSync; PeerSync: > core=executions url=http://a2:8211/solr START > replicas=[http://a1:8111/solr/executions/] > nUpdates=100 INFO - 2014-11-04 11:50:43.076; > org.apache.solr.update.PeerSync; PeerSync: core=executions url= > http://a2:8211/solr Received 98 versions from > a1:8111/solr/executions/ INFO - 2014-11-04 11:50:43.076; > org.apache.solr.update.PeerSync; PeerSync: > core=executions url=http://a2:8211/solr Our versions are newer. > ourLowThreshold=1483859630192852992 otherHigh=1483859633446584320 INFO > - > 2014-11-04 11:50:43.077; org.apache.solr.update.PeerSync; PeerSync: > core=executions url=http://a2:8211/solr DONE. sync succeeded > > > And I end up with a different set of documents in each node (actually > A1 has all the documents but A2 misses some). > > Is my understanding wrong and is it a completely nonsense to start A2 > before A1? > > If my understanding right, what could cause the desync? (I can provide > more logs) And is there a way to force A2 to index the missing > documents? I have try the FORCERECOVERY command but it generates the > same result as shown above. > > Thanks > francois > > ___ > > This message is for information purposes only, it is not a > recommendation, advice, offer or solicitation to buy or sell a product > or service nor an official confirmation of any transaction. It is > directed at persons who are professionals and is not intended for > retail customer use. Intended for recipient only. This message is subject to > the terms at: > www.barclays.com/emaildisclaimer. > > For important disclosures, please see: > www.barclays.com/salesandtradingdisclaimer regarding market commentary > from Barclays Sales and/or Trading, who are active market > participants; and in respect of Barclays Research, including > disclosures relating to specific issuers, please see > http://publicresearch.barclays.com. > > ___ > ___ > > This message is for information purposes only, it is not a > recommendation, advice, offer or solicitation to buy or sell a product > or service nor an official confirmation of any transaction. It is > directed at persons who are professionals and is not intended for > retail customer use. Intended for recipient only. This message is subject to > the terms at: > www.barclays.com/emaildisclaimer. > > For important disclosures, please see: > www.barclays.com/salesandtradingdisclaimer regarding market commentary > from Barclays Sales and/or Trading, who are active market > participants; and in respect of Barclays Research, including > disclosures relating to specific issuers, please see > http://publicresearch.barclays.com. > > ___ > _
SegmentInfos exposed to /admin/luke
Dear All, We have a high percentage of deleted docs which do not go away because there are several huge ancient segments that do not merge with anything else naturally. Our use case in constant reindexing of same data - ~100 gb, 12 000 000 real records, 20 000 000 total records in index, which is ~80% deletes. We plan to deal with situation by playing with mergeFactor, reclaimDeletesWeight and maxSegmentSizeMB settings to optimize for our re-indexing rate and data size. And in order to do it with eyes-opened we want to see a picture similar to http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html with columns of segment size and %of deletes. The plan is to expose SegmentInfos via /admin/luke handler and draw column bars in Solr admin. Is there an easier way to achieve that? Even in raw Luke we didn't' found these data. We'd be happy to push the changes to Solr afterwards. Thank you, Alexey Kozhemiakin
Re: Problem with additional Servlet Filter (SolrRequestParsers Exception)
Hi again,
just for reference, here is my filter class (taken from the example
posted earlier) - as soon as I iterate over the request parameters, Solr
gets angry... :(
I have also tried HttpServletRequestWrapper, but that didn't help
either... nor did this:
http://ocpsoft.org/opensource/how-to-safely-add-modify-servlet-request-parameter-values/
(because I don't want to only add some static values, I still have to
iterate over the original request's parameters to get my desired data
out of the request's POST data)
Here goes...
public final class PostDataDumperFilter implements Filter {
private FilterConfig filterConfig = null;
public void init(FilterConfig filterConfig) throws ServletException {
this.filterConfig = filterConfig;
}
public void destroy() {
this.filterConfig = null;
}
public void setFilterConfig(FilterConfig fc) {
filterConfig=fc;
}
public FilterConfig getFilterConfig() {
return filterConfig;
}
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
if (filterConfig == null)
return;
HttpServletRequest httpRequest = (HttpServletRequest) request;
// The variable "postdata" is used in the Solr Tomcat Valve
Enumeration names = httpRequest.getParameterNames();
StringBuffer output = new StringBuffer();
boolean skipRequest = false;
while (names.hasMoreElements()) {
String name = (String) names.nextElement();
output.append(name + "=");
String values[] = httpRequest.getParameterValues(name);
for (int i = 0; i < values.length; i++) {
if (i > 0) {
output.append("&" + name + "=");
}
output.append(values[i]);
// ignore paging requests, where request parameter
"start" is > 0, e.g. "&start=12":
if(name.equalsIgnoreCase("start") &&!
values[i].equals("0")) {
skipRequest = true;
}
}
if (names.hasMoreElements())
output.append("&");
}
if(!skipRequest) {
request.setAttribute("postdata", output);
}
chain.doFilter(request, response);
}
}
Thanks,
Stefan
Am 03.12.2014 um 09:47 schrieb Stefan Moises:
Hi Folks,
I have a problem with an additional servlet filter defined in my
web.xml (Tomcat 7.x).
In Solr 4.2.1. we've successfully used a filter for processing POST
request data (basically, the filter reads the POST data, collects some
parameters from it and writes it back to the request, based on this
example:
http://www.coderanch.com/t/484631/Tomcat/configure-Tomcat-log-POST-data)
To make this work, the filter has to be the first one defined in the
web.xml.
But now in Solr 4.8.0, if we define that filter, Solr complains that
there is a filter before it and claims that we have to remove it:
null:org.apache.solr.common.SolrException: Solr requires that request
parameters sent using application/x-www-form-urlencoded content-type
can be read through the request input stream. Unfortunately, the
stream was empty / not available. This may be caused by another
servlet filter calling ServletRequest.getParameter*() before
SolrDispatchFilter, please remove it.
at
org.apache.solr.servlet.SolrRequestParsers$FormDataRequestParser.getParameterIncompatibilityException(SolrRequestParsers.java:622)
Here is my web.xml:
post-data-dumper-filter
filters.PostDataDumperFilter
post-data-dumper-filter
/*
SolrRequestFilter
org.apache.solr.servlet.SolrDispatchFilter
SolrRequestFilter
/*
Any idea how to solve this? Why does Solr have a problem now if there
is any pre-filter defined?
Thanks a lot,
Stefan
--
Mit den besten Grüßen aus Nürnberg,
Stefan Moises
***
Stefan Moises
Senior Softwareentwickler
Leiter Modulentwicklung
shoptimax GmbH
Ulmenstrasse 52 H
90443 Nürnberg
Amtsgericht Nürnberg HRB 21703
GF Friedrich Schreieck
Fax: 0911/25566-29
moi...@shoptimax.de
http://www.shoptimax.de
***
Too much Lucene code to refactor but I like SolrCloud
I have an existing application that includes Lucene code. I want to add high availability. From what I have read SolrCloud looks like an effective approach. My problem is that there is a lot of Lucene code; out of 100+ java files in the application more than 20 of them are focused on Lucene code. Refactoring this much code seems very risky. My thought was to migrate the index from Lucene 35 to Solr/Lucene 4.10 then after making sure everything still works I would add in the HA. I looked at solrj with the hope that it would look like Lucene but it did not look like it would simplify the transition. So the question is can I leave most of the existing Lucene code and just make small changes to get the benefit of HA from SolrCloud. Is there a better approach?
Alternative searching
Hello, is possible searching by Solr search alternative words from some field? For example if i want search some phrase from range: At first position i want to have probably in one field hello,hi,cheerio. At second my At third name At fourth is At fifth Tomas, John, Paul. And if i send query "My Tomas"~3 / "My John"~3 / "Hi Paul"~4 So that always will find the required query. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SegmentInfos exposed to /admin/luke
You can't use grouping aliases and do full rebuilds on a separate core + substitutions? Might be a better strategy for nearly complete replacement. Regards, Alex. P.s. But I like your proposal anyway. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 3 December 2014 at 06:35, Alexey Kozhemiakin wrote: > Dear All, > > We have a high percentage of deleted docs which do not go away because there > are several huge ancient segments that do not merge with anything else > naturally. Our use case in constant reindexing of same data - ~100 gb, 12 000 > 000 real records, 20 000 000 total records in index, which is ~80% deletes. > > We plan to deal with situation by playing with mergeFactor, > reclaimDeletesWeight and maxSegmentSizeMB settings to optimize for our > re-indexing rate and data size. > And in order to do it with eyes-opened we want to see a picture similar to > http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html > with columns of segment size and %of deletes. > The plan is to expose SegmentInfos via /admin/luke handler and draw column > bars in Solr admin. > > Is there an easier way to achieve that? Even in raw Luke we didn't' found > these data. > > We'd be happy to push the changes to Solr afterwards. > > > Thank you, > Alexey Kozhemiakin >
Re: Alternative searching
Have you looked at the synonyms token filter that is shipped in the default example? Might be the first step. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 3 December 2014 at 08:39, tomas.kalas wrote: > Hello, > is possible searching by Solr search alternative words from some field? > For example if i want search some phrase from range: > > At first position i want to have probably in one field hello,hi,cheerio. > At second my > At third name > At fourth is > At fifth Tomas, John, Paul. > > And if i send query "My Tomas"~3 / "My John"~3 / "Hi Paul"~4 > So that always will find the required query. > > Thanks. > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339.html > Sent from the Solr - User mailing list archive at Nabble.com.
RE: SegmentInfos exposed to /admin/luke
Hi Alexandre, our rebuilds are not like 'full rebuilds' - it's a constant massive flow of price updates in ecommerce marketplace. Unfortunately "substitution" option is not working for us :( -Original Message- From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Sent: Wednesday, December 3, 2014 16:39 To: solr-user Subject: Re: SegmentInfos exposed to /admin/luke You can't use grouping aliases and do full rebuilds on a separate core + substitutions? Might be a better strategy for nearly complete replacement. Regards, Alex. P.s. But I like your proposal anyway. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 3 December 2014 at 06:35, Alexey Kozhemiakin wrote: > Dear All, > > We have a high percentage of deleted docs which do not go away because there > are several huge ancient segments that do not merge with anything else > naturally. Our use case in constant reindexing of same data - ~100 gb, 12 000 > 000 real records, 20 000 000 total records in index, which is ~80% deletes. > > We plan to deal with situation by playing with mergeFactor, > reclaimDeletesWeight and maxSegmentSizeMB settings to optimize for our > re-indexing rate and data size. > And in order to do it with eyes-opened we want to see a picture similar to > http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html > with columns of segment size and %of deletes. > The plan is to expose SegmentInfos via /admin/luke handler and draw column > bars in Solr admin. > > Is there an easier way to achieve that? Even in raw Luke we didn't' found > these data. > > We'd be happy to push the changes to Solr afterwards. > > > Thank you, > Alexey Kozhemiakin >
Re: Alternative searching
Hi Tomas, I dont fully follow but, with ComplexPhraseQueryParser, you can use parenthesis inside phrases. q="(hello hi cheerio) my name is (Tomas John Paul)"&defType=complexphrase https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-ComplexPhraseQueryParser Ahmet On Wednesday, December 3, 2014 3:40 PM, tomas.kalas wrote: Hello, is possible searching by Solr search alternative words from some field? For example if i want search some phrase from range: At first position i want to have probably in one field hello,hi,cheerio. At second my At third name At fourth is At fifth Tomas, John, Paul. And if i send query "My Tomas"~3 / "My John"~3 / "Hi Paul"~4 So that always will find the required query. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Alternative searching
Ok and how do you think how i get data into to fields? And how it recognize so how it is one term? -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339p4172349.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Alternative searching
Its ok, when i use the example by synonym filter, so it wokrs, but i don´t know how i have transfer this text to the schema. -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339p4172356.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Alternative searching
Hi Tomas, What do you mean by "get data into fields"? Do you mean indexing? Following solr search URL will return all possible 9 documents. q=someField:"(hello hi cheerio) my name is (Tomas John Paul)"&defType=complexphrase for example, doc1 = hello my name is Tomas doc2 = hi my name is Paul ... doc9 = cheerio my name is John. Isn't that what you want? On Wednesday, December 3, 2014 4:13 PM, tomas.kalas wrote: Ok and how do you think how i get data into to fields? And how it recognize so how it is one term? -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339p4172349.html Sent from the Solr - User mailing list archive at Nabble.com. On Wednesday, December 3, 2014 4:13 PM, tomas.kalas wrote: Ok and how do you think how i get data into to fields? And how it recognize so how it is one term? -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339p4172349.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Alternative searching
I think something like that: First Position Second Position Third Position Fourth Position Fift Position - -- --- Hello MyName Is Paul Hi Tomas Cherio John And it is like one sentence, and i thinks so it don¨t bee in more docs. -- View this message in context: http://lucene.472066.n3.nabble.com/Alternative-searching-tp4172339p4172361.html Sent from the Solr - User mailing list archive at Nabble.com.
Group by on multiple fields and Stats Solr
Hi All, I need to perform the following queries into Solr: SELECT a, b, Sum(a) FROM tbl GROUP BY a, b. -- Thanks, Ankit Jain
Refreshed Solr Clients page
Hello, Just a heads up that the Solr clients page on the Wiki has been substantially refreshed and brought up to date: https://wiki.apache.org/solr/IntegratingSolr If you think you know an obscure Solr client (Dart?), now would be good time to add it to the Wiki or at least let me know and I'll do it. Also, if you know of a particularly successful fork of a client library that's gone stale, let's put that in as well. It's quite hard to figure that out otherwise. And if you think there is no client for your language of choice, check it out, you might be surprised. Some real research went into this cleanup :-) Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
Re: Get list of collection
On 12/3/2014 2:51 AM, Ankit Jain wrote: > Hi Erick, > > We are using the 4.7.2 version of solr and no getCollectionList() method is > present in CloudSolrServer class. That method is private, so you can't use it in a user program. I think this would probably do what you want, if server is an instance of CloudSolrServer: Set list = server.getZkStateReader().getClusterState().getCollections(); Thanks, Shawn
Dynamic boosting with functions
I need to boost some of document with with a function. I can give boosting for field "myField" with bq = myField^10 But I like to give dynamic boosting based on whether or not other field "someotherfield" exists in a document or not. I like to give boosting if (exists(termfreq(someotherfield,"green")),100,1) to "myField" field Is this possible? If so, how? Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Dynamic-boosting-with-functions-tp4172387.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with additional Servlet Filter (SolrRequestParsers Exception)
Stefan I had problems like this -- and the short answer is -- it's a
PITA. Solr is not really designed to be extended in this way. In fact
I believe they are moving towards an architecture where this is even
less possible - folks will be encouraged to run solr using a bundled
exe, perhaps with jetty embedded (I'm not all up to speed on this, maybe
someone will correct me), and no war file shipped.
So perhaps a better strategy is to wrap the service at the HTTP layer
using a proxy.
Still, you can probably fix your immediate problem by extending Solr's
SolrDispatchFilter class. Here's how I did that:
https://github.com/msokolov/lux/blob/master/src/main/java/lux/solr/LuxDispatchFilter.java
-Mike
On 12/03/2014 08:02 AM, Stefan Moises wrote:
Hi again,
just for reference, here is my filter class (taken from the example
posted earlier) - as soon as I iterate over the request parameters,
Solr gets angry... :(
I have also tried HttpServletRequestWrapper, but that didn't help
either... nor did this:
http://ocpsoft.org/opensource/how-to-safely-add-modify-servlet-request-parameter-values/
(because I don't want to only add some static values, I still have to
iterate over the original request's parameters to get my desired data
out of the request's POST data)
Here goes...
public final class PostDataDumperFilter implements Filter {
private FilterConfig filterConfig = null;
public void init(FilterConfig filterConfig) throws ServletException {
this.filterConfig = filterConfig;
}
public void destroy() {
this.filterConfig = null;
}
public void setFilterConfig(FilterConfig fc) {
filterConfig=fc;
}
public FilterConfig getFilterConfig() {
return filterConfig;
}
public void doFilter(ServletRequest request, ServletResponse
response,
FilterChain chain) throws IOException, ServletException {
if (filterConfig == null)
return;
HttpServletRequest httpRequest = (HttpServletRequest) request;
// The variable "postdata" is used in the Solr Tomcat Valve
Enumeration names = httpRequest.getParameterNames();
StringBuffer output = new StringBuffer();
boolean skipRequest = false;
while (names.hasMoreElements()) {
String name = (String) names.nextElement();
output.append(name + "=");
String values[] = httpRequest.getParameterValues(name);
for (int i = 0; i < values.length; i++) {
if (i > 0) {
output.append("&" + name + "=");
}
output.append(values[i]);
// ignore paging requests, where request parameter
"start" is > 0, e.g. "&start=12":
if(name.equalsIgnoreCase("start") &&!
values[i].equals("0")) {
skipRequest = true;
}
}
if (names.hasMoreElements())
output.append("&");
}
if(!skipRequest) {
request.setAttribute("postdata", output);
}
chain.doFilter(request, response);
}
}
Thanks,
Stefan
Am 03.12.2014 um 09:47 schrieb Stefan Moises:
Hi Folks,
I have a problem with an additional servlet filter defined in my
web.xml (Tomcat 7.x).
In Solr 4.2.1. we've successfully used a filter for processing POST
request data (basically, the filter reads the POST data, collects
some parameters from it and writes it back to the request, based on
this example:
http://www.coderanch.com/t/484631/Tomcat/configure-Tomcat-log-POST-data)
To make this work, the filter has to be the first one defined in the
web.xml.
But now in Solr 4.8.0, if we define that filter, Solr complains that
there is a filter before it and claims that we have to remove it:
null:org.apache.solr.common.SolrException: Solr requires that request
parameters sent using application/x-www-form-urlencoded content-type
can be read through the request input stream. Unfortunately, the
stream was empty / not available. This may be caused by another
servlet filter calling ServletRequest.getParameter*() before
SolrDispatchFilter, please remove it.
at
org.apache.solr.servlet.SolrRequestParsers$FormDataRequestParser.getParameterIncompatibilityException(SolrRequestParsers.java:622)
Here is my web.xml:
post-data-dumper-filter
filters.PostDataDumperFilter
post-data-dumper-filter
/*
SolrRequestFilter
org.apache.solr.servlet.SolrDispatchFilter
SolrRequestFilter
/*
Any idea how to solve this? Why does Solr have a problem now if there
is any pre-filter defined?
Thanks a lot,
Stefan
