how to scan dynamic field without specifying each field in query

2009-09-03 Thread gdeconto

say I have a dynamic field called Foo* (where * can be in the hundreds) and
want to search Foo* for a value of 3 (for example)

I know I can do this via this:

http://localhost:8994/solr/select?q=(Foo1:3 OR Foo2:3 OR Foo3:3 OR …
Foo999:3)

However, is there a better way?  i.e. is there some way to query by a
function I create, possibly something like this:

http://localhost:8994/solr/select?q=myfunction(‘Foo’, 3)

where myfunction itself iterates thru all the instances of Foo*

any help appreciated

-- 
View this message in context: 
http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280228.html
Sent from the Solr - User mailing list archive at Nabble.com.



RE: how to scan dynamic field without specifying each field in query

2009-09-03 Thread gdeconto

thx for the reply.

you mean into a multivalue field?  possible, but was wondering if there was
something more flexible than that.  the ability to use a function (ie
myfunction) would open up some possibilities for more complex searching and
search syntax.

I could write my own query parser with special extended syntax, but that is
farther than I wanted to go.



Manepalli, Kalyan wrote:
> 
> You can copy the dynamic fields value into a different field and query on
> that field.
> 
> Thanks,
> Kalyan Manepalli
> 
> 

-- 
View this message in context: 
http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280669.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how to scan dynamic field without specifying each field in query

2009-09-03 Thread gdeconto

I am thinking that my example was too simple/generic :-U.  It is possible for
more several dynamic fields to exist and other functionality to be required.
i.e. what about if my example had read:

http://localhost:8994/solr/select?q=((Foo1:3 OR Foo2:3 OR Foo3:3 OR …
Foo999:3) AND (Bar1:1 OR Bar2:1 OR Bar3:1...Bar999:1) AND (Etc1:7 OR Etc2:7
OR Etc3:7...Etc:999:7)

obviously a nasty query (and care would be needed for MAX_BOOLEAN_CLAUSES). 
that said, are there other mechanisms to better handle that type of query,
i.e.:

http://localhost:8994/solr/select?q=(myfunction(‘Foo’, 3) AND
myfunction('Bar', 1) AND (myfunction('Etc', 7))


gdeconto wrote:
> 
> say I have a dynamic field called Foo* (where * can be in the hundreds)
> and want to search Foo* for a value of 3 (for example)
> 
> I know I can do this via this:
> 
> http://localhost:8994/solr/select?q=(Foo1:3 OR Foo2:3 OR Foo3:3 OR …
> Foo999:3)
> 
> However, is there a better way?  i.e. is there some way to query by a
> function I create, possibly something like this:
> 
> http://localhost:8994/solr/select?q=myfunction(‘Foo’, 3)
> 
> where myfunction itself iterates thru all the instances of Foo*
> 
> any help appreciated
> 
> 

-- 
View this message in context: 
http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25283094.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how to scan dynamic field without specifying each field in query

2009-09-04 Thread gdeconto

I dont have that answer as I was asking a general question, not one for a
specific situation I am encountering).

what I am essentially asking for is: is there a short, simple and generic
method/technique to deal with large numbers of dynamic fields (rather than
having to specify each and every test on each and every dynamic field) in a
query

what originally prompted this question is I was looking at FunctionQueries
(http://wiki.apache.org/solr/FunctionQuery) and started to wonder if there
was some way to create my own functions to handle dynamic fields.


Aakash Dharmadhikari wrote:
> 
> what all other searches you would like to perform on these fields?
> 
> ...
> 

-- 
View this message in context: 
http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25297439.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how to scan dynamic field without specifying each field in query

2009-09-04 Thread gdeconto

Thanks Avlesh.

I was only thinking of something 'like' function queries (since they
appeared to have similar behavior).

Agree that custom QueryParser is looking like my only choice.  Now have to
figure out how to do that :-)



Avlesh Singh wrote:
> 
> I don't think you need function queries here. Function queries are
> supposed
> to return score for a document based on their ValueSource. What you
> probably
> need is a custom QueryParser.
> 
> 

-- 
View this message in context: 
http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25300855.html
Sent from the Solr - User mailing list archive at Nabble.com.



how to create a custom queryparse to handle new functions

2009-09-04 Thread gdeconto

Can someone point me in the general direction of how to create a custom
queryparser that would allow me to create custom query commands like this:

http://localhost:8994/solr/select?q=myfunction(‘Foo’, 3)

or point me towards an example?

note that the actual functionality of myfunction is not defined.  I am just
wondering if this sort of extensibility is possible.
-- 
View this message in context: 
http://www.nabble.com/how-to-create-a-custom-queryparse-to-handle-new-functions-tp25301698p25301698.html
Sent from the Solr - User mailing list archive at Nabble.com.



about modifying querystring parameters submitted to solr

2009-10-05 Thread gdeconto

sorry if this is a very simple question, but I am stuck (and online searches
for this info havent been fruitful).

Lets say that, in certain circumstances, I want to change the field names
and/or field query values being passed to SOLR.

For example, lets say my unmodified query is
"http://localhost:8994/solr/select?q=xxx:[* TO 3] AND yyy:[3 TO
*]&defType=myQParser" and (JUST for the sake of argument) lets say I want to
rewrite it as "http://localhost:8994/solr/select?q=aaa:[1 TO 2] AND bbb:[3
TO 10]&defType=myQParser".

I think I can do it by extending QParserPlugin, and overriding the
createParser method (see my code snippet below). The qstr parameter
apparently contains the parts I want to examine and/or modify.

now to my questions:
1. is that the correct location to do this sort of manipulation?
2. is there an existing method for parsing out the fields and their
parameters? i.e. to break a qstr of "xxx:[* TO 3] AND yyy:[3 TO *]" into an
array something like  x[0][0] = "xxx", x[0][1]="* TO 3", x[1][0] = "yyy",
x[1][1]="3 TO *".  Or possibly even finer granularity than that.  I could
write it myself but its much nicer not to have to (especially since the
queries could be very complex).

thanks in advance for any help.

package com.topproducer.rentals.solr.search;

import org.apache.lucene.queryParser.ParseException;
import org.apache.lucene.search.Query;
import org.apache.solr.common.params.SolrParams;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.search.QParser;
import org.apache.solr.search.QParserPlugin;

public class myQParserPlugin extends QParserPlugin {

@Override
public QParser createParser(String qstr, SolrParams localParams,
SolrParams params, SolrQueryRequest req)
{
return new QParser(qstr, localParams, params, req) {
  QParser baseParser;

  public Query parse() throws ParseException {
StringBuilder queryBuilder = new StringBuilder();

// extract and/or view and/or change qstr content
here
// ..
// is there an existing function/method to parse
qstr into its component parts?
// i.e. to break "?q=xxx:[1 TO 3] AND yyy:[3 TO *]"
into something like:
// x[0][0] = "xxx", x[0][1]="1 TO 3"
// x[1][0] = "xxx", x[1][1]="3 TO *"

// after modifying qstr, store it into queryBuilder
here
queryBuild.append(new_qstr);


// prepare queryBuilder for any additional solr
handling
baseParser = subQuery(queryBuilder.toString(),
null);
Query q = baseParser.parse();
return q;
  }


  public String[] getDefaultHighlightFields() {
return baseParser.getDefaultHighlightFields();
  }

   
  public Query getHighlightQuery() throws ParseException {
return baseParser.getHighlightQuery();
  }

  public void addDebugInfo(NamedList debugInfo) {
baseParser.addDebugInfo(debugInfo);
  }
};
  }

@Override
public void init(NamedList arg0) {
   
// TODO Auto-generated method stub
   
}
} 
-- 
View this message in context: 
http://www.nabble.com/about-modifying-querystring-parameters-submitted-to-solr-tp25752898p25752898.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: about modifying querystring parameters submitted to solr

2009-10-05 Thread gdeconto

Note that there is a similar question in 
http://www.nabble.com/TermsComponent-to25302503.html#a25312549
http://www.nabble.com/TermsComponent-to25302503.html#a25312549 


-- 
View this message in context: 
http://www.nabble.com/about-modifying-querystring-parameters-submitted-to-solr-tp25752898p25754727.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: AW: AW: Concept Expansion

2009-10-05 Thread gdeconto

i had a similar question in my post 
http://www.nabble.com/forum/ViewPost.jtp?post=25752898&framed=y
http://www.nabble.com/forum/ViewPost.jtp?post=25752898&framed=y 

since queries can be quite complex, how would we parse the q string so that
we could identify and expand specific terms (ie is there an existing method)
in a custom QParserPlugin?



polx wrote:
> 
> 
> Le 05-sept.-09 à 23:26, Villemos, Gert a écrit :
> 
>> - As part of the construction the plugin parses the q string and  
>> extracts the parameters, ading them as TermQuery(s) to the parser
>  
> 

-- 
View this message in context: 
http://www.nabble.com/TermsComponent-tp25302503p25754730.html
Sent from the Solr - User mailing list archive at Nabble.com.



how can I use debugQuery if I have extended QParserPlugin?

2009-10-07 Thread gdeconto

in a previous post, I asked how I would go about creating virtual function in
my solr query; ie: http://127.0.0.1:8994/solr/select...@myfunc(1,2,3,4)

I was trying to find a way to more easily/cleanly perform queries against
large numbers of dynamic fields (ie field1, field2, field3...field99).

I have extended QParserPlugin so that I can do this. the extended method
replaces the virtual function section of the query with an expanded set of
fields; "@myFunc(1,2,3,4)" can become something like "(A1:1 AND B1:2 AND
C1:3 AND D1:4) OR (A2:1 AND B2:2 AND C2:3 AND D2:4) OR ... (A99:1 AND B99:2
AND C99:3 AND D99:4))"

one thing I noticed is that if I append debugQuery to a query that includes
the virtual function, I get a NullPointerException, likely because the
debugging code looks at the query passed in and not the expanded list that
my code generates.

I would like to be able to use debugQuery to analyse my queries, including
those with the virtual function.

What would I have to modify to get debugQuery to work??

thx in advance.
-- 
View this message in context: 
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tp25789546p25789546.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how can I use debugQuery if I have extended QParserPlugin?

2009-10-08 Thread gdeconto

I did check the other posts, as well as whatever I could find on the net but
didnt find anything.

Has anyone encountered this type of issue, or is what I am doing (extending
QParserPlugin) that unusual??



gdeconto wrote:
> 
> ...
> one thing I noticed is that if I append "debugQuery=true" to a query that
> includes the virtual function, I get a NullPointerException, likely
> because the debugging code looks at the query passed in and not the
> expanded query that my code generates and that gets used by solr for
> retrieving data.
> ...
> 

-- 
View this message in context: 
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tp25789546p25803277.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how can I use debugQuery if I have extended QParserPlugin?

2009-10-08 Thread gdeconto

Hi Yonik;

My original post (
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tt25789546.html
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tt25789546.html
) has the stack trace.  =^D

I am having trouble reproducing this issue consistently (I sometimes dont
get the NPE) so will have to track this down a bit more.  Luckily, someone
just showed me how to debug the core solr files with Eclipse.  Hopefully I
can now figure this out on my own.

thx


Yonik Seeley-2 wrote:
> 
> I think you need to provide some more information such as a stack
> trace for the NPE, or a more elaborate description of what you think
> the problem is with the debug component.
> You said "because the debugging code looks at the query passed in and
> not the expanded query", but I don't understand that.  The debug
> component is passed the actual Query object that the QParserPlugin
> created.
> 

-- 
View this message in context: 
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tp25789546p25812899.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: about modifying querystring parameters submitted to solr

2009-10-09 Thread gdeconto

Thanks for the reply hossman.

I have successfully created a custom QParserPlugin that implements some
special syntax to help me handle my many dynamic fields.

The syntax looks something like this:

  http://127.0.0.1:8994/solr/select...@fn:(1,2,3,4)

Had to create java code to extract the virtual function and its parameters. 
Will look at your comment further to see if it can help me any with parsing.



hossman wrote:
> 
> 
> : 2. is there an existing method for parsing out the fields and their
> : parameters? i.e. to break a qstr of "xxx:[* TO 3] AND yyy:[3 TO *]" into
> an
> : array something like  x[0][0] = "xxx", x[0][1]="* TO 3", x[1][0] =
> "yyy",
> : x[1][1]="3 TO *".  Or possibly even finer granularity than that.  I
> could
> : write it myself but its much nicer not to have to (especially since the
> : queries could be very complex).
> 
> if you wnat to reuse the existing Lucene syntax (and based on your example 
> here it seems like you do) then your best bet is probably to have your 
> QParser create a subclass of the Lucen QueryParser where you override each 
> of the get methods to inspect the "field" argument, change it as you see 
> fit, and then delegate to the superclass to build the actual query object.  
> that's the closest thing to what you describe.
> 
> 
> -Hoss
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/about-modifying-querystring-parameters-submitted-to-solr-tp25752898p25827695.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how can I use debugQuery if I have extended QParserPlugin?

2009-10-13 Thread gdeconto

my apologies, you are correct; I put the stack trace in an edit of the post
and not in the original post.

re version info: 

Solr Specification Version: 1.3.0.2009.07.08.08.05.45
Solr Implementation Version: nightly exported - yonik - 2009-07-08 08:05:45 

NOTE: I have some more info on this NPE problem.  I get the NPE error
whenever I use debugQuery and the query range has an asterix in it, even tho
the query itself should work.  For example:

These work ok:

http://127.0.0.1:8994/solr/select?q=myfield:[* TO 1]
http://127.0.0.1:8994/solr/select?q=myfield:[1 TO *]
http://127.0.0.1:8994/solr/select?q=myfield:[1 TO 1000]
http://127.0.0.1:8994/solr/select?q=myfield:[1 TO 1000]&debugQuery=true

These do not work ok:

http://127.0.0.1:8994/solr/select?q=myfield:[* TO 1]&debugQuery=true
http://127.0.0.1:8994/solr/select?q=myfield:[1 TO *]&debugQuery=true
http://127.0.0.1:8994/solr/select?q=myfield:*
http://127.0.0.1:8994/solr/select?q=myfield:*&debugQuery=true

Not sure if the * gets translated somewhere into a null value parameter (I
am just starting to look at the solr code) per your comment
-- 
View this message in context: 
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tp25789546p25878964.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how can I use debugQuery if I have extended QParserPlugin?

2009-10-19 Thread gdeconto

awesome.  Thanks for figuring this out guys



wojtekpia wrote:
> 
> Good catch. I was testing on a nightly build from mid-July. I just tested
> on a similar deployment with nightly code from Oct 5th and everything
> seems to work. 
> 
> My mid-July deployment breaks on sints, integers, sdouble, doubles, slongs
> and longs. My more recent deployment works with tints, sints, integers,
> tdoubles, sdoubles, doubles, tlongs, slongs, and longs. (I don't have any
> floats in my schema so I didn't test those). Sounds like another reason to
> upgrade to 1.4.
> 
> Wojtek
> 

-- 
View this message in context: 
http://www.nabble.com/how-can-I-use-debugQuery-if-I-have-extended-QParserPlugin--tp25789546p25959707.html
Sent from the Solr - User mailing list archive at Nabble.com.



having solr generate and execute other related queries automatically

2009-11-12 Thread gdeconto

Scenario:

1. I have a query I want to execute; I would be using the results and facets
returned
2. I also have a couple of dozen other queries that are closely related to
the first query and to the facets returned by that query.  For each query, I
would only be using the total number of results documents (aka numFound)
3. steps 1 and 2 would occur for each page load/view

while I could run these queries individually, I am concerned about the
overhead of having to run +20 solr queries.

I was wondering if it was a better idea to extend the search or facet
component somehow so that:

1. I send solr the first query
2. solr executes the query
3. solr then creates (programatically, based on the parameters I pass in the
first query as well as some of the facet results) and executes +20 other
queries
4. solr finally returns me the results and facets of the first query, as
well as the counts/numFounds obtained from the +20 other queries

I was thinking that this would save me the time and roundtrip overhead of
+20 solr queries per page, but I am unsure of how to proceed.

hopefully this question is not too vague.  any help/example appreciated.
-- 
View this message in context: 
http://old.nabble.com/having-solr-generate-and-execute-other-related-queries-automatically-tp26327032p26327032.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: having solr generate and execute other related queries automatically

2009-11-12 Thread gdeconto



tpunder wrote:
> 
> Could you use the facet.query feature
> (http://wiki.apache.org/solr/SimpleFacetParameters#facet.query_:_Arbitrary_Query_Faceting)
> to reduce it to 2 queries?
> 
> So you'd:
> 
> 1. Send solr the first query
> 2. Solr executes and returns the query to you
> 3. You then use the facet results to create a 2nd query with your +20
> facet.query queries
> 3. Solr executes and returns the results of your +20 facet.query queries
> 
> Then you'd only have 2 roundtrips.
> 
> -Tim
> 

Unfortunately no.  the +20 queries are distinct from each other, even tho
they share some of the original query parameters (and some facet information
from the original query facets).

what I was envisioning was something that works like a facet, but instead of
returning information about the first query, it would return information
about queries similar to the first query.
-- 
View this message in context: 
http://old.nabble.com/having-solr-generate-and-execute-other-related-queries-automatically-tp26327032p26328314.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: having solr generate and execute other related queries automatically

2009-11-13 Thread gdeconto


tpunder wrote:
> 
> Maybe I misunderstand what you are trying to do (or the facet.query
> feature).  If I did an initial query on my data-set that left me with the
> following questions:
> ...
> http://localhost:8983/solr/select/?q=*%3A*&start=0&rows=0&facet=on&facet.query=brand_id:1&facet.query=brand_id:2&facet.query=+%2Bbrand_id:5+%2Bcategory_id:4051
> ...
> 

Thanks for the reply Tim.

I can't provide you with an example as I dont have anything prototyped as
yet; I am still trying to work things thru in my head.  The +20 queries
would allow us to suggest other possibilities to users in a facet-like way
(but not returning the exact same info as facets).

With the technique you mention I would have to specify the list of query
params for each facet.query.  That would work for relatively simple queries. 
Unfortunately, the queries I was looking at doing would be fairly long (say
hundreds of AND/OR statements).   That said, I dont think solr would be able
to handle the query size I would end up with (at least not efficiently),
because the resulting query would consist of thousands of AND/OR statements
(isnt there a limit of sorts in Solr?)

I think that my best bet would be to extend the SearchComponent and perform
the additional query generation and execution in the extension.  That
approach should also allow me to have access to the facet values that the
base query would generate (which would allow me to generate and execute the
other queries).

thx again.
-- 
View this message in context: 
http://old.nabble.com/having-solr-generate-and-execute-other-related-queries-automatically-tp26327032p26343409.html
Sent from the Solr - User mailing list archive at Nabble.com.



question re parserPlugin.createParser parameters

2009-11-17 Thread gdeconto

I am looking at executing a single solr query and having solr automatically
execute one (or more) additional solr queries (inside solr) as a way to save
some overhead/time.  I am doing this by overriding the SearchComponent.  My
code works and I was looking at ways to optimize the code.

the original query has facets turned on, but I dont want the additional
queries to have facets or sorting turned on.  per the code snippets below,
would the additional queries be executed with facets on or with facets off?

note: I put null in the localparams and params fields of the createParser
(thinking that is where solr would look for the facet parameter, but was
wondering if the facet setting in the req param might somehow get used.

any help appreciated. thx.

public class MyFacetComponent extends SearchComponent implements
...
  @Override
  public void process(ResponseBuilder rb) throws IOException {
SolrParams p = rb.req.getParams();
SolrIndexSearcher searcher = rb.req.getSearcher();
MyFacetComponent.MyFacetProcessor mfp = new MyFacetProcessor(p, 
searcher);

// execute code to perform the additional queries
mfp.getMyFacets(rb, rb.getFieldFlags(), analyzer);
  }
...
  public static class MyFacetProcessor {
// TODO: why are these class members but other required variables
aren't?
final SolrIndexSearcher searcher;
final SolrParams params;

public MyFacetProcessor(SolrParams params, SolrIndexSearcher searcher) {
  this.searcher = searcher;
  this.params = params;
}

public void getMyFacets(ResponseBuilder rb, int flags, Analyzer
analyzer, boolean debugQuery)
throws IOException {

SolrQueryRequest req = rb.req;
...
for (int n = 0; n < myquerylist.length; n++) {
// code for prepping for new search goes here
...
MyQParserPlugin parserPlugin = new MyQParserPlugin();
QParser parser = 
parserPlugin.createParser(mynewsearchstr, null, null,
req);
DocList results = searcher.getDocList(pfQuery, 
emptyFilterList, null, 0,
0, flags);
...
}
}

-- 
View this message in context: 
http://old.nabble.com/question-re-parserPlugin.createParser-parameters-tp26399496p26399496.html
Sent from the Solr - User mailing list archive at Nabble.com.



how is score computed with hsin functionquery?

2009-11-24 Thread gdeconto

I was looking at functionqueries, and noticed that:

1. if I use the sum functionquery, the score in the results is the sum of
the values I want to sum (all well and good and expected):

http://127.0.0.1:8080/solr/select?q=(*:*)^0%20%20_val_:"sum(1,2,3,4,5)"&fl=score,Latitude,Longitude&sort=score%20asc

2. if I use the hsin functionquery (i.e.
hsin(45.67890,-123.456789,Latitude,Longitude,10)"http://127.0.0.1:8080/solr/select?q=(*:*)^0%20%20_val_:"hsin(45.67890,-123.456789,Latitude,Longitude,10)"&fl=score,Latitude,Longitude&sort=score%20asc

assuming this is not a quirk in 1.5, is there some way to convert the hsin
value to distance?

thx
-- 
View this message in context: 
http://old.nabble.com/how-is-score-computed-with-hsin-functionquery--tp26504265p26504265.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how is score computed with hsin functionquery?

2009-11-24 Thread gdeconto


gdeconto wrote:
> 
> ...
> is there some way to convert the hsin value to distance?
> ...
> 

I just noticed that the solr wiki states "Values must be in Radians" and all
my test values were in degrees.

-- 
View this message in context: 
http://old.nabble.com/how-is-score-computed-with-hsin-functionquery--tp26504265p26505091.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how is score computed with hsin functionquery?

2009-11-25 Thread gdeconto


Grant Ingersoll-6 wrote:
> 
> ...
> Yep.  Also note that I added deg() and rad() functions, but for the most
> part is probably better to do the conversion during indexing.
> ...
> 

Thanks Grant.  I hadnt seen the deg and rad functions.  Conversion would be
difficult since I typically work with degrees.  Once I get a bit more
experienced with the solr code, maybe I can contribute a degree version of
hsin  :-)
-- 
View this message in context: 
http://old.nabble.com/how-is-score-computed-with-hsin-functionquery--tp26504265p26515157.html
Sent from the Solr - User mailing list archive at Nabble.com.



RE: question about schemas

2009-12-02 Thread gdeconto

I dont believe there is any way to link values in one multivalue field to
values in other multivalue fields.

Re "where each doc contains the customer info and info for ALL products that
the  customer might have (likely done via dynamicfields)":

one thing you might want to consider is that this solution might lead to
performance issues if you need to do range queries such as q=((Width1:[50 TO
*] AND Density1:[7 to *]) OR (Width2:[50 TO *] AND Density2:[7 TO *]) OR …) 

I had a similar problem a while back, and basically had similar options.  In
my tests, this particular option became slower as I increased the number of
"products" (and so the number of unique values for each "product" field).

If you come up with a solution, let me know.

also, another option might be to encode the "product" information (ie using
a field delimiter, something like CSV) and then storing it into a multivalue
field for each customer.  I dont know how you would search that data tho
(maybe by having a unique delimiter for each field?)
-- 
View this message in context: 
http://old.nabble.com/question-about-schemas-tp26600956p26611997.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how is score computed with hsin functionquery?

2009-12-02 Thread gdeconto


Grant Ingersoll-6 wrote:
> 
> ...
> Yep.  Also note that I added deg() and rad() functions, but for the most
> part is probably better to do the conversion during indexing.
> ...
> 

as it is not possible for me to convert my data from deg to rad during
import (since queries are done using degrees), and manually putting in rad()
stmts into my hsin query seems awkward, I was looking at ways to have solr
do it for me.  

One idea I had was to leverage the existing hsin code (see below).  My
problem is that HavesineFunction expects ValueSource and I do not see a way
to convert my radian values back to ValueSource (pls excuse my ignorance on
this).  Any help/ideas appreciated

public class MyValueSourceParser extends ValueSourceParser {
public ValueSource parse(FunctionQParser fp) throws ParseException {
ValueSource source = fp.parseValueSource();

ValueSource x1 = fp.parseValueSource();
ValueSource y1 = fp.parseValueSource();
ValueSource x2 = fp.parseValueSource();
ValueSource y2 = fp.parseValueSource();
double radius = fp.parseDouble();

return HaversineFunction(Math.toRadians(x1), Math.toRadians(y1),
Math.toRadians(x2), Math.toRadians(y2), radius); // ** how do I convert the
rad param values to ValueSource **
}
}
-- 
View this message in context: 
http://old.nabble.com/how-is-score-computed-with-hsin-functionquery--tp26504265p26612289.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: question about schemas (and SOLR-1131?)

2009-12-02 Thread gdeconto

I saw an interesting thread in the solr-dev forum about multiple fields per
fieldtype (https://issues.apache.org/jira/browse/SOLR-1131)

from the sounds of it, it might be of interest and/or use in these types of
problems;  for your example, you might be able to define a fieldtype that
houses the product data.

note that I only skimmed the thread. hopefully, I'll get get some time to
look at it more closely
-- 
View this message in context: 
http://old.nabble.com/question-about-schemas-tp26600956p26619190.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how is score computed with hsin functionquery?

2009-12-03 Thread gdeconto


Lance Norskog-2 wrote:
> 
> If you use the DataImportHandler you can add your own Javascript code to
> do the degree->radian conversion.
> 

Thx Lance, but I am not sure what you mean
-- 
View this message in context: 
http://old.nabble.com/how-is-score-computed-with-hsin-functionquery--tp26504265p26634948.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: how is score computed with hsin functionquery?

2009-12-04 Thread gdeconto

Thanks Lance, I appreciate your response.  

I know what a DIH is and have already written custom transformers.  I just
misunderstood your response to my message (I wasnt aware that we could use
JS to create transformers).

Anyhow, my intent is to change the tool (create a variation of hsin to
support degrees) rather than change the data (which introduces other issues,
such as having to support most systems in degrees and this one system in
radians)

any ideas/advice in that regard?
-- 
View this message in context: 
http://old.nabble.com/how-is-score-computed-with-hsin-functionquery--tp26504265p26638720.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr vs Sphinx

2009-05-14 Thread gdeconto



Yonik Seeley-2 wrote:
> 
> It's probably the case that every search engine out there is faster
> than Solr at one thing or another, and that Solr is faster or better
> at some other things.
> 
> I prefer to spend my time improving Solr rather than engage in
> benchmarking wars... and Solr 1.4 will have a ton of speed
> improvements over Solr 1.3.
> 
> -Yonik
> http://www.lucidimagination.com
> 
> 

Solr is very fast even with 1.3 and the developers have done an incredible
job.

However, maybe the next Solr improvement should be the creation of a
configuration manager and/or automated tuning tool.  I know that optimizing
Solr performance can be time consuming and sometimes frustrating.


-- 
View this message in context: 
http://www.nabble.com/Solr-vs-Sphinx-tp23524676p23544492.html
Sent from the Solr - User mailing list archive at Nabble.com.



need help with feb 3/2010 trunk and solr-236

2010-02-03 Thread gdeconto

I got latest trunk (feb3/2010) using svn and applied solr-236.

did an "ant clean" and it seems to build fine with no errors or warnings.

however, when I start solr I get an error (here is a snippet):

SEVERE: org.apache.solr.common.SolrException: Error loading class
'org.apache.so
lr.handler.component.CollapseComponent'
at
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.
java:373)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:422)
at
org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:444)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1499)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1493)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1526)
at
org.apache.solr.core.SolrCore.loadSearchComponents(SolrCore.java:824)
...

I can see CollapseComponent.class in apache-solr-1.5-dev.war inside
apache-solr-core-1.5-dev.jar\org\apache\solr\handler\component, so it seems
to be finding and building the java file ok.

any ideas?
-- 
View this message in context: 
http://old.nabble.com/need-help-with-feb-3-2010-trunk-and-solr-236-tp27446001p27446001.html
Sent from the Solr - User mailing list archive at Nabble.com.



question/suggestion for Solr-236 patch

2010-02-10 Thread gdeconto

I have been able to apply and use the solr-236 patch (field collapsing)
successfully.

Very, very cool and powerful.

My one comment/concern is that the collapseCount and aggregate function
values in the collapse_counts list only represent the collapsed documents
(ie the ones that are not shown in results).

Are there any plans to include the non-collapsed (?) document in the
collapseCount and aggregate function values (ie so that it includes ALL
documents, not just the collapsed ones)?  Possibly via some parameter like
collapse.includeAll?

I think this would be a great addition to the collapse code (and solr
functionality) via what I would think is a small change.
-- 
View this message in context: 
http://old.nabble.com/question-suggestion-for-Solr-236-patch-tp27533695p27533695.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: question/suggestion for Solr-236 patch

2010-02-10 Thread gdeconto


Joe Calderon-2 wrote:
> 
> you can do that very easily yourself in a post processing step after
> you receive the solr response
> 

true (and am already doing so).

was thinking that having this done as part of the field collapsing code, it
might be faster than doing so via post processing (ie no need to navigate
the xml results for two different values for each collapsed set, adding the
numbers to get the total, etc)

it was just a suggestion.  field collapsing is a great feature.
-- 
View this message in context: 
http://old.nabble.com/question-suggestion-for-Solr-236-patch-tp27533695p27535153.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Dynamic fields with more than 100 fields inside

2010-02-11 Thread gdeconto


Xavier Schepler wrote:
> 
> for example, "concept_user_*", and I will have maybe more than 200 users 
> using this feature.
> 

I've done tests with many hundred dynamically created fields (ie foo_1 thru
f_400).  generally speaking, I havent noticed any noticeable performance
issues from having that many fields

the only exception: I have noticed performance issues if you try to query 
large numbers of fields (ie q=(foo_1:123 AND foo_2:234 ... AND foo_400:912)
-- 
View this message in context: 
http://old.nabble.com/Dynamic-fields-with-more-than-100-fields-inside-tp27502271p27554402.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: getting unexpected statscomponent values

2010-02-23 Thread gdeconto


Erick Erickson wrote:
> 
> It's especially helpful if you can take a bit of time to pare away
> all the unnecessary stuff in your example files and/or comment
> what you think the important bits are.
> 

entered as SOLR-1782 in jira
-- 
View this message in context: 
http://old.nabble.com/getting-unexpected-statscomponent-values-tp27599248p27710509.html
Sent from the Solr - User mailing list archive at Nabble.com.