Getting dynamicFields using QueryResponse

2009-04-24 Thread Avlesh Singh
Is their a way to dynamicFields obtained in my response using the QueryResponse.getBeans() method? Cheers Avlesh

Re: PageRank sort

2009-04-24 Thread Marcus Herou
Meant this part: m...@mahe-laptop:~$ GET ' http://127.0.0.1:8110/solr/test/select?indent=on&start=0&rows=100&q={!boostb=blogRank v=$qq}&qq=title:solr&debugQuery=on' 0 121 title:solr 0 on {!boost b=blogRank v=$qq} on 100 3 4 1 1 1 2 4 5 2

Re: PageRank sort

2009-04-24 Thread Marcus Herou
Cool! GET ' http://127.0.0.1:8110/solr/test/select?indent=on&start=0&rows=100&q={!boostb=blogRank v=$qq}&qq=title:solr&debugQuery=on' On Sat, Apr 25, 2009 at 12:43 AM, Marcus Herou wrote: > That seems wise... PageRank * Text-based Scoring. > > So you mean in my stupid case that: > GET ' > http:/

Re: PageRank sort

2009-04-24 Thread Marcus Herou
That seems wise... PageRank * Text-based Scoring. So you mean in my stupid case that: GET ' http://127.0.0.1:8110/solr/test/select?indent=on&start=0&rows=100&q={!boostb=blogRank v=$qq}&qq=*:*' would yield the same results as: GET "http://127.0.0.1:8110/solr/test/select?indent=on&start=0&rows=100&

Re: PageRank sort

2009-04-24 Thread Yonik Seeley
You probably want to mix the custom score with the normal relevancy score... to add, use a normal boolean query. To multiply, check out boosted query: http://lucene.apache.org/solr/api/org/apache/solr/search/BoostQParserPlugin.html For other options, use a more complex function query with the new

Re: PageRank sort

2009-04-24 Thread Marcus Herou
That is fantastic, I am creating a really small index right now trying to figure out howto implement the FunctionQuery for this. //Marcus On Fri, Apr 24, 2009 at 10:55 PM, Yonik Seeley wrote: > On Fri, Apr 24, 2009 at 1:39 PM, Marcus Herou > wrote: > > Great! That seems like something that coul

Re: PageRank sort

2009-04-24 Thread Marcus Herou
Works like a charm! Thank you sir. //Marcus On Fri, Apr 24, 2009 at 11:01 PM, Marcus Herou wrote: > That is fantastic, I am creating a really small index right now trying to > figure out howto implement the FunctionQuery for this. > > //Marcus > > > On Fri, Apr 24, 2009 at 10:55 PM, Yonik Seele

Re: Change boost of documents / single fields / external scoring ?

2009-04-24 Thread Marcus Herou
Yep it is / was, solved now. Thanks Grant and Yonik. Published the setup here: http://dev.tailsweep.com/solr-external-scoring/en/ //Marcus On Fri, Apr 24, 2009 at 6:34 PM, Grant Ingersoll wrote: > See also my comment on your PageRank question, which I take is the same > issue. Namely, you migh

Re: PageRank sort

2009-04-24 Thread Marcus Herou
And I published the setup here: http://dev.tailsweep.com/solr-external-scoring/en/ /M On Sat, Apr 25, 2009 at 12:01 AM, Marcus Herou wrote: > Works like a charm! > > Thank you sir. > > //Marcus > > > On Fri, Apr 24, 2009 at 11:01 PM, Marcus Herou > wrote: > >> That is fantastic, I am creating a

RE: fl parameter

2009-04-24 Thread Harsch, Timothy J. (ARC-SC)[PEROT SYSTEMS]
Anyone able to help with the question below? Yonik, I couldn't find the issues you speak of can you point me in the right direction? Thanks Tim -Original Message- From: Harsch, Timothy J. (ARC-SC)[PEROT SYSTEMS] [mailto:timothy.j.har...@nasa.gov] Sent: Wednesday, April 22, 2009 1:36 P

Re: PageRank sort

2009-04-24 Thread Yonik Seeley
On Fri, Apr 24, 2009 at 1:39 PM, Marcus Herou wrote: > Great! That seems like something that could work. Depends on how that field > get's re-read/indexed I guess. http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html It's a separate *text* file that just contains id/va

Re: Copy field : String Vs Text

2009-04-24 Thread Senthil Kumar
Hi Timothy, In copyfield I used the name that was not part of the doc and matching to the dynamic text fieldType declaration. It worked. I searched on Q=gudactor:"xxx". It is able to search on actor with full text... thanks, Senthil On Fri, Apr 24, 2009 at 4:08 PM, Harsch, Timothy J. (

RE: Copy field : String Vs Text

2009-04-24 Thread Harsch, Timothy J. (ARC-SC)[PEROT SYSTEMS]
I'd be curious to hear someone explain the mechanics of a String field. I'm not entirely sure but I believe a String field is not tokenized and thus only exact matches work. I think in your case you should be searching on your "name" field and faceting on the "actor" field. -Original Mess

Copy field : String Vs Text

2009-04-24 Thread Senthil Kumar
Hi, I have a requirement as below. The field needs to be available for facet. The value will not be a single letter word. But I should be able to search like a text fields. Silver stolen Schema.xml I should be able to search for the word "silver" on actor field and should get the

Re: using boolean operators with different fields for querying

2009-04-24 Thread Rakesh Sinha
On Fri, Apr 24, 2009 at 11:51 AM, Shalin Shekhar Mangar wrote: > On Fri, Apr 24, 2009 at 9:08 PM, Rakesh Sinha wrote: > >>  How do I specify boolean operators on a given field to search with >> the 'q' parameter. >> >> For a given index - I have a different documents , document 1 with a >> field c

Re: PageRank sort

2009-04-24 Thread Marcus Herou
Hi. Comments inline. On Fri, Apr 24, 2009 at 1:00 PM, Grant Ingersoll wrote: > How often are you updating the rank? The goal is to optimize the pagerank calculating algorithm so can have continuous updates (1 blogs at a time 24/7) but more likely we'll end up refreshing the index once a we

Re: Delete from Solr index...

2009-04-24 Thread Shalin Shekhar Mangar
Try solrServer.deleteByQuery("id:2354345"); This will delete all documents which have id=2354345. OR solrServer.deleteById("2354345"); This works if id is uniqueKey in schema.xml id On Fri, Apr 24, 2009 at 10:43 PM, lupiss wrote: > > You can use solrServer.deleteByQuery("*:*") and then call

Re: Delete from Solr index...

2009-04-24 Thread lupiss
You can use solrServer.deleteByQuery("*:*") and then call commit by solrServer.commit(true, true); This will erase the index. -- Regards, Shalin Shekhar Mangar. hola gracias por contestar! ese comando ya lo había visto, pero borra todo los índices verdad?, yo quisiera borrar solo un índice de

Re: Custom score for a id field

2009-04-24 Thread Shalin Shekhar Mangar
On Fri, Apr 24, 2009 at 7:38 PM, Raju444us wrote: > > Thanks,Yonik for quick reply. But How can I override the relevancy score > with > my fieldScore for that document when I get back the results. If possible an > example code is helpfull. > > Note: Iam storing the fieldScore. so that I can retri

spellcheck.collate causes StringIndexOutOfBoundsException during startup.

2009-04-24 Thread Eric Sabourin
I’m using SOLR 1.3.0 (from download, not a nightly build) apache-tomcat-5.5.27 on Windows XP. When I add true to my requestHandler in my solrconfig.xml, I get the StringIndexOutOfBoundsException stacktrace below on startup. Removing the element, or setting it to false, causes the exception to

Solr index

2009-04-24 Thread aidahaj
Hi , I'm using Nutch to crawl a list of web sites. Solr is my index(Nutch-1.0 integration with solr). I'm working on detecting web site defacement(if there's any changes in the text of a web page). I want to know if solr may give me the possibility to detect the changes in the Documents in the ind

Re: Change boost of documents / single fields / external scoring ?

2009-04-24 Thread Grant Ingersoll
See also my comment on your PageRank question, which I take is the same issue. Namely, you might have a look into the external file stuff. I've used that to store oft-changing boost information that then factors into the score. -Grant On Apr 24, 2009, at 12:03 PM, Michael McCandless wrot

Re: Solr index

2009-04-24 Thread Shalin Shekhar Mangar
Solr 1.4 (trunk) has a similar functionality. http://wiki.apache.org/solr/Deduplication On Fri, Apr 24, 2009 at 9:53 PM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > Hi, > > Solr doesn't include such functionality. But Nutch has: > > [o...@localhost src]$ ff \*Signature\*java > ./t

Re: multifield queries dismax or...

2009-04-24 Thread Otis Gospodnetic
Note that not only is pulling 10K documents out of the index (or even documentCache) slowing you down, but so is writing out and reading the response. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Alessandro F > To: solr-user@lucene.ap

Re: Solr index

2009-04-24 Thread Otis Gospodnetic
Hi, Solr doesn't include such functionality. But Nutch has: [o...@localhost src]$ ff \*Signature\*java ./test/org/apache/nutch/crawl/TestSignatureFactory.java ./java/org/apache/nutch/crawl/SignatureFactory.java ./java/org/apache/nutch/crawl/MD5Signature.java ./java/org/apache/nutch/crawl/Signat

Re: Solr Performance bottleneck

2009-04-24 Thread Jon Bodner
Grant Ingersoll-6 wrote: > > Hi Jon, > > Can you share the stack traces for the exceptions? Also, I don't know > what it is about it, but an index with non-stored items of only 925K > items being about 2.3GB seems weird to me for some reason. How many > unique terms do you have? > Ind

Re: using boolean operators with different fields for querying

2009-04-24 Thread Shalin Shekhar Mangar
On Fri, Apr 24, 2009 at 9:08 PM, Rakesh Sinha wrote: > How do I specify boolean operators on a given field to search with > the 'q' parameter. > > For a given index - I have a different documents , document 1 with a > field course:"Applied Statistics" > document 2 with a field course: "Applied P

Re: Change boost of documents / single fields / external scoring ?

2009-04-24 Thread Michael McCandless
I think something like this (NOTE: not at all tested, and I have no real experience with function queries): ValueSource vals = new MyPageRankScores(...); ValueSourceQuery prQuery = new ValueSourceQuery(vals); Query realQuery = get-user's-query; Query q = new CustomScoreQuery(realQuery, prQ

using boolean operators with different fields for querying

2009-04-24 Thread Rakesh Sinha
How do I specify boolean operators on a given field to search with the 'q' parameter. For a given index - I have a different documents , document 1 with a field course:"Applied Statistics" document 2 with a field course: "Applied Probability" document 3 with a field course:"Discrete math" I ne

Re: multifield queries dismax or...

2009-04-24 Thread Alessandro F
Hi Erik, thank you for the fast answere! I didn't take care abut the number of the output records. I know 1 recs are a lot, but i need these. Actually if I reduce the number of the records the query is faster. so the problem seams releted to loading the content from the index, at that poit I

Re: How to indexing non-english html text in unicode with Solr?

2009-04-24 Thread Shalin Shekhar Mangar
On Fri, Apr 24, 2009 at 6:11 PM, ahmed baseet wrote: > > What is the maximum field length supported by Solr [Is it 1 by default? > I think so]. Will this make sense during searching? > Yes, it is 1 by default. You can change it in solrconfig.xml with the parameter. -- Regards, Shalin S

Re: replicated index files have incorrect timestamp

2009-04-24 Thread Shalin Shekhar Mangar
2009/4/24 Noble Paul നോബിള്‍ नोब्ळ् > looks like a bug. > https://issues.apache.org/jira/browse/SOLR-1126 > > This should be fixed in trunk and the latest nightly build. -- Regards, Shalin Shekhar Mangar.

Re: Query | Solr conf and data (index) distribution using master slave configuration

2009-04-24 Thread Shalin Shekhar Mangar
On Fri, Apr 24, 2009 at 4:31 PM, Vicky_Dev wrote: > > Thanks Shalin for your reply > > Is it possible to replicate conf folder from master to slave? > > It should be possible but I've never tried to replicate config with the script based replication myself. The script based replication calls comm

Re: Get date facet counts per month

2009-04-24 Thread Raju444us
Thanks, Shalin.That worked. Shalin Shekhar Mangar wrote: > > On Thu, Apr 23, 2009 at 2:36 AM, Raju444us > wrote: > >> >> In the example on the wiki at gives the facet counts for date per day.How >> should the query look like to get date facets by month. >> >> >> >> http://wiki.apache.org/solr/

Re: Custom score for a id field

2009-04-24 Thread Raju444us
Thanks,Yonik for quick reply. But How can I override the relevancy score with my fieldScore for that document when I get back the results. If possible an example code is helpfull. Note: Iam storing the fieldScore. so that I can retrieve the score. Thanks, Raju Yonik Seeley-2 wrote: > > On Th

Re: Custom score for a id field

2009-04-24 Thread Yonik Seeley
On Thu, Apr 23, 2009 at 1:19 PM, Raju444us wrote: > I have a requirement.I index a field "id" and a calculated score for that > field named "fieldScore". > > Note:I have many other fields which are also indexed.But only for this id > field i want a custom calculated score. > > So when I search for

Re: Custom score for a id field

2009-04-24 Thread Raju444us
Before I index this field I classify this field with some other mechanism and build some score.So When I index this id field i also index score(which is float) for this field.So when i search for this id field in the results i need the score i indexed for this field instead of default score for th

Re: Is solr right for this scenario?

2009-04-24 Thread Eric Pugh
On Apr 24, 2009, at 7:54 AM, Developer In London wrote: Thanks for the fast reply. Wow this seems a very active community. I have a few more questions in that case: 1) If Solr is going to be file-based, os it then preferable to run multiple Solrs with Shards? How can I determine what capac

Re: How to indexing non-english html text in unicode with Solr?

2009-04-24 Thread ahmed baseet
Grant, thanks for your quick response. In the mean time I did a bit googling and found that there are java swing html parsers that can extract the plain text from the html page. I tried running sample examples with non-english pages and found that its working fine. Then I thought of putting this w

Re: Is solr right for this scenario?

2009-04-24 Thread Developer In London
Thanks for the fast reply. Wow this seems a very active community. I have a few more questions in that case: 1) If Solr is going to be file-based, os it then preferable to run multiple Solrs with Shards? How can I determine what capacity 1 Solr can cope? 2) I am presuming there is already tokeni

Re: multifield queries dismax or...

2009-04-24 Thread Erik Hatcher
Returning 1 rows/documents - maybe that is the speed issue you're encountering? Try adding &debugQuery=true to your request and look at the timings in the response. What is the QueryComponent time? Try returning fewer rows and see how the speed changes. Erik On Apr 24, 2009

multifield queries dismax or...

2009-04-24 Thread Alessandro F
Hi all, I'm working to replace my old lucene java-based search functionalities with Solr. I implemented multi-field query search using the "dismax", but the query seams to take to much time (if I compare with my old lucene queries). There is another way to perform multi-field query in solr?

Re: How to indexing non-english html text in unicode with Solr?

2009-04-24 Thread Grant Ingersoll
See the Solr Cell contrib: http://wiki.apache.org/solr/ExtractingRequestHandler . Note, it's 1.4-dev only. If you want it for 1.3, you'll have to use Tika on the client side. Solr does support Unicode indexing. On Apr 24, 2009, at 2:22 AM, ahmed baseet wrote: Hi All, I'm trying to index s

Re: Query | Solr conf and data (index) distribution using master slave configuration

2009-04-24 Thread Vicky_Dev
Thanks Shalin for your reply Is it possible to replicate conf folder from master to slave? ~Vikrant Shalin Shekhar Mangar wrote: > > On Thu, Apr 23, 2009 at 10:10 AM, Vicky_Dev > wrote: > >> >> 1. Please confirm whether the tag entry : >> In solrconfig.xml should match for the Slave sol

Re: PageRank sort

2009-04-24 Thread Grant Ingersoll
How often are you updating the rank? You might also be able to keep the rank info in a flat file via the ExternalFileField and the FileFloatSource and do FunctionQuery stuff that way. However, I don't know how that handles refreshing data or if it would be efficient in your case. On Apr

Re: Is solr right for this scenario?

2009-04-24 Thread Eric Pugh
It seems like you have three components to your system: 1) Data indexing from multiple sources 2) Search for specific words in documents 3) Preserve rating and search term. I think that Solr comes into play on #1 and #2. You can index content in any number of approaches, either via the new

Re: Solr Performance bottleneck

2009-04-24 Thread Grant Ingersoll
Hi Jon, Can you share the stack traces for the exceptions? Also, I don't know what it is about it, but an index with non-stored items of only 925K items being about 2.3GB seems weird to me for some reason. How many unique terms do you have? Also, in Lucene there is a standalone program

Is solr right for this scenario?

2009-04-24 Thread Developer In London
Hi All, I am new to the whole Solr/Lucene community. But I think this might be the solution ot what I am looking to do. I would appreciate any feedback on how I can go about doing this with Solr: I am looking to make a system where - a) mainly lots of different blog sites, web journals, articles

Indexing tempary data on lucene

2009-04-24 Thread Atlantis69
I have a list of public profiles of my site user's on solr index. There is also a community around them, which is currently not their in Index. While searching, I have to give an option to search only my community (friends and friends of friends). I could do it from data base query or storing con

Re: Change boost of documents / single fields / external scoring ?

2009-04-24 Thread Marcus Herou
Yes I am thinking of something like that. Could you elaborate on how that would look like pseudo wise ? Kindly //Marcus On Fri, Apr 24, 2009 at 9:05 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > Could function queries be used here? EG you could implement a > ValueSource that pu

Re: SOLR-769 clustering

2009-04-24 Thread Stanislaw Osinski
> > How would we enable people via SOLR-769 to do this? Good point, Grant! To apply the modified stopwords.* and stoplabels.* files to Solr, simply make them available in the classpath. For the example Solr runner scripts that would be something like: java -cp -Dsolr.solr.home=./clustering/solr

Re: autowarmcount how to check if cache has been warmed up

2009-04-24 Thread sunnyfr
Ok I've an error : Apr 24 09:28:41 search-01 jsvc.exec[8033]: :907) ^Iat java.lang.Thread.run(Thread.java:619) Apr 24 09:29:20 search-01 jsvc.exec[8033]: Apr 24, 2009 9:29:20 AM org.apache.solr.common.SolrException log SEVERE: Error during auto-warming of key:org.apache.solr.search.queryresult.

Re: MLT for sorting results?

2009-04-24 Thread Shrutipriya
righto. feel like a bummer for asking that in retrospect. thanks for the clarification :-) On Thu, Apr 23, 2009 at 5:30 PM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > That is true *only if* you combine those 2 clauses with AND. It's not true > with OR. > > > Otis -- > Sematext --

Re: Change boost of documents / single fields / external scoring ?

2009-04-24 Thread Michael McCandless
Could function queries be used here? EG you could implement a ValueSource that pulls in the external scores? Mike On Thu, Apr 23, 2009 at 4:01 PM, Marcus Herou wrote: > Hi. > > Confusing subject eh ? Trying to become a little clearer in a few sentences. > > We have a Solr/Lucene index where eac