SpellCheckComponent questions

2010-06-16 Thread Blargy
Is it generally wiser to build the dictionary from the existing index? Search Log? Other? For "Did you mean" does one usually just use collate=true and then return that string? Should I be using a separate spellchecker handler to should I just always include spellcheck=true in my original searc

Re: SpellCheckComponent questions

2010-06-16 Thread Blargy
Follow up question. How can I influence the "scoring" of results that comeback either through term frequency (if i build of an index) or through # of search results returned (if using a search log)? Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/SpellCheckComponent-

Re: how to apply patch SOLR-1316

2010-06-16 Thread Blargy
Im trying to apply this via the command line "patch -p0 < SOLR-1316.patch". When patching against trunk I get the following errors. ~/workspace $ patch -p0 < SOLR-1316.patch patching file dev/trunk/solr/src/java/org/apache/solr/handler/component/SpellCheckComponent.java Hunk #2 succeeded at 575

Autsuggest/autocomplete/spellcheck phrases

2010-06-17 Thread Blargy
How can I preserve phrases for either autosuggest/autocomplete/spellcheck? For example we have a bunch of product listings and I want if someone types: "louis" for it to common up with "Louis Vuitton". "World" ... "World cup". Would I need n-grams? Shingling? Thanks -- View this message in con

Re: Autsuggest/autocomplete/spellcheck phrases

2010-06-17 Thread Blargy
Thanks for the reply Michael. Ill definitely try that out and let you know how it goes. Your solution sounds similar to the one I've read here: http://www.lucidimagination.com/blog/2009/09/08/auto-suggest-from-popular-queries-using-edgengrams/ There are some good comments in there too. I think

Re: Autsuggest/autocomplete/spellcheck phrases

2010-06-17 Thread Blargy
Ok that makes perfect sense. "What I did was use a combination of the two running the indexed terms through " - I initially read this as you used your current index and use the terms from that to buildup your dictionary. -- View this message in context: http://lucene.472066.n3.nabble.com/Au

DismaxRequestHandler

2010-06-17 Thread Blargy
I have a title field and a description filed. I am searching across both fields but I don't want description matches unless they are within some slop of each other. How can I query for this? It seems that im getting back crazy results when there are matches that are nowhere each other -- View th

defType=Dismax questions

2010-06-17 Thread Blargy
Sorry for the repost but I posted under DismaxRequestHandler when I should have listed it as DismaxQueryParser.. ie im using defType=dismax I have a title field and a description filed. I am searching across both fields but I don't want description matches unless they are within some slop of each

Peformance tuning

2010-06-17 Thread Blargy
After indexing our item descriptions our index grew from around 3gigs to now 17.5 and I can see our search has deteriorated from sub 50ms searches to over 500ms now. The sick thing is I'm not even searching across that field at the moment but I plan to in the near future as well as include highlig

Re: Peformance tuning

2010-06-17 Thread Blargy
Is there an alternative for highlighting on a large stored field? I thought for highlighting you needed the field stored? I really just need the excerpting feature for highlighting relevant portions of our item descriptions. Not sure if this is because of the index size (17.5G) or because of high

Re: Peformance tuning

2010-06-17 Thread Blargy
Blargy - Please try to quote the mail you're responding to, at least > the relevant piece. It's nice to see some context to the discussion. No problem ;) Depends - if you optimize the index on the master, then the entire index is replicated. If you simply commit and let Luc

Re: Peformance tuning

2010-06-17 Thread Blargy
> first step is to do an &debugQuery=true and see where the time is > going on the server-side. If you're doing highlighting of a stored > field, that can be a biggie. The timings will be in the debug output > - be sure to look at both sections of the timings. > Looks like the majori

Re: Peformance tuning

2010-06-18 Thread Blargy
Otis Gospodnetic-2 wrote: > > Smaller merge factor will make things worse - > - Whoops... Ill guess Ill change it from 5 to the default 10 -- View this message in context: http://lucene.472066.n3.nabble.com/Peformance-tuning-tp904540p905726.html Sent from the Solr - User mailing list archiv

Re: Peformance tuning

2010-06-18 Thread Blargy
Otis Gospodnetic-2 wrote: > > You may want to try the RPM tool, it will show you what inside of that > QueryComponent is really slow. > We are already using it :) Where should I be concentrating on? Transaction trace? -- View this message in context: http://lucene.472066.n3.nabble.com/Pefo

jdbc4.CommunicationsException

2010-06-20 Thread Blargy
Does anyone know a solution to this problem? I've already tried autoReconnect=true and it doesn't appear to help. This happened 34 hours into my full-import... ouch! org.apache.solr.handler.dataimport.DataImportHandlerException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last p

IDH - "Total Documents Processed" is missing

2010-06-20 Thread Blargy
It seems that when importing via DIH the "Total Documents Processed" status message does not appear when there are two entities for a given document. Is this by design? -- View this message in context: http://lucene.472066.n3.nabble.com/IDH-Total-Documents-Processed-is-missing-tp909325

LocalParams?

2010-06-21 Thread Blargy
Huh? Read through the wiki: See http://wiki.apache.org/solr/LocalParams but I still don't understand its utility? Can someone explain to me why this would even be used? Any examples to help clarify? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/LocalParams-tp91318

Re: anyone use hadoop+solr?

2010-06-22 Thread Blargy
Need, Seems like we are in the same boat. Our index consist of 5M records which roughly equals around 30 gigs. All in all thats not too bad however our indexing process (we use DIH but I'm now revisiting that idea) takes a whopping 30+ hours!!! I just bought the Hadoop In Action early edition b

Re: anyone use hadoop+solr?

2010-06-22 Thread Blargy
Muneeb Ali wrote: > > Hi Blargy, > > Nice to hear that I am not alone ;) > > Well we have been using Hadoop for other data-intensive services, those > that can be done in parallel. We have multiple nodes, which are used by > Hadoop for all our MapReduce jobs. I pe

Similarity

2010-06-24 Thread Blargy
Can someone explain how I can override the default behavior of the tf contributing a higher score for documents with repeated words? For example: Query: "foo" Doc1: "foo bar" score 1.0 Doc2: "foo foo bar" score 1.1 Doc2 contains "foo" twice so it is scored higher. How can I override this behavi

Re: Similarity

2010-06-24 Thread Blargy
Yonik Seeley-2-2 wrote: > > Depends on the larger context of what you are trying to do. > Do you still want the idf and length norm relevancy factors? If not, > use a filter, or boost the particular clause with 0. > I do want the other relevancy factors.. ie boost, phrase-boosting etc but I j

SweetSpotSimilarity

2010-06-25 Thread Blargy
Would someone mind explaining how this differs from the DefaultSimilarity? Also how would one replace the use of the DefaultSimilarity class with this one? I can't seem to find any such configuration in solrconfig.xml. Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/S

Re: SweetSpotSimilarity

2010-06-25 Thread Blargy
iorixxx wrote: > > it is in schema.xml: > > > Thanks. Im guessing this is all or nothing.. ie you can't you one similarity class for one request handler and another for a separate request handler. Is that correct? -- View this message in context: http://lucene.472066.n3.nabble.com/Sweet

Re: SweetSpotSimilarity

2010-06-28 Thread Blargy
iorixxx wrote: > > it is in schema.xml: > > > How would you configure the tfBaselineTfFactors and LengthNormFactors when configuring via schema.xml? Do I have to create a subclass that hardcodes these values? -- View this message in context: http://lucene.472066.n3.nabble.com/SweetSpotSimi

Re: SweetSpotSimilarity

2010-06-28 Thread Blargy
iorixxx wrote: > > CustomSimilarityFactory that extends > org.apache.solr.schema.SimilarityFactory should do it. There is an example > CustomSimilarityFactory.java under src/test/org... > This is exactly what I was looking for... this is very similar ( no put intended ;) ) to the updateProcess

Optimizing cache

2010-06-28 Thread Blargy
Here is a screen shot for our cache from New Relic. http://s4.postimage.org/mmuji-31d55d69362066630eea17ad7782419c.png Query cache: 55-65% Filter cache: 100% Document cache: 63% Cache size is 512 for above 3 caches. How do I interpret this data? What are some optimal configuration changes give

Custom PhraseQuery

2010-06-29 Thread Blargy
Is there anyway to override/change up the default PhraseQuery class that is used... similar to how you can change out the Similarity class? Let me explain what I am trying to do. I would like to override the TF is calculated... always returning a max of 1 for phraseFreq. For example: Query: "fo

ValueSource/Function questions

2010-07-01 Thread Blargy
Can someone explain what the createWeight methods should do? And one someone mind explaining what the hashCode method is doing in this use case? public int hashCode() { int h = a.hashCode(); h ^= (h << 13) | (h >>> 20); h += b.hashCode(); h ^= (h << 23) | (h >>> 10); h += n

MLT with boost capability

2010-07-09 Thread Blargy
I've asked this question in the past without too much success. I figured I would try to revive it. Is there a way I can incorporate boost functions with a MoreLikeThis search? Can it be accomplished at the MLT request handler level or would I need to create a custom request handler which in turn

Re: Custom PhraseQuery

2010-07-09 Thread Blargy
Oh.. i didnt know about the different signatures to tf. Thanks for that clarification. It sounds like all I need to do is actually override tf(float) in the SweetSpotSimilarity class to delegate to baselineTF just like tf(int) does. Is this correct? Thanks -- View this message in context: http

Foreign characters question

2010-07-13 Thread Blargy
I am trying to add the following synonym while indexing/searching swimsuit, bañadores, bañador I testing searching for "bañadores" however it didn't return any results. After further inspection I noticed in the field analysis admin that swimsuit gets expanded to ba�adores. Not sure if it will sh

RE: Foreign characters question

2010-07-14 Thread Blargy
Thanks for the reply but that didnt help. Tomcat is accepting foreign characters but for some reason when it reads the synonyms file and it encounters that character ñ it doesnt appear correctly in the Field Analysis admin. It shows up as �. If I query exactly for ñ it will work but the synonyms

Re: Foreign characters question

2010-07-14 Thread Blargy
How can I tell and/or create a UTF-8 synonyms file? Do I have to instruct solr that this file is UTF-8? -- View this message in context: http://lucene.472066.n3.nabble.com/Foreign-characters-question-tp964078p967037.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Foreign characters question

2010-07-14 Thread Blargy
Nevermind. Apparently my IDE (Netbeans) was set to "No encoding"... wtf. Changed it to UTF-8 and recreated the file and all is good now. Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Foreign-characters-question-tp964078p967058.html Sent from the Solr - User mailing

Stemming

2010-07-20 Thread Blargy
I am using the LucidKStemmer and I noticed that it doesnt stem certain words... for example "bags". How could I create a list of explicit words to stem... ie sort of the opposite of protected words. I know this can be accomplished using the synonyms file but I want to know how to just replace one

Re: Stemming

2010-07-20 Thread Blargy
Perfect! Is there an associated JIRA ticket/patch for this so I can patch my 4.1 build? -- View this message in context: http://lucene.472066.n3.nabble.com/Stemming-tp982690p982786.html Sent from the Solr - User mailing list archive at Nabble.com.

Architectural help

2010-03-09 Thread blargy
I was wondering if someone could be so kind to give me some architectural guidance. A little about our setup. We are RoR shop that is currently using Ferret (no laughs please) as our search technology. Our indexing process at the moment is quite poor as well as our search results. After some deli

Re: Architectural help

2010-03-10 Thread blargy
; Worked when i tested it ;) > > On Wed, Mar 10, 2010 at 1:56 AM, blargy wrote: > >> >> I was wondering if someone could be so kind to give me some architectural >> guidance. >> >> A little about our setup. We are RoR shop that is currently using Ferret >>

DIH field options

2010-03-11 Thread blargy
How can you simply add a static value like?  How does one add a static multi-value field? Is there any documentation on all the options for the field tag in data-config.xml? Thanks for the help -- View this message in context: http://old.nabble.com/DIH-field-options-tp27873996p27873996.html

Re: DIH field options

2010-03-12 Thread blargy
define it with > multiValued="true" > > > On 3/11/10 10:58 PM, blargy wrote: >> How can you simply add a static value like?> value="123"/> >> How does one add a static multi-value field?> values="123, 456"/> >> >>

Re: DIH field options

2010-03-12 Thread blargy
I feel like the default option is a little hacky plus I'll probably be sharing my schema.xml for multiple cores using dynamic field types. I can't believe there isnt an easy way to specify this. So my only options is something like this? What if I don't need a templa

Re: DIH field options

2010-03-12 Thread blargy
Im still having a problem with this... for example I would assume this would index the value Item into the field called type However I receive this error when starting up Solr: Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Field must have a co

DIH template multivalued fields

2010-03-12 Thread blargy
How can I manually specify a static multiple value field in the DataImportHandler? I finally figured out the answer of how to statically define a value from this FAQ: http://wiki.apache.org/solr/DataImportHandlerFaq which basically states to use the TemplateTransformer. My question is what do I

Hardware Recommendations

2010-03-12 Thread blargy
Ill have about 5m documents indexed (ranging in size) with an expected amount of searches to be between 750k and 1m per day. Ill be using a master/slave setup with an unknown number of slaves. What hardware requirements would you recommend/suggest? Thoughts? -- View this message in context:

Re: DIH template multivalued fields

2010-03-12 Thread blargy
I was actually able to accomplish (althought not pretty) what I wanted using a regex transformer. blargy wrote: > > How can I manually specify a static multiple value field in the > DataImportHandler? > > I finally figured out the answer of how to statically defi

Solr Logging XML

2010-03-13 Thread blargy
How can I enable logging of all the xml posted to my Solr server? Is this possible? As of right now all I see in the logs are the request params when querying. While I am on the topic of logging I have one other question too. Is it possible to use custom variables in the logging.properties file s

DataImportHandler development console

2010-03-13 Thread blargy
Is there any documentation on this screen? (and dont point me http://wiki.apache.org/solr/DataImportHandler) When using the Full-import, Status, Reload-Config, Document-Count and Full Import With Cleaning everything works as expected but when I use any of the following I get an exception: Debug N

Re: DataImportHandler development console

2010-03-13 Thread blargy
Also how would one auto-commit after a delta-import? I click on the commit, clean and verbose checkboxes but those seem to have no affect. blargy wrote: > > Is there any documentation on this screen? (and dont point me > http://wiki.apache.org/solr/DataImportHandler) > > When

Managing configuration files/Environment variables

2010-03-13 Thread blargy
How are you guys solving the problem with managing all of your configuration difference between development and production. For example when deploying to production I need to change the data-config.xml (DataImportHandler) database settings. I also have some ant scripts to start/stop tomcat as wel

DIH datasource configuration

2010-03-14 Thread blargy
My current DIH is configured via the requestHandler block in solrconfig.xml data-config.xml ${datasource.driver} ${datasource.url} ${datasource.user} ${datasource.password} -1 true My question is, does the batchsize a

RegexTransformer

2010-03-14 Thread blargy
How would I go about splitting a column by a certain delimiter AND ignore all empty matches. For example: I have a some columns that dont have a value for values but so its getting actually index as blank. I just want to totally ignore those values. Is this possible? -- View this message in

Re: RegexTransformer

2010-03-15 Thread blargy
Thanks for the replies. Ill just roll out my own transformer for this. Shalin Shekhar Mangar wrote: > > On Mon, Mar 15, 2010 at 2:53 PM, Michael Kuhlmann < > michael.kuhlm...@zalando.de> wrote: > >> On 03/15/10 08:56, Shalin Shekhar Mangar wrote: >> > On Mon

Re: DIH datasource configuration

2010-03-15 Thread blargy
? blargy wrote: > > My current DIH is configured via the requestHandler block in > solrconfig.xml > > class="org.apache.solr.handler.dataimport.DataImportHandler"> > > data-config.xml > > ${datasource.driver} > ${

Re: DIH datasource configuration

2010-03-15 Thread blargy
Thanks but I was thinking more of a way to check the datasource's current internal configuration. IE, i wanted to check if I put batchsize -1 in the sorlconfig.xml that it was actually set. Anyway, back to my previous question. Is there a way to dynamically set the values for the datasource in da

Dyanmic variables/properties

2010-03-15 Thread blargy
Can someone point me in the right direction as where to find some documentation on how and where I can configure dynamic variables/properties to be used throughout the solr configuration files. Also what is the correct term for these dynamic variables? For example in solrconfig.xml there is this

Stemming suggestions

2010-03-16 Thread blargy
Most of our documents will be in English but not all and we are certain in the process of acquiring more international content. Does anyone have any experience using all of the different stemmers for languages of unknown origin? Which ones perform the best? Give the most relevant results? What are

Re: LucidWorks Solr

2010-03-16 Thread blargy
KStemmer, but I also liked the fact that it included > about a dozen or so stable patches since Solr 1.4 was released. We just > use the included WAR in our project however. We don't use the installer or > anything like that. > > > > > > ___

Stopwords

2010-03-16 Thread blargy
I was reading "Scaling Lucen and Solr" (http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr/) and I came across the section StopWords. In there it mentioned that its not recommended to remove stop words at index time. Why is this the case? Don't all t

APR setup

2010-03-16 Thread blargy
[java] INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: .:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java What the heck is this and why is it recommended for production setti

Recommended OS

2010-03-18 Thread blargy
Does anyone have any recommendations on which OS to use when setting up Solr search server? Any memory/disk space recommendations? Thanks -- View this message in context: http://old.nabble.com/Recommended-OS-tp27948306p27948306.html Sent from the Solr - User mailing list archive at Nabble.com

Re: Recommended OS

2010-03-18 Thread blargy
y. > > Would you mind explaining why you would stick with Jetty instead of > Tomcat? > > >> You really will need to read the docs to get the settings right as >> there is no one-size-fits-all setting. (re your mem/dsk question) >> >> K >> >> &g

Omitting norms question

2010-03-18 Thread blargy
Should I include not omit-norms on any fields that I would like to boost via a boost-query/function query? For example I have a created_on field on one of my documents and I would like to add some sort of function query to this field when querying. In this case does this mean I need to have the n

Re: Omitting norms question

2010-03-19 Thread blargy
rtable fields? Facetable fields? > You can use both without setting norms aswell. > > See what norms are for: > http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/Similarity.html#lengthNorm%28java.lang.String,%20int%29 > > > blargy wrote: >> >> Should I

Delta-Import quick question

2010-03-19 Thread blargy
Does the DIH delta-import automatically commit and optimize after its done? ... 8120 0 ... What is the difference between these? Usually I see the Total Documents Processed. -- View this message in context: http://old.nabble.com/Delta-Import-quick-question-tp27951022p27951022.html Sent from th

MLT question

2010-03-20 Thread blargy
Im playing around with MLT and I am getting back decent results when searching against a particular document. My question is how can I paginate the results of this query? For example instead of setting rows you must specify mlt.count in the params. But how can I set the offset? mlt.offset? Thank

Re: DIH - Deleting documents

2010-03-23 Thread blargy
Are there any examples out there for using these special commands? Im not quite sure of the syntax. Any simple example will suffice. Thanks mausch wrote: > > Take a look at the DIH special commands: > http://wiki.apache.org/solr/DataImportHandler#Special_Commands >

Impossible Boost Query?

2010-03-23 Thread blargy
I was wondering if this is even possible. I'll try to explain what I'm trying to do to the best of my ability. Ok, so our site has a bunch of products that are sold by any number of sellers. Currently when I search for some product I get back all products matching that search term but the proble

Re: Impossible Boost Query?

2010-03-23 Thread blargy
arch :: http://search-hadoop.com/ > > > > - Original Message >> From: blargy >> To: solr-user@lucene.apache.org >> Sent: Tue, March 23, 2010 2:39:48 PM >> Subject: Impossible Boost Query? >> >> > I was wondering if this is even pos

Re: Impossible Boost Query?

2010-03-23 Thread blargy
Maybe a better question is... how can I install this and will it work with 1.4? Thanks blargy wrote: > > Possibly. How can I install this as a contrib or do I need to actually > perform the patch? > > > Otis Gospodnetic wrote: >> >> Would Field Collapsing fr

Re: Impossible Boost Query?

2010-03-23 Thread blargy
the latest patch and trunk and try applying. > > Otis > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Hadoop ecosystem search :: http://search-hadoop.com/ > > > > - Original Message >> From: blargy >> To: solr-user@lucene.

SOLR-236 patch with version 1.4

2010-03-23 Thread blargy
Is the field collapsing patch (236) not compatible with Solr 1.4? $ patch -p0 -i ~/Desktop/SOLR-236.patch patching file src/test/test-files/solr/conf/solrconfig-fieldcollapse.xml patching file src/java/org/apache/solr/search/fieldcollapse/collector/DocumentGroupCountCollapseCollectorFactory.java

Re: Impossible Boost Query?

2010-03-24 Thread blargy
urce >> patch. You have to check out the Solr trunk from the Apache subversion >> server, apply the patch with the 'patch' command, and build the new >> Solr with 'ant'. >> >> On Tue, Mar 23, 2010 at 4:13 PM, blargy wrote: >>> >>>

Field Collapsing SOLR-236

2010-03-24 Thread blargy
Has anyone had any luck with the field collapsing patch (SOLR-236) with Solr 1.4? I tried patching my version of 1.4 with no such luck. Thanks -- View this message in context: http://old.nabble.com/Field-Collapsing-SOLR-236-tp28019949p28019949.html Sent from the Solr - User mailing list archive

Re: Impossible Boost Query?

2010-03-25 Thread Blargy
Ok so this is basically just a random sort. Anyway I can get this to randomly sort documents that closely related and not the rest of the results? -- View this message in context: http://n3.nabble.com/Impossible-Boost-Query-tp472080p580214.html Sent from the Solr - User mailing list archive at

DIH best pratices question

2010-03-26 Thread Blargy
I have a items table on db1 and and item_descriptions table on db2. The items table is very small in the sense that it has small columns while the item_descriptions table has a very large text field column. Both tables are around 7 million rows What is the best way to import these into one docu

Multicore process

2010-03-28 Thread Blargy
I was hoping someone could explain to me how your Solr multicore process currently operates. This is what I am thinking about and I was hoping I could get some ideas/suggestions. I have a master/slave setup where the master will be doing all the indexing via DIH. Ill be doing a full-import ever

Re: Multicore process

2010-03-28 Thread Blargy
Also, how do I share the same schema and config files? -- View this message in context: http://n3.nabble.com/Multicore-process-tp681929p681936.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Multicore process

2010-03-28 Thread Blargy
Mark, first off thanks for the response. Im glad someone is around today ;) So this is what I have so far: So my directory structure is: home/items/data/live/index home/items/data/offline/index So after playing around I see that swap literally just swaps the dataDir in solr.

Re: Multicore process

2010-03-28 Thread Blargy
Ok great... its starting to make sense. Now Im just a little confused on replication. So I had previously had my slave configuration as follows commit startup schema.xml,stopwords.txt http://${replication.host}:8983/solr/${solr.core.instanceDir}r

Re: Multicore process

2010-03-28 Thread Blargy
I just thought about this... Im guessing my slaves should always be replicating from the "live" master core: (http://localhost:8983/solr/items-live/replication). So my master solr will have a directory structure like this: home/items/data/core0/index home/items/data/core1/index and at any poi

Re: Multicore process

2010-03-28 Thread Blargy
Nice. Almost there... So it appears then that I will need two different solr.xml configurations. One for the master defining core0 and core1 and one for the slave with the default configuration. Is there anyway to specify master/slave specific settings in solr.xml or will I have to have 2 differe

Re: Multicore process

2010-03-28 Thread Blargy
Thanks that makes perfect sense for solrconfig.xml however I dont see that sort of functionality for solr.xml. Im guessing Ill need to manage 2 different versions of solr.xml Version 1 master Version 2 slave And my app will always be pointing to http://slave-host

Re: Multicore process

2010-03-28 Thread Blargy
Mark Miller-3 wrote: > > Hmmm...but isn't your slave on a different machine? Every install is > going to need a solr.xml, no way around that.. > Of course its on another machine. I was just hoping to only have 1 version of solr.xml checked into our source control and that I can change which co

Optimize after delta-import (DIH)

2010-03-29 Thread Blargy
According to the wiki: http://wiki.apache.org/solr/DataImportHandler#Commands the delta-import command will accept the same clean, commit and optimize parameters that the full-import command takes but I am my index keeps saying its not optimized. [java] INFO: [items] webapp=/solr path=/dataimport

DIH after import hooks

2010-03-30 Thread Blargy
Can you use a RunExecutableListener on DIH to run external scripts after a full-import/delta-import just like you can use on the DirectUpdateHandler2? If not, is there any alternative way to achieve this functionality? Thanks -- View this message in context: http://n3.nabble.com/DIH-after-impor

MoreLikeThis function queries

2010-04-01 Thread Blargy
Are function queries possible using the MLT request handler? How about using the _val_ hack? Thanks for your help -- View this message in context: http://n3.nabble.com/MoreLikeThis-function-queries-tp692377p692377.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: MoreLikeThis function queries

2010-04-02 Thread Blargy
Bueller? Anyone? :) -- View this message in context: http://n3.nabble.com/MoreLikeThis-function-queries-tp692377p693648.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: MoreLikeThis function queries

2010-04-02 Thread Blargy
Fair enough :) -- View this message in context: http://n3.nabble.com/MoreLikeThis-function-queries-tp692377p693872.html Sent from the Solr - User mailing list archive at Nabble.com.

Related terms/combined terms

2010-04-02 Thread Blargy
Not sure of the exact vocabulary I am looking for so I'll try to explain myself. Given a search term is there anyway to return back a list of related/grouped keywords (based on the current state of the index) for that term. For example say I have a sports catalog and I search for "Callaway". Is

an OR filter query

2010-04-04 Thread Blargy
Is there anyway to use a filter query as an OR clause? For example I have product listings and I want to be able to filter out mature items by default. To do this I added: mature:false But then I can never return any mature items because appending fq=mature:true will obviously return 0 res

Re: MoreLikeThis function queries

2010-04-05 Thread Blargy
Ok its now monday and everyone should have had their nice morning cup of coffee :) -- View this message in context: http://n3.nabble.com/MoreLikeThis-function-queries-tp692377p698304.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Related terms/combined terms

2010-04-05 Thread Blargy
Thanks for the response Mitch. I'm not too sure how well this will work for my needs but Ill certainly play around with it. I think something more along the lines of Ahmet's solution is what I was looking for. -- View this message in context: http://n3.nabble.com/Related-terms-combined-terms-

Re: Related terms/combined terms

2010-04-05 Thread Blargy
Ahmet thanks, this sounds like what I was looking for. Would one recommend using the TermsComponent prefix search or the Faceted prefix search for this sort of functionality. I know for auto-suggest functionality the generally consensus has been leaning towards the Faceted prefix search over the

DIH multiple queries per sub-entity?

2010-04-06 Thread Blargy
I am going through some of my DIH verbose output and I noticed that for each sub entity it appear to be query the DB multiple times and it keeps increasing at a linear fashion! For example: . select * from item_categories where item_id=1 ... .

Bucketing a price field

2010-04-06 Thread Blargy
What would be the best way to do range bucketing on a price field? I'm sort of taking the example from the Solr 1.4 book and I was thinking about using a PatternTokenizerFactory with a SynonymFilterFactory. Is there a better way? Thanks -- View this message in context: http://n3.nabble.com

Re: Bucketing a price field

2010-04-07 Thread Blargy
Duh, didnt even think of that. This will probably be the easy way for now since we are only using a small number of predefined ranges. Thanks for the reply -- View this message in context: http://n3.nabble.com/Bucketing-a-price-field-tp701801p703169.html Sent from the Solr - User mailing list a

Best practice to handle misspellings

2010-04-07 Thread Blargy
Whats is the best way to handle misspellings? Complete ignore them and suggest alternative searches or some sort of fuzzy matching? Also, is it possible to use fuzzy matching using the dismax request handler? Thanks -- View this message in context: http://n3.nabble.com/Best-practice-to-handl

Need help with StackOverflowError

2010-04-07 Thread Blargy
My last few delta-imports via DIH have been failing with a StackOverFlow error. Has anyone else encountered this why trying to importing? I don't even see any relevant information in the stack trace. Can anyone lend some suggestions. Thanks... pr 7, 2010 2:13:34 PM org.apache.solr.handler.dataimp

Re: Need help with StackOverflowError

2010-04-07 Thread Blargy
If it helps at all to mention, I manually updated the last_index_time in conf/dataimport.properties so I could select a smaller subset and the delta-import worked which leads me to believe there is nothing wrong with my DIH delta queries themselves. There must be something wrong with my dataset th

Re: Need help with StackOverflowError

2010-04-08 Thread Blargy
Also, If i remove my deletedPkQuery on the root entity the delta-import will complete successfully. Does anyone have any idea how a deletedPkQuery would end up in this circular StackOverflowError? FYI. I have a logical model called "item" and whenever an item gets deleted it gets moved over to t

  1   2   >