Re: http replication transfer speed

2009-10-29 Thread Shalin Shekhar Mangar
On Sun, Oct 11, 2009 at 4:11 AM, Mark Miller wrote: > Anyone know why you would see a transfer speed of just 10-20MB over a > gigbit network connection? > > Even with standard drives, I would expect to at least see around 40MB. > Has anyone seen over 10-20 using replication? > > Any ideas on what

Re: Issues with SolrJ and IndexReader reopening

2009-10-29 Thread Shalin Shekhar Mangar
On Fri, Oct 30, 2009 at 2:37 AM, Simon Wistow wrote: > We've been trying to build an indexing pipeline using SolrJ but we've > run into a couple of issues - namely that IndexReaders don't seem to get > reopened after a commit(). > > After an index or delete the change doesn't show up until I rest

solr search

2009-10-29 Thread manishkbawne
I am using solr search to search through xml files. As I am working on millions of data, the result output is slower. Can anyone please suggest me some way, by which I can increase the search result output? -- View this message in context: http://old.nabble.com/solr-search-tp26125183p26125183.h

RE: uppercase and wildcards/truncation

2009-10-29 Thread Bernadette Houghton
Sorry, answering my own question - I see it has been marked for 1.5 :-) -Original Message- From: Bernadette Houghton [mailto:bernadette.hough...@deakin.edu.au] Sent: Friday, 30 October 2009 2:54 PM To: 'solr-user@lucene.apache.org' Subject: uppercase and wildcards/truncation Looks like s

uppercase and wildcards/truncation

2009-10-29 Thread Bernadette Houghton
Looks like solr 1.3 at least is still not lowercasing wildcarded/truncated queries. I see from http://issues.apache.org/jira/browse/SOLR-219 that this has been an issue for a while. Anyone know if this has yet been solved? TIA Bernadette Houghton, Library Business Applications Developer Deaki

Re: Weird Facet and KeywordTokenizerFactory Issue

2009-10-29 Thread Ravi Kiran
Thank you very much...I shall try out the tokenizerFactory attribute on SynonymFilterFactory On Tue, Oct 13, 2009 at 12:27 AM, Chris Hostetter wrote: > > : I had to be brief as my facets are in the order of 100K over 800K > documents > : and also if I give the complete schema.xml I was afraid nob

Re: Is it possible to a Transform handle data from 2 rows?

2009-10-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
how do you know that all the rows with the same group id has already come? transformer can store the row and return null till you collect all the rows and then you may return one consolidated row. But the problem is it does not know when the row ends. you can override the SqlEntityProcessor and ju

solr web ui

2009-10-29 Thread scabbage
Hi, I'm a new solr user. I would like to know if there are any easy to setup web UIs for solr. It can be as simple as a search box, term highlighting and basic faceting. Basically I'm using solr to store all our automation testing logs and would like to have a simple searchable UI. I don't wanna

Re: Indexing multiple entities

2009-10-29 Thread Avlesh Singh
> > One thing I thought about is if I can define my own > DocumentObjectBinder, so I can concatenate my entity names with the > IDs in the XML creation. > > Anyone knows if something like this can be done without modifying > Solrj sources? Is there any injection or plugin mecanism for this? > More

Re: multiple sql queries for one index?

2009-10-29 Thread Avlesh Singh
Read this example fully - http://wiki.apache.org/solr/DataImportHandler#Full_Import_Example "nested entities" is an answer to your question. The example has a sample. Cheers Avlesh On Fri, Oct 30, 2009 at 2:58 AM, Joel Nylund wrote: > Hi, Its been hurting my brain all day to try to build 1 quer

Re: best way to model 1-N

2009-10-29 Thread Avlesh Singh
> > In the database this is modeled a a 1-N where category table has the > mapping of feed to category > I need to be able to query , give me all the feeds in any given category. > How can I best model this in solr? > Seems like multiValued field might help, but how would I populate it, and > would

best way to model 1-N

2009-10-29 Thread Joel Nylund
Hi, I have one index so far which contains feeds. I have been able to de- normalize several tables and map this data onto the feed entity. There is one tricky problem that I need help on. Feeds have 1 - many categories. So Lets say we have Category1, Category2 and Category3 Feed 1 - is in

Re: Faceting within one document

2009-10-29 Thread Lance Norskog
Sorry, forgot that part. On Thu, Oct 29, 2009 at 1:37 PM, Andrew Clegg wrote: > > Actually Avlesh pointed me at that, earlier in the thread. But thanks :-) > > > Yonik Seeley-2 wrote: >> >> On Wed, Oct 28, 2009 at 2:02 PM, Andrew Clegg >> wrote: >>> If I give a query that matches a single docume

RE: "begins with" searches

2009-10-29 Thread Bernadette Houghton
G'day Avlesh, converting the "all" field to type "edgytext" doesn't work as expected as the various "text" analysers etc don't get to work on that field, so I get less results than expected. And adding the edgy filter into the text field also yields less results. I can work around the issue by s

multiple sql queries for one index?

2009-10-29 Thread Joel Nylund
Hi, Its been hurting my brain all day to try to build 1 query for my index (joins upon joins upon joins). Is there a way I can do multiple queries to populate the same index? I have one main table that I can join everything back via ID, it should be theoretically possible If this can be

Re: Facets - ORing attribute values

2009-10-29 Thread Jay Hill
1.4 has a good chance of being released next week. There was a hope that it might make it this week, but another bug in Lucene 2.9.1 was found, pushing things back just a little bit longer. -Jay http://www.lucidimagination.com On Thu, Oct 29, 2009 at 11:43 AM, beaviebugeater wrote: > > Do you h

Issues with SolrJ and IndexReader reopening

2009-10-29 Thread Simon Wistow
We've been trying to build an indexing pipeline using SolrJ but we've run into a couple of issues - namely that IndexReaders don't seem to get reopened after a commit(). After an index or delete the change doesn't show up until I restart solr. I've tried commit() and commit(true, true) just to

Re: TermVector (TF-IDF Scores) From Subset of Documents

2009-10-29 Thread Grant Ingersoll
On Oct 29, 2009, at 11:10 AM, peelman wrote: Indeed I have used this already, buy unless I am missing something this will always return scores based on the entire index. I see now way from the documentation to have it recalculate TF-IDF scores using only a subset of documents. Am I mi

Re: Indexing multiple entities

2009-10-29 Thread Christian López Espínola
Hi Israel, Thanks for your suggestion, On Thu, Oct 29, 2009 at 9:37 PM, Israel Ekpo wrote: > On Thu, Oct 29, 2009 at 3:31 PM, Christian López Espínola < > penyask...@gmail.com> wrote: > >> Hi, my name is Christian and I'm a newbie introducing to solr (and solrj). >> >> I'm working on a website w

Re: Indexing multiple entities

2009-10-29 Thread Israel Ekpo
On Thu, Oct 29, 2009 at 3:31 PM, Christian López Espínola < penyask...@gmail.com> wrote: > Hi, my name is Christian and I'm a newbie introducing to solr (and solrj). > > I'm working on a website where I want to index multiple entities, like > Book or Magazine. > The issue I'm facing is both of the

Re: Faceting within one document

2009-10-29 Thread Andrew Clegg
Actually Avlesh pointed me at that, earlier in the thread. But thanks :-) Yonik Seeley-2 wrote: > > On Wed, Oct 28, 2009 at 2:02 PM, Andrew Clegg > wrote: >> If I give a query that matches a single document, and facet on a >> particular >> field, I get a list of all the terms in that field whi

Is it possible to a Transform handle data from 2 rows?

2009-10-29 Thread Lucas F. A. Teixeira
Hello all, Is it possible to have my own implemented transformer that generates some output data from data of 2 different rows? My intention is to generate pre-processed HTML from a set of rows. Eg. SQL RESULT: GROUP | ITEM_ID | DESCRIPTION 1 | 8 | desc1 1 | 23 | desc2 1 |

Re: Faceting within one document

2009-10-29 Thread Yonik Seeley
On Wed, Oct 28, 2009 at 2:02 PM, Andrew Clegg wrote: > If I give a query that matches a single document, and facet on a particular > field, I get a list of all the terms in that field which appear in that > document. > > (I also get some with a count of zero, I don't really understand where they >

Re: Faceting within one document

2009-10-29 Thread Andrew Clegg
Are you sure? I've *never* explicitly deleted a document, I only ever rebuild the entire index with the data import handler's "full import with cleaning" operation. Lance Norskog-2 wrote: > > 0-value facets are left behind by docs which you have deleted. If you > optimize, there should be no 0

Re: Faceting within one document

2009-10-29 Thread Lance Norskog
0-value facets are left behind by docs which you have deleted. If you optimize, there should be no 0-value facets. On Wed, Oct 28, 2009 at 11:36 AM, Andrew Clegg wrote: > > > Isn't the TermVectorComponent more for one document at a time, and the > TermsComponent for the whole index? > > Actually

Indexing multiple entities

2009-10-29 Thread Christian López Espínola
Hi, my name is Christian and I'm a newbie introducing to solr (and solrj). I'm working on a website where I want to index multiple entities, like Book or Magazine. The issue I'm facing is both of them have an attribute ID, which I want to use as the uniqueKey on my schema, so I cannot identify uni

Re: data import with transformer

2009-10-29 Thread Shalin Shekhar Mangar
On Thu, Oct 29, 2009 at 9:48 PM, Joel Nylund wrote: > Hi, I have been reading the solr book and wiki, but I cant find any similar > examples to what Im looking for. > > I have a database field called category, this field needs some text > manipulation before it goes in the index > > here is the j

Re: Plugin Performance Issues

2009-10-29 Thread entdeveloper
Here is where our custom class is referenced in the schema: As you can see, we built our own field type to be used at index time to essentially act as a sort of KeywordTokenizer, but removing stopwords. We share a schema.xml for both master and slave servers for conv

Re: Facets - ORing attribute values

2009-10-29 Thread beaviebugeater
Do you have any (educated) guess on when 1.4 will be officially released? Weeks? Months? Years? Yonik Seeley-2 wrote: > > Perhaps something like this that's actually running Solr w/ multi-selecti? > http://search.lucidimagination.com/ > > http://wiki.apache.org/solr/SimpleFacetParameters#Ta

Re: dismax and query analysis

2009-10-29 Thread Andrew Clegg
Thanks, that demonstrates it really nicely. Now if only dismax did wildcards too... :-) Cheers, Andrew. ANithian wrote: > > The best way to get started with answering this is to pass the > &debugQuery=true and to scroll down the results page. Here, you will see a > breakdown of how the quer

Re: dismax and query analysis

2009-10-29 Thread Amit Nithian
The best way to get started with answering this is to pass the &debugQuery=true and to scroll down the results page. Here, you will see a breakdown of how the query you entered in the q field is being parsed and sent to lucene via the pf,qf, and bf. You can also see how the weights affect the diffe

Re: Facets - ORing attribute values

2009-10-29 Thread beaviebugeater
I'll dive in. On the surface this looks like exactly what I described. Thanks for the quick reply!! Yonik Seeley-2 wrote: > > Perhaps something like this that's actually running Solr w/ multi-selecti? > http://search.lucidimagination.com/ > > http://wiki.apache.org/solr/SimpleFacetParameter

Re: Facets - ORing attribute values

2009-10-29 Thread Yonik Seeley
Perhaps something like this that's actually running Solr w/ multi-selecti? http://search.lucidimagination.com/ http://wiki.apache.org/solr/SimpleFacetParameters#Tagging_and_excluding_Filters You just need a recent version of Solr 1.4 -Yonik http://www.lucidimagination.com On Thu, Oct 29, 2009

Facets - ORing attribute values

2009-10-29 Thread beaviebugeater
I have implemented faceting with Solr for an ecommerce project. However, I'd like to change the default behavior somewhat. Visualize with me the left nav that contains: Attribute A value1 (count) value2 (count) value3 (count) Attribute B value4 (count) value5 (count) The user interface has a

Re: data import with transformer

2009-10-29 Thread Chantal Ackermann
Another option is the RegexTransformer in DIH: http://wiki.apache.org/solr/DataImportHandler?highlight=%28regex%29#RegexTransformer Chantal William Pierce schrieb: I'd recommend two ways: The way I do it in my app is that I have written a MySql function to transform the column as part of the

Re: data import with transformer

2009-10-29 Thread William Pierce
I'd recommend two ways: The way I do it in my app is that I have written a MySql function to transform the column as part of the select statement. In this approach, your select query would like so: select col1, col2, col3, spPrettyPrintCategory(category) as X, col4, col5, from table

data import with transformer

2009-10-29 Thread Joel Nylund
Hi, I have been reading the solr book and wiki, but I cant find any similar examples to what Im looking for. I have a database field called category, this field needs some text manipulation before it goes in the index here is the java code for what im trying to do: // categories look like

Re: TermVector (TF-IDF Scores) From Subset of Documents

2009-10-29 Thread peelman
Indeed I have used this already, buy unless I am missing something this will always return scores based on the entire index. I see now way from the documentation to have it recalculate TF-IDF scores using only a subset of documents. Am I missing something? Are you saying I can do a filter query

Annotations and reference types

2009-10-29 Thread M. Tinnemeyer
Dear listusers, Is there a way to store an instance of class A (including the fields from "myB") via solr using annotations ? The index should look like : id; name; b_id; b_name -- Class A { @Field private String id; @Field private String name; @Field private B myB; } -- Cla

New solr-driven site

2009-10-29 Thread Steinar Asbjørnsen
Hi all. Just wanted to inform you that a new solr-driven website is up and running, and to say thanks to you guys for helping out. A little info: Bedriftsøket.no (http://www.bedriftsoket.no/) is a Norwegian company catalogue using solr and SolrNet for all search/faceting functionality. Th

Re: weird problem with letters S and T

2009-10-29 Thread Joel Nylund
Hey everyone thanks for the help, it seems to be working this am after a restart & reindex (maybe I was just too sleepy last night), and using field type of text_ws. Im curios about the pro's and cons of Michel's approach below, this seems like another good way to do it, is there any differ

Re: TermVector (TF-IDF Scores) From Subset of Documents

2009-10-29 Thread Grant Ingersoll
Have a look at the TermVectorComponent: http://wiki.apache.org/solr/TermVectorComponent . That might help. On Oct 28, 2009, at 10:30 PM, peelman wrote: I have an index of about 3 million documents, and specific list of document ids that belong in that 3 million (somewhere around 20-50 doc

Re: Plugin Performance Issues

2009-10-29 Thread Grant Ingersoll
I would guess that your code is being used. I'm not sure what you mean by it "was only referenced in the schema". That implies usage to me. Is it a new field type? What is your plugin doing? Have you tried setting breakpoints at method entry points in your plugin and starting up Solr w/

Re: weird problem with letters S and T

2009-10-29 Thread Michel Bottan
Hi Joel, If you intend querying for the TITLE which starts with specifics letters, I have another solution which seems to be easier, since you don't need a specific field for the first letter. 1. Create a new type in your schema.xml using the following analyzer

dismax and query analysis

2009-10-29 Thread Andrew Clegg
Morning, Can someone clarify how dismax queries work under the hood? I couldn't work this particular point out from the documentation... I get that they pretty much issue the user's query against all of the fields in the schema -- or rather, all of the fields you've specified in the qf parameter