DataImportHanler JDBC case problems
I tried to perform a DataImportHandler where the column name "user" and the field name "User" are the same but the case of the first letter, when performing a full import, I was getting different sorts of errors, on that field depending on the cases of the names, I tried the four possible combinations, and none worked. At the end, I changed column name in the database to be "User" and everthing was fine. -- View this message in context: http://www.nabble.com/DataImportHanler-JDBC-case-problems-tp20617216p20617216.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unique id
Hello again. I'm getting a bit confused by your questions, and I believe it would be easier for us to help you if you could post the field definitions from your schema.xml and the structure of your two database views. ie. table 1: (id (int), subject (string) -.--) table 2: (category (string), other fields ..) So please post this and we can try to help you. - Aleks On Fri, 21 Nov 2008 07:49:31 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Thanks Erik. If I convert that to a string then id field defined in schema.xml would fail as I have that as integer. If I change that to string then first view would fail as it is Integer there. What to do in such scenarios? Do I need to define multiple schema.xml or multiple unique key definitions in same schema. How does this work? Pls explain. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Thursday, November 20, 2008 6:40 PM To: solr-user@lucene.apache.org Subject: Re: Unique id I'd suggest aggregating those three columns into a string that can serve as the Solr uniqueKey field value. Erik On Nov 20, 2008, at 1:10 AM, Raghunandan Rao wrote: Basically, I am working on two views. First one has an ID column. The second view has no unique ID column. What to do in such situations? There are 3 other columns where I can make a composite key out of those. I have to index these two views now. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 19, 2008 5:24 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Technically, no, a uniqueKey field is NOT required. I've yet to run into a situation where it made sense not to use one though. As for indexing database tables - if one of your tables doesn't have a primary key, does it have an aggregate unique "key" of some sort? Do you plan on updating the rows in that table and reindexing them? Seems like some kind of unique key would make sense for updating documents. But yeah, a more detailed description of your table structure and searching needs would be helpful. Erik On Nov 19, 2008, at 5:18 AM, Aleksander M. Stensby wrote: Yes it is. You need a unique id because the add method works as and "add or update" method. When adding a document whose ID is already found in the index, the old document will be deleted and the new will be added. Are you indexing two tables into the same index? Or does one entry in the index consist of data from both tables? How are these linked together without an ID? - Aleksander On Wed, 19 Nov 2008 10:42:00 +0100, Raghunandan Rao <[EMAIL PROTECTED] wrote: Hi, Is the uniqueKey in schema.xml really required? Reason is, I am indexing two tables and I have id as unique key in schema.xml but id field is not there in one of the tables and indexing fails. Do I really require this unique field for Solr to index it better or can I do away with this? Thanks, Rahgu -- Aleksander M. Stensby Senior software developer Integrasco A/S www.integrasco.no
RE: Unique id
View structure is: 1. Task(id* (int), name (string), start (timestamp), end (timestamp)) 2. Team(person_id (int), deptId (int), isManager (int)) * is primary key In schema.xml I have id -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 2:56 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Hello again. I'm getting a bit confused by your questions, and I believe it would be easier for us to help you if you could post the field definitions from your schema.xml and the structure of your two database views. ie. table 1: (id (int), subject (string) -.--) table 2: (category (string), other fields ..) So please post this and we can try to help you. - Aleks On Fri, 21 Nov 2008 07:49:31 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: > Thanks Erik. > If I convert that to a string then id field defined in schema.xml would > fail as I have that as integer. If I change that to string then first > view would fail as it is Integer there. What to do in such scenarios? Do > I need to define multiple schema.xml or multiple unique key definitions > in same schema. How does this work? Pls explain. > > -Original Message- > From: Erik Hatcher [mailto:[EMAIL PROTECTED] > Sent: Thursday, November 20, 2008 6:40 PM > To: solr-user@lucene.apache.org > Subject: Re: Unique id > > I'd suggest aggregating those three columns into a string that can > serve as the Solr uniqueKey field value. > > Erik > > > On Nov 20, 2008, at 1:10 AM, Raghunandan Rao wrote: > >> Basically, I am working on two views. First one has an ID column. The >> second view has no unique ID column. What to do in such situations? >> There are 3 other columns where I can make a composite key out of >> those. >> I have to index these two views now. >> >> >> -Original Message- >> From: Erik Hatcher [mailto:[EMAIL PROTECTED] >> Sent: Wednesday, November 19, 2008 5:24 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Unique id >> >> Technically, no, a uniqueKey field is NOT required. I've yet to run >> into a situation where it made sense not to use one though. >> >> As for indexing database tables - if one of your tables doesn't have a >> primary key, does it have an aggregate unique "key" of some sort? Do >> you plan on updating the rows in that table and reindexing them? >> Seems like some kind of unique key would make sense for updating >> documents. >> >> But yeah, a more detailed description of your table structure and >> searching needs would be helpful. >> >> Erik >> >> >> On Nov 19, 2008, at 5:18 AM, Aleksander M. Stensby wrote: >> >>> Yes it is. You need a unique id because the add method works as and >>> "add or update" method. When adding a document whose ID is already >>> found in the index, the old document will be deleted and the new >>> will be added. Are you indexing two tables into the same index? Or >>> does one entry in the index consist of data from both tables? How >>> are these linked together without an ID? >>> >>> - Aleksander >>> >>> On Wed, 19 Nov 2008 10:42:00 +0100, Raghunandan Rao >> <[EMAIL PROTECTED] wrote: >>> Hi, Is the uniqueKey in schema.xml really required? Reason is, I am indexing two tables and I have id as unique key in schema.xml but id field is not there in one of the tables and indexing fails. Do I really require this unique field for Solr to index it better or can I do away with this? Thanks, Rahgu >>> >>> >>> >>> -- >>> Aleksander M. Stensby >>> Senior software developer >>> Integrasco A/S >>> www.integrasco.no >
Re: solrQueryParser does not take effect - nightly build
That sounds a bit strange. Did you do the changes in the schema.xml before starting the server? Because if you change it while it is running, it will by default delete and replace the file (discarding any changes you make). In other words, make sure the server is not running, make your changes and then start up the server. Apart from that, I can't really see any reason for this to not work... - Aleks On Thu, 20 Nov 2008 22:03:30 +0100, ashokc <[EMAIL PROTECTED]> wrote: Hi, I have set but it is not taking effect. It continues to take it as OR. I am working with the latest nightly build 11/20/2008 For a querry like term1 term2 Debug shows content:term1 content:term2>/str> Bug? Thanks - ashok
Re: Unique id
Ok, this brings me to the question; how are the two view's connected to each other (since you are indexing partly view 1 and partly view 2 into a single index structure? If they are not at all connected I believe you have made a fundamental mistake / misunderstand the use of your index... I assume that a Task can be assigned to a person, and your Team view displays that person, right? Maybe you are doing something like this: View 1 1, somename, sometimestamp, someothertimestamp 2, someothername, somethirdtimestamp, timetamp4 ... View 2 1, 58, 0 2, 58, 1 3, 52, 0 ... I'm really confused about your database structure... To me, It would be logical to add a team_id field to the team table, and add a third table to link tasks to a team (or to individual persons). Once you have that information (because I do assume there MUST be some link there) you would do: insert into your index: (id from the task), (name of the task), (id of the person assigned to this task), (id of the departement that this person works in). I guess that you _might_ be thinking a bit wrong and trying to do something like this: Treat each view as independent views, and inserting values from each table as separate documents in the index so you would do: insert into your index: (id from the task), (name of the task), (no value), (no value) which will be ok to do (no value), (no value), (id of the person), (id of the departement) --- which makes no sense to me... So, can you clearify the relationship between the two views, and how you are thinking of inserting entries into your index? - Aleks On Fri, 21 Nov 2008 10:33:28 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: View structure is: 1. Task(id* (int), name (string), start (timestamp), end (timestamp)) 2. Team(person_id (int), deptId (int), isManager (int)) * is primary key In schema.xml I have id -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 2:56 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Hello again. I'm getting a bit confused by your questions, and I believe it would be easier for us to help you if you could post the field definitions from your schema.xml and the structure of your two database views. ie. table 1: (id (int), subject (string) -.--) table 2: (category (string), other fields ..) So please post this and we can try to help you. - Aleks On Fri, 21 Nov 2008 07:49:31 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Thanks Erik. If I convert that to a string then id field defined in schema.xml would fail as I have that as integer. If I change that to string then first view would fail as it is Integer there. What to do in such scenarios? Do I need to define multiple schema.xml or multiple unique key definitions in same schema. How does this work? Pls explain. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Thursday, November 20, 2008 6:40 PM To: solr-user@lucene.apache.org Subject: Re: Unique id I'd suggest aggregating those three columns into a string that can serve as the Solr uniqueKey field value. Erik On Nov 20, 2008, at 1:10 AM, Raghunandan Rao wrote: Basically, I am working on two views. First one has an ID column. The second view has no unique ID column. What to do in such situations? There are 3 other columns where I can make a composite key out of those. I have to index these two views now. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 19, 2008 5:24 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Technically, no, a uniqueKey field is NOT required. I've yet to run into a situation where it made sense not to use one though. As for indexing database tables - if one of your tables doesn't have a primary key, does it have an aggregate unique "key" of some sort? Do you plan on updating the rows in that table and reindexing them? Seems like some kind of unique key would make sense for updating documents. But yeah, a more detailed description of your table structure and searching needs would be helpful. Erik On Nov 19, 2008, at 5:18 AM, Aleksander M. Stensby wrote: Yes it is. You need a unique id because the add method works as and "add or update" method. When adding a document whose ID is already found in the index, the old document will be deleted and the new will be added. Are you indexing two tables into the same index? Or does one entry in the index consist of data from both tables? How are these linked together without an ID? - Aleksander On Wed, 19 Nov 2008 10:42:00 +0100, Raghunandan Rao <[EMAIL PROTECTED] wrote: Hi, Is the uniqueKey in schema.xml really required? Reason is, I am indexing two tables and I have id as unique key in schema.xml but id field is not there in one of the tables and indexing fails. Do I really require this unique field for Solr to index it better o
Highlighting wildcards
Hi, i'm using solr 1.3.0 and SolrJ for my java application I need to highlight my query words even if I use wildcards for example q=tele* i need to highlight words as "television", "telephone", etc I found this thread http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200704.mbox/[EMAIL PROTECTED] but i have not understood ho to solve my problem could anyone tell me how to solve the problem with SolrJ and with solr web (by url)? thanks in advance, Revenge Unisciti alla community di Io fotografo e video, il nuovo corso di fotografia di Gazzetta dello sport: http://www.flickr.com/groups/iofotografoevideo
RE: Unique id
Ok. There is common column in two views called queueId. I query second view first and get all the queueids for a person. And having queueIds I get all the ids from first view. Sorry for missing that column earlier. I think it should make sense now. -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:18 PM To: solr-user@lucene.apache.org Subject: Re: Unique id And in case that wasn't clear, the reason for it failing then would obviously be because you define the id field with required="true", and you try inserting a document where this field is missing... - Aleks On Fri, 21 Nov 2008 10:46:10 +0100, Aleksander M. Stensby <[EMAIL PROTECTED]> wrote: > Ok, this brings me to the question; how are the two view's connected to > each other (since you are indexing partly view 1 and partly view 2 into > a single index structure? > > If they are not at all connected I believe you have made a fundamental > mistake / misunderstand the use of your index... > I assume that a Task can be assigned to a person, and your Team view > displays that person, right? > > Maybe you are doing something like this: > View 1 > 1, somename, sometimestamp, someothertimestamp > 2, someothername, somethirdtimestamp, timetamp4 > ... > > View 2 > 1, 58, 0 > 2, 58, 1 > 3, 52, 0 > ... > > I'm really confused about your database structure... > To me, It would be logical to add a team_id field to the team table, and > add a third table to link tasks to a team (or to individual persons). > Once you have that information (because I do assume there MUST be some > link there) you would do: > insert into your index: > (id from the task), (name of the task), (id of the person assigned to > this task), (id of the departement that this person works in). > > I guess that you _might_ be thinking a bit wrong and trying to do > something like this: > Treat each view as independent views, and inserting values from each > table as separate documents in the index > so you would do: > insert into your index: > (id from the task), (name of the task), (no value), (no value) > which will be ok to do > (no value), (no value), (id of the person), (id of the departement) > --- which makes no sense to me... > > So, can you clearify the relationship between the two views, and how you > are thinking of inserting entries into your index? > > - Aleks > > > > On Fri, 21 Nov 2008 10:33:28 +0100, Raghunandan Rao > <[EMAIL PROTECTED]> wrote: > >> View structure is: >> >> 1. >> Task(id* (int), name (string), start (timestamp), end (timestamp)) >> >> 2. >> Team(person_id (int), deptId (int), isManager (int)) >> >> * is primary key >> >> In schema.xml I have >> >> > required="true"/> >> >> >> >> >> id >> >> >> -Original Message- >> From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] >> Sent: Friday, November 21, 2008 2:56 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Unique id >> >> Hello again. I'm getting a bit confused by your questions, and I believe >> >> it would be easier for us to help you if you could post the field >> definitions from your schema.xml and the structure of your two database >> >> views. >> ie. >> table 1: (id (int), subject (string) -.--) >> table 2: (category (string), other fields ..) >> >> >> So please post this and we can try to help you. >> >> - Aleks >> >> >> On Fri, 21 Nov 2008 07:49:31 +0100, Raghunandan Rao >> <[EMAIL PROTECTED]> wrote: >> >>> Thanks Erik. >>> If I convert that to a string then id field defined in schema.xml >> would >>> fail as I have that as integer. If I change that to string then first >>> view would fail as it is Integer there. What to do in such scenarios? >> Do >>> I need to define multiple schema.xml or multiple unique key >> definitions >>> in same schema. How does this work? Pls explain. >>> >>> -Original Message- >>> From: Erik Hatcher [mailto:[EMAIL PROTECTED] >>> Sent: Thursday, November 20, 2008 6:40 PM >>> To: solr-user@lucene.apache.org >>> Subject: Re: Unique id >>> >>> I'd suggest aggregating those three columns into a string that can >>> serve as the Solr uniqueKey field value. >>> >>> Erik >>> >>> >>> On Nov 20, 2008, at 1:10 AM, Raghunandan Rao wrote: >>> Basically, I am working on two views. First one has an ID column. The second view has no unique ID column. What to do in such situations? There are 3 other columns where I can make a composite key out of those. I have to index these two views now. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 19, 2008 5:24 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Technically, no, a uniqueKey field is NOT required. I've yet to run into a situation where it made sense not to use one though. As for indexing database tables - if one of your tables doesn't have >> a primary key, do
Re: Unique id
And in case that wasn't clear, the reason for it failing then would obviously be because you define the id field with required="true", and you try inserting a document where this field is missing... - Aleks On Fri, 21 Nov 2008 10:46:10 +0100, Aleksander M. Stensby <[EMAIL PROTECTED]> wrote: Ok, this brings me to the question; how are the two view's connected to each other (since you are indexing partly view 1 and partly view 2 into a single index structure? If they are not at all connected I believe you have made a fundamental mistake / misunderstand the use of your index... I assume that a Task can be assigned to a person, and your Team view displays that person, right? Maybe you are doing something like this: View 1 1, somename, sometimestamp, someothertimestamp 2, someothername, somethirdtimestamp, timetamp4 ... View 2 1, 58, 0 2, 58, 1 3, 52, 0 ... I'm really confused about your database structure... To me, It would be logical to add a team_id field to the team table, and add a third table to link tasks to a team (or to individual persons). Once you have that information (because I do assume there MUST be some link there) you would do: insert into your index: (id from the task), (name of the task), (id of the person assigned to this task), (id of the departement that this person works in). I guess that you _might_ be thinking a bit wrong and trying to do something like this: Treat each view as independent views, and inserting values from each table as separate documents in the index so you would do: insert into your index: (id from the task), (name of the task), (no value), (no value) which will be ok to do (no value), (no value), (id of the person), (id of the departement) --- which makes no sense to me... So, can you clearify the relationship between the two views, and how you are thinking of inserting entries into your index? - Aleks On Fri, 21 Nov 2008 10:33:28 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: View structure is: 1. Task(id* (int), name (string), start (timestamp), end (timestamp)) 2. Team(person_id (int), deptId (int), isManager (int)) * is primary key In schema.xml I have id -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 2:56 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Hello again. I'm getting a bit confused by your questions, and I believe it would be easier for us to help you if you could post the field definitions from your schema.xml and the structure of your two database views. ie. table 1: (id (int), subject (string) -.--) table 2: (category (string), other fields ..) So please post this and we can try to help you. - Aleks On Fri, 21 Nov 2008 07:49:31 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Thanks Erik. If I convert that to a string then id field defined in schema.xml would fail as I have that as integer. If I change that to string then first view would fail as it is Integer there. What to do in such scenarios? Do I need to define multiple schema.xml or multiple unique key definitions in same schema. How does this work? Pls explain. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Thursday, November 20, 2008 6:40 PM To: solr-user@lucene.apache.org Subject: Re: Unique id I'd suggest aggregating those three columns into a string that can serve as the Solr uniqueKey field value. Erik On Nov 20, 2008, at 1:10 AM, Raghunandan Rao wrote: Basically, I am working on two views. First one has an ID column. The second view has no unique ID column. What to do in such situations? There are 3 other columns where I can make a composite key out of those. I have to index these two views now. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 19, 2008 5:24 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Technically, no, a uniqueKey field is NOT required. I've yet to run into a situation where it made sense not to use one though. As for indexing database tables - if one of your tables doesn't have a primary key, does it have an aggregate unique "key" of some sort? Do you plan on updating the rows in that table and reindexing them? Seems like some kind of unique key would make sense for updating documents. But yeah, a more detailed description of your table structure and searching needs would be helpful. Erik On Nov 19, 2008, at 5:18 AM, Aleksander M. Stensby wrote: Yes it is. You need a unique id because the add method works as and "add or update" method. When adding a document whose ID is already found in the index, the old document will be deleted and the new will be added. Are you indexing two tables into the same index? Or does one entry in the index consist of data from both tables? How are these linked together without an ID? - Aleksander On Wed, 19 Nov 2008 10:42:00 +0100, Raghunan
Re: Unique id
Well, In that case, what do you want to search for? If I were you, I would make my index consist of tasks (and I assume that is what you are trying to do). So why don't you just use your schema.xml as you have right now, and do the following: Pick a person (let's say he has person_id=42 and deptId=3), get his queue of tasks, then for each task in queue do: insert into index: (id from the task), (name of the task), (id of the person), (id of the departement) an example: 3, "this is a very important task", 42, 3 4, "this one is also important", 42, 3 5, "this one is low priority", 42, 3 And then for the next person you do the same, (person_id=58 and deptId=5) insert: 6, "this is about solr", 58, 5 7, "this is about lucene", 58, 5 etc. Now you can search for all tasks in departement 5 by doing "deptId:5". If you want to search for all the tasks assigned to a specific person you just enter the query "personId:42". And you could also search for all tasks containing certain keywords by doing the query "name:solr" OR "name:lucene". Do you understand now, or is it still unclear? - Aleks On Fri, 21 Nov 2008 10:56:38 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Ok. There is common column in two views called queueId. I query second view first and get all the queueids for a person. And having queueIds I get all the ids from first view. Sorry for missing that column earlier. I think it should make sense now. -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:18 PM To: solr-user@lucene.apache.org Subject: Re: Unique id And in case that wasn't clear, the reason for it failing then would obviously be because you define the id field with required="true", and you try inserting a document where this field is missing... - Aleks On Fri, 21 Nov 2008 10:46:10 +0100, Aleksander M. Stensby <[EMAIL PROTECTED]> wrote: Ok, this brings me to the question; how are the two view's connected to each other (since you are indexing partly view 1 and partly view 2 into a single index structure? If they are not at all connected I believe you have made a fundamental mistake / misunderstand the use of your index... I assume that a Task can be assigned to a person, and your Team view displays that person, right? Maybe you are doing something like this: View 1 1, somename, sometimestamp, someothertimestamp 2, someothername, somethirdtimestamp, timetamp4 ... View 2 1, 58, 0 2, 58, 1 3, 52, 0 ... I'm really confused about your database structure... To me, It would be logical to add a team_id field to the team table, and add a third table to link tasks to a team (or to individual persons). Once you have that information (because I do assume there MUST be some link there) you would do: insert into your index: (id from the task), (name of the task), (id of the person assigned to this task), (id of the departement that this person works in). I guess that you _might_ be thinking a bit wrong and trying to do something like this: Treat each view as independent views, and inserting values from each table as separate documents in the index so you would do: insert into your index: (id from the task), (name of the task), (no value), (no value) which will be ok to do (no value), (no value), (id of the person), (id of the departement) --- which makes no sense to me... So, can you clearify the relationship between the two views, and how you are thinking of inserting entries into your index? - Aleks On Fri, 21 Nov 2008 10:33:28 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: View structure is: 1. Task(id* (int), name (string), start (timestamp), end (timestamp)) 2. Team(person_id (int), deptId (int), isManager (int)) * is primary key In schema.xml I have id -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 2:56 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Hello again. I'm getting a bit confused by your questions, and I believe it would be easier for us to help you if you could post the field definitions from your schema.xml and the structure of your two database views. ie. table 1: (id (int), subject (string) -.--) table 2: (category (string), other fields ..) So please post this and we can try to help you. - Aleks On Fri, 21 Nov 2008 07:49:31 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Thanks Erik. If I convert that to a string then id field defined in schema.xml would fail as I have that as integer. If I change that to string then first view would fail as it is Integer there. What to do in such scenarios? Do I need to define multiple schema.xml or multiple unique key definitions in same schema. How does this work? Pls explain. -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: Thursday, November 20, 2008 6:40 PM To: solr-user@lucene.apache.org Subject: Re: Unique i
RE: Unique id
Ok. I got your point. So I need not require ID field in the second view. I will hence remove required="true" in schema.xml. What I thought was unique ID makes indexing easier or used to maintain doc. Thanks a lot. -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:36 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Well, In that case, what do you want to search for? If I were you, I would make my index consist of tasks (and I assume that is what you are trying to do). So why don't you just use your schema.xml as you have right now, and do the following: Pick a person (let's say he has person_id=42 and deptId=3), get his queue of tasks, then for each task in queue do: insert into index: (id from the task), (name of the task), (id of the person), (id of the departement) an example: 3, "this is a very important task", 42, 3 4, "this one is also important", 42, 3 5, "this one is low priority", 42, 3 And then for the next person you do the same, (person_id=58 and deptId=5) insert: 6, "this is about solr", 58, 5 7, "this is about lucene", 58, 5 etc. Now you can search for all tasks in departement 5 by doing "deptId:5". If you want to search for all the tasks assigned to a specific person you just enter the query "personId:42". And you could also search for all tasks containing certain keywords by doing the query "name:solr" OR "name:lucene". Do you understand now, or is it still unclear? - Aleks On Fri, 21 Nov 2008 10:56:38 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: > Ok. There is common column in two views called queueId. I query second > view first and get all the queueids for a person. And having queueIds I > get all the ids from first view. > > Sorry for missing that column earlier. I think it should make sense now. > > > -Original Message- > From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] > Sent: Friday, November 21, 2008 3:18 PM > To: solr-user@lucene.apache.org > Subject: Re: Unique id > > And in case that wasn't clear, the reason for it failing then would > obviously be because you define the id field with required="true", and > you > try inserting a document where this field is missing... > > - Aleks > > On Fri, 21 Nov 2008 10:46:10 +0100, Aleksander M. Stensby > <[EMAIL PROTECTED]> wrote: > >> Ok, this brings me to the question; how are the two view's connected > to >> each other (since you are indexing partly view 1 and partly view 2 > into >> a single index structure? >> >> If they are not at all connected I believe you have made a fundamental > >> mistake / misunderstand the use of your index... >> I assume that a Task can be assigned to a person, and your Team view >> displays that person, right? >> >> Maybe you are doing something like this: >> View 1 >> 1, somename, sometimestamp, someothertimestamp >> 2, someothername, somethirdtimestamp, timetamp4 >> ... >> >> View 2 >> 1, 58, 0 >> 2, 58, 1 >> 3, 52, 0 >> ... >> >> I'm really confused about your database structure... >> To me, It would be logical to add a team_id field to the team table, > and >> add a third table to link tasks to a team (or to individual persons). >> Once you have that information (because I do assume there MUST be some > >> link there) you would do: >> insert into your index: >> (id from the task), (name of the task), (id of the person assigned > to >> this task), (id of the departement that this person works in). >> >> I guess that you _might_ be thinking a bit wrong and trying to do >> something like this: >> Treat each view as independent views, and inserting values from each >> table as separate documents in the index >> so you would do: >> insert into your index: >> (id from the task), (name of the task), (no value), (no value) > >> which will be ok to do >> (no value), (no value), (id of the person), (id of the departement) > >> --- which makes no sense to me... >> >> So, can you clearify the relationship between the two views, and how > you >> are thinking of inserting entries into your index? >> >> - Aleks >> >> >> >> On Fri, 21 Nov 2008 10:33:28 +0100, Raghunandan Rao >> <[EMAIL PROTECTED]> wrote: >> >>> View structure is: >>> >>> 1. >>> Task(id* (int), name (string), start (timestamp), end (timestamp)) >>> >>> 2. >>> Team(person_id (int), deptId (int), isManager (int)) >>> >>> * is primary key >>> >>> In schema.xml I have >>> >>> >> required="true"/> >>> >>> >>> >>> >>> id >>> >>> >>> -Original Message- >>> From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] >>> Sent: Friday, November 21, 2008 2:56 PM >>> To: solr-user@lucene.apache.org >>> Subject: Re: Unique id >>> >>> Hello again. I'm getting a bit confused by your questions, and I > believe >>> >>> it would be easier for us to help you if you could post the field >>> definitions from your schema.xml and the structure of your two > database >>> >>> views. >>> ie. >>> table 1: (id (int), subject (string) -.--) >>> tab
Re: DataImportHanler JDBC case problems
The column value should be same as it is in the DB and the name value should be same as it is in the schema. BTW, if you are using a latest nightly all this must be fixed. The case does not matter. On Fri, Nov 21, 2008 at 2:13 PM, JCodina <[EMAIL PROTECTED]> wrote: > > I tried to perform a DataImportHandler where the column name "user" and the > field name "User" are the same but the case of the first letter, > when performing a full import, I was getting different sorts of errors, on > that field depending on the cases of the names, I tried the four possible > combinations, and none worked. > At the end, I changed column name in the database to be "User" and everthing > was fine. > > > > > > > > > -- > View this message in context: > http://www.nabble.com/DataImportHanler-JDBC-case-problems-tp20617216p20617216.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
RE: Unique id
Can you also let me know how I join two search indices in one query? That means, in this case I have two diff search indices and I need to join by queueId and get all the tasks in one SolrQuery. I am creating queries in Solrj. -Original Message- From: Raghunandan Rao [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:45 PM To: solr-user@lucene.apache.org Subject: RE: Unique id Ok. I got your point. So I need not require ID field in the second view. I will hence remove required="true" in schema.xml. What I thought was unique ID makes indexing easier or used to maintain doc. Thanks a lot. -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:36 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Well, In that case, what do you want to search for? If I were you, I would make my index consist of tasks (and I assume that is what you are trying to do). So why don't you just use your schema.xml as you have right now, and do the following: Pick a person (let's say he has person_id=42 and deptId=3), get his queue of tasks, then for each task in queue do: insert into index: (id from the task), (name of the task), (id of the person), (id of the departement) an example: 3, "this is a very important task", 42, 3 4, "this one is also important", 42, 3 5, "this one is low priority", 42, 3 And then for the next person you do the same, (person_id=58 and deptId=5) insert: 6, "this is about solr", 58, 5 7, "this is about lucene", 58, 5 etc. Now you can search for all tasks in departement 5 by doing "deptId:5". If you want to search for all the tasks assigned to a specific person you just enter the query "personId:42". And you could also search for all tasks containing certain keywords by doing the query "name:solr" OR "name:lucene". Do you understand now, or is it still unclear? - Aleks On Fri, 21 Nov 2008 10:56:38 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: > Ok. There is common column in two views called queueId. I query second > view first and get all the queueids for a person. And having queueIds I > get all the ids from first view. > > Sorry for missing that column earlier. I think it should make sense now. > > > -Original Message- > From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] > Sent: Friday, November 21, 2008 3:18 PM > To: solr-user@lucene.apache.org > Subject: Re: Unique id > > And in case that wasn't clear, the reason for it failing then would > obviously be because you define the id field with required="true", and > you > try inserting a document where this field is missing... > > - Aleks > > On Fri, 21 Nov 2008 10:46:10 +0100, Aleksander M. Stensby > <[EMAIL PROTECTED]> wrote: > >> Ok, this brings me to the question; how are the two view's connected > to >> each other (since you are indexing partly view 1 and partly view 2 > into >> a single index structure? >> >> If they are not at all connected I believe you have made a fundamental > >> mistake / misunderstand the use of your index... >> I assume that a Task can be assigned to a person, and your Team view >> displays that person, right? >> >> Maybe you are doing something like this: >> View 1 >> 1, somename, sometimestamp, someothertimestamp >> 2, someothername, somethirdtimestamp, timetamp4 >> ... >> >> View 2 >> 1, 58, 0 >> 2, 58, 1 >> 3, 52, 0 >> ... >> >> I'm really confused about your database structure... >> To me, It would be logical to add a team_id field to the team table, > and >> add a third table to link tasks to a team (or to individual persons). >> Once you have that information (because I do assume there MUST be some > >> link there) you would do: >> insert into your index: >> (id from the task), (name of the task), (id of the person assigned > to >> this task), (id of the departement that this person works in). >> >> I guess that you _might_ be thinking a bit wrong and trying to do >> something like this: >> Treat each view as independent views, and inserting values from each >> table as separate documents in the index >> so you would do: >> insert into your index: >> (id from the task), (name of the task), (no value), (no value) > >> which will be ok to do >> (no value), (no value), (id of the person), (id of the departement) > >> --- which makes no sense to me... >> >> So, can you clearify the relationship between the two views, and how > you >> are thinking of inserting entries into your index? >> >> - Aleks >> >> >> >> On Fri, 21 Nov 2008 10:33:28 +0100, Raghunandan Rao >> <[EMAIL PROTECTED]> wrote: >> >>> View structure is: >>> >>> 1. >>> Task(id* (int), name (string), start (timestamp), end (timestamp)) >>> >>> 2. >>> Team(person_id (int), deptId (int), isManager (int)) >>> >>> * is primary key >>> >>> In schema.xml I have >>> >>> >> required="true"/> >>> >>> >>> >>> >>> id >>> >>> >>> -Original Message- >>> From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] >>> Sent: Friday
Re: Highlighting wildcards
To do it now, you'd have to switch the query parser to using the old style wildcard (and/or prefix) query, which is slower on large indexes and has max clause issues. I think I can make it work out of the box for the next release again though. see https://issues.apache.org/jira/browse/SOLR-825 Antonio Zippo wrote: Hi, i'm using solr 1.3.0 and SolrJ for my java application I need to highlight my query words even if I use wildcards for example q=tele* i need to highlight words as "television", "telephone", etc I found this thread http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200704.mbox/[EMAIL PROTECTED] but i have not understood ho to solve my problem could anyone tell me how to solve the problem with SolrJ and with solr web (by url)? thanks in advance, Revenge Unisciti alla community di Io fotografo e video, il nuovo corso di fotografia di Gazzetta dello sport: http://www.flickr.com/groups/iofotografoevideo
Re: Unique id
I still don't understand why you want two different indexes if you want to return the linked information each time anyways... I would say the easiest way is just to index all data (all columns from your views) into the index like this: taskid - taskname - start - end - personid - deptid - ismanager then you can just search like I already explained earlier. This way, you have already joined by queue-id when you insert it into the index and thus you get both results from one single search. (if you also want to have the ability to search on the queueID, just add a column for that. In general, your questions doesn't really have anything to do with solr, but architecture, db-design and what you want to search on. - A. 1. Task(id* (int), name (string), start (timestamp), end (timestamp)) 2. Team(person_id (int), deptId (int), isManager (int)) * is primary key In schema.xml I have On Fri, 21 Nov 2008 11:59:56 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Can you also let me know how I join two search indices in one query? That means, in this case I have two diff search indices and I need to join by queueId and get all the tasks in one SolrQuery. I am creating queries in Solrj. -Original Message- From: Raghunandan Rao [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:45 PM To: solr-user@lucene.apache.org Subject: RE: Unique id Ok. I got your point. So I need not require ID field in the second view. I will hence remove required="true" in schema.xml. What I thought was unique ID makes indexing easier or used to maintain doc. Thanks a lot. -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:36 PM To: solr-user@lucene.apache.org Subject: Re: Unique id Well, In that case, what do you want to search for? If I were you, I would make my index consist of tasks (and I assume that is what you are trying to do). So why don't you just use your schema.xml as you have right now, and do the following: Pick a person (let's say he has person_id=42 and deptId=3), get his queue of tasks, then for each task in queue do: insert into index: (id from the task), (name of the task), (id of the person), (id of the departement) an example: 3, "this is a very important task", 42, 3 4, "this one is also important", 42, 3 5, "this one is low priority", 42, 3 And then for the next person you do the same, (person_id=58 and deptId=5) insert: 6, "this is about solr", 58, 5 7, "this is about lucene", 58, 5 etc. Now you can search for all tasks in departement 5 by doing "deptId:5". If you want to search for all the tasks assigned to a specific person you just enter the query "personId:42". And you could also search for all tasks containing certain keywords by doing the query "name:solr" OR "name:lucene". Do you understand now, or is it still unclear? - Aleks On Fri, 21 Nov 2008 10:56:38 +0100, Raghunandan Rao <[EMAIL PROTECTED]> wrote: Ok. There is common column in two views called queueId. I query second view first and get all the queueids for a person. And having queueIds I get all the ids from first view. Sorry for missing that column earlier. I think it should make sense now. -Original Message- From: Aleksander M. Stensby [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 3:18 PM To: solr-user@lucene.apache.org Subject: Re: Unique id And in case that wasn't clear, the reason for it failing then would obviously be because you define the id field with required="true", and you try inserting a document where this field is missing... - Aleks On Fri, 21 Nov 2008 10:46:10 +0100, Aleksander M. Stensby <[EMAIL PROTECTED]> wrote: Ok, this brings me to the question; how are the two view's connected to each other (since you are indexing partly view 1 and partly view 2 into a single index structure? If they are not at all connected I believe you have made a fundamental mistake / misunderstand the use of your index... I assume that a Task can be assigned to a person, and your Team view displays that person, right? Maybe you are doing something like this: View 1 1, somename, sometimestamp, someothertimestamp 2, someothername, somethirdtimestamp, timetamp4 ... View 2 1, 58, 0 2, 58, 1 3, 52, 0 ... I'm really confused about your database structure... To me, It would be logical to add a team_id field to the team table, and add a third table to link tasks to a team (or to individual persons). Once you have that information (because I do assume there MUST be some link there) you would do: insert into your index: (id from the task), (name of the task), (id of the person assigned to this task), (id of the departement that this person works in). I guess that you _might_ be thinking a bit wrong and trying to do something like this: Treat each view as independent views, and inserting values from each table as separate documents in the index so you would do: insert into your index
Re: Upgrade from 1.2 to 1.3 gives 3x slowdown
On Nov 20, 2008, at 9:18 AM, Fergus McMenemie wrote: Hello Grant, Were you overwriting the existing index or did you also clean out the Solr data directory, too? In other words, was it a fresh index, or an existing one? And was that also the case for the 22 minute time? No in each case it was a new index. I store the indexes (the "data" dir) outside the solr home directory. For the moment I, rm -rf the index dir after each edit to the solrconfig.sml or schema.xml file and reindex from scratch. The relaunch of tomcat recreates the index dir. Would it be possible to profile the two instance and see if you notice anything different? I dont understand this. Do mean run a profiler against the tomcat image as indexing takes place, or somehow compare the indexes? Something like JProfiler or any other Java profiler. I was think of making a short script that replicates the results, and posting it here, would that help? Very much so. Thanks, Grant On Nov 19, 2008, at 8:25 AM, Fergus McMenemie wrote: Hello, I have a CSV file with 6M records which took 22min to index with solr 1.2. I then stopped tomcat replaced the solr stuff inside webapps with version 1.3, wiped my index and restarted tomcat. Indexing the exact same content now takes 69min. My machine has 2GB of RAM and tomcat is running with $JAVA_OPTS -Xmx512M -Xms512M. Are there any tweaks I can use to get the original index time back. I read through the release notes and was expecting a speed up. I saw the bit about increasing ramBufferSizeMB and set it to 64MB; it had no effect. -- === Fergus McMenemie Email:[EMAIL PROTECTED] Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer === -- === Fergus McMenemie Email:[EMAIL PROTECTED] Techmore Ltd Phone:(UK) 07721 376021 Unix/Mac/Intranets Analyst Programmer === -- Grant Ingersoll Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ
Re: Boosting by field contents
Have a look at the FunctionQuery stuff: http://wiki.apache.org/solr/FunctionQuery On Nov 20, 2008, at 4:42 PM, Lance Norskog wrote: Is it possible to boost a document by the contents of a field? Given the query: text field:value I want to return all documents with 'text'. Documents where 'field = value' boosted over documents where 'field = some other value'. This query does it: (text field:value)^100 (text -field:value) Is there a simpler way? (I might have left out crucial AND/OR clauses, but you get the picture.) I don't remember much on this topic. Does it appear in the Lucene lists? Is there a Lucene feature or open issue? Thanks, Lance -- Grant Ingersoll Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ
Re: Highlighting wildcards
On 21-Nov-08, at 3:45 AM, Mark Miller wrote: To do it now, you'd have to switch the query parser to using the old style wildcard (and/or prefix) query, which is slower on large indexes and has max clause issues. An alternative is to query for q=tele?*, which forces wildcardquery -Mike I think I can make it work out of the box for the next release again though. see https://issues.apache.org/jira/browse/SOLR-825 Antonio Zippo wrote: Hi, i'm using solr 1.3.0 and SolrJ for my java application I need to highlight my query words even if I use wildcards for example q=tele* i need to highlight words as "television", "telephone", etc I found this thread http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200704.mbox/[EMAIL PROTECTED] but i have not understood ho to solve my problem could anyone tell me how to solve the problem with SolrJ and with solr web (by url)? thanks in advance,Revenge Unisciti alla community di Io fotografo e video, il nuovo corso di fotografia di Gazzetta dello sport: http://www.flickr.com/groups/iofotografoevideo
Re: Question about dismax 'mm' - give boost to searches by location
Since i didn't receive any response, i think i my question wasn't very clear. If the phrase has 4 words (last will and testament florida - and will be removed by stopwordfilter), right now solr matches docs with atleast 3 out of those 4 words. So whats happening is last will and testament from all states are returned although user specifically asked for florida will. I don't want to alter the 'mm' either because its working fine for other searches. Just for the search terms with a 'location' , i want to be able to match all words. Any easy way to do this? Someone please? anuvenk wrote: > > I use the 'dismax handler' for my phrase matching. And i have the 'mm' set > this way: > Up to 3 words, match all > up to 4, match 3 > up to 4, match 3 & so on > Its been working fine, but for certain phrases like 'san diego drunk > driving defense attorney', its brings up dui attorneys for other cities > first because the way i've set the 'mm' its trying to match docs with any > 4 words from the search phrase. In such cases how do i make solr return > the san diego listing first. I don't want to make phrase matching stricter > either (i.e don't want to change the current 'mm' configuration) > Any way to solve for this? > -- View this message in context: http://www.nabble.com/Question-about-dismax-%27mm%27---give-boost-to-searches-by-location-tp20606730p20628404.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Pagination with Solr
Hola! soy nueva en solr, lo estoy usando para mi proyecto de residencias, el problema que tengo es similar al que dice david, necesito enviar todos los resultados de cierta consulta al cliente, sin embargo siempre puedo obtener solamente 10, los diez primeros, y yo necesito obtener todos, yo estoy usando solrj y puedo obtener el numfound, sin embargo no puedo obtener todos los resultados =(, hay alguien que pueda ayudarme? gracias =) -- View this message in context: http://www.nabble.com/Pagination-with-Solr-tp13847908p20628504.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Pagination with Solr
Yo no suelo usar solrj, pero creo que quiere investigar SolrQuery.setStart() y SolrQuery.setRows(). Creo que el primero es para indicar el numero del primero resultado que quere obtener y el segundo es para indicar cuantos resultados quiere obtener. Si quere todos los resultados, probablemente va a querer hacer un loop, pidiendo 10 o 100 1000 cada vez. 2008/11/21 lupiss <[EMAIL PROTECTED]>: > > Hola! > soy nueva en solr, lo estoy usando para mi proyecto de residencias, el > problema que tengo es similar al que dice david, necesito enviar todos los > resultados de cierta consulta al cliente, sin embargo siempre puedo obtener > solamente 10, los diez primeros, y yo necesito obtener todos, yo estoy > usando solrj y puedo obtener el numfound, sin embargo no puedo obtener todos > los resultados =(, hay alguien que pueda ayudarme? > > gracias =) > -- > View this message in context: > http://www.nabble.com/Pagination-with-Solr-tp13847908p20628504.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Re: solr search
Hola!
soy nueva en solr y tengo el mismo problema, necesito obtener todos los
resultados de cierta consulta y si se puede paginados, de 10 en 10, yo estoy
usando jsolr y jsp en una aplicación web, entonces no sabria donde poner la
nueva url, con el start = 10 y el rows = 10 o algo así, estoy usando struts
entonces la consulta ocurre en una action que ha obtenido los parámetros de
un jsp pegaré parte del código:
la clase que hace la conexión, consulta y añade, etc...
public class TestSolrClient {
CommonsHttpSolrServer server;
SolrInputDocument doc;
UpdateRequest req;
UpdateResponse rsp;
SolrQuery query;
QueryResponse qrsp;
SolrDocumentList docs;
SolrDocument documento;
public TestSolrClient() throws MalformedURLException,
SolrServerException,
IOException {
String url = "http://localhost:8080/solr";;
server = new CommonsHttpSolrServer( url );
server.setParser(new XMLResponseParser());
}
public void añadir(Campo[] Campos) throws SolrServerException,
IOException {
doc = new SolrInputDocument();
for(int y=0; y
solo puse una parte donde se recogen los valores introducidos por el
usuario, en realidad la jsp pide un tipo de documento a consultar, luego
sobre ese documento pide que se seleccionen los campos que se consultara y
por ultimo pide que se introduzcan valores para esos campos, despues esos
valores se quedan en memoria en un bean que es accedido desde la action, a
continuación la action que hace la consulta:
public class ConsultarSolrAction extends Action {
/**This is the main action called from the Struts framework.
* @param mapping The ActionMapping used to select this instance.
* @param form The optional ActionForm bean for this request.
* @param request The HTTP Request we are processing.
* @param response The HTTP Response we are processing.
*/
public ActionForward execute(ActionMapping mapping, ActionForm form,
HttpServletRequest request,
HttpServletResponse response) throws
IOException,
ServletException,
SolrServerException {
// forma es el bean
ConsultaSolr forma = (ConsultaSolr)form;
List aux=forma.getSeleccionados();
String[] Valores= request.getParameterValues("valores");
forma.añadirConsulta("tipo: "+forma.getTipo()+" ");
if(Valores!=null){
for(int i=0; i0)
{
Campo campo= (Campo)aux.get(i);
System.out.println("Valor: "+ Valores[i]);
if(campo.getNombre().compareTo("general")!=0)
forma.añadirConsulta(campo.getNombre()+": "+Valores[i]+"
");
else
forma.añadirConsulta(Valores[i]+" ");
}
}
}
System.out.println("la consulta a solr sera:" +
forma.getConsulta());
TestSolrClient Solr = new TestSolrClient();
forma.resetValores();
forma.resetSeleccionados();
forma.setListSolr(Solr.consultar(forma.getConsulta()));
forma.reset(mapping,request);
System.out.println("num de resultados:
"+forma.getListSolr().size());
if(forma.getListSolr().isEmpty())
return mapping.findForward( "Error");
else
return mapping.findForward( "success");
}
}
despues de aqui se va a otra jsp que muestra los resultados, el problema es
que hago una consulta que tiene numfound=600 y solo muestra los 10 primeros
y yo quiero todos!!
help please!!
--
View this message in context:
http://www.nabble.com/solr-search-tp17249602p20628977.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Query two fields OR other AND
: I have problem with two fields. In query I am use q.op=AND but I would like : use in two first fields OR. : Example: : : iTitle:(+keyword 1 +keyword2) OR iDescription:(+keyword 1 : +keyword2)+iUser:"12345"+iType:"1" : : This query not work :( someplaces in your query sting it seems like you are using "+" to indicate a mandatory term, but in the second line it looks like the "+" characters are URL escaped whitespace ... what does debugQUery=true show you for the parsed query? does it match what you expect? i would also strongly recomment that you don't mix/match the differnet syntaxes (AND/OR vs +/-) in a single query ... it will only confuse you ... stick with q.op=OR and use +/- and things will make a lot more sense (since that's the most direct representation of hte underlying query classes) -Hoss
Re: Newbie Question - getting search results from dataimport request handler
: > it might be worth considering a new @attribute for to indicate : > that they are going to be used purely as "component" fields (ie: your : > first-name/last-name example) and then have DIH pass all non-component : > fields along and error if undefined in the schema just like other updating : > RequestHandlers do. : > : > either that, or require that people declaure indexed="false" : > stored="false" fields in the schema for these intermediate component : > fields so that we can properly warn then when DIH is getting data it : > doesn't know what to do with -- protecting people from field name typos : > and returning errors instead of silently ignoring unexpected input is : > fairly important behavir -- especially for new users. : Actually it is done by DIH . When the dataconfig is loaded DIH reports : these information on the console. though it is limited , it helps to a : certain extent Hmmm. Logging an error and returning successfully (without adding any docs) is still inconsistent with the way all other RequestHandlers work: fail the request. I know DIH isn't a typical RequestHandler, but some things (like failing on failure) seem like they should be a given. -Hoss
Re: Updating schema.xml without deleting index?
: I've tried searching for this answer all over but have found no results : thus far. I am trying to add a new field to my schema.xml with a : default value of 0. I have a ton of data indexed right now and it would : be very hard to retrieve all of the original sources to rebuild my : index. So my question is...is there any way to send a command to SOLR : that tells it to re-index everything it has and include the new field I : added? default values only affect documents as they are indexed. you can add that field, without reindexing anything, and solr will work, but those docs still won't have that value. you have to manually re-index everything for them to get it. -Hoss
RE: Updating schema.xml without deleting index?
Yep seems this is the case. I found a work-around in case anyone wants to know. I adjusted my query to include a fq of '-myfield:1'. So pretty much, all my data that was in there already would have had a value of '0' for myfield so I can treat either myfield:0 or the absence of myfield as the same result. I wish there was an option to just completely reindex all data..i suppose optimize may do that a little bit? --Jeff -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Friday, November 21, 2008 4:47 PM To: solr-user@lucene.apache.org Subject: Re: Updating schema.xml without deleting index? : I've tried searching for this answer all over but have found no results : thus far. I am trying to add a new field to my schema.xml with a : default value of 0. I have a ton of data indexed right now and it would : be very hard to retrieve all of the original sources to rebuild my : index. So my question is...is there any way to send a command to SOLR : that tells it to re-index everything it has and include the new field I : added? default values only affect documents as they are indexed. you can add that field, without reindexing anything, and solr will work, but those docs still won't have that value. you have to manually re-index everything for them to get it. -Hoss
New user questions
I've gotten Solr up and running, I can ingest the demo objects and query them via the admin tool, so far so good. Now, how do I ingest some basic XML, how can I pull from an existing MySQL database, what about pulling records in via OAI? I'm assuming I need to write some schemas for that, can someone point me to some documentation or tutorial for that? Thanks P -- http://fak3r.com dim high beams for oncoming traffic http://lefttochance.com know your rights, don't lose them
Re: New user questions
I guess you may have already gone through the Solr tutorial. Also look at http://wiki.apache.org/solr/UpdateXmlMessages for general description of add/delete commands Look at the DataImportHandler which can help in indexing databases and your custom XML -- http://wiki.apache.org/solr/DataImportHandler On Sat, Nov 22, 2008 at 3:31 AM, phil cryer <[EMAIL PROTECTED]> wrote: > I've gotten Solr up and running, I can ingest the demo objects and > query them via the admin tool, so far so good. Now, how do I ingest > some basic XML, how can I pull from an existing MySQL database, what > about pulling records in via OAI? I'm assuming I need to write some > schemas for that, can someone point me to some documentation or > tutorial for that? > > Thanks > > P > -- > http://fak3r.com dim high beams for oncoming traffic > http://lefttochance.com know your rights, don't lose them > -- Regards, Shalin Shekhar Mangar.
RE: :TODO: we should try to make a DTD for the schema, DONE as XSD instead
: There were a few comments about schema files in Mark Mail between you : and Grant a couple of months ago, no big demand for them for the : schema.xml file. Before I drop this would you consider taking a look at : XSD file below for the schema.xml and perhaps submit the XSD file the : SVN system? I can send it to you in a zip file, so it is formatted : nicer. contributions have to be made as Jira attachments so that there is an audit trail of exactly where it came from and that the contributor "Grant license to ASF for inclusion in ASF works" I would definitely open an issue for this and attach it since it could be useful for a lot of people as a way of documenting most of what's possible... ...but... Your tests showcase why trying to make a DTD or XSD for the schema isn't something anyone has tackled before... : There were a couple of files that contained attributes that the : documentation was not complete for. ...the crux of the issue is that any FieldType, or analysis factory can define whatever custom attributes it wants. if we had been smart about this when Solr was first being built, we would have used XML namespaces so that we could have one XSD for the "core" Solr namespace, and people writing custom plugins could have used their own namespace (and their own XSD) for their plugin and whatever custom attributes it expects. Since we didn't do that, i don't think it's actually possible to create an XSD that correctly identifies all valid schema.xml as valid (unless there's a way in XSD to indicate that any attribute is legal for certain elements) This is just one of those things where until we completley overhaul the way SOlr is configured, I don't think we can have robust validation of the configs. -Hoss
Re: Advice for indexing page numbers
: Possible ranges are numbers (1 to 1000s) but also could include appendix : page that include roman and alphabet characters (i, ii, iii, iv, as well as : a, b, c, etc). : : It makes sense people would want to search for things between page 1 to 5 : but I cannot really see how someone would search for page iv to 50. : : I was thinking to split this into two fields, one is just a string for exact : matching (maybe case insensitive) and the other as a number for ranges. This : would allow search for page ranges as well as exact matches. Alternately: 3 fields, where each "page" only has a value in one field... numeric_pagenum ... SortableInt roman_pagenum ... SortableInt alpha_pagenum ... String where you app (or a custom subclass of SortableInt) would convert from roman numerals to integers, and whever someone does a page range search, just check of the ranges look like numbers, letters, or roman numerals, and query the appropriate field. pagenum:[i TO v] is ambigious, by as long as you translate that into... +(roman_pagenum:[1 TO 5] alpha_pagenum:[i TO v]) ...it will work correctly. sorting by page numbers is a completley differnet ball of wax. : Has anyone had experience with pages or the like in solr? Is splitting it : into two fields like this needed or can I do that with one of the standard : filters that I have missed? logic like what you describe (or what i described) isn't possible out of the box ... you'd need to do it in the client hwen sending updates, or in an UpdateProcessor plugin. -Hoss
Re: solrQueryParser does not take effect - nightly build
ashok, Hmm, this is a bug. It was accidentally introduced when SOLR-657 was committed. http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/search/LuceneQParserPlugin.java?p2=%2Flucene%2Fsolr%2Ftrunk%2Fsrc%2Fjava%2Forg%2Fapache%2Fsolr%2Fsearch%2FLuceneQParserPlugin.java&p1=%2Flucene%2Fsolr%2Ftrunk%2Fsrc%2Fjava%2Forg%2Fapache%2Fsolr%2Fsearch%2FLuceneQParserPlugin.java&r1=701485&r2=701484&view=diff&pathrev=701485 I'll fix it shortly. Thank you, Koji ashokc wrote: Hi, I have set but it is not taking effect. It continues to take it as OR. I am working with the latest nightly build 11/20/2008 For a querry like term1 term2 Debug shows content:term1 content:term2>/str> Bug? Thanks - ashok
Re: Is there a DTD/XSD for XML response?
: > I assume there is a schema definition or DTD for XML response but could not : > find it anywhere. : > : > Is there one? there was an attempt at one a while back, it got to the point where it was mostly correct, but no one really had any specific ideas of what to do with it (just commiting it could be missleading if people forget to upgrade it when making enhancements to the XmlResponseWriter and forgets to update it). https://issues.apache.org/jira/browse/SOLR-17 If someone wanted to get it uptodate, and then write a patch to the AbstractSolrTest so every assertQ validated the response aganst the XSD (to demonstrate it's correctness and help ensure that any hcanges to XmlResponseWriter get made to the XSD as well) i'd be in favor of committing it then. -Hoss
Re: solrQueryParser does not take effect - nightly build
Committed revision 719793. Thank you for reporting this, ashok! Koji Koji Sekiguchi wrote: ashok, Hmm, this is a bug. It was accidentally introduced when SOLR-657 was committed. http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/search/LuceneQParserPlugin.java?p2=%2Flucene%2Fsolr%2Ftrunk%2Fsrc%2Fjava%2Forg%2Fapache%2Fsolr%2Fsearch%2FLuceneQParserPlugin.java&p1=%2Flucene%2Fsolr%2Ftrunk%2Fsrc%2Fjava%2Forg%2Fapache%2Fsolr%2Fsearch%2FLuceneQParserPlugin.java&r1=701485&r2=701484&view=diff&pathrev=701485 I'll fix it shortly. Thank you, Koji
Re: Newbie Question - getting search results from dataimport request handler
On Sat, Nov 22, 2008 at 3:10 AM, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > : > it might be worth considering a new @attribute for to indicate > : > that they are going to be used purely as "component" fields (ie: your > : > first-name/last-name example) and then have DIH pass all non-component > : > fields along and error if undefined in the schema just like other updating > : > RequestHandlers do. > : > > : > either that, or require that people declaure indexed="false" > : > stored="false" fields in the schema for these intermediate component > : > fields so that we can properly warn then when DIH is getting data it > : > doesn't know what to do with -- protecting people from field name typos > : > and returning errors instead of silently ignoring unexpected input is > : > fairly important behavir -- especially for new users. > > : Actually it is done by DIH . When the dataconfig is loaded DIH reports > : these information on the console. though it is limited , it helps to a > : certain extent > > Hmmm. > > Logging an error and returning successfully (without adding any docs) is > still inconsistent with the way all other RequestHandlers work: fail the > request. > > I know DIH isn't a typical RequestHandler, but some things (like failing > on failure) seem like they should be a given. SOLR-842 . DIH is an ETL tool pretending to be a RequestHandler. Originally it was built to run outside of Solr using SolrJ. For better integration and ease of use we changed it later. SOLR-853 aims to achieve the oroginal goal The goal of DIH is to become a full featured ETL tool. > > > > -Hoss > > -- --Noble Paul
facet sort by ranking
Hi All, Is there any way to sort the facet values by other own ranking value instead of only count? Thanks and Regards, Amit No virus found in this outgoing message. Checked by AVG. Version: 7.5.549 / Virus Database: 270.9.9/1804 - Release Date: 21-11-2008 18:24
Re: facet sort by ranking
On Sat, Nov 22, 2008 at 12:31 PM, Amit <[EMAIL PROTECTED]> wrote: > > Is there any way to sort the facet values by other own ranking value > instead > of only count? > > Facets do not have a "score" associated with them of their own. Not sure what you mean exactly. -- Regards, Shalin Shekhar Mangar.
