Re: Records skipped when using DataImportHandler

2011-08-08 Thread Erick Erickson
Spend some time in the admin/analysis page, that'll show you what part of the analysis chain is doing what to your data. It'll save you a world of headache... But at a guess WordDelimiterFilterFactory is your culprit... Best Erick On Thu, Aug 4, 2011 at 6:08 PM, anand sridhar wrote: > Ok. After

Re: Records skipped when using DataImportHandler

2011-08-05 Thread Shalin Shekhar Mangar
On Fri, Aug 5, 2011 at 3:38 AM, anand sridhar wrote: > Ok. After analysis, I narrowed the reduced results set to the fact that the > zipcode field is not indexed 'as is'. i.e the zipcode field values are > broken down into tokens and then stored. Hence, if there are 10 documents > with zipcode fie

Re: Records skipped when using DataImportHandler

2011-08-04 Thread anand sridhar
Ok. After analysis, I narrowed the reduced results set to the fact that the zipcode field is not indexed 'as is'. i.e the zipcode field values are broken down into tokens and then stored. Hence, if there are 10 documents with zipcode fields varying from 91000-91009, then the zipcode fields are not

Re: Records skipped when using DataImportHandler

2011-08-03 Thread Erick Erickson
Sorry, I'm on a restricted machine so can't get the precise URL. But, there's a debug page for DIH that might allow you to see what the query actually returns. I'd guess one of two things: 1> you aren't getting the number of rows you think. 2> you aren't committing the documents you add. But that'