RE: DataImportHandler - very slow delta import

2009-08-18 Thread Matthew Painter
Of Noble Paul ??? ?? Sent: Tuesday, 18 August 2009 5:11 p.m. To: solr-user@lucene.apache.org Subject: Re: DataImportHandler - very slow delta import delta imports are likely to be far slower that the full imports because it makes one db call per changed row. if you can write the "que

RE: DataImportHandler - very slow delta import

2009-08-18 Thread Matthew Painter
, 18 August 2009 5:11 p.m. To: solr-user@lucene.apache.org Subject: Re: DataImportHandler - very slow delta import delta imports are likely to be far slower that the full imports because it makes one db call per changed row. if you can write the "query" in such a way that it gives

Re: DataImportHandler - very slow delta import

2009-08-17 Thread Noble Paul നോബിള്‍ नोब्ळ्
delta imports are likely to be far slower that the full imports because it makes one db call per changed row. if you can write the "query" in such a way that it gives only the changed rows, then write a separate entity (directly under ) and just run a full-import with that entity only. On Tue, Aug

DataImportHandler - very slow delta import

2009-08-17 Thread Matthew Painter
Hi, We are using Solr's DataImportHandler to populate the Solr index from a SQL Server database of nearly 4,000,000 rows. Whereas the population itself is very fast (around 1000 rows per second), the delta import is only processing around one row a second. Is this a known performance issue? We