did you put in the data? On Mon, Dec 29, 2008 at 3:07 PM, Manupriya <manupriya.si...@gmail.com> wrote: > > Hi Shalin, > > I am actually trying out the example provided at > http://wiki.apache.org/solr/DataImportHandler. > > My db-data-config.xml is as following - > > <dataConfig> > <dataSource driver="com.mysql.jdbc.Driver" > url="jdbc:mysql://localhost/solr_demo_db" user="root" password="root" /> > <document name="products"> > <entity name="item" pk="ID" query="select * from item" > deltaQuery="select id from item where last_modified > > '${dataimporter.last_index_time}'"> > <field column="NAME" name="name" /> > <field column="NAME" name="nameSort" /> > <field column="NAME" name="alphaNameSort" /> > > <entity name="feature" pk="ITEM_ID" > query="select DESCRIPTION from FEATURE where > ITEM_ID='${item.ID}'" > deltaQuery="select ITEM_ID from FEATURE where > last_modified > '${dataimporter.last_index_time}'" > parentDeltaQuery="select ID from item where > ID=${feature.ITEM_ID}"> > <field name="features" column="DESCRIPTION" /> > </entity> > > <entity name="item_category" pk="ITEM_ID, CATEGORY_ID" > query="select CATEGORY_ID from item_category where > ITEM_ID='${item.ID}'" > deltaQuery="select ITEM_ID, CATEGORY_ID from > item_category where last_modified > '${dataimporter.last_index_time}'" > parentDeltaQuery="select ID from item where > ID=${item_category.ITEM_ID}"> > <entity name="category" pk="ID" > query="select DESCRIPTION from category where ID = > '${item_category.CATEGORY_ID}'" > deltaQuery="select ID from category where > last_modified > '${dataimporter.last_index_time}'" > parentDeltaQuery="select ITEM_ID, CATEGORY_ID from > item_category where CATEGORY_ID=${category.ID}"> > <field column="description" name="cat" /> > </entity> > </entity> > </entity> > </document> > </dataConfig> > --------------------------------------------------------------------------------------------------- > > My DB structure is as following - > > DROP TABLE IF EXISTS solr_demo_db.item; > CREATE TABLE `item` ( > `ID` int(2) NOT NULL auto_increment, > `name` varchar(100) default NULL, > `manu` varchar(20) default NULL, > `weight` varchar(20) default NULL, > `price` varchar(20) default NULL, > `popularity` varchar(2) default NULL, > `includes` varchar(10) default NULL, > `last_modified` datetime default NULL, > PRIMARY KEY (`ID`) > ) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=latin1; > > DROP TABLE IF EXISTS solr_demo_db.feature; > CREATE TABLE `feature` ( > `DESCRIPTION` varchar(100) default NULL, > `ITEM_ID` int(2) NOT NULL, > `last_modified` datetime default NULL, > PRIMARY KEY (`ITEM_ID`) > ) ENGINE=InnoDB DEFAULT CHARSET=latin1; > > DROP TABLE IF EXISTS solr_demo_db.category; > CREATE TABLE `category` ( > `ID` int(2) NOT NULL auto_increment, > `DESCRIPTION` varchar(100) default NULL, > `last_modified` date default NULL, > PRIMARY KEY (`ID`) > ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1; > > DROP TABLE IF EXISTS solr_demo_db.item_category; > CREATE TABLE `item_category` ( > `ITEM_ID` int(2) NOT NULL default '0', > `CATEGORY_ID` int(2) NOT NULL default '0', > `last_modified` date default NULL, > PRIMARY KEY (`ITEM_ID`,`CATEGORY_ID`) > ) ENGINE=InnoDB DEFAULT CHARSET=latin1; > > Am I doing something wrong here? > > Thanks, > Manu > > > Shalin Shekhar Mangar wrote: >> >> What does your data-config look like? especially the delta query part. >> >> On Mon, Dec 29, 2008 at 2:35 PM, Manupriya >> <manupriya.si...@gmail.com>wrote: >> >>> >>> Hi, >>> >>> I am using Delta DataImport feature to partially refresh the indexes. >>> >>> Note - full-import is working perfectly fine. But I need to do >>> delta-import >>> as I do not want to rebuild all the indexes. >>> >>> My database structure is same as mentioned in the example at >>> http://wiki.apache.org/solr/DataImportHandler. >>> >>> http://www.nabble.com/file/p21200498/example-schema.png >>> --------------------------------------------------------------------------- >>> The dataimport.properties has the value as - >>> >>> #Mon Dec 29 14:08:12 IST 2008 >>> last_index_time=2008-12-29 14\:08\:12 >>> --------------------------------------------------------------------------- >>> I have changed the last_modified for the 'tem' table as current >>> timestamp. >>> And when I query the DB, I get the following result - >>> >>> http://www.nabble.com/file/p21200498/data.jpg >>> >>> ------------------------------------------------------------------------------------------------- >>> >>> But when I call delta data-import, it returns me the following response - >>> >>> Server Response: >>> >>> {responseHeader={status=0,QTime=16},initArgs={defaults={config=db-data-config.xml}},command=delta-import,status=idle,importResponse=,statusMessages={},WARNING=This >>> response format is experimental. It is likely to change in the future.} >>> >>> >>> --------------------------------------------------------------------------------------------------- >>> And when I query for data for '*:*', I get the following - >>> Note - Data returned by query doesnt reflect the correct timestamp from >>> database. >>> >>> SolrDocument(1) >>> Field Name & Value: includes=12 >>> Field Name & Value: features=[demo feature, demo feature 1] >>> Field Name & Value: price=12.0 >>> Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008 >>> Field Name & Value: manu=manu-12 >>> Field Name & Value: sku=1 >>> Field Name & Value: id=1 >>> Field Name & Value: popularity=1 >>> Field Name & Value: weight=12.0 >>> SolrDocument(2) >>> Field Name & Value: includes=43 >>> Field Name & Value: features=demo feature 2 >>> Field Name & Value: price=10.0 >>> Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008 >>> Field Name & Value: manu=Demo - 12 >>> Field Name & Value: sku=2 >>> Field Name & Value: id=2 >>> Field Name & Value: popularity=5 >>> Field Name & Value: weight=12.0 >>> SolrDocument(3) >>> Field Name & Value: includes=1 >>> Field Name & Value: features=demo -3 >>> Field Name & Value: price=1.0 >>> Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008 >>> Field Name & Value: manu=manu - 36 >>> Field Name & Value: sku=3 >>> Field Name & Value: id=3 >>> Field Name & Value: popularity=1 >>> Field Name & Value: weight=1.0 >>> SolrDocument(4) >>> Field Name & Value: includes=2 >>> Field Name & Value: features=demo - 4 >>> Field Name & Value: price=2.0 >>> Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008 >>> Field Name & Value: manu=manu - 46 >>> Field Name & Value: sku=4 >>> Field Name & Value: id=4 >>> Field Name & Value: popularity=2 >>> Field Name & Value: weight=2.0 >>> SolrDocument(5) >>> Field Name & Value: includes=3 >>> Field Name & Value: features=demo - 5 >>> Field Name & Value: price=3.0 >>> Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008 >>> Field Name & Value: manu=manu - 56 >>> Field Name & Value: sku=5 >>> Field Name & Value: id=5 >>> Field Name & Value: popularity=3 >>> Field Name & Value: weight=3.0 >>> SolrDocument(6) >>> Field Name & Value: includes=4 >>> Field Name & Value: features=demo - 66 >>> Field Name & Value: price=4.0 >>> Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008 >>> Field Name & Value: manu=manu - 66 >>> Field Name & Value: sku=6 >>> Field Name & Value: id=6 >>> Field Name & Value: popularity=4 >>> Field Name & Value: weight=4.0 >>> >>> >>> ------------------------------------------------------------------------------------------ >>> My Java code that calls delta import is as follows - >>> >>> public static void main(String[] args) throws IOException, >>> SolrServerException { >>> CommonsHttpSolrServer server = new >>> CommonsHttpSolrServer("http://localhost:8983/solr";); >>> BinaryResponseParser parser = new BinaryResponseParser(); >>> server.setParser(parser); >>> >>> ModifiableSolrParams params = new ModifiableSolrParams(); >>> params.set("qt", "/dataimport"); >>> params.set("command", "delta-import"); >>> params.set("commit", "true"); >>> params.set("wt","json"); >>> >>> try{ >>> >>> QueryResponse response = server.query(params); >>> server.commit(); >>> System.out.println("Server Response: " + response); >>> >>> SolrQuery query = new SolrQuery("*:*"); >>> QueryResponse results = server.query(query); >>> >>> SolrDocumentList list = results.getResults(); >>> >>> int index = 1; >>> for (SolrDocument solrDocument : list) { >>> >>> System.out.println("SolrDocument("+index+")"); >>> Iterator<Entry<String, Object>> iterator = >>> solrDocument.iterator(); >>> >>> while(iterator.hasNext()){ >>> System.out.println("Field Name & Value: >>> "+iterator.next()); >>> } >>> >>> index++; >>> } >>> >>> }catch(Exception ex){ >>> >>> System.out.println("Exception Occured:"+ex); >>> } >>> >>> } >>> --------------------------------------------------------------------------- >>> >>> Does SOLR do some kind of caching? I dont understand as why the updated >>> values are not queried correctly from DB? >>> >>> There are no errors in the logs. But I can see an error on the server >>> console - >>> >>> SEVERE: Delta Import Failed >>> java.lang.NullPointerException >>> at >>> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.getDeltaImportQuery(SqlEntityProcessor.java:153) >>> at >>> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.getQuery(SqlEntityProcessor.java:125) >>> at >>> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73) >>> at >>> >>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285) >>> at >>> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:211) >>> at >>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:133) >>> at >>> >>> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:359) >>> at >>> >>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:388) >>> at >>> >>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) >>> >>> >>> What can be the reason for this error? >>> >>> Thanks, >>> Manu >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Delta-DataImport-is-not-picking-the-modified-value-in-DB-tp21200498p21200498.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- > View this message in context: > http://www.nabble.com/Delta-DataImport-is-not-picking-the-modified-value-in-DB-tp21200498p21200786.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
-- --Noble Paul
