Hi Noble, I had forgotten to also list comboId as a uniqueKey in the schema.xml file. But that didn't make a difference. It still complained about the "Document [null] missing required field: id" for each row it ran into of the outer entity.
If you look at the debug output of the entity:pets (see below on original message). The query looks like this: "SELECT id,name,birth_date,type_id FROM pets WHERE owner_id='owners-1' This is the problem lies, because, the owner_id in the pets table is currently a number and thus will not match the modified combo id generated for the owners' id column. So, somehow, I need to be able to either remove the 'owners-' suffix before comparing, or append the same suffix to the pets.owner_id value prior to comparing. Thanks ** julio -----Original Message----- From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED] Sent: Monday, June 02, 2008 9:20 PM To: solr-user@lucene.apache.org Subject: Re: How to describe 2 entities in dataConfig for the DataImporter? hi Julio, delete my previous response. In your schema , 'id' is the uniqueKey. make 'comboid' the unique key. Because that is the target field name coming out of the entity 'owners' --Noble On Tue, Jun 3, 2008 at 9:46 AM, Noble Paul ??????? ?????? <[EMAIL PROTECTED]> wrote: > The field 'id' is repeated for pet also rename it to something else > say <entity name="pets" pk="id" > query="SELECT id,name,birth_date,type_id FROM pets WHERE > owner_id='${owners.id}'" > parentDeltaQuery="SELECT id FROM owners WHERE > id=${pets.owner_id}"> > <field column="id" name="petid"/> > </entity> > > --Noble > > On Tue, Jun 3, 2008 at 3:28 AM, Julio Castillo <[EMAIL PROTECTED]> wrote: >> Shalin, >> I experimented with it, and the null pointer exception has been taken >> care of. Thank you. >> >> I have a different problem now. I believe it is a >> syntax/specification problem. >> >> When importing data, I got the following exceptions: >> SEVERE: Exception while adding: >> SolrInputDocumnt[{comboId=comboId(1.0)={owners-9}, >> userName=userName(1.0)={[David, Schroeder]}}] >> >> org.apache.solr.common.SolrException: Document [null] missing >> required >> field: id >> at >> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289) >> at >> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImp >> ortHand >> ler.java:263) >> ... >> >> The problem arises the moment I try to include nested entities (e.g. >> pets -the problem does not occur if I don't use the transformer, but >> I have to use the transformer because other unrelated entities also have id's). >> My data config file looks as follows. >> >> <dataConfig> >> <document name="doc-1"> >> <entity name="owners" pk="id" >> query="select id,first_name,last_name FROM owners" >> transformer="TemplateTransformer"> >> <field column="id" name="comboId" template="owners-${owners.id}"/> >> <field column="first_name" name="userName"/> >> <field column="last_name" name="userName"/> >> >> <entity name="pets" pk="id" >> query="SELECT id,name,birth_date,type_id FROM pets >> WHERE owner_id='${owners.id}'" >> parentDeltaQuery="SELECT id FROM owners WHERE >> id=${pets.owner_id}"> >> <field column="id" name="id"/> >> <field column="name" name="name"/> >> <field column="birth_date" name="birthDate"/> >> </entity> >> </entity> >> </document> >> </dataConfig> >> >> The debug output of the data import looks as follows: >> >> .... >> - <lst name="verbose-output"> >> - <lst name="entity:owners"> >> - <lst name="document#1"> >> <str name="query">select id,first_name,last_name FROM owners</str> >> <str name="time-taken">0:0:0.15</str> >> <str>----------- row #1-------------</str> >> <int name="id">1</int> >> <str name="first_name">George</str> >> <str name="last_name">Franklin</str> >> <str>---------------------------------------------</str> >> - <lst name="transformer:TemplateTransformer"> >> <str>---------------------------------------------</str> >> <str name="id">owners-1</str> >> <str name="first_name">George</str> >> <str name="last_name">Franklin</str> >> <str>---------------------------------------------</str> >> - <lst name="entity:pets"> >> <str name="query">SELECT id,name,birth_date,type_id FROM >> pets WHERE owner_id='owners-1'</str> >> <str name="time-taken">0:0:0.0</str> >> </lst> >> </lst> >> </lst> >> + <lst name="document#1"> >> .... >> >> Thanks again >> >> ** julio >> >> >> -----Original Message----- >> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] >> Sent: Saturday, May 31, 2008 10:26 AM >> To: solr-user@lucene.apache.org >> Subject: Re: How to describe 2 entities in dataConfig for the DataImporter? >> >> Hi Julio, >> >> I've fixed the bug, can you please replace the exiting >> TemplateTransformer.java in the SOLR-469.patch and use the attached >> TemplateTransformer.java file. We'll add the changes to our next patch. >> Sorry for all the trouble. >> >> On Sat, May 31, 2008 at 10:31 PM, Noble Paul ??????? ?????? >> <[EMAIL PROTECTED]> wrote: >>> julio, >>> Looks like it is a bug. >>> We can give u a new TemplateTransformer.java which we will >>> incorporate in the next patch --Noble >>> >>> On Sat, May 31, 2008 at 12:24 AM, Julio Castillo >>> <[EMAIL PROTECTED]> wrote: >>>> I'm sorry Shalin, but I still get the same Null Pointer exception. >>>> This is my complete dataconfig.xml (I remove the parallel entity to >>>> narrow down the scope of the problem). >>>> <dataConfig> >>>> <document name="doc-1"> >>>> <entity name="vets" pk="id" >>>> query="select id as idAlias,first_name,last_name FROM vets" >>>> deltaQuery="SELECT id as idAlias FROM vets WHERE >>>> last_modified > '${dataimporter.last_index_time}'" >>>> transformer="TemplateTransformer"> >>>> <field column="id" name="id" >>>> template="vets-${vets.idAlias}"/> >>>> <field column="first_name" name="userName"/> >>>> <field column="last_name" name="userName"/> >>>> </entity> >>>> </document> >>>> </dataConfig> >>>> >>>> Thanks again. >>>> >>>> ** julio >>>> >>>> -----Original Message----- >>>> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] >>>> Sent: Friday, May 30, 2008 11:38 AM >>>> To: solr-user@lucene.apache.org >>>> Subject: Re: How to describe 2 entities in dataConfig for the >> DataImporter? >>>> >>>> The surname is used just as an example of a field. >>>> >>>> The NullPointerException is because the same field "id" tries to >>>> use it's own value in a template. The template cannot contain the >>>> same field on which it is being applied. I'd suggest that you get >>>> the id aliased to another name, for example using a query "select >>>> id as idAlias from vets" and then >>>> use: >>>> <field column="id" template="vets-${vets.idAlias}" /> >>>> >>>> That should work, let me know if you face a problem. >>>> >>>> On Fri, May 30, 2008 at 10:40 PM, Julio Castillo >>>> <[EMAIL PROTECTED]> >>>> wrote: >>>>> Thanks for all the leads. >>>>> I did get however a null pointer exception while implementing it: >>>>> >>>>> May 30, 2008 9:57:50 AM >>>>> org.apache.solr.handler.dataimport.EntityProcessorBase applyTransformer >>>>> WARNING: transformer threw error >>>>> java.lang.NullPointerException >>>>> at >>>>> org.apache.solr.handler.dataimport.TemplateTransformer.transformRow(Template Transformer.java:55) >>>>> at >>>>> org.apache.solr.handler.dataimport.EntityProcessorBase.applyTransformer(Enti tyProcessorBase.java:186) >>>>> >>>>> Looking at the source code, it appears that the resolverMap is null. >>>>> The resolver returned null given the entityName. >>>>> >>>>> >>>>> ** julio >>>>> >>>>> -----Original Message----- >>>>> From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED] >>>>> Sent: Thursday, May 29, 2008 11:10 PM >>>>> To: solr-user@lucene.apache.org >>>>> Subject: Re: How to describe 2 entities in dataConfig for the >>>> DataImporter? >>>>> >>>>> Sorry I forgot to mention that. >>>>> http://wiki.apache.org/solr/DataImportHandler#head-a6916b30b5d7605 >>>>> a9 >>>>> 90 >>>>> fb03c4 >>>>> ff461b3736496a9 >>>>> --Noble >>>>> >>>>> On Fri, May 30, 2008 at 11:37 AM, Shalin Shekhar Mangar >>>>> <[EMAIL PROTECTED]> wrote: >>>>>> You need to enable TemplateTransformer for your entity. For example: >>>>>> <entity name="owners" pk="id" query="...." >>>>>> transformer="TemplateTransformer"> >>>>>> >>>>>> On Fri, May 30, 2008 at 11:31 AM, Julio Castillo >>>>>> <[EMAIL PROTECTED]> wrote: >>>>>>> Noble, >>>>>>> I tried the template setting for the "id" field, but I didn't >>>>>>> notice any different behavior. I also didn't see where this >>>>>>> would be >> reflected. >>>>>>> I looked at the fields and the debug output for the dataImporter >>>>>>> and couldn't see any reference to a modified id name (per the >>>>>>> template instructions). >>>>>>> >>>>>>> The behavior in the end seemed to be the same. Did I miss anything? >>>>>>> I assume that the <uniqueKey>id</uniqueKey> setting in the >>>>>>> schema.xml remains the same? >>>>>>> >>>>>>> Thanks again >>>>>>> >>>>>>> ** julio >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED] >>>>>>> Sent: Thursday, May 29, 2008 9:46 PM >>>>>>> To: solr-user@lucene.apache.org >>>>>>> Subject: Re: How to describe 2 entities in dataConfig for the >>>>> DataImporter? >>>>>>> >>>>>>> Consider constructing the id concatenating an extra string for >>>>>>> each document . You can construct that field using the >> TeplateTransformer. >>>>>>> in the entity owners keep the id as >>>>>>> >>>>>>> <field column="id" name="id" template="owners-${owners.id}"/> >>>>>>> and in vets <field column="id" name="id" >>>>>>> template="vets-${vets.id}"/> >>>>>>> >>>>>>> or anything else which can make it unique >>>>>>> >>>>>>> --Noble >>>>>>> >>>>>>> On Fri, May 30, 2008 at 10:05 AM, Shalin Shekhar Mangar >>>>>>> <[EMAIL PROTECTED]> wrote: >>>>>>>> That will happen only if id is the uniqueKey in Solr and the id >>>>>>>> coming from both your tables have same values. In that case, >>>>>>>> they will overwrite each other. You will need a separate >>>>>>>> uniqueKey (on other than id field).
