The field 'id' is repeated for pet also rename it to something else
say
<entity name="pets" pk="id"
query="SELECT id,name,birth_date,type_id FROM pets WHERE
owner_id='${owners.id}'"
parentDeltaQuery="SELECT id FROM owners WHERE
id=${pets.owner_id}">
<field column="id" name="petid"/>
</entity>
--Noble
On Tue, Jun 3, 2008 at 3:28 AM, Julio Castillo <[EMAIL PROTECTED]> wrote:
> Shalin,
> I experimented with it, and the null pointer exception has been taken care
> of. Thank you.
>
> I have a different problem now. I believe it is a syntax/specification
> problem.
>
> When importing data, I got the following exceptions:
> SEVERE: Exception while adding:
> SolrInputDocumnt[{comboId=comboId(1.0)={owners-9},
> userName=userName(1.0)={[David, Schroeder]}}]
>
> org.apache.solr.common.SolrException: Document [null] missing required
> field: id
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
> at
> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHand
> ler.java:263)
> ...
>
> The problem arises the moment I try to include nested entities (e.g. pets
> -the problem does not occur if I don't use the transformer, but I have to
> use the transformer because other unrelated entities also have id's).
> My data config file looks as follows.
>
> <dataConfig>
> <document name="doc-1">
> <entity name="owners" pk="id"
> query="select id,first_name,last_name FROM owners"
> transformer="TemplateTransformer">
> <field column="id" name="comboId" template="owners-${owners.id}"/>
> <field column="first_name" name="userName"/>
> <field column="last_name" name="userName"/>
>
> <entity name="pets" pk="id"
> query="SELECT id,name,birth_date,type_id FROM pets WHERE
> owner_id='${owners.id}'"
> parentDeltaQuery="SELECT id FROM owners WHERE
> id=${pets.owner_id}">
> <field column="id" name="id"/>
> <field column="name" name="name"/>
> <field column="birth_date" name="birthDate"/>
> </entity>
> </entity>
> </document>
> </dataConfig>
>
> The debug output of the data import looks as follows:
>
> ....
> - <lst name="verbose-output">
> - <lst name="entity:owners">
> - <lst name="document#1">
> <str name="query">select id,first_name,last_name FROM owners</str>
> <str name="time-taken">0:0:0.15</str>
> <str>----------- row #1-------------</str>
> <int name="id">1</int>
> <str name="first_name">George</str>
> <str name="last_name">Franklin</str>
> <str>---------------------------------------------</str>
> - <lst name="transformer:TemplateTransformer">
> <str>---------------------------------------------</str>
> <str name="id">owners-1</str>
> <str name="first_name">George</str>
> <str name="last_name">Franklin</str>
> <str>---------------------------------------------</str>
> - <lst name="entity:pets">
> <str name="query">SELECT id,name,birth_date,type_id FROM pets
> WHERE owner_id='owners-1'</str>
> <str name="time-taken">0:0:0.0</str>
> </lst>
> </lst>
> </lst>
> + <lst name="document#1">
> ....
>
> Thanks again
>
> ** julio
>
>
> -----Original Message-----
> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
> Sent: Saturday, May 31, 2008 10:26 AM
> To: solr-user@lucene.apache.org
> Subject: Re: How to describe 2 entities in dataConfig for the DataImporter?
>
> Hi Julio,
>
> I've fixed the bug, can you please replace the exiting
> TemplateTransformer.java in the SOLR-469.patch and use the attached
> TemplateTransformer.java file. We'll add the changes to our next patch.
> Sorry for all the trouble.
>
> On Sat, May 31, 2008 at 10:31 PM, Noble Paul ??????? ??????
> <[EMAIL PROTECTED]> wrote:
>> julio,
>> Looks like it is a bug.
>> We can give u a new TemplateTransformer.java which we will incorporate
>> in the next patch --Noble
>>
>> On Sat, May 31, 2008 at 12:24 AM, Julio Castillo
>> <[EMAIL PROTECTED]> wrote:
>>> I'm sorry Shalin, but I still get the same Null Pointer exception.
>>> This is my complete dataconfig.xml (I remove the parallel entity to
>>> narrow down the scope of the problem).
>>> <dataConfig>
>>> <document name="doc-1">
>>> <entity name="vets" pk="id"
>>> query="select id as idAlias,first_name,last_name FROM vets"
>>> deltaQuery="SELECT id as idAlias FROM vets WHERE
>>> last_modified > '${dataimporter.last_index_time}'"
>>> transformer="TemplateTransformer">
>>> <field column="id" name="id"
>>> template="vets-${vets.idAlias}"/>
>>> <field column="first_name" name="userName"/>
>>> <field column="last_name" name="userName"/>
>>> </entity>
>>> </document>
>>> </dataConfig>
>>>
>>> Thanks again.
>>>
>>> ** julio
>>>
>>> -----Original Message-----
>>> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
>>> Sent: Friday, May 30, 2008 11:38 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: How to describe 2 entities in dataConfig for the
> DataImporter?
>>>
>>> The surname is used just as an example of a field.
>>>
>>> The NullPointerException is because the same field "id" tries to use
>>> it's own value in a template. The template cannot contain the same
>>> field on which it is being applied. I'd suggest that you get the id
>>> aliased to another name, for example using a query "select id as
>>> idAlias from vets" and then
>>> use:
>>> <field column="id" template="vets-${vets.idAlias}" />
>>>
>>> That should work, let me know if you face a problem.
>>>
>>> On Fri, May 30, 2008 at 10:40 PM, Julio Castillo
>>> <[EMAIL PROTECTED]>
>>> wrote:
>>>> Thanks for all the leads.
>>>> I did get however a null pointer exception while implementing it:
>>>>
>>>> May 30, 2008 9:57:50 AM
>>>> org.apache.solr.handler.dataimport.EntityProcessorBase
>>>> applyTransformer
>>>> WARNING: transformer threw error
>>>> java.lang.NullPointerException
>>>> at
>>>> org.apache.solr.handler.dataimport.TemplateTransformer.transformRow(
>>>> Te
>>>> mplate
>>>> Transformer.java:55)
>>>> at
>>>> org.apache.solr.handler.dataimport.EntityProcessorBase.applyTransfor
>>>> me
>>>> r(Enti
>>>> tyProcessorBase.java:186)
>>>>
>>>> Looking at the source code, it appears that the resolverMap is null.
>>>> The resolver returned null given the entityName.
>>>>
>>>> Looking at the documentation, there is the reference to a
> eparent.surname.
>>>> The example says:
>>>> <field column="namedesc" template="hello${e.name},${eparent.surname}"
>>>> />
>>>>
>>>> I'm afraid, I don't know what an eparent.surname is. This is my
>>>> current dataconfig.xml configuration excerpt:
>>>>
>>>> <entity name="vets" pk="id" query="..." deltaQuery="..."
>>>> transformer="TemplateTransformer">
>>>> <field column="id" name="id" template="vets-${vets.id}"/> ...
>>>>
>>>> Am I missing a surname? Whatever that may be?
>>>>
>>>> Thanks
>>>>
>>>> ** julio
>>>>
>>>> -----Original Message-----
>>>> From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED]
>>>> Sent: Thursday, May 29, 2008 11:10 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>>> DataImporter?
>>>>
>>>> Sorry I forgot to mention that.
>>>> http://wiki.apache.org/solr/DataImportHandler#head-a6916b30b5d7605a9
>>>> 90
>>>> fb03c4
>>>> ff461b3736496a9
>>>> --Noble
>>>>
>>>> On Fri, May 30, 2008 at 11:37 AM, Shalin Shekhar Mangar
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> You need to enable TemplateTransformer for your entity. For example:
>>>>> <entity name="owners" pk="id" query="...."
>>>>> transformer="TemplateTransformer">
>>>>>
>>>>> On Fri, May 30, 2008 at 11:31 AM, Julio Castillo
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> Noble,
>>>>>> I tried the template setting for the "id" field, but I didn't
>>>>>> notice any different behavior. I also didn't see where this would be
> reflected.
>>>>>> I looked at the fields and the debug output for the dataImporter
>>>>>> and couldn't see any reference to a modified id name (per the
>>>>>> template instructions).
>>>>>>
>>>>>> The behavior in the end seemed to be the same. Did I miss anything?
>>>>>> I assume that the <uniqueKey>id</uniqueKey> setting in the
>>>>>> schema.xml remains the same?
>>>>>>
>>>>>> Thanks again
>>>>>>
>>>>>> ** julio
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED]
>>>>>> Sent: Thursday, May 29, 2008 9:46 PM
>>>>>> To: solr-user@lucene.apache.org
>>>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>>>> DataImporter?
>>>>>>
>>>>>> Consider constructing the id concatenating an extra string for
>>>>>> each document . You can construct that field using the
> TeplateTransformer.
>>>>>> in the entity owners keep the id as
>>>>>>
>>>>>> <field column="id" name="id" template="owners-${owners.id}"/> and
>>>>>> in vets <field column="id" name="id" template="vets-${vets.id}"/>
>>>>>>
>>>>>> or anything else which can make it unique
>>>>>>
>>>>>> --Noble
>>>>>>
>>>>>> On Fri, May 30, 2008 at 10:05 AM, Shalin Shekhar Mangar
>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>> That will happen only if id is the uniqueKey in Solr and the id
>>>>>>> coming from both your tables have same values. In that case, they
>>>>>>> will overwrite each other. You will need a separate uniqueKey (on
>>>>>>> other than id field).
>>>>>>>
>>>>>>> On Fri, May 30, 2008 at 6:34 AM, Julio Castillo
>>>>>>> <[EMAIL PROTECTED]>
>>>>>> wrote:
>>>>>>>> Thanks Shalin,
>>>>>>>> I tried putting everything under the same document (two
>>>>>>>> different unrelated entities), and got a bit further.
>>>>>>>>
>>>>>>>> My problem now appears to be both of them stepping on each other
>>>>>>>> due to
>>>>>> "id"
>>>>>>>> conflicts. Currently my id is defined in my schema as <field
>>>>>>>> name="id" type="long" indexed="true" stored="true"
>>>>>>>> required="true"/>
>>>>>>>>
>>>>>>>> Do I have to create a new "id" field?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> ** julio
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
>>>>>>>> Sent: Thursday, May 29, 2008 11:40 AM
>>>>>>>> To: solr-user@lucene.apache.org
>>>>>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>>>>>> DataImporter?
>>>>>>>>
>>>>>>>> Hi Julio,
>>>>>>>>
>>>>>>>> The first data-config is correct.
>>>>>>>>
>>>>>>>> You're running DataImportHandler in debug mode which creates
>>>>>>>> only the first 10 documents by default. You can also add count=N
>>>>>>>> to index only the first N documents. But this is intended only
>>>>>>>> for debugging purposes. If you want to do a full-import just use
>>>>>>>> /dataimport?command=full-import. By default, a full-import
>>>>>>>> automatically
>>>>>> cleans and commits.
>>>>>>>>
>>>>>>>> Hope that helps.
>>>>>>>>
>>>>>>>> On Thu, May 29, 2008 at 11:42 PM, Julio Castillo
>>>>>>>> <[EMAIL PROTECTED]>
>>>>>>>> wrote:
>>>>>>>>> I have 2 dB tables unrelated to each other that I want to index.
>>>>>>>>>
>>>>>>>>> I have tried 2 approaches for specifying them in my
>>>>>>>>> data-config.xml
>>>>>> file.
>>>>>>>>> None of them seem to work (it seems I can only get data for the
>>>>>>>>> first one listed).
>>>>>>>>>
>>>>>>>>> CASE 1)
>>>>>>>>> <dataConfig>
>>>>>>>>> <document name="test">
>>>>>>>>> <entity name="owners" pk="id" query="....">
>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>> </entity>
>>>>>>>>> <entity name="vets" pk="id" query="....">
>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>> </entity>
>>>>>>>>> </document>
>>>>>>>>> </dataConfig>
>>>>>>>>>
>>>>>>>>> CASE 2)
>>>>>>>>> <dataConfig>
>>>>>>>>> <document name="test-1">
>>>>>>>>> <entity name="owners" pk="id" query="....">
>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>> </entity>
>>>>>>>>> </document>
>>>>>>>>> <document name="test-2">
>>>>>>>>> <entity name="vets" pk="id" query="....">
>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>> </entity>
>>>>>>>>> </document>
>>>>>>>>> </dataConfig>
>>>>>>>>>
>>>>>>>>> I ran the dataImporter as follows
>>>>>>>>> http://localhost:2455/solr/dataimport?command=full-import&debug
>>>>>>>>> =o
>>>>>>>>> n
>>>>>>>>> &v
>>>>>>>>> er
>>>>>>>>> bose=t
>>>>>>>>> rue&clean=true&commit=true
>>>>>>>>>
>>>>>>>>> After running Case 1 the output only shows documents under
>>>>>> entity:owners.
>>>>>>>>> And an empty reference to entity:vets (no data).
>>>>>>>>>
>>>>>>>>> After running Case 2, the output only shows documents under
>>>>>> entity:owners.
>>>>>>>>> No references to entity:vets.
>>>>>>>>>
>>>>>>>>> In either case, I have commented out the entity:owners and I do
>>>>>>>>> see the entity:vets data then.
>>>>>>>>>
>>>>>>>>> Thanks for your assistance
>>>>>>>>>
>>>>>>>>> Julio Castillo
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Shalin Shekhar Mangar.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Shalin Shekhar Mangar.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> --Noble Paul
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Shalin Shekhar Mangar.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --Noble Paul
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Shalin Shekhar Mangar.
>>>
>>>
>>
>>
>>
>> --
>> --Noble Paul
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
--
--Noble Paul
