hi Julio,
delete my previous response. In your schema , 'id' is the uniqueKey.
make 'comboid' the unique key. Because that is the target field name
coming out of the entity 'owners'
--Noble
On Tue, Jun 3, 2008 at 9:46 AM, Noble Paul നോബിള് नोब्ळ्
<[EMAIL PROTECTED]> wrote:
> The field 'id' is repeated for pet also rename it to something else
> say
> <entity name="pets" pk="id"
> query="SELECT id,name,birth_date,type_id FROM pets WHERE
> owner_id='${owners.id}'"
> parentDeltaQuery="SELECT id FROM owners WHERE
> id=${pets.owner_id}">
> <field column="id" name="petid"/>
> </entity>
>
> --Noble
>
> On Tue, Jun 3, 2008 at 3:28 AM, Julio Castillo <[EMAIL PROTECTED]> wrote:
>> Shalin,
>> I experimented with it, and the null pointer exception has been taken care
>> of. Thank you.
>>
>> I have a different problem now. I believe it is a syntax/specification
>> problem.
>>
>> When importing data, I got the following exceptions:
>> SEVERE: Exception while adding:
>> SolrInputDocumnt[{comboId=comboId(1.0)={owners-9},
>> userName=userName(1.0)={[David, Schroeder]}}]
>>
>> org.apache.solr.common.SolrException: Document [null] missing required
>> field: id
>> at
>> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
>> at
>> org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHand
>> ler.java:263)
>> ...
>>
>> The problem arises the moment I try to include nested entities (e.g. pets
>> -the problem does not occur if I don't use the transformer, but I have to
>> use the transformer because other unrelated entities also have id's).
>> My data config file looks as follows.
>>
>> <dataConfig>
>> <document name="doc-1">
>> <entity name="owners" pk="id"
>> query="select id,first_name,last_name FROM owners"
>> transformer="TemplateTransformer">
>> <field column="id" name="comboId" template="owners-${owners.id}"/>
>> <field column="first_name" name="userName"/>
>> <field column="last_name" name="userName"/>
>>
>> <entity name="pets" pk="id"
>> query="SELECT id,name,birth_date,type_id FROM pets WHERE
>> owner_id='${owners.id}'"
>> parentDeltaQuery="SELECT id FROM owners WHERE
>> id=${pets.owner_id}">
>> <field column="id" name="id"/>
>> <field column="name" name="name"/>
>> <field column="birth_date" name="birthDate"/>
>> </entity>
>> </entity>
>> </document>
>> </dataConfig>
>>
>> The debug output of the data import looks as follows:
>>
>> ....
>> - <lst name="verbose-output">
>> - <lst name="entity:owners">
>> - <lst name="document#1">
>> <str name="query">select id,first_name,last_name FROM owners</str>
>> <str name="time-taken">0:0:0.15</str>
>> <str>----------- row #1-------------</str>
>> <int name="id">1</int>
>> <str name="first_name">George</str>
>> <str name="last_name">Franklin</str>
>> <str>---------------------------------------------</str>
>> - <lst name="transformer:TemplateTransformer">
>> <str>---------------------------------------------</str>
>> <str name="id">owners-1</str>
>> <str name="first_name">George</str>
>> <str name="last_name">Franklin</str>
>> <str>---------------------------------------------</str>
>> - <lst name="entity:pets">
>> <str name="query">SELECT id,name,birth_date,type_id FROM pets
>> WHERE owner_id='owners-1'</str>
>> <str name="time-taken">0:0:0.0</str>
>> </lst>
>> </lst>
>> </lst>
>> + <lst name="document#1">
>> ....
>>
>> Thanks again
>>
>> ** julio
>>
>>
>> -----Original Message-----
>> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
>> Sent: Saturday, May 31, 2008 10:26 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: How to describe 2 entities in dataConfig for the DataImporter?
>>
>> Hi Julio,
>>
>> I've fixed the bug, can you please replace the exiting
>> TemplateTransformer.java in the SOLR-469.patch and use the attached
>> TemplateTransformer.java file. We'll add the changes to our next patch.
>> Sorry for all the trouble.
>>
>> On Sat, May 31, 2008 at 10:31 PM, Noble Paul ??????? ??????
>> <[EMAIL PROTECTED]> wrote:
>>> julio,
>>> Looks like it is a bug.
>>> We can give u a new TemplateTransformer.java which we will incorporate
>>> in the next patch --Noble
>>>
>>> On Sat, May 31, 2008 at 12:24 AM, Julio Castillo
>>> <[EMAIL PROTECTED]> wrote:
>>>> I'm sorry Shalin, but I still get the same Null Pointer exception.
>>>> This is my complete dataconfig.xml (I remove the parallel entity to
>>>> narrow down the scope of the problem).
>>>> <dataConfig>
>>>> <document name="doc-1">
>>>> <entity name="vets" pk="id"
>>>> query="select id as idAlias,first_name,last_name FROM vets"
>>>> deltaQuery="SELECT id as idAlias FROM vets WHERE
>>>> last_modified > '${dataimporter.last_index_time}'"
>>>> transformer="TemplateTransformer">
>>>> <field column="id" name="id"
>>>> template="vets-${vets.idAlias}"/>
>>>> <field column="first_name" name="userName"/>
>>>> <field column="last_name" name="userName"/>
>>>> </entity>
>>>> </document>
>>>> </dataConfig>
>>>>
>>>> Thanks again.
>>>>
>>>> ** julio
>>>>
>>>> -----Original Message-----
>>>> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
>>>> Sent: Friday, May 30, 2008 11:38 AM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>> DataImporter?
>>>>
>>>> The surname is used just as an example of a field.
>>>>
>>>> The NullPointerException is because the same field "id" tries to use
>>>> it's own value in a template. The template cannot contain the same
>>>> field on which it is being applied. I'd suggest that you get the id
>>>> aliased to another name, for example using a query "select id as
>>>> idAlias from vets" and then
>>>> use:
>>>> <field column="id" template="vets-${vets.idAlias}" />
>>>>
>>>> That should work, let me know if you face a problem.
>>>>
>>>> On Fri, May 30, 2008 at 10:40 PM, Julio Castillo
>>>> <[EMAIL PROTECTED]>
>>>> wrote:
>>>>> Thanks for all the leads.
>>>>> I did get however a null pointer exception while implementing it:
>>>>>
>>>>> May 30, 2008 9:57:50 AM
>>>>> org.apache.solr.handler.dataimport.EntityProcessorBase
>>>>> applyTransformer
>>>>> WARNING: transformer threw error
>>>>> java.lang.NullPointerException
>>>>> at
>>>>> org.apache.solr.handler.dataimport.TemplateTransformer.transformRow(
>>>>> Te
>>>>> mplate
>>>>> Transformer.java:55)
>>>>> at
>>>>> org.apache.solr.handler.dataimport.EntityProcessorBase.applyTransfor
>>>>> me
>>>>> r(Enti
>>>>> tyProcessorBase.java:186)
>>>>>
>>>>> Looking at the source code, it appears that the resolverMap is null.
>>>>> The resolver returned null given the entityName.
>>>>>
>>>>> Looking at the documentation, there is the reference to a
>> eparent.surname.
>>>>> The example says:
>>>>> <field column="namedesc" template="hello${e.name},${eparent.surname}"
>>>>> />
>>>>>
>>>>> I'm afraid, I don't know what an eparent.surname is. This is my
>>>>> current dataconfig.xml configuration excerpt:
>>>>>
>>>>> <entity name="vets" pk="id" query="..." deltaQuery="..."
>>>>> transformer="TemplateTransformer">
>>>>> <field column="id" name="id" template="vets-${vets.id}"/> ...
>>>>>
>>>>> Am I missing a surname? Whatever that may be?
>>>>>
>>>>> Thanks
>>>>>
>>>>> ** julio
>>>>>
>>>>> -----Original Message-----
>>>>> From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED]
>>>>> Sent: Thursday, May 29, 2008 11:10 PM
>>>>> To: solr-user@lucene.apache.org
>>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>>>> DataImporter?
>>>>>
>>>>> Sorry I forgot to mention that.
>>>>> http://wiki.apache.org/solr/DataImportHandler#head-a6916b30b5d7605a9
>>>>> 90
>>>>> fb03c4
>>>>> ff461b3736496a9
>>>>> --Noble
>>>>>
>>>>> On Fri, May 30, 2008 at 11:37 AM, Shalin Shekhar Mangar
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> You need to enable TemplateTransformer for your entity. For example:
>>>>>> <entity name="owners" pk="id" query="...."
>>>>>> transformer="TemplateTransformer">
>>>>>>
>>>>>> On Fri, May 30, 2008 at 11:31 AM, Julio Castillo
>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>> Noble,
>>>>>>> I tried the template setting for the "id" field, but I didn't
>>>>>>> notice any different behavior. I also didn't see where this would be
>> reflected.
>>>>>>> I looked at the fields and the debug output for the dataImporter
>>>>>>> and couldn't see any reference to a modified id name (per the
>>>>>>> template instructions).
>>>>>>>
>>>>>>> The behavior in the end seemed to be the same. Did I miss anything?
>>>>>>> I assume that the <uniqueKey>id</uniqueKey> setting in the
>>>>>>> schema.xml remains the same?
>>>>>>>
>>>>>>> Thanks again
>>>>>>>
>>>>>>> ** julio
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Noble Paul ??????? ?????? [mailto:[EMAIL PROTECTED]
>>>>>>> Sent: Thursday, May 29, 2008 9:46 PM
>>>>>>> To: solr-user@lucene.apache.org
>>>>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>>>>> DataImporter?
>>>>>>>
>>>>>>> Consider constructing the id concatenating an extra string for
>>>>>>> each document . You can construct that field using the
>> TeplateTransformer.
>>>>>>> in the entity owners keep the id as
>>>>>>>
>>>>>>> <field column="id" name="id" template="owners-${owners.id}"/> and
>>>>>>> in vets <field column="id" name="id" template="vets-${vets.id}"/>
>>>>>>>
>>>>>>> or anything else which can make it unique
>>>>>>>
>>>>>>> --Noble
>>>>>>>
>>>>>>> On Fri, May 30, 2008 at 10:05 AM, Shalin Shekhar Mangar
>>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>>> That will happen only if id is the uniqueKey in Solr and the id
>>>>>>>> coming from both your tables have same values. In that case, they
>>>>>>>> will overwrite each other. You will need a separate uniqueKey (on
>>>>>>>> other than id field).
>>>>>>>>
>>>>>>>> On Fri, May 30, 2008 at 6:34 AM, Julio Castillo
>>>>>>>> <[EMAIL PROTECTED]>
>>>>>>> wrote:
>>>>>>>>> Thanks Shalin,
>>>>>>>>> I tried putting everything under the same document (two
>>>>>>>>> different unrelated entities), and got a bit further.
>>>>>>>>>
>>>>>>>>> My problem now appears to be both of them stepping on each other
>>>>>>>>> due to
>>>>>>> "id"
>>>>>>>>> conflicts. Currently my id is defined in my schema as <field
>>>>>>>>> name="id" type="long" indexed="true" stored="true"
>>>>>>>>> required="true"/>
>>>>>>>>>
>>>>>>>>> Do I have to create a new "id" field?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> ** julio
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
>>>>>>>>> Sent: Thursday, May 29, 2008 11:40 AM
>>>>>>>>> To: solr-user@lucene.apache.org
>>>>>>>>> Subject: Re: How to describe 2 entities in dataConfig for the
>>>>>>> DataImporter?
>>>>>>>>>
>>>>>>>>> Hi Julio,
>>>>>>>>>
>>>>>>>>> The first data-config is correct.
>>>>>>>>>
>>>>>>>>> You're running DataImportHandler in debug mode which creates
>>>>>>>>> only the first 10 documents by default. You can also add count=N
>>>>>>>>> to index only the first N documents. But this is intended only
>>>>>>>>> for debugging purposes. If you want to do a full-import just use
>>>>>>>>> /dataimport?command=full-import. By default, a full-import
>>>>>>>>> automatically
>>>>>>> cleans and commits.
>>>>>>>>>
>>>>>>>>> Hope that helps.
>>>>>>>>>
>>>>>>>>> On Thu, May 29, 2008 at 11:42 PM, Julio Castillo
>>>>>>>>> <[EMAIL PROTECTED]>
>>>>>>>>> wrote:
>>>>>>>>>> I have 2 dB tables unrelated to each other that I want to index.
>>>>>>>>>>
>>>>>>>>>> I have tried 2 approaches for specifying them in my
>>>>>>>>>> data-config.xml
>>>>>>> file.
>>>>>>>>>> None of them seem to work (it seems I can only get data for the
>>>>>>>>>> first one listed).
>>>>>>>>>>
>>>>>>>>>> CASE 1)
>>>>>>>>>> <dataConfig>
>>>>>>>>>> <document name="test">
>>>>>>>>>> <entity name="owners" pk="id" query="....">
>>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>>> </entity>
>>>>>>>>>> <entity name="vets" pk="id" query="....">
>>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>>> </entity>
>>>>>>>>>> </document>
>>>>>>>>>> </dataConfig>
>>>>>>>>>>
>>>>>>>>>> CASE 2)
>>>>>>>>>> <dataConfig>
>>>>>>>>>> <document name="test-1">
>>>>>>>>>> <entity name="owners" pk="id" query="....">
>>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>>> </entity>
>>>>>>>>>> </document>
>>>>>>>>>> <document name="test-2">
>>>>>>>>>> <entity name="vets" pk="id" query="....">
>>>>>>>>>> <field column="id" name="id"/>
>>>>>>>>>> <field column=name" name="userName"/>
>>>>>>>>>> </entity>
>>>>>>>>>> </document>
>>>>>>>>>> </dataConfig>
>>>>>>>>>>
>>>>>>>>>> I ran the dataImporter as follows
>>>>>>>>>> http://localhost:2455/solr/dataimport?command=full-import&debug
>>>>>>>>>> =o
>>>>>>>>>> n
>>>>>>>>>> &v
>>>>>>>>>> er
>>>>>>>>>> bose=t
>>>>>>>>>> rue&clean=true&commit=true
>>>>>>>>>>
>>>>>>>>>> After running Case 1 the output only shows documents under
>>>>>>> entity:owners.
>>>>>>>>>> And an empty reference to entity:vets (no data).
>>>>>>>>>>
>>>>>>>>>> After running Case 2, the output only shows documents under
>>>>>>> entity:owners.
>>>>>>>>>> No references to entity:vets.
>>>>>>>>>>
>>>>>>>>>> In either case, I have commented out the entity:owners and I do
>>>>>>>>>> see the entity:vets data then.
>>>>>>>>>>
>>>>>>>>>> Thanks for your assistance
>>>>>>>>>>
>>>>>>>>>> Julio Castillo
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Shalin Shekhar Mangar.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Shalin Shekhar Mangar.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> --Noble Paul
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Shalin Shekhar Mangar.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --Noble Paul
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Shalin Shekhar Mangar.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --Noble Paul
>>>
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>>
>
>
>
> --
> --Noble Paul
>
--
--Noble Paul
