It is strange that LogTransformer did not log the data. .
On Fri, Oct 16, 2009 at 5:54 PM, William Pierce <evalsi...@hotmail.com> wrote:
> Folks:
>
> Continuing my saga with DIH and use of its special commands. I have
> verified that the script functionality is indeed working. I also verified
> that '$skipRow' is working. But I don't think that '$deleteDocById' is
> working.
>
> My script now looks as follows:
>
> <script>
> <![CDATA[
> function DeleteRow(row) {
> var jid = row.get('Id');
> var jis = row.get('IndexingStatus');
> if ( jis == 4 ) {
> row.put('$deleteDocById', jid);
> row.remove('Col1');
> row.put('Col1', jid);
> }
> return row;
> }
> ]]>
> </script>
>
> The theory is that rows whose 'IndexingStatus' value is 4 should be deleted
> from solr index. Just to be sure that javascript syntax was correct and
> checked out, I intentionally overwrite a field called 'Col1' in my schema
> with primary key of the document to be deleted.
>
> On a clean and empty index, I import 47 rows from my dummy db. Everything
> checks out correctly since IndexingStatus for each row is 1. There are no
> rows to delete. I then go into the db and set one row with the
> IndexingStatus = 4. When I execute the dataimport, I find that all 47
> documents are imported correctly. However, for the row for which
> 'IndexingStatus' was set to 4, the Col1 value is set correctly by the
> script transformer to be the primary key value for that row/document.
> However, I should not be seeing that document since the '$deleteDocById
> should have deleted this from solr.
>
> Could this be a bug in solr? Or, am I misunderstanding how $deleteDocById
> works?
>
> By the way, Noble, I tried to set the LogTransformer, and add logging per
> your suggestion. That did not work either. I set logLevel="debug", and
> also turned on solr logging in the admin console to be the max value
> (finest) and still no output.
>
> Thanks,
>
> - Bill
>
>
>
> --------------------------------------------------
> From: "Noble Paul ??????? ??????" <noble.p...@corp.aol.com>
> Sent: Thursday, October 15, 2009 10:05 PM
> To: <solr-user@lucene.apache.org>
> Subject: Re: Using DIH's special commands....Help needed
>
>> use LogTransformer to see if the value is indeed set
>>
>> <entity name="post" transformer="script:DeleteRow,
>> RegexTransformer,LogTransformer"
>> logTemplate="${post}"
>> query=" select Id, a, b, c, IndexingStatus from prod_table
>> where (IndexingStatus = 1 or IndexingStatus = 4) ">
>>
>> this should print out the entire row after the transformations
>>
>>
>>
>> On Fri, Oct 16, 2009 at 3:04 AM, William Pierce <evalsi...@hotmail.com>
>> wrote:
>>>
>>> Thanks for your reply! I tried your suggestion. No luck. I have
>>> verified
>>> that I have version 1.6.0_05-b13 of java installed. I am running with
>>> the
>>> nightly bits of October 7. I am pretty much out of ideas at the present
>>> time....I'd appreciate any tips/pointers.
>>>
>>> Thanks,
>>>
>>> - Bill
>>>
>>> --------------------------------------------------
>>> From: "Shalin Shekhar Mangar" <shalinman...@gmail.com>
>>> Sent: Thursday, October 15, 2009 1:42 PM
>>> To: <solr-user@lucene.apache.org>
>>> Subject: Re: Using DIH's special commands....Help needed
>>>
>>>> On Fri, Oct 16, 2009 at 12:46 AM, William Pierce
>>>> <evalsi...@hotmail.com>wrote:
>>>>
>>>>> Thanks for your help. Here is my DIH config file....I'd appreciate any
>>>>> help/pointers you may give me. No matter what I do the documents are
>>>>> not
>>>>> getting deleted from the index. My db has rows whose 'IndexingStatus'
>>>>> field
>>>>> has values of either 1 (which means add it to solr), or 4 (which means
>>>>> delete the document with the primary key from SOLR index). I have two
>>>>> transformers running. Not sure what I am doing wrong.
>>>>>
>>>>> <dataConfig>
>>>>> <script><![CDATA[
>>>>> function DeleteRow(row) {
>>>>> var jis = row.get('IndexingStatus');
>>>>> var jid = row.get('Id');
>>>>> if ( jis == 4 ) {
>>>>> row.put('$deleteDocById', jid);
>>>>> }
>>>>> return row;
>>>>> }
>>>>> ]]></script>
>>>>>
>>>>> <dataSource type="JdbcDataSource"
>>>>> driver="com.mysql.jdbc.Driver"
>>>>> url="jdbc:mysql://localhost/db"
>>>>> user="******"
>>>>> password="*******"/>
>>>>> <document>
>>>>> <entity name="post" transformer="script:DeleteRow, RegexTransformer"
>>>>> query=" select Id, a, b, c, IndexingStatus from prod_table
>>>>> where (IndexingStatus = 1 or IndexingStatus = 4) ">
>>>>> <field column="ptype" splitBy="," sourceColName="a" />
>>>>> <field column="wauth" splitBy="," sourceColName="b" />
>>>>> <field column="miles" splitBy="," sourceColName="c" />
>>>>> </entity>
>>>>> </document>
>>>>> </dataConfig>
>>>>>
>>>>>
>>>> One thing I'd try is to use '4' for comparison rather than the number 4
>>>> (the
>>>> type would depend on the sql type). Also, for javascript transformers to
>>>> work, you must use JDK 6 which has javascript support. Rest looks fine
>>>> to
>>>> me.
>>>>
>>>> --
>>>> Regards,
>>>> Shalin Shekhar Mangar.
>>>>
>>>
>>
>>
>>
>> --
>> -----------------------------------------------------
>> Noble Paul | Principal Engineer| AOL | http://aol.com
>>
>
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com
