Lance, Noble:
I set logLevel="debug" in my dihconfig.xml at the entity level. Got no
output! I then gave up digging into this further because I was pressed for
time to dig into how to increase the speed of importing into solr with
dih...
Cheers,
- Bill
--------------------------------------------------
From: "Noble Paul നോബിള് नोब्ळ्" <noble.p...@corp.aol.com>
Sent: Monday, October 19, 2009 1:05 AM
To: <solr-user@lucene.apache.org>
Subject: Re: Using DIH's special commands....Help needed
The accepted logLevel values are
error, deubug,warn,trace,info
2009/10/18 Noble Paul നോബിള് नोब्ळ् <noble.p...@corp.aol.com>:
On Sun, Oct 18, 2009 at 4:16 AM, Lance Norskog <goks...@gmail.com> wrote:
I had this problem also, but I was using the Jetty exampl. I fail at
logging configurations about 90% of the time, so I assumed it was my
fault.
did you set the logLevel atribute also in the entity? if you set
logLevel="severe" it should definitely be printed
2009/10/17 Noble Paul നോബിള് नोब्ळ् <noble.p...@corp.aol.com>:
It is strange that LogTransformer did not log the data. .
On Fri, Oct 16, 2009 at 5:54 PM, William Pierce <evalsi...@hotmail.com>
wrote:
Folks:
Continuing my saga with DIH and use of its special commands. I have
verified that the script functionality is indeed working. I also
verified
that '$skipRow' is working. But I don't think that '$deleteDocById'
is
working.
My script now looks as follows:
<script>
<![CDATA[
function DeleteRow(row) {
var jid = row.get('Id');
var jis = row.get('IndexingStatus');
if ( jis == 4 ) {
row.put('$deleteDocById', jid);
row.remove('Col1');
row.put('Col1', jid);
}
return row;
}
]]>
</script>
The theory is that rows whose 'IndexingStatus' value is 4 should be
deleted
from solr index. Just to be sure that javascript syntax was correct
and
checked out, I intentionally overwrite a field called 'Col1' in my
schema
with primary key of the document to be deleted.
On a clean and empty index, I import 47 rows from my dummy db.
Everything
checks out correctly since IndexingStatus for each row is 1. There
are no
rows to delete. I then go into the db and set one row with the
IndexingStatus = 4. When I execute the dataimport, I find that all
47
documents are imported correctly. However, for the row for which
'IndexingStatus' was set to 4, the Col1 value is set correctly by the
script transformer to be the primary key value for that row/document.
However, I should not be seeing that document since the
'$deleteDocById
should have deleted this from solr.
Could this be a bug in solr? Or, am I misunderstanding how
$deleteDocById
works?
By the way, Noble, I tried to set the LogTransformer, and add logging
per
your suggestion. That did not work either. I set logLevel="debug",
and
also turned on solr logging in the admin console to be the max value
(finest) and still no output.
Thanks,
- Bill
--------------------------------------------------
From: "Noble Paul ??????? ??????" <noble.p...@corp.aol.com>
Sent: Thursday, October 15, 2009 10:05 PM
To: <solr-user@lucene.apache.org>
Subject: Re: Using DIH's special commands....Help needed
use LogTransformer to see if the value is indeed set
<entity name="post" transformer="script:DeleteRow,
RegexTransformer,LogTransformer"
logTemplate="${post}"
query=" select Id, a, b, c, IndexingStatus from prod_table
where (IndexingStatus = 1 or IndexingStatus = 4) ">
this should print out the entire row after the transformations
On Fri, Oct 16, 2009 at 3:04 AM, William Pierce
<evalsi...@hotmail.com>
wrote:
Thanks for your reply! I tried your suggestion. No luck. I have
verified
that I have version 1.6.0_05-b13 of java installed. I am running
with
the
nightly bits of October 7. I am pretty much out of ideas at the
present
time....I'd appreciate any tips/pointers.
Thanks,
- Bill
--------------------------------------------------
From: "Shalin Shekhar Mangar" <shalinman...@gmail.com>
Sent: Thursday, October 15, 2009 1:42 PM
To: <solr-user@lucene.apache.org>
Subject: Re: Using DIH's special commands....Help needed
On Fri, Oct 16, 2009 at 12:46 AM, William Pierce
<evalsi...@hotmail.com>wrote:
Thanks for your help. Here is my DIH config file....I'd
appreciate any
help/pointers you may give me. No matter what I do the documents
are
not
getting deleted from the index. My db has rows whose
'IndexingStatus'
field
has values of either 1 (which means add it to solr), or 4 (which
means
delete the document with the primary key from SOLR index). I have
two
transformers running. Not sure what I am doing wrong.
<dataConfig>
<script><![CDATA[
function DeleteRow(row) {
var jis = row.get('IndexingStatus');
var jid = row.get('Id');
if ( jis == 4 ) {
row.put('$deleteDocById', jid);
}
return row;
}
]]></script>
<dataSource type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/db"
user="******"
password="*******"/>
<document>
<entity name="post" transformer="script:DeleteRow,
RegexTransformer"
query=" select Id, a, b, c, IndexingStatus from
prod_table
where (IndexingStatus = 1 or IndexingStatus = 4) ">
<field column="ptype" splitBy="," sourceColName="a" />
<field column="wauth" splitBy="," sourceColName="b" />
<field column="miles" splitBy="," sourceColName="c" />
</entity>
</document>
</dataConfig>
One thing I'd try is to use '4' for comparison rather than the
number 4
(the
type would depend on the sql type). Also, for javascript
transformers to
work, you must use JDK 6 which has javascript support. Rest looks
fine
to
me.
--
Regards,
Shalin Shekhar Mangar.
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com
--
Lance Norskog
goks...@gmail.com
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com
