[
https://issues.apache.org/jira/browse/FLUME-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466170#comment-15466170
]
Siddharth Ahuja commented on FLUME-2966:
----------------------------------------
To provide a bit more context on this as requested, I raised this JIRA based on
my investigation of the NPE stacktrace encountered by one of the customers
running IBM Websphere MQ with Flume 1.5.0 as per below:
{code}
ERROR org.apache.flume.source.jms.JMSSource Unexpected error processing events
java.lang.NullPointerException
at
org.apache.flume.source.jms.DefaultJMSMessageConverter.convert(DefaultJMSMessageConverter.java:101)
at
org.apache.flume.source.jms.JMSMessageConsumer.take(JMSMessageConsumer.java:124)
at org.apache.flume.source.jms.JMSSource.doProcess(JMSSource.java:261)
at
org.apache.flume.source.AbstractPollableSource.process(AbstractPollableSource.java:58)
at
org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:137)
at java.lang.Thread.run(Thread.java:745)
{code}
Please find my analysis for the above as per below:
a) Based on code inspection at
https://github.com/apache/flume/blob/flume-1.5/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/DefaultJMSMessageConverter.java#L101,
event and textMessage cannot be NULL as a brand new event has been created
just before at
https://github.com/apache/flume/blob/flume-1.5/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/DefaultJMSMessageConverter.java#L74
and message has been used earlier at
https://github.com/apache/flume/blob/flume-1.5/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/DefaultJMSMessageConverter.java#L77.
Therefore, if any of them were null, we would have NPE'd with a different
stacktrace. So the only remaining possibility is that textMessage.getText() is
null.
b) To confirm if we can have code using MQ frameworks send null text
messages to Flume I got ActiveMQ going in Eclipse where I created a sample
producer and consumer based on http://activemq.apache.org/hello-world.html.
The Producer creates a text message (similar to this case) with a "null" text
string for the destination queue as per below:
{code}
…
// Create the destination (Topic or Queue)
Destination destination = session.createQueue("TEST.FOO");
// Create a MessageProducer from the Session to the Topic or
Queue
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
// Create a null text message
// String text = "Hello world! From: " +
Thread.currentThread().getName() + " : " + this.hashCode();
String text = null;
TextMessage message = session.createTextMessage(text);
…
{code}
When I run the ActiveMQ application with this Producer, there are no
errors/exceptions creating that text message with a null string. However, my
consumer fails with an NPE while trying to get bytes off the null text as per
below:
{code}
…
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
String text = textMessage.getText();
text.getBytes(Charset.defaultCharset()); <------------FAILS
here because text is NULL
System.out.println("Received: " + text);
} else {
System.out.println("Received: " + message);
}
…
{code}
The above test confirms that code using MQ frameworks can allow for a null text
message to be sent through, as such, we need to check for this possibility in
Flume so that these messages are ignored.
I have also added a new Junit test : testNullTextMessage() that will correctly
throw an NPE if my code from "DefaultJMSMessageConverter" in the patch is
removed further confirming the fix.
I have attached the sample ActiveMQ code used for my testing to the JIRA as
well.
Please let me know if this resolves your request [~bessbd] and if the latest
patch (FLUME-2966-1.patch) conforms to your expectation (I have removed the
trailing whitespaces and used assertEquals instead of assertTrue as per the
other test cases).
Thanks again!
> NULL text in a TextMessage from a JMS source in Flume can lead to NPE
> ---------------------------------------------------------------------
>
> Key: FLUME-2966
> URL: https://issues.apache.org/jira/browse/FLUME-2966
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.5.0
> Reporter: Siddharth Ahuja
> Assignee: Siddharth Ahuja
> Attachments: App.java, FLUME-2966-0.patch, FLUME-2966-1.patch
>
>
> Code at
> https://github.com/apache/flume/blob/trunk/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/DefaultJMSMessageConverter.java#L103
> does not check for a NULL text in a TextMessage from a Flume JMS source.
> This can lead to a NullPointerException here:
> {code}textMessage.getText().getBytes(charset){code} while trying to
> de-reference a null text from the textmessage.
> We should probably skip these like the NULL Objects in the ObjectMessage just
> below at:
> https://github.com/apache/flume/blob/trunk/flume-ng-sources/flume-jms-source/src/main/java/org/apache/flume/source/jms/DefaultJMSMessageConverter.java#L107.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
