Hi Mark, I read through the EvaluateXPath.java yesterday. I have few (I guess newbie) questions. I hope you don't mind.
I have created the new EvaluateJSON.java file and started to write the parser. I am wondering what is the best way of recompiling the standard processors and try out the newly created feature. I did a mvn -T C2.0 -DskipTests=true install but the new file is not showing up in the menu when I drop the processor to the worksheet. I was wondering what makes Nifi to accept a new processor. Do I need to add anything else somewhere to let the system know about a new file like that? Thank you in advance. Best regards, Istvan On Wed, Feb 4, 2015 at 2:22 PM, Mark Payne <[email protected]> wrote: > Istvan, > I think this is a great use case for NiFi! The TransformXml processor > could be applicable, but I would actually guide you more toward the > EvaluateXPath. Specifically, I would break the problem into two processors: > EvaluateJsonPath and RouteOnAttribute. > EvaluateJsonPath would allow you to specify one or more "JsonPath" > expressions and put the results into user-defined attributes.Then, you can > route and filter on those attributes via the RouteOnAttribute processor. > I have actually considered doing something like this a few times, but I > know there are a few different "JsonPath" types of languages/libs out there > and wasn't sure which library made most sense to use. I know there is one > at https://github.com/jayway/JsonPath that has an Apache 2.0 License. I > would want to look a bit closer to make sure that all of its dependencies > are also Apache-compatible, but that may be a good starting point. > Then, the user would configure this processor by adding their own > properties. For instance, a property named "orderId" could have value > "/order/orderId". That would create an attribute on the FlowFile named > "orderId" whose value would be the result of evaluating the > "/order/orderId" JsonPath. Then RouteOnAttribute could route on that > attribute. > Does this make sense at all? > I would be happy to go into more detail or explain further with better > examples if it doesn't make sense. > Thanks-Mark > > > Date: Wed, 4 Feb 2015 14:08:22 -0800 > > Subject: Creating a custom processor for JSON documents > > From: [email protected] > > To: [email protected] > > > > Hi, > > > > I was wondering what would be the best way of doing this. The basic > > scenario is that we have a Kafka cluster that has lots of messages (JSON > > documents). I would like to go though all of those messages and parse > each > > individual message to extract fields from it, like if you are thinking > > about the HashMap representation of a JSON document I need a value of a > > certain key. I need some filtering capabilities as well, only process > > messages that has a certain key and the value of that key is configurable > > in the UI. > > > > I have seen that there are standard processors here: > > > > > https://github.com/apache/incubator-nifi/tree/develop/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard > > > > Which one should I use as a kind of example to do this? Am I on the right > > track that this is achievable with Nifi and I am not trying to do > something > > silly? > > > > It seems that TransformXml.java would be a good start, using it as an > > example to implement something like this. > > > > Thank you in advance, > > Istvan > > > > -- > > the sun shines for all > > -- the sun shines for all
