Thank you Bryan, Much appreciated!
I. On Wed, Feb 4, 2015 at 2:51 PM, Bryan Bende <[email protected]> wrote: > Not sure if this is helpful, but I recently attempted writing a processor > to rename and/or exclude fields from JSON documents. My thought being that > one system is producing documents that need to be slightly modified before > sending to another system. > > > https://github.com/bbende/nifi-example-bundles/blob/master/nifi-example-utils-bundle/nifi-example-utils-processors/src/main/java/org/apache/nifi/processors/example/utils/RenameJSONFields.java > > I'll put a big disclaimer on this and say that it hasn't really been tested > other than the unit tests, but it might be helpful in getting you started > writing your own EvaluateJsonPath processor. > > On Wed, Feb 4, 2015 at 5:25 PM, István <[email protected]> wrote: > > > Mark, > > > > This is exactly the feedback I was looking for. Thank you very much! I am > > trying to play with it for a while and get back if I have any questions. > > > > Regards, > > Istvan > > > > > > On Wed, Feb 4, 2015 at 2:22 PM, Mark Payne <[email protected]> wrote: > > > > > Istvan, > > > I think this is a great use case for NiFi! The TransformXml processor > > > could be applicable, but I would actually guide you more toward the > > > EvaluateXPath. Specifically, I would break the problem into two > > processors: > > > EvaluateJsonPath and RouteOnAttribute. > > > EvaluateJsonPath would allow you to specify one or more "JsonPath" > > > expressions and put the results into user-defined attributes.Then, you > > can > > > route and filter on those attributes via the RouteOnAttribute > processor. > > > I have actually considered doing something like this a few times, but I > > > know there are a few different "JsonPath" types of languages/libs out > > there > > > and wasn't sure which library made most sense to use. I know there is > > one > > > at https://github.com/jayway/JsonPath that has an Apache 2.0 License. > I > > > would want to look a bit closer to make sure that all of its > dependencies > > > are also Apache-compatible, but that may be a good starting point. > > > Then, the user would configure this processor by adding their own > > > properties. For instance, a property named "orderId" could have value > > > "/order/orderId". That would create an attribute on the FlowFile named > > > "orderId" whose value would be the result of evaluating the > > > "/order/orderId" JsonPath. Then RouteOnAttribute could route on that > > > attribute. > > > Does this make sense at all? > > > I would be happy to go into more detail or explain further with better > > > examples if it doesn't make sense. > > > Thanks-Mark > > > > > > > Date: Wed, 4 Feb 2015 14:08:22 -0800 > > > > Subject: Creating a custom processor for JSON documents > > > > From: [email protected] > > > > To: [email protected] > > > > > > > > Hi, > > > > > > > > I was wondering what would be the best way of doing this. The basic > > > > scenario is that we have a Kafka cluster that has lots of messages > > (JSON > > > > documents). I would like to go though all of those messages and parse > > > each > > > > individual message to extract fields from it, like if you are > thinking > > > > about the HashMap representation of a JSON document I need a value > of a > > > > certain key. I need some filtering capabilities as well, only process > > > > messages that has a certain key and the value of that key is > > configurable > > > > in the UI. > > > > > > > > I have seen that there are standard processors here: > > > > > > > > > > > > > > https://github.com/apache/incubator-nifi/tree/develop/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard > > > > > > > > Which one should I use as a kind of example to do this? Am I on the > > right > > > > track that this is achievable with Nifi and I am not trying to do > > > something > > > > silly? > > > > > > > > It seems that TransformXml.java would be a good start, using it as an > > > > example to implement something like this. > > > > > > > > Thank you in advance, > > > > Istvan > > > > > > > > -- > > > > the sun shines for all > > > > > > > > > > > > > > -- > > the sun shines for all > > > -- the sun shines for all
