In general, I would recommend looking to the NiFi Developer Guide (http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html ) and the standard processors as your examples. These processors have all been used in production for quite a while.
From: Phillip G Sent: Wednesday, February 11, 2015 7:06 PM To: [email protected] Cc: Mark Payne Just for future use bryanbende.com who wrote the arche type has a post on developing processors, and there is another on Developing A Custom Apache Nifi Processor (JSON) - NiFi.rocks | | | | | | | | | | | bryanbende.comThis post will cover the initial steps to get started developing a custom processor for Apache NiFi. The focus will be on setting up a new processor bundle and and... | | | | View on bryanbende.com | Preview by Yahoo | | | | | | | | | | | | | | Developing A Custom Apache Nifi Processor (JSON) - NiFi.rocksHow to develop a custom Apache Nifi processor. The example in this post will utilize json, and json path | | | | View on www.nifi.rocks | Preview by Yahoo | | | | | On Wednesday, February 11, 2015 2:26 PM, István <[email protected]> wrote: Aldrin, I think this might be it. I forgot to do that. Thanks! On Wed, Feb 11, 2015 at 10:51 AM, Aldrin Piri <[email protected]> wrote: > Istvan, > > Have you included the processor class in the > org.apache.nifi.processor.Processor > file in the services directory? The accompanying reference in the > developer guide can be found at > > http://nifi.incubator.apache.org/docs/nifi-docs/developer-guide.html#processor_api > . > > > > On Wed, Feb 11, 2015 at 1:31 PM, István <[email protected]> wrote: > > > Hi Mark, > > > > I read through the EvaluateXPath.java yesterday. I have few (I guess > > newbie) questions. I hope you don't mind. > > > > I have created the new EvaluateJSON.java file and started to write the > > parser. I am wondering what is the best way of recompiling the standard > > processors and try out the newly created feature. I did a mvn -T C2.0 > > -DskipTests=true install but the new file is not showing up in the menu > > when I drop the processor to the worksheet. I was wondering what makes > Nifi > > to accept a new processor. Do I need to add anything else somewhere to > let > > the system know about a new file like that? > > > > Thank you in advance. > > > > Best regards, > > Istvan > > > > On Wed, Feb 4, 2015 at 2:22 PM, Mark Payne <[email protected]> wrote: > > > > > Istvan, > > > I think this is a great use case for NiFi! The TransformXml processor > > > could be applicable, but I would actually guide you more toward the > > > EvaluateXPath. Specifically, I would break the problem into two > > processors: > > > EvaluateJsonPath and RouteOnAttribute. > > > EvaluateJsonPath would allow you to specify one or more "JsonPath" > > > expressions and put the results into user-defined attributes.Then, you > > can > > > route and filter on those attributes via the RouteOnAttribute > processor. > > > I have actually considered doing something like this a few times, but I > > > know there are a few different "JsonPath" types of languages/libs out > > there > > > and wasn't sure which library made most sense to use. I know there is > > one > > > at https://github.com/jayway/JsonPath that has an Apache 2.0 License. > I > > > would want to look a bit closer to make sure that all of its > dependencies > > > are also Apache-compatible, but that may be a good starting point. > > > Then, the user would configure this processor by adding their own > > > properties. For instance, a property named "orderId" could have value > > > "/order/orderId". That would create an attribute on the FlowFile named > > > "orderId" whose value would be the result of evaluating the > > > "/order/orderId" JsonPath. Then RouteOnAttribute could route on that > > > attribute. > > > Does this make sense at all? > > > I would be happy to go into more detail or explain further with better > > > examples if it doesn't make sense. > > > Thanks-Mark > > > > > > > Date: Wed, 4 Feb 2015 14:08:22 -0800 > > > > Subject: Creating a custom processor for JSON documents > > > > From: [email protected] > > > > To: [email protected] > > > > > > > > Hi, > > > > > > > > I was wondering what would be the best way of doing this. The basic > > > > scenario is that we have a Kafka cluster that has lots of messages > > (JSON > > > > documents). I would like to go though all of those messages and parse > > > each > > > > individual message to extract fields from it, like if you are > thinking > > > > about the HashMap representation of a JSON document I need a value > of a > > > > certain key. I need some filtering capabilities as well, only process > > > > messages that has a certain key and the value of that key is > > configurable > > > > in the UI. > > > > > > > > I have seen that there are standard processors here: > > > > > > > > > > > > > > https://github.com/apache/incubator-nifi/tree/develop/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard > > > > > > > > Which one should I use as a kind of example to do this? Am I on the > > right > > > > track that this is achievable with Nifi and I am not trying to do > > > something > > > > silly? > > > > > > > > It seems that TransformXml.java would be a good start, using it as an > > > > example to implement something like this. > > > > > > > > Thank you in advance, > > > > Istvan > > > > > > > > -- > > > > the sun shines for all > > > > > > > > > > > > > > -- > > the sun shines for all > > > -- the sun shines for all
