@Mark, "Regarding the idea of a testing infrastructure that runs against a template, we also have to remember that if that template communicates with an external service, you will then have test failures (and build failures) when that external service is not available. This is probably not a good thing.". -> True, what I was planning to do was to mock those services. The template will not be pointing to the production endpoints, but to some mocked ones. Most of the interaction with external systems in my case are done through HTTP, so that is something easy to mock for my unit tests. I'm already doing something similar when testing a custom HTTP Post Processor I've implemented with TestRunner.
@Joe, thanks for the advice! I'll have it in mind. Regards, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Esteban Aliverti - Blog @ http://ilesteban.wordpress.com On Wed, Feb 25, 2015 at 2:58 AM, Joe Gresock <[email protected]> wrote: > For what it's worth, our team has developed an extensive set of integration > tests against a live instance using an "integration.test" flow file > attribute to route data to local disk in addition to the actual endpoints. > We're able to retrieve files from these directories and verify some basic > things about their filenames, contents, etc. > > This has proved invaluable and has caught many, many flow configuration > regression errors, as well as detected problems occurring in the external > systems. With Nifi as it currently is, I'd strongly recommend an approach > like this. > > Incidentally, the Nifi mock framework is quite powerful for simple > processor chaining in unit tests if you need it; the output of one > processor can be captured in a MockFlowFile, which can be the input to > another processor. > > Joe > > On Tue, Feb 24, 2015 at 11:31 AM, Mark Payne <[email protected]> wrote: > > > Joe, > > > > > > Regarding the idea of a testing infrastructure that runs against a > > template, we also have to remember that if that template communicates > with > > an external service, you will then have test failures (and build > failures) > > when that external service is not available. This is probably not a good > > thing. > > > > > > Thanks > > > > -Mark > > > > > > > > > > > > > > From: Joe Witt > > Sent: Tuesday, February 24, 2015 11:27 AM > > To: [email protected] > > > > > > > > > > > > Esteban, > > > > Understood. This is really a key strength of a flow-based programming > > approach like NiFi takes. Each component is a cohesive and loosely > > coupled entity. You should be able to produce very effective unit > > tests of those components against their well defined interface of > > inputs, outputs. Then in an actual dataflow one can freely compose > > those together in new and interesting ways provided it adheres to the > > purpose of those components. > > > > When creating unit tests that hit external services it is definitely a > > good idea to mock them out. The same is true when building any > > processor. You want to essentially mock input data and output data > > which adheres to or violates your preconditions and validate the > > output is as expected. > > > > Integration testing is still of value though of course. But I'd think > > the integration tests would be having your process actually hit a real > > instance of the external service. I think there is likely less value > > in an integration test of a preconfigured series of processors. At > > that point your testing the flow and not the processes themselves. If > > you do indeed want to test the behavior of a configured flow itself > > that i think is best done on a live instance. > > > > Having said all that I wonder if there is value in us enabling a > > developer to programatically generate templates and then providing an > > automated testing infrastructure around those templates. Will have to > > think about that. > > > > Thanks > > Joe > > > > On Tue, Feb 24, 2015 at 11:13 AM, Esteban Aliverti > > <[email protected]> wrote: > > > Thanks for the clarification. I'll try to see what other options do I > > have. > > > My scenario is not a simple ETL where everything gets resolved using > > local > > > data. The flow interacts with a bunch of external systems to both, > > retrieve > > > and send data. I was planning to use mocked services in those cases. > > > > > > Thanks for your time! > > > > > > Regards, > > > > > > > > > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > > > > > > Esteban Aliverti > > > - Blog @ http://ilesteban.wordpress.com > > > > > > On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[email protected]> > > wrote: > > > > > >> Esteban, > > >> > > >> > > >> > > >> > > >> OK, thanks for clarifying. This notion of how to test your flow has > > >> starting coming up a lot in the last week or so on the dev list. I’ll > > take > > >> this to be a good thing, meaning that people are really starting to > > think > > >> about how they can use it! 😊 > > >> > > >> > > >> > > >> > > >> We have actually intentionally avoided creating anything to allow > > >> developers to create unit/integration tests for flows. Creating these > > sorts > > >> of tests are very common for most ETL types of applications. However, > > NiFi > > >> was designed specifically so that there was no need to do so. The > > real-time > > >> command and control that NiFi affords you is very different than that > > >> provided by typical ETL applications. Because of this real-time > command > > and > > >> control, we push operators to build a “test" flow on the production > > >> instance and run the live data through the test flow at the same time > > that > > >> it is processed by the operational flow. In the test flow, though, the > > >> operator will likely terminate the data at the end, rather than > sending > > to > > >> any other system. This allows the operator to verify that everything > is > > in > > >> good shape with the test flow. Once it has been determined that > > everything > > >> is in good shape, the operator should then enable sending the data to > > the > > >> appropriate destinations. This tends to provide much faster > > time-to-market > > >> of flow changes. > > >> > > >> > > >> Over the past several days there has been another thread discussing > the > > >> notion of testing a flow. This thread can be viewed at > > >> > > > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811 > > >> > > >> > > >> If you have specific questions/concerns, I’d be happy to discuss in > more > > >> detail. > > >> > > >> > > >> Thanks > > >> > > >> -Mark > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> From: Esteban Aliverti > > >> Sent: Tuesday, February 24, 2015 10:52 AM > > >> To: [email protected] > > >> > > >> > > >> > > >> > > >> > > >> Thanks for the prompt response Mark. > > >> My idea was to have a NiFi workflow defined and to be able to run some > > >> automated tests on it. I thought that maybe there was a way to start > an > > >> embedded version of NiFi programatically and to interact with it. > > >> I did see the TestRunner class and I even used it for some of the unit > > >> tests of some custom processors I've implemented. TestRunner seems to > be > > >> exactly what I'm looking for, but instead of being focused in a single > > >> processor, I need something that allows me to run multiple processors > > that > > >> compose a flow. > > >> I do agree that each individual processor must be tested using a > > >> TestRunner, but I was planning on doing something more like an > > integration > > >> test of an entire flow. > > >> > > >> Regards, > > >> > > >> > > >> > > >> > > >> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > > >> > > >> Esteban Aliverti > > >> - Blog @ http://ilesteban.wordpress.com > > >> > > >> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[email protected]> > > wrote: > > >> > > >> > Esteban, > > >> > > > >> > > > >> > NiFi does not allow you to run it embedded. However, it does > provide a > > >> > “Mock Framework” that we use very heavily for doing unit testing of > > NiFi > > >> > components. > > >> > > > >> > > > >> > Can you explain your use case a bit further? What exactly are you > > trying > > >> > to unit test? > > >> > > > >> > > > >> > Thanks > > >> > > > >> > -Mark > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > From: Esteban Aliverti > > >> > Sent: Tuesday, February 24, 2015 8:34 AM > > >> > To: [email protected] > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > Hello, > > >> > My name is Esteban Aliverti and I'm currently trying to use NiFi in > > one > > >> of > > >> > my company's projects. The idea is to use NiFi behind a java API. > > >> > What I was wondering is if there's any way to run NiFi embedded so I > > can > > >> > easily build unit tests without having to start a separate NiFi > server > > >> > before I run the tests. Another important thing is that each test > > should > > >> be > > >> > self-contained and thus I would like them to always run in a "clean" > > NiFi > > >> > instance. > > >> > What I was thinking was a way to programatically configure and start > > NiFi > > >> > and maybe import a workflow from a template definition I can create > > >> > beforehand. > > >> > Is this currently possible? If not, any suggestion? > > >> > > > >> > Regards, > > >> > > > >> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > > >> > > > >> > Esteban Aliverti > > >> > - Blog @ http://ilesteban.wordpress.com > > >> > > > >> > > > > > > -- > I know what it is to be in need, and I know what it is to have plenty. I > have learned the secret of being content in any and every situation, > whether well fed or hungry, whether living in plenty or in want. I can do > all this through him who gives me strength. *-Philippians 4:12-13* >
