Hi Naveen, Thanks for your feedback and suggestions. I have updated the document addressing the feedback and concerns, alternate solutions pros/cons are added. https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks
Best, Sandeep On Mon, May 13, 2019 at 1:20 PM Naveen Swamy <[email protected]> wrote: > Sandeep, > > Thanks for initiating work on individual operator performance. However I > find the proposed approach(ie., a separate libary/framework) to unnecessary > and increases maintenance overhead for the project. > Also, have you considered alternate approaches to achieve the same goal.? > > Many of the requirements/motivations you have mentioned typically should be > covered in unit-tests(different data-types/ different dimensions), so > instead of having to rewrite for all operators measuring performance, > consider writing a @timeit routine(using Python decorators) which can be > called on individual unit tests. Also even if you call the performance > script from Python, typically you want to measure as close to the kernel as > possible and avoid any other variables. > > I left some comments on the doc itself. > > Happy to discuss further. > > -Naveen > > > On Mon, Apr 29, 2019 at 1:57 PM sandeep krishnamurthy < > [email protected]> wrote: > > > Hello Community, > > > > I am currently working on building a utility/library to help us easily do > > individual operator benchmarking in MXNet. I have documented the proposal > > in > > this cwiki > > < > > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+Operator+Benchmarks > > >, > > and staging the current development in this github repository > > <https://github.com/sandeep-krishnamurthy/dl-operator-benchmark>. > Proposal > > is to get this library under incubator-mxnet/benchmark/ > > <https://github.com/apache/incubator-mxnet/tree/master/benchmark>. > Please > > do review and provide your feedback and suggestions. > > > > Thanks to fellow MXNet community members - Lin, Sam, Rohit for providing > > initial ideas and suggestion. > > > > Best, > > Sandeep > > > > > > > > > > -- > > Sandeep Krishnamurthy > > > -- Sandeep Krishnamurthy
