On Fri, Feb 19, 2016 at 11:10 AM, Eugene Zelenko <eugene.zele...@gmail.com> wrote:
> On Fri, Feb 19, 2016 at 11:06 AM, David Blaikie <dblai...@gmail.com> > wrote: > > > > Could we take this conversation back to the list? (better to discuss > things > > with everyone) > > > > On Fri, Feb 19, 2016 at 11:02 AM, Eugene Zelenko < > eugene.zele...@gmail.com> > > wrote: > >> > >> On Fri, Feb 19, 2016 at 10:54 AM, David Blaikie <dblai...@gmail.com> > >> wrote: > >> > Ah, I don't think that's the case/not the testing I'm suggesting - I > >> > think > >> > the std::set example is a good one to follow (& while it's laudible to > >> > add > >> > missing coverage, it's not a requirement & shouldn't be an impediment > to > >> > adding /some/ testing, especially for any new code/changes) > >> > > >> > You could stamp out the types with a macro to save some code (but I > >> > imagine > >> > the /usage/ would need to be separate, for the fixits to work > properly, > >> > etc) > >> > >> Sorry, I'm not expert in testing, but anything that doesn't test real > >> implementation is flawed by design for me. This is my conclusion after > >> I discovered that container check didn't work with own STL > >> implementation. > > > > > > It's a tradeoff, to be sure - but we can't test with every STL > > implementation, and even testing with one may add a lot of overhead to > > tests. The same is true for other aspects of the LLVM project testing - > in > > our regression suite we don't test LLVM with Clang, we test LLVM and > Clang > > each in isolation. This ensures the testing is targeted and fast, but can > > lead to divergence on the boundaries (testing that Clang produces A, but > > testing that LLVM handles B, instead of A). > > > > - David > > Sure, it's not possible to test with every STL implementation, but > it's reasonable to test with implementation used for particular build. > If LLVM/Clang could be built with it, why not Clang-tidy regressions? > They can, but it's a tradeoff as I said - makes the test much broader than is necessarily ideal. (this is the usual tradeoff between unit/feature/integration testing) I wouldn't be opposed to integration testing, but it wouldn't generally be run under "make check" (instead as part of some larger feature testing mode run on buildbots, etc) and wouldn't be a substitute for these more targeted unit-like tests.
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
