On Mon, 03/12 18:17, Marc-André Lureau wrote: > Hi > > On Mon, Mar 12, 2018 at 6:11 PM, Fam Zheng <[email protected]> wrote: > > On Mon, 03/12 15:11, Marc-André Lureau wrote: > >> Hi Fam, > >> > >> It would be great if patchew could run the debug/sanitizer build. > >> Unfortunately, except x86 target, the run is full of ASAN leaks or > >> warnings. > >> > >> Do you think patchew could learn to do regression testing? By that I > >> mean that patchew could compare a patchset output with the current > >> master (that output should be cached for multiple tests). The output > >> to compare here would be the errors reported by ASAN during the build. > >> (eventually, this could be used for other metrics some day). If so, > >> would you be willing to help me implementing it? > > > > The question is can we simply rely on the exit code of cmp or diff for that > > kind > > of comparison? If there is any out-of-order texts, that simple diff won't > > work. > > If the tests are run sequentially, i think it should produce the same > output when run multiple times (to be verified).
In that case the test will last for a long time. Currently patchew isn't very good as multi-tasking. But that is one option we can explore (like we can devote one VM for asan test). > > However, if there are leaks already, and the address of allocation or > symbols change, this may produce diff on existing lines. Maybe we can come up with a 'make docker-test-debug-filtered@fedora' or a relatively simple 'make docker-test-debug@fedora | sed -e "..."' command that can be fed to diff. > > What I would like to see is the diff of added (or removed) lines. > > I suppose that should be possible, I'll do some experiments. > > > Anything more intelligent/sensible should go as a policy, and patchew should > > continue focus on mechanisms. > > hmm ok :) Like said above, I think it's possible to postprocess the output to make it easy to diff/cmp. By mechanism/policy I mean the place to draw a line to code Patchew. However patchew has a database where we can write test script snippets (as configurations) that is "the policy". Fam
