Re: Split Gecko in standalone fuzzing-friendly programs.
> To what extent are we fuzzing things like our url parser and other necko > bits via our existing shell (xpcshell) that exposes all that stuff? > > -Boris Not at all. xpcshell is not very useful for fuzzing. It is slow, the sample tests (xpcshell-tests) are extremely complicated to adapt and we can't easily use it with AFL. As discussed in #developers, we could maybe have xpcshell work with AFL persistent mode (although it would be complicated I assume), but that still doesn't solve the problem that people have to write the necessary code that we can fuzz then. xpcshell-tests are not that code (I did some fuzzing with an xpcshell-test when we added the new X509 validation code and it was horrible). Right now, the best approach to me seems having gtests and other compiled tests. The have the advantage that they are relatively simple to adapt (I was able to transform some tests image and mp3 decoding in less than 30 minutes) but they often still reflect what we do in the browser. For example for image decoding I saw we have multi-chunk decoding. If we just fuzz libjpeg or libpng we would just miss this part. As for speed, on my gtest I get 500-1000 iterations per second on the GIF decoder for example. That's a good speed for fuzzing and very useful. The only thing that I haven't looked into yet is how I can package the resulting tests easily so they can be sent to fuzzing machines, but that's solvable. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Split Gecko in standalone fuzzing-friendly programs.
> > > the sample tests (xpcshell-tests) are extremely complicated to adapt > > That seems like it would be a problem in any new thing too, right? Actually no. I adapted our gtests in less than an hour. > > > and we can't easily use it with AFL. > > Just to satisfy my curiosity, what is AFL? http://lcamtuf.coredump.cx/afl/ > > > but that still doesn't solve the problem that people have to write the > > necessary code that we can fuzz then. > > OK. This is a problem, certainly, and pretty independent of both the > "split Gecko" thing and the existence of shells, right? Not really no. Because some shells and tests we have are very straightforward to use and we can figure it out ourselves. xpcshell is not such an example. > > What are the necessary qualities for things you can fuzz? It depends on the type of fuzzing. Let's stick to AFL: - Program is easy to start (doesn't need profiles or long initialization) and can be packaged - Has AFL persistent mode support (requires support on C++ level) - Exercises the targeted feature in a similar way compared to how Firefox would do it - Optionally has some extra testing features (e.g. gczeal, ion-eager, extra-checks for the JS shell) that make bug finding easier - Can be compiled with all sanitizer types (although MSan is not going to work for some stuff even in shells) That's just a dump out of my head, might be missing some stuff. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
