Adding elfutils to OSS-Fuzz
Hi all Wanted to add elfutils to OSS-Fuzz. libelf is used in many distributions and seems like a good target to have some fuzz testing on https://github.com/google/oss-fuzz/pull/6670 Would love to hear your thoughts. Thanks!
[Bug debuginfod/28514] New: limit grooming time for many stale files
https://sourceware.org/bugzilla/show_bug.cgi?id=28514
Bug ID: 28514
Summary: limit grooming time for many stale files
Product: elfutils
Version: unspecified
Status: NEW
Severity: normal
Priority: P2
Component: debuginfod
Assignee: unassigned at sourceware dot org
Reporter: fche at redhat dot com
CC: elfutils-devel at sourceware dot org
Target Milestone: ---
When a big debuginfod server starts grooming, and starts finding stale data
(archives or files being removed), its self-cleaning efforts can take a long
time. It's been observed to take O(seconds) to do a single sqlite query. In
the metrics, see the sqlite3_milliseconds_count...{"nuke..."} ones. And the
groom() function will check every file for staleness, until interrupted by a
SIGUSR1, so that O(5) stale files could take a whole day.
During all this time, the server can service buildid requests, so it's not that
bad, but it cannot scan for new files.
We should investigate whether a more time-bounded groom operation could serve
about as well. We could limit groom to a certain percentage of time, like 1
hr/day, then abort. (We'd have to traverse the file list in some stateful or
random way in order not to just recheck the same ones over and over.) The
post-loop cleanup ops ("nuke orphan buildids" ... end of function) are
relatively quick and not worth worrying about at this time.
Alternately, there may be a way to accelerate the individual nuke queries,
maybe with more indexes, at the cost of more storage.
--
You are receiving this mail because:
You are on the CC list for the bug.
Re: Adding elfutils to OSS-Fuzz
Hi Murtaza, On Thu, Oct 28, 2021 at 09:47:40AM -0700, Murtaza Izzee via Elfutils-devel wrote: > Wanted to add elfutils to OSS-Fuzz. libelf is used in many distributions > and seems like a good target to have some fuzz testing on > > https://github.com/google/oss-fuzz/pull/6670 > > Would love to hear your thoughts. We have been using afl (American Fuzzy Lop) in an ad-hoc way to find issues. I have used OSS-Fuzz for some other projects, but found it very painful to get any results out if you don't have a google account. If you can set it up so that it posts the results and artifacts to the mailinglists that would be great. If possible I would try write something a bit more specific than just reusing eu-readelf. When using eu-readelf you'll basically first have to go through libdwfl and libdw initialization, there are various sanity checks that probably mean the fuzzer will not reach libelf for more interesting input files. Take a peek at some of the simpler elf tests (under test) if you want to really fuzz libelf itself. Maybe using elfcpy and then elfcmp to make sure the copy is really identical would make a fun fuzzcase. Cheers, Mark
Re: Adding elfutils to OSS-Fuzz
Hey Mark This is useful. I did have to filter out some of the debug parsing from the `libreadelf` I put together. I figured I could save some time by using some existing client of libelf. I will take a look at those simpler tests, which might make for a cleaner OSS-Fuzz target as well. I can work on getting the results published. I have a Google account so I can look into this. Thanks for the response Murtaza On Thu, Oct 28, 2021 at 12:34 PM Mark Wielaard wrote: > Hi Murtaza, > > On Thu, Oct 28, 2021 at 09:47:40AM -0700, Murtaza Izzee via Elfutils-devel > wrote: > > Wanted to add elfutils to OSS-Fuzz. libelf is used in many distributions > > and seems like a good target to have some fuzz testing on > > > > https://github.com/google/oss-fuzz/pull/6670 > > > > Would love to hear your thoughts. > > We have been using afl (American Fuzzy Lop) in an ad-hoc way to find > issues. I have used OSS-Fuzz for some other projects, but found it > very painful to get any results out if you don't have a google > account. If you can set it up so that it posts the results and > artifacts to the mailinglists that would be great. > > If possible I would try write something a bit more specific than just > reusing eu-readelf. When using eu-readelf you'll basically first have > to go through libdwfl and libdw initialization, there are various > sanity checks that probably mean the fuzzer will not reach libelf for > more interesting input files. Take a peek at some of the simpler elf > tests (under test) if you want to really fuzz libelf itself. Maybe > using elfcpy and then elfcmp to make sure the copy is really identical > would make a fun fuzzcase. > > Cheers, > > Mark > >
