rfc/patch: user-agent distro-description for debuginfod http traffic

[Frank Ch. Eigler]

Hi -

commit 53aac40e9577168b886dfeecb77e5a72d2666fc9 (HEAD -> 
fche/debuginfod-useragent, origin/fche/debuginfod-useragent)
Author: Frank Ch. Eigler 
Date:   Mon Jan 6 04:29:21 2020 -0500

debuginfod: pass a distro-summary User-Agent request header

It may be useful for a debuginfod server operator to know what kinds
of clients make webapi requests.  This is mainly as a
telemetry/diagnostic (though the data cannot be really trusted).  It
may also be useful to automate downloading of distro packages to a
debuginfod server in the case of an unknown hex buildid.  doc/testing
not affected as these are diagnostics.

diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 1582eba5bc0e..bed3ad7814ff 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -1,3 +1,10 @@
+2020-01-06  Frank Ch. Eigler  
+
+   * Makefile.am: Create a build-time header with lsb/uname info.
+   * debuginfod-client.c: Pass above as additional User-Agent request 
header.
+   * debuginfod.cxx (conninfo): Print this and X-Forwarded-For, as per
+   apache httpd reverse-proxy chain.
+
 2019-12-22  Frank Ch. Eigler  
 
* debuginfod.cxx (*_rpm_*): Rename to *_archive_* throughout.
diff --git a/debuginfod/Makefile.am b/debuginfod/Makefile.am
index 52ead30aebf8..20a70545755a 100644
--- a/debuginfod/Makefile.am
+++ b/debuginfod/Makefile.am
@@ -67,6 +67,20 @@ debuginfod_find_LDADD = $(libeu) $(libdebuginfod)
 noinst_LIBRARIES = libdebuginfod.a
 noinst_LIBRARIES += libdebuginfod_pic.a
 
+debuginfod-client-useragent.h:
+   if type uname 2>/dev/null; then \
+   echo '#define DEBUGINFOD_CLIENT_USERAGENT_1 "'`uname -sm | sed 
-e 's, ,/,g'`'"' > $@; \
+   else \
+   echo '#define DEBUGINFOD_CLIENT_USERAGENT_1 ""' > $@; \
+   fi
+   if type lsb_release 2>/dev/null; then \
+   echo '#define DEBUGINFOD_CLIENT_USERAGENT_2 "'`lsb_release -sir 
| sed -e 's, ,/,g'`'"' >> $@; \
+   else \
+   echo '#define DEBUGINFOD_CLIENT_USERAGENT_2 ""' > $@; \
+   fi
+   echo '#define DEBUGINFOD_CLIENT_USERAGENT DEBUGINFOD_CLIENT_USERAGENT_1 
"," DEBUGINFOD_CLIENT_USERAGENT_2' >> $@
+
+libdebuginfod: debuginfod-client-useragent.h
 libdebuginfod_a_SOURCES = debuginfod-client.c
 libdebuginfod_pic_a_SOURCES = debuginfod-client.c
 am_libdebuginfod_pic_a_OBJECTS = $(libdebuginfod_a_SOURCES:.c=.os)
@@ -98,7 +112,7 @@ uninstall: uninstall-am
 
 EXTRA_DIST = libdebuginfod.map
 MOSTLYCLEANFILES = $(am_libdebuginfod_pic_a_OBJECTS) 
libdebuginfod.so.$(VERSION)
-CLEANFILES += $(am_libdebuginfod_pic_a_OBJECTS) libdebuginfod.so
+CLEANFILES += $(am_libdebuginfod_pic_a_OBJECTS) libdebuginfod.so 
debuginfod-client-useragent.h
 
 # automake std-options override: arrange to pass LD_LIBRARY_PATH
 installcheck-binPROGRAMS: $(bin_PROGRAMS)
diff --git a/debuginfod/debuginfod-client.c b/debuginfod/debuginfod-client.c
index 9a4a0e0fc421..3c3610023e46 100644
--- a/debuginfod/debuginfod-client.c
+++ b/debuginfod/debuginfod-client.c
@@ -40,6 +40,7 @@
 
 #include "config.h"
 #include "debuginfod.h"
+#include "debuginfod-client-useragent.h"
 #include "system.h"
 #include 
 #include 
@@ -514,7 +515,7 @@ debuginfod_query_server (debuginfod_client *c,
   curl_easy_setopt(data[i].handle, CURLOPT_NOSIGNAL, (long) 1);
   curl_easy_setopt(data[i].handle, CURLOPT_AUTOREFERER, (long) 1);
   curl_easy_setopt(data[i].handle, CURLOPT_ACCEPT_ENCODING, "");
-  curl_easy_setopt(data[i].handle, CURLOPT_USERAGENT, (void*) 
PACKAGE_STRING);
+  curl_easy_setopt(data[i].handle, CURLOPT_USERAGENT, (void*) PACKAGE_NAME 
"/" PACKAGE_VERSION "," DEBUGINFOD_CLIENT_USERAGENT);
 
   curl_multi_add_handle(curlm, data[i].handle);
   server_url = strtok_r(NULL, url_delim, &strtok_saveptr);
diff --git a/debuginfod/debuginfod-find.c b/debuginfod/debuginfod-find.c
index 8bd3a3dba7a0..fc7c373c93d1 100644
--- a/debuginfod/debuginfod-find.c
+++ b/debuginfod/debuginfod-find.c
@@ -1,6 +1,6 @@
 /* Command-line frontend for retrieving ELF / DWARF / source files
from the debuginfod.
-   Copyright (C) 2019 Red Hat, Inc.
+   Copyright (C) 2019-2020 Red Hat, Inc.
This file is part of elfutils.
 
This file is free software; you can redistribute it and/or modify
diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 70cb95fecd65..2c1a77876677 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -1,5 +1,5 @@
 /* Debuginfo-over-http server.
-   Copyright (C) 2019 Red Hat, Inc.
+   Copyright (C) 2019-2020 Red Hat, Inc.
This file is part of elfutils.
 
This file is free software; you can redistribute it and/or modify
@@ -751,7 +751,12 @@ conninfo (struct MHD_Connection * conn)
 hostname[0] = servname[0] = '\0';
   }
 
-  return string(hostname) + string(":") + string(servname);
+  // extract headers relevant to administration
+  const char* user_agent = MHD_lookup_connection_value (conn, MHD_HEADER_KIND, 
"User-Agent")

Re: oss-fuzz

[Matthias Maennich via elfutils-devel]

Hi everyone!

On Thu, Dec 26, 2019 at 11:50:48PM +0100, Mark Wielaard wrote:

Hi Berkeley,

On Mon, Dec 23, 2019 at 08:06:54AM +0200, Berkeley Churchill wrote:

Great, thanks for the feedback!

One of my first tasks will be to support llvm/clang builds.  I've seen some
prior discussion on what's needed for that, but if you have any extra tips
I'll take them.  I'll be sure to create a build target for the fuzzers so
they can be run standalone.


clang is slightly inconvenient because it doesn't implement various
GNU C extensions. We even have a configure check for them now so it is
clear what we require from a C/gnu99 compiler:
https://sourceware.org/git/?p=elfutils.git;a=blob;f=configure.ac;hb=HEAD#l98

In theory when clang support that, everything should just compile.

There have been some attempts to rewrite some source code to get clang
to accept it:
https://sourceware.org/git/?p=elfutils.git&a=search&h=HEAD&st=commit&s=clang

But there is just too much code clang simply doesn't parse.

I don't know how much work there is left to get clang to accept
everything. But Matthias (CCed) said he got somewhat further on irc
once. Maybe he can share his patches.


The version of elfutils that I maintain is incomplete, but sufficient to
run libabigail on top of it and to compile everything with clang. So, my
local modifications can be summarized as:
- I use a different build system (similar to Bazel (https://bazel.build/)
- build a completely static version
- disable some warnings to build with -Werror otherwise
- Use a static config.h that is created with
   ./configure --without-lzma --without-bzlib and modified to
  - #undef HAVE_FALLTHROUGH
  - #undef HAVE_GCC_STRUCT

The source code itself is not modified for building with clang. Though I
adopted some patches from the DTS branch to build a version with builtin
ebl backends. A comparable series is merged upstream for the next
release as far as I can see.

The most tricky to address are the nested function definitions. It is
not entirely clear to me how to resolve these without completely messing
up the existing code. Most of them would end up as macro or factored out
function that takes a lot of arguments. I did not find the time to work
on a solution that is suitable upstream.



A simpler approach would be to see if oss-fuzz really needs clang at
all. As far as I know the only thing needed is a compiler supporting
inserting tracing calls into every basic block and/or comparison
operations and linking to some (C++) library that intercepts those. gcc
can do that with -fsanitize-coverage=trace-pc and/or
-fsanitize-coverage=trace-cmp (which I believe is command line
compatible with what clang uses).


I run a fuzzing project against libabigail which indirectly also fuzzes
some parts of elfutils (the subset I am compiling). So far, I did not
come across a finding in elfutils. But again, I am covering only a very
small subset of the code.

Cheers,
Matthias



Cheers,

Mark


On Mon, Dec 23, 2019 at 3:12 AM Mark Wielaard  wrote:

> Hi Berkeley,
>
> On Fri, 2019-12-20 at 17:21 +0200, Berkeley Churchill wrote:
> > Any interest in integrating with oss-fuzz?  It's a google project
> > that supports open source projects by fuzzing. It allows Google to
> > find and report bugs, especially security bugs, to the project.
> > I'm willing to work on writing fuzzers and performing the integration,
> > if this would be welcome by the maintainers.   Thoughts?
>
> Certainly interested. I have been running afl-fuzz on various utilities
> and test cases. That has found lots of issues. But it isn't very
> structured. And it often needs to go through a completely valid ELF
> file before fuzzing the more interesting data structures inside it.
>
> The only request I would have is that if the fuzzer targets are added
> to elfutils itself then they should also be made to work locally. So
> someone could also use them with e.g. afl-fuzz or some other fuzzing
> framework, or simply as extra testcase.
>
> Please also see:
> https://sourceware.org/git/?p=elfutils.git;f=CONTRIBUTING;hb=HEAD
>
> Cheers,
>
> Mark
>