Hi, everyone.
After a week of hard work, I'd like to request your comments
on the draft of GLEP 74. This GLEP aims to replace the old tree-signing
GLEPs 58 and 60 with a superior implementation and more complete
specification.
The original tree-signing GLEPs were accepted a few years back but they
have never been implemented. This specification, on the other hand,
comes with a working reference implementation for the verification
algorithm. I expect to finish the update/generation part in a few days,
then work on additional optimizations (threading, incremental
verification, incremental updates).
ReST: https://dev.gentoo.org/~mgorny/tmp/glep-0074.rst
HTML: https://dev.gentoo.org/~mgorny/tmp/glep-0074.html
impl: https://github.com/mgorny/gemato/
Full text following for inline comments.
---
GLEP: 74
Title: Full-tree verification using Manifest files
Author: Michał Górny
Type: Standards Track
Status: Draft
Version: 1
Created: 2017-10-21
Last-Modified: 2017-10-26
Post-History: 2017-10-26
Content-Type: text/x-rst
Requires: 59, 61
Replaces: 44, 58, 60
---
Abstract
This GLEP extends the Manifest file format to cover full-tree file
integrity and authenticity checks.The format aims to be future-proof,
efficient and provide means of backwards compatibility.
Motivation
==
The Manifest files as defined by GLEP 44 [#GLEP44]_ provide the current
means of verifying the integrity of distfiles and package files
in Gentoo. Combined with OpenPGP signatures, they provide means to
ensure the authenticity of the covered files. However, as noted
in GLEP 57 [#GLEP57]_ they lack the ability to provide full-tree
authenticity verification as they do not cover any files outside
the package directory. In particular, they provide multiple ways
for a third party to inject malicious code into the ebuild environment.
Historically, the topic of providing authenticity coverage for the whole
repository has been mentioned multiple times. The most noteworthy effort
are GLEPs 58 [#GLEP58]_ and 60 [#GLEP60]_ by Robin H. Johnson from 2008.
They were accepted by the Council in 2010 but have never been
implemented. When potential implementation work started in 2017, a new
discussion about the specification arose. It prompted the creation
of a competing GLEP that would provide a redesigned alternative to
the old GLEPs.
This specification is designed with the following goals in mind:
1. It should provide means to ensure the authenticity of the complete
repository, including preventing the injection of additional files.
2. Alike the original Manifest2, the files should be split into two
groups — files whose authenticity is critical, and those whose
mismatch may be accepted in non-strict mode. The same classification
should apply both to files listed in Manifests, and to stray files
present only in the repository.
3. The format should be universal enough to work both for the Gentoo
repository and third-party repositories of different characteristics.
4. The Manifest files should be verifiable stand-alone, that is without
knowing any details about the underlying repository format.
Specification
=
Manifest file format
This specification reuses and extends the Manifest file format defined
in GLEP 44 [#GLEP44]_. For the purpose of it, the *file type* field is
repurposed as a generic *tag* that could also indicate additional
(non-checksum) metadata. Appropriately, those tags can be followed by
other space-separated values.
Unless specified otherwise, the paths used in the Manifest files
are relative to the directory containing the Manifest file. The paths
must not reference the parent directory (``..``).
Manifest file locations and nesting
---
The ``Manifest`` file located in the root directory of the repository
is called top-level Manifest, and it is used to perform the full-tree
verification. In order to verify the authenticity, it must be signed
using OpenPGP, using the armored cleartext format.
The top-level Manifest may reference sub-Manifests contained
in subdirectories of the repository. The sub-Manifests are traditionally
named ``Manifest``; however, the implementation must support arbitrary
names, including the possibility of multiple (split) Manifests
for a single directory. The sub-Manifest can only cover the files inside
the directory tree where it resides.
The sub-Manifest can also be signed using OpenPGP armored cleartext
format. However, the signature verification can be omitted if it is
covered by a signed top-level Manifest.
The Manifest files can also specify ``IGNORE`` entries to skip Manifest
verification of subdirectories and/or files. Files and directories
starting with a dot are always implicitly ignored. All files that
are not ignored must be covered by at least one of the Manifests.
A single file may be matched by multiple identical or equivalent
Manifest entries, if and only if the entries have the sa
