Attached is a substantially reworked draft. I've restructured the whole thing, fleshed it out in places, clarified some parts and incorporated the useful stuff from previous discussions.
Note: this is now GLEP 42 as allocated by Grant. AFAIK ChrisWhite's GLEP of the same number never made it to official status. Feedback from people who have something useful to say would be very much welcomed, assuming of course that they've read the GLEP. If you're going to rant on about XML or anything that assumes we have a static release Debian-style against which we somehow make 'corrections', you'll be ignored unless you come up with a very good justification based upon requirements and implementation rather than hand waving and incoherent rants. If you don't like it, you're welcome to write a competing GLEP. To give those who like arguing endlessly something else to go on about, the eselect news module is now upgraded to the 'suggested' tool for displaying news items. Look! I'm sneakily and evilly pushing a sekrit agenda here! Also, I murdered three puppies last night. Grant, please commit this to CVS. I'm too scared of docutils to do it myself. -- Ciaran McCreesh : Gentoo Developer (Anti-XML, anti-newbie conspiracy) Mail : ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm
GLEP: 42
Title: Critical News Reporting
Version: $Revision: $
Author: Ciaran McCreesh <[EMAIL PROTECTED]>
Last-Modified: $Date: $
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 31-October-2005
Post-Date: 1-November-2005, 5-November-2005
Abstract
========
This GLEP proposes a new way of informing users about important updates and news
regarding tree-related items.
Motivation
==========
There are currently several ways of getting news out to our users, none of them
particularly effective:
* Gentoo Weekly News
* The ``gentoo-announce`` mailing list
* The Gentoo Forums
* The main Gentoo website
* RSS feeds of Gentoo news
A more reliable way of getting news of critical updates out to users is required
to avoid repeats of the various recent upgrade debacles. This GLEP proposes a
solution based around pushing news items out to the user via the ``rsync`` tree.
Requirements
============
An adequate solution must meet all of the following requirements:
Preemptive
Users should be told of changes *before* they break the user's system,
after the damage has already been done.
No user subscription required
It has already been demonstrated [#forums-whining]_ that many users do not
read the ``gentoo-announce`` mailing list or ``RSS`` feeds. A solution which
requires subscription has no advantage over current methods.
No user monitoring required
It has already been demonstrated [#forums-whining]_ that many users do not
read news items posted to the Gentoo website, or do not read news items
until it is too late. A solution that relies upon active monitoring of a
particular source has no advantage over current methods.
Relevant
System administrators who do not use a particular package should not have to
read news items which affect purely that package. Some news items may be of
relevance to most or all users, but those that are not should not be forced
upon users unnecessarily.
Lightweight
It is not reasonable to expect all users to have an MTA, web browser, email
client, cron daemon or text processing suite available on their system.
No privacy violations
Users of the solution should not be required to provide information about
their systems (for example, IP addresses or installed packages).
Multiple delivery method support
Some users may wish to view news items via email, some via a terminal and
some via a web browser. A solution should either support all of these
methods or (better still) make it trivial to write clients for displaying
news items in different ways.
The following characteristics would be desirable:
Internationalisable
Being able to provide messages in multiple languages may be beneficial.
Quality control
There should be some way to ensure that badly written or irrelevant messages
are not sent out, for example by inexperienced developers, those whose
English language skills are below par or morons.
Simple for developers
Posting news items should be as simple as is reasonably possible.
Simple for users
Reading relevant news items should be as simple as is reasonably possible.
Compatibility with existing and future news sources
A news system would ideally be able to be integrated with existing news
sources (for example, Forums, GWN, the main Gentoo website) without
excessive difficulty. Similarly, easy interoperation with any future news
sources should not be precluded.
Specification
=============
Overview
--------
News items are published and delivered to users as follows:
1. A news item is written. The format to be used is described in
`News Item File Format`_.
2. The news item is reviewed, following the process described in
`News Item Quality Control`_.
3. The news item is committed to a CVS (or Subversion [#glep-36]_) repository.
From here, it is merged with the rsync tree. This is described in `News Item
Distribution`_.
4. The news item is merged with the rsync tree. Implementation details of this
point are beyond the scope of this GLEP; existing solutions are in place
for merging GLSAs to the tree.
5. Users fetch the news item when they sync. This ensures that the news items in
question are pushed to the user before the user accidentally makes an
unwanted change. No changes to the existing rsync process are required by
this GLEP.
6. Portage filters the news item and, if it is relevant, installs it in a
special location to be read by a news item reader. Messages are also
displayed to inform the user that news is available.
7. The news item is handled by the user's choice of news item reader. See `News
Item Clients`_.
News Item File Format
---------------------
Each news item will be represented by a single text file. This file will be
encoded using UTF-8 for compatibility with and for the same reason as existing
Gentoo documentation [#docs-policy]_ and tree [#glep-31]_ practices.
The news item will be named in the form ``yyyy-mm-dd-item-name.en.txt``, where
``item-name`` is a very short name (e.g. ``apache-updates``) and ``en`` is the
two letter ISO 639 [#iso-639]_ language code for the news item. The short name
must consist only of characters ``a-z``, ``A-Z``, ``0-9`` and ``-`` (hyphen).
News items may be signed using GPG. If this is done, a detached signature should
be used.
The directory and file name rules are designed specifically to allow easy
sorting by date.
An English (``en``) version must be available for all news items. Other
languages may be provided either by the original author or by other translators
who have commit access. This anglocentricity is justified on the grounds that
nobody objected to it with GLEP 34 [#glep-34]_.
A news item's content will consist of an RFC 822 [#rfc-822]_ style header
followed by the main body of the message as plain text. This GLEP defines
various optional and mandatory headers. Future GLEPs may propose new headers --
tools handling these news items must ignore any unrecognised header.
News Item Headers
'''''''''''''''''
The following headers describe the purpose and format of the news item:
``Title:``
A short (maximum 44 characters) descriptive title. Mandatory.
``Author:``
Author's name and email address, in the form ``Real Name <[EMAIL
PROTECTED]>``.
Mandatory, multiple author fields may be specified if appropriate.
``Translator:``
For translated news items, the translator's name and email address.
``Content-Type:``
Must be ``text/plain``. Mandatory.
``Posted:``
Date of posting, in ``dd-mmm-yyyy`` format (e.g. 14-Aug-2001). UTC time in
``hh-mm-ss +0000`` format may also be included. This field is mandatory.
``Version:``
Initially 1. Incremented every time a non-trivial change is made. Changes
which require a re-read of the news item should instead use a new news item
file.
``News-Item-Format:``
Must be ``1.0``. Future revisions to the format may increment the minor
number for backwards-compatible changes, or the major number for major
changes.
The following headers are used for filtering. If none of these headers are
specified, the news item is displayed for all users. Otherwise, the news item is
displayed if *at least one* header matches.
``Display-If-Installed:``
A dependency atom or simple package name (for example,
``<dev-lang/php-5_alpha`` or ``net-www/apache``). If the user has the
package specified installed, the news item should be displayed.
``Display-If-Keyword:``
A keyword [#glep-22]_ name, for example ``mips``. If the user is on the arch
in question, the news item should be displayed.
``Display-If-Profile:``
A profile path, for example ``default-linux/sparc/sparc64/server``. If the
user is using the exact profile in question, the news item should be
displayed. This header may be used to replace ``deprecated`` files in the
future.
News Item Body
''''''''''''''
The header section must be followed by a blank line, then the main body of the
text.
The text body should be wrapped at 72 characters. No fancy formatting or tab
characters should be used -- the news item may be being displayed directly to a
terminal. Paragraphs should be separated by two blank lines.
Hyperlinks may be used to refer to further information (for example, an upgrade
guide). However, the main body of the news item should be descriptive and not
simply a "read this link" text. It is assumed that the user will have access to
a web browser *somewhere*, but not necessarily on the box which is being
administrated -- this will be the case on may servers and routers, for example.
Example News Item
'''''''''''''''''
The following hypothetical news item could be used for an upgrade to the
``YourSQL`` database format which breaks forward compatibility. It should be
named ``2005-11/2005-11-01-yoursql-upgrades.en.txt``.
::
Title: YourSQL Upgrades from 4.0 to 4.1
Author: Ciaran McCreesh <[EMAIL PROTECTED]>
Content-Type: text/plain
Posted: 01-Nov-2005
Display-If-Installed: <dev-db/yoursql-4.1_alpha
YourSQL databases created using YourSQL version 4.0 are incompatible
with YourSQL version 4.1 or later. There is no reliable way to
automate the database format conversion, so action from the system
administrator is required before an upgrade can take place.
Please see the Gentoo YourSQL Upgrade Guide for instructions:
http://www.gentoo.org/doc/en/yoursql-upgrading.xml
Also see the official YourSQL documentation:
http://dev.yoursql.com/doc/refman/4.1/en/upgrading-from-4-0.html
After upgrading, you should also recompile any packages which link
against YourSQL:
revdep-rebuild --library=libyoursqlclient.so.12
The revdep-rebuild tool is provided by app-portage/gentoolkit.
News Item Quality Control
-------------------------
There have been complaints regarding the comprehensibility of some upgrade
notices and news items in the past. This is understandable -- not every Gentoo
developer speaks English as a first language. However, for the sake of clarity
and professionalism it is important that any language problems be corrected
before inflicting a news item upon end users.
Thus, all proposed news items must be posted to the ``gentoo-dev`` or
``gentoo-core`` mailing list, and ``Cc:``\ed to [EMAIL PROTECTED] at least 72
hours before being committed (exceptions may be made in exceptional
circumstances). Any complaints regarding wording or clarity **must** be
addressed before the news item goes live.
News items must only be for **important** changes that may cause serious upgrade
or compatibility problems. Ordinary upgrade messages and non-critical news items
should remain in ``einfo`` notices. The importance of the message to its
intended audience should be justified with the proposal.
.. Important:: The filtering system means that it is appropriate to send out
news items which are aimed at users of an uncommon package or architecture.
Thus, the justification should be in the form "this message is important to
YourSQL users because ...", not "YourSQL is important because ...".
News Item Distribution
----------------------
Server Side
'''''''''''
News items are to be made available via the Portage tree. This removes any
need for polling of a remote source.
A new repository will be created for news items. The type (CVS or Subversion),
location and access controls on this repository are beyond the scope of this
GLEP.
.. Note:: A previous draft of this GLEP instead used the main ``gentoo-x86``
tree. This was changed following advice from Infrastructure
[#ramereth-repo]_. Both solutions have the same end result.
This repository will contain directories named ``yyyy-mm/``, where ``yyyy`` is
the current year and ``mm`` is the current month number (01 for January through
12 for December). This separation will help keep news items more manageable.
The contents of this repository will automatically be merged with the main rsync
tree, placing the items under a top-level ``news/`` directory. The method used
for merging these items is beyond the scope of this GLEP -- a similar setup is
already used for merging GLSAs into the rsync tree.
.. Note:: The ``profiles/`` directory will *not* be used. This fits in with
the Portage team's future plans for ``profiles/`` and ``metadata/``
separation.
Client Side
'''''''''''
Whenever relevant unread news items are found, ``emerge`` will copy or symlink
the news file into ``/var/lib/gentoo/news/``.
Notification that new relevant news items will be displayed via the
``emerge`` tool in a similar way to the existing "configuration files need
updating" messages:
::
* Important: 3 config files in /etc need updating.
* Type emerge --help config to learn how to update config files.
* Important: there are 5 unread news items.
* Type emerge --help news to learn how to read news files.
The unread news message will also be displayed immediately after an
``emerge sync``.
Portage may also warn of unread news items using, for example, a red flashy
before actions such as merging a package.
Portage must keep track of news items which have already been installed to avoid
repeatedly reinstalling a deleted news item.
Users who really don't care about news items can use ``rsync_excludes`` to
filter out the ``news/`` directory.
News Item Clients
-----------------
Once a news item is 'installed', third party tools (or a traditional Unix pager
and ``rm``) can be used to display and view the news files. An ``eselect``
[#eselect]_ module shall be created as the 'suggested' display tool; other
display tools (for example, a news to email forwarder, which would be ideal for
users who sync on a cron) are left as options for those who desire them -- the
simple file format make this relatively simple.
News Item Removal
-----------------
News items can be removed (by removing the news file from the main tree) when
they are no longer relevant, if they are made obsolete by a future news item or
after a long period of time. This is the same as the method used for ``updates``
entries.
Integration with Existing Systems
=================================
It would be trivial to convert these news items into the format used for news
items on the Gentoo website or posts for the ``gentoo-announce`` mailing list.
There is an existing automated tool [#forums-glsa]_ for posting GLSAs to the
forums. A similar tool can be used for these news items.
Backwards Compatibility
=======================
Backwards compatibility is not a concern here. Existing tools will simply ignore
the ``news/`` directory.
Reference Implementation
========================
Portage Code
------------
TODO
Simple ``eselect`` News Client
------------------------------
A demonstration ``eselect`` news display script follows:
::
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id: $
DESCRIPTION="Read important Gentoo news items"
MAINTAINER="[EMAIL PROTECTED]"
SVN_DATE='$Date: $'
VERSION=$(svn_date_to_version "${SVN_DATE}" )
get_news_item_list() {
[[ -d "${ROOT}/var/lib/gentoo/news" ]] || return
(
local news
for news in "${ROOT}/var/lib/gentoo/news/"*.txt ; do
echo $(basename ${news%%.*} )
done
) | sort -u
}
get_filename() {
local best_lang=${LANG%%_*} lang= filename="${1}"
[[ -e "/${filename}" ]] || for lang in "${best_lang}" "en" ; do
filename="${ROOT}/var/lib/gentoo/news/${1}.${lang}.txt"
[[ -e "${filename}" ]] && break
done
[[ -e "${filename}" ]] && echo "${filename}" || die -q "Can't find
'${1}'"
}
get_all_filenames() {
if [[ -z "${1}" ]] ; then
echo "${ROOT}/var/lib/gentoo/news/"*.??".txt"
else
echo "${ROOT}/var/lib/gentoo/news/${1}."??".txt"
fi
}
get_headers() {
sed -e '/^$/Q' < $(get_filename "${1}" )
}
get_body() {
sed -e '1,/^$/d' < $(get_filename "${1}" )
}
get_title() {
get_headers "${1}" | sed -n -e '/^[Tt]itle: /s/^[^:]\+:[ \t]*//p'
}
### list action
## {{{ list stuff
describe_list() {
echo "List available news items"
}
do_list() {
write_list_start "Available news items:"
local empty=yes n=1
for news in $(get_news_item_list ) ; do
write_numbered_list_entry ${n} "$(highlight $(get_title ${news} ) )
(${news})"
n=$(( n + 1 ))
done
}
## }}}
## {{{ show stuff
describe_show(){
echo "Show a news item"
}
do_show() {
[[ -z "${1}" ]] && die -q "You didn't tell me which news item to read"
local target=${1} filename=
if is_number "${target}" && [[ ${target} -ge 1 ]] ; then
targets=( $(get_news_item_list ) )
target=${targets[$(( ${target} - 1 ))]}
fi
(
get_headers "${target}" | grep -v '^Display-If-\|Content-Type:'
echo
get_body "${target}"
) | ${PAGER:-cat}
}
## }}}
# {{{ delete stuff
describe_delete() {
echo "Delete a news item"
}
do_delete() {
[[ -z "${1}" ]] && die -q "You didn't tell me which news item to delete"
for target in $@ ; do
if is_number "${target}" && [[ ${target} -ge 1 ]] ; then
targets=( $(get_news_item_list ) )
target=${targets[$(( ${target} - 1 ))]}
fi
rm $(get_all_filenames "${target}" ) || die -q "Couldn't delete
${target}"
done
}
# }}}
# {{{ delete-all
describe_delete-all() {
echo "Delete all news items"
}
do_delete-all() {
rm $(get_all_filenames ) || die -q "Couldn't delete news items"
}
# }}}
Simple News to Mail Forwarder
-----------------------------
A demonstration shell script which delivers news items via email follows:
::
#!/bin/bash
to_address="[EMAIL PROTECTED]"
mail="mailx"
best_lang="${LANG%%_*}"
for news in /var/lib/gentoo/news/*.*.txt ; do
[[ -z "${best_lang}" ]] || news="${news%%.*}.${best_lang}.txt"
[[ -f "${news}" ]] || news="${news%%.*}.en.txt"
[[ -f "${news}" ]] || continue
title=$(sed -n -e '/^Title:/ { s,^[^:]\+:[ \t]\+,,p ; q }' < ${news} )
sed -e '1,/^$/ { /^Display-If\|Content-Type/d }' < "${news}" | \
${mail} -s "Gentoo news: ${title}" "${to_address}" && \
for file in ${news%%.*}.*.txt ; do rm "${file}" ; done
done
Credits
=======
The idea behind notifying users of news updates via Portage comes from Stuart
Herbert [#stuart-blog]_.
Thanks to Lance Albertson, Donnie Berkholz, Grant Goodyear, Brian Harring, Dan
Meltzer, Paul de Vrieze and Alec Warner for input. Some of the ideas presented
here are theirs, others go completely against their suggestions.
References
==========
.. [#docs-policy] Gentoo XML Guide, Daniel Robbins et al.,
http://www.gentoo.org/doc/en/xml-guide.xml
.. [#eselect] eselect modular framework for configuration and
administration utilities,
http://www.gentoo.org/proj/en/eselect/index.xml
.. [#forums-glsa] Forums user GLSA,
http://forums.gentoo.org/profile.php?mode=viewprofile&u=55648
.. [#forums-whining] Forums thread "Gentoo Apache2 Config Change Idiocy",
http://forums.gentoo.org/viewtopic-t-384368.html
.. [#glep-22] GLEP 22: "New "keyword" system to incorporate various
userlands/kernels/archs", Grant Goodyear,
http://www.gentoo.org/proj/en/glep/glep-0022.html
.. [#glep-31] GLEP 31: "Character Sets for Portage Tree Items", Ciaran
McCreesh,
http://www.gentoo.org/proj/en/glep/glep-0031.html
.. [#glep-34] GLEP 34: "Per-Category metadata.xml Files", Ciaran McCreesh,
http://www.gentoo.org/proj/en/glep/glep-0034.html
.. [#glep-36] GLEP 36: "Subversion/CVS for Gentoo Hosted Projects", Aaron
Walker,
http://www.gentoo.org/proj/en/glep/glep-0036.html
.. [#iso-639] ISO 639 "Code for the representation of names of languages"
.. [#ramereth-repo] "Re: [gentoo-dev] GLEP ??: Critical News Reporting", Lance
Albertson,
http://marc.theaimsgroup.com/?l=gentoo-dev&m=113111585907703&w=2
.. [#rfc-822] RFC 822 "Standard for the format of ARPA Internet text messages"
.. [#stuart-blog] "Favouring an automatic news mechanism", Stuart Herbert,
http://stu.gnqs.org/diary/gentoo.php/2005/10/28/favouring_an_automatic_news_mechanism
Copyright
=========
This document has been placed in the public domain.
.. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :
pgpZIXGs79KC1.pgp
Description: PGP signature
