Hi! Ok, here's the updated patch series for the conversion omitting the actual conversion commit. Attached and in the pu/markup-singularity branch at <https://git.hadrons.org/cgit/debian/policy.git/>.
A summary of the current status: * Switched from OpenJade to xsltproc, and added xmllint. * The Abstract now has a heading again (!). * The License and Copyright information although w/o heading are more distinguishible now (in all output formats). * Turned the upgrading-checklist into a proper appendix. * The standalone upgrading-checklist outputs have no TOC anymore, to preserve compatibility and avoid clutter, but perhaps we want them? * Removed the tidy(1) usage, see the commit message, but left the first fix in, because I'm not sure if you agree with this or not, otherwise the first commit could be removed. * Reordered the commits so that the conversion script makes more sense now on its own. * Do not forget to remove the recent ghostscript entry from the changelog, as it's now gone! * Commit 6/8 can be found in the git repo, but it should be reproducible by running the conversion script. So, the HTML and txt outputs look pretty good to me (I think even better than before). The PDF/PS output looks in general way better than the current ones, but they have some issues mostly with the heading/preface/etc that need to be addressed, and that will need LaTeX tunning I think: * The release information is not present in the PDF/PS output (the current version). This only gets displayed with draft.mode=yes, and in the wrong place anyway. :/ * The authors information is (IMO) attrociously presented (this at least includes the release date). This can be very easily disabled for now though with doc.collab.show=0. At least this information is somewhat present already in ยง1.3. Thanks, Guillem
From ee07a4c3c8fcbcc7c05c7cec7f97b6e6690cf54a Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Sun, 26 Mar 2017 20:24:16 +0200 Subject: [PATCH 1/8] Make tidy not convert character entities to their UTF-8 form The output documents do not have any encoding specified, so programs handling them will assume ASCII, and when finding UTF-8 characters will recode those, producing garbage on output. --- copyright-format/Makefile | 2 +- debconf_spec/Makefile | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/copyright-format/Makefile b/copyright-format/Makefile index 6d26018..31d6e7a 100644 --- a/copyright-format/Makefile +++ b/copyright-format/Makefile @@ -3,7 +3,7 @@ all: copyright-format-1.0.txt.gz copyright-format-1.0.html copyright-format-1.0.html: copyright-format-1.0.xml html.dsl openjade -V nochunks -t sgml -d html.dsl \ /usr/share/xml/declaration/xml.dcl $< > $@ - -tidy -q -i -m -f /dev/null $@ + -tidy -raw -q -i -m -f /dev/null $@ copyright-format-1.0.txt: copyright-format-1.0.html links -dump $< | perl -pe 's/[\r\0]//g' > $@ diff --git a/debconf_spec/Makefile b/debconf_spec/Makefile index cdfd135..bc6614d 100644 --- a/debconf_spec/Makefile +++ b/debconf_spec/Makefile @@ -3,7 +3,7 @@ all: debconf_specification.txt.gz debconf_specification.html %.html: %.xml html.dsl openjade -V nochunks -t sgml -d html.dsl \ /usr/share/xml/declaration/xml.dcl $< > $@ - -tidy -q -i -m -f /dev/null $@ + -tidy -raw -q -i -m -f /dev/null $@ %.txt: %.html links -dump $< | perl -pe 's/[\r\0]//g' > $@ -- 2.12.2.564.g063fe858b8
From eb3cb1a4943934bf6f38035ad93301213799146a Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Mon, 9 Jan 2017 05:22:59 +0100 Subject: [PATCH 2/8] Do not ship redundant SGML and XML sources We already ship the sources in the source package, and there's nothing requiring us to duplicate these files. --- debian-menu-policy.desc | 3 --- debian-perl-policy.desc | 3 --- debian-policy.desc | 3 --- debian/rules | 24 ++++++------------------ 4 files changed, 6 insertions(+), 27 deletions(-) diff --git a/debian-menu-policy.desc b/debian-menu-policy.desc index 9757253..a71e36b 100644 --- a/debian-menu-policy.desc +++ b/debian-menu-policy.desc @@ -6,9 +6,6 @@ Abstract: This manual describes the policy requirements for the Menu hierarchical structure of the menu sections. Section: Debian -Format: debiandoc-sgml -Files: /usr/share/doc/debian-policy/menu-policy.sgml.gz - Format: text Files: /usr/share/doc/debian-policy/menu-policy.txt.gz diff --git a/debian-perl-policy.desc b/debian-perl-policy.desc index cb6337c..0aa1831 100644 --- a/debian-perl-policy.desc +++ b/debian-perl-policy.desc @@ -7,9 +7,6 @@ Abstract: This manual describes the policy requirements for the Perl using Perl and Perl modules. Section: Debian -Format: debiandoc-sgml -Files: /usr/share/doc/debian-policy/perl-policy.sgml.gz - Format: text Files: /usr/share/doc/debian-policy/perl-policy.txt.gz diff --git a/debian-policy.desc b/debian-policy.desc index 745538e..fc3d79a 100644 --- a/debian-policy.desc +++ b/debian-policy.desc @@ -8,9 +8,6 @@ Abstract: This manual describes the policy requirements for the Debian included in the distribution. Section: Debian -Format: debiandoc-sgml -Files: /usr/share/doc/debian-policy/policy.sgml.gz - Format: text Files: /usr/share/doc/debian-policy/policy.txt.gz diff --git a/debian/rules b/debian/rules index 03cea36..88ca46e 100755 --- a/debian/rules +++ b/debian/rules @@ -17,8 +17,7 @@ DOCDIR := $(TMPTOP)/usr/share/doc/$(package) LIBDIR := $(TMPTOP)/usr/share/doc-base # SGML source files in the top-level directory. We do some common actions -# with each of these: validate, build text, HTML, and one-page HTML output, -# and generate a tarball of the source. +# with each of these: validate, build text, HTML, and one-page HTML output. SGML_FILES := policy menu-policy perl-policy upgrading-checklist # Markdown source files in the top-level directory. We generate text and @@ -35,10 +34,10 @@ FHS_ARCHIVE := fhs-2.3-source.tar.gz FHS_FILES := fhs-2.3.html fhs-2.3.ps.gz fhs-2.3.txt.gz fhs-2.3.pdf.gz # A list of the simple Policy files that we include in the documentation -# directory of the generated package. The tarballs of source are handled -# separately, as are the directories of HTML output. -POLICY_FILES := $(SGML_FILES:=.sgml) $(SGML_FILES:=.txt.gz) \ - $(SGML_FILES:=-1.html) version.ent \ +# directory of the generated package. The directories of HTML output are +# handled separately. +POLICY_FILES := $(SGML_FILES:=.txt.gz) \ + $(SGML_FILES:=-1.html) \ virtual-package-names-list.txt \ copyright-format/copyright-format-1.0.html \ copyright-format/copyright-format-1.0.txt.gz \ @@ -64,8 +63,6 @@ FILES_TO_CLEAN := $(SGML_FILES:=.txt) $(SGML_FILES:=.txt.gz) \ copyright-format/version.xml \ autopkgtest/version.txt \ debconf_spec/include/version.xml version.ent \ - copyright-format.xml.tar.gz \ - debconf_specification.xml.tar.gz \ debian/files # Install files and directories with the correct ownership and permissions. @@ -97,10 +94,6 @@ stamp-build: $(MDWN_FILES:=.md) \ $(MAKE) -C copyright-format all $(MAKE) -C autopkgtest all $(MAKE) -C debconf_spec all - cd copyright-format && \ - GZIP=-n9 tar -zcf ../copyright-format.xml.tar.gz * - cd debconf_spec && \ - GZIP=-n9 tar -zcf ../debconf_specification.xml.tar.gz * touch stamp-build # Create the version files for inclusion in the various documents. We want @@ -160,11 +153,6 @@ stamp-binary: stamp-build $(install) debian/changelog $(DOCDIR)/ $(install) debian/copyright $(DOCDIR)/ # -# Install DocBook XML source. -# - $(install) copyright-format.xml.tar.gz $(DOCDIR)/ - $(install) debconf_specification.xml.tar.gz $(DOCDIR)/ -# # Install generated HTML directories. # @set -ex; for file in $(SGML_FILES); do \ @@ -186,7 +174,7 @@ stamp-binary: stamp-build # # Compress files and build MD5 checksums. # - gzip -fn9 $(DOCDIR)/*.sgml $(DOCDIR)/changelog + gzip -fn9 $(DOCDIR)/changelog gzip -fn9 $(DOCDIR)/*.txt @set -ex; cd debian/tmp; \ find . -path './DEBIAN' -prune -o -type f -printf '%P\0' \ -- 2.12.2.564.g063fe858b8
From 22947ee50ac2375511e85bd04dde026e9f4b77a8 Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Tue, 28 Mar 2017 06:18:32 +0200 Subject: [PATCH 3/8] Remove unused Build-Depends These are ancient dependencies that never got cleaned up. --- debian/control | 3 --- 1 file changed, 3 deletions(-) diff --git a/debian/control b/debian/control index 49a8aa4..666ec76 100644 --- a/debian/control +++ b/debian/control @@ -8,17 +8,14 @@ Uploaders: Section: doc Priority: optional Build-Depends: - bsdmainutils, debiandoc-sgml (>= 1.1.47), docbook-dsssl, docbook-xml, ghostscript, - groff, libtext-multimarkdown-perl, links | elinks, openjade, opensp, - pstoedit, sgml-data, texlive, texlive-latex-extra, -- 2.12.2.564.g063fe858b8
From 0d0e39c4e53c02251785a53924112ae36236be9d Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Tue, 28 Mar 2017 03:09:48 +0200 Subject: [PATCH 4/8] Switch from OpenJade to xsltproc for DocBook processing OpenJade seems to have stagnated upstream, and xsltproc is considered one of the fastest XML processors out there, and more importantly it is maintained, and used more widely. --- copyright-format/Makefile | 11 +---------- copyright-format/html.dsl | 29 ----------------------------- debconf_spec/Makefile | 11 +---------- debconf_spec/html.dsl | 29 ----------------------------- debian/control | 6 +++--- debian/rules | 24 +++++++++--------------- docbook.mk | 13 +++++++++++++ xsl/common.xsl | 15 +++++++++++++++ xsl/html-chunk.xsl | 16 ++++++++++++++++ xsl/html-single.xsl | 10 ++++++++++ xsl/text.xsl | 10 ++++++++++ 11 files changed, 78 insertions(+), 96 deletions(-) delete mode 100644 copyright-format/html.dsl delete mode 100644 debconf_spec/html.dsl create mode 100644 docbook.mk create mode 100644 xsl/common.xsl create mode 100644 xsl/html-chunk.xsl create mode 100644 xsl/html-single.xsl create mode 100644 xsl/text.xsl diff --git a/copyright-format/Makefile b/copyright-format/Makefile index 31d6e7a..36568e1 100644 --- a/copyright-format/Makefile +++ b/copyright-format/Makefile @@ -1,15 +1,6 @@ all: copyright-format-1.0.txt.gz copyright-format-1.0.html -copyright-format-1.0.html: copyright-format-1.0.xml html.dsl - openjade -V nochunks -t sgml -d html.dsl \ - /usr/share/xml/declaration/xml.dcl $< > $@ - -tidy -raw -q -i -m -f /dev/null $@ - -copyright-format-1.0.txt: copyright-format-1.0.html - links -dump $< | perl -pe 's/[\r\0]//g' > $@ - -copyright-format-1.0.txt.gz: copyright-format-1.0.txt - gzip -ncf9 $< > $@ +include ../docbook.mk clean: rm -f *.css *.html *.txt *.txt.gz diff --git a/copyright-format/html.dsl b/copyright-format/html.dsl deleted file mode 100644 index 2774825..0000000 --- a/copyright-format/html.dsl +++ /dev/null @@ -1,29 +0,0 @@ -<!DOCTYPE style-sheet PUBLIC "-//James Clark//DTD DSSSL Style Sheet//EN" [ -<!ENTITY docbook.dsl PUBLIC "-//Norman Walsh//DOCUMENT DocBook HTML Stylesheet//EN" CDATA dsssl> -]> -<style-sheet> - -<style-specification id="html" use="docbook"> -<style-specification-body> - -(define %generate-article-toc% #t) -(define %generate-article-titlepage% #t) -(define %generate-legalnotice-link% #t) -(define (article-titlepage-recto-elements) - (list (normalize "title") - (normalize "subtitle") - (normalize "authorgroup") - (normalize "author") - (normalize "releaseinfo") - (normalize "copyright") - (normalize "pubdate") - (normalize "revhistory") - (normalize "legalnotice") - (normalize "abstract"))) - -</style-specification-body> -</style-specification> - -<external-specification id="docbook" document="docbook.dsl"> - -</style-sheet> diff --git a/debconf_spec/Makefile b/debconf_spec/Makefile index bc6614d..3e03c49 100644 --- a/debconf_spec/Makefile +++ b/debconf_spec/Makefile @@ -1,15 +1,6 @@ all: debconf_specification.txt.gz debconf_specification.html -%.html: %.xml html.dsl - openjade -V nochunks -t sgml -d html.dsl \ - /usr/share/xml/declaration/xml.dcl $< > $@ - -tidy -raw -q -i -m -f /dev/null $@ - -%.txt: %.html - links -dump $< | perl -pe 's/[\r\0]//g' > $@ - -%.txt.gz: %.txt - gzip -ncf9 $< > $@ +include ../docbook.mk clean: rm -f *.css *.html *.txt *.txt.gz diff --git a/debconf_spec/html.dsl b/debconf_spec/html.dsl deleted file mode 100644 index 2774825..0000000 --- a/debconf_spec/html.dsl +++ /dev/null @@ -1,29 +0,0 @@ -<!DOCTYPE style-sheet PUBLIC "-//James Clark//DTD DSSSL Style Sheet//EN" [ -<!ENTITY docbook.dsl PUBLIC "-//Norman Walsh//DOCUMENT DocBook HTML Stylesheet//EN" CDATA dsssl> -]> -<style-sheet> - -<style-specification id="html" use="docbook"> -<style-specification-body> - -(define %generate-article-toc% #t) -(define %generate-article-titlepage% #t) -(define %generate-legalnotice-link% #t) -(define (article-titlepage-recto-elements) - (list (normalize "title") - (normalize "subtitle") - (normalize "authorgroup") - (normalize "author") - (normalize "releaseinfo") - (normalize "copyright") - (normalize "pubdate") - (normalize "revhistory") - (normalize "legalnotice") - (normalize "abstract"))) - -</style-specification-body> -</style-specification> - -<external-specification id="docbook" document="docbook.dsl"> - -</style-sheet> diff --git a/debian/control b/debian/control index 666ec76..0f59405 100644 --- a/debian/control +++ b/debian/control @@ -9,17 +9,17 @@ Section: doc Priority: optional Build-Depends: debiandoc-sgml (>= 1.1.47), - docbook-dsssl, docbook-xml, + docbook-xsl, ghostscript, libtext-multimarkdown-perl, + libxml2-utils, links | elinks, - openjade, opensp, - sgml-data, texlive, texlive-latex-extra, tidy, + xsltproc, Standards-Version: 3.9.9 Vcs-Browser: https://anonscm.debian.org/git/dbnpolicy/policy.git Vcs-Git: https://anonscm.debian.org/git/dbnpolicy/policy.git diff --git a/debian/rules b/debian/rules index 88ca46e..d21ef57 100755 --- a/debian/rules +++ b/debian/rules @@ -20,6 +20,8 @@ LIBDIR := $(TMPTOP)/usr/share/doc-base # with each of these: validate, build text, HTML, and one-page HTML output. SGML_FILES := policy menu-policy perl-policy upgrading-checklist +XML_VERSION := copyright-format/version.xml debconf_spec/include/version.xml + # Markdown source files in the top-level directory. We generate text and # HTML versions from these. MDWN_FILES := Process README @@ -60,9 +62,9 @@ FILES_TO_CLEAN := $(SGML_FILES:=.txt) $(SGML_FILES:=.txt.gz) \ $(MDWN_FILES:=.html) $(MDWN_FILES:=.txt) \ policy.pdf.gz policy.ps.gz \ policy.pdf policy.ps policy.tpt policy.txt \ - copyright-format/version.xml \ autopkgtest/version.txt \ - debconf_spec/include/version.xml version.ent \ + $(XML_VERSION) \ + version.ent \ debian/files # Install files and directories with the correct ownership and permissions. @@ -74,16 +76,15 @@ all build build-indep: stamp-build build-arch: stamp-build: $(MDWN_FILES:=.md) \ $(SGML_FILES:=.sgml) \ + $(XML_VERSION) \ version.ent \ copyright-format/copyright-format-1.0.xml \ - copyright-format/version.xml \ autopkgtest/autopkgtest.md \ autopkgtest/version.txt \ debconf_spec/include/commands.xml \ debconf_spec/include/priorities.xml \ debconf_spec/include/statuscodes.xml \ - debconf_spec/include/types.xml \ - debconf_spec/include/version.xml + debconf_spec/include/types.xml $(MAKE) $(SGML_FILES:=.sgml.validate) \ $(SGML_FILES:=.html.tar.gz) \ $(SGML_FILES:=-1.html) \ @@ -99,21 +100,14 @@ stamp-build: $(MDWN_FILES:=.md) \ # Create the version files for inclusion in the various documents. We want # to put the Policy version and date in each document, even if they # separately have their own versions. -configure: version.ent copyright-format/version.xml \ - autopkgtest/version.txt \ - debconf_spec/include/version.xml +configure: version.ent $(XML_VERSION) autopkgtest/version.txt version.ent: debian/changelog rm -f $@ echo "<!entity version \"$(version)\">" >> $@ echo "<!entity date \"$(date)\">" >> $@ -copyright-format/version.xml: debian/changelog - rm -f $@ - echo '<?xml version="1.0" standalone="no"?>' > $@ - echo '<!ENTITY version "$(version)">' >> $@ - echo '<!ENTITY date "$(date)">' >> $@ -debconf_spec/include/version.xml: debian/changelog +$(XML_VERSION): debian/changelog rm -f $@ - echo '<?xml version="1.0" standalone="no"?>' > $@ + echo '<?xml version="1.0" encoding="utf-8"?>' > $@ echo '<!ENTITY version "$(version)">' >> $@ echo '<!ENTITY date "$(date)">' >> $@ autopkgtest/version.txt: debian/changelog diff --git a/docbook.mk b/docbook.mk new file mode 100644 index 0000000..18e09b7 --- /dev/null +++ b/docbook.mk @@ -0,0 +1,13 @@ +XSLDIR ?= ../xsl +XSLTPROC = xsltproc --nonet --xinclude + +%.html: %.xml $(XSLDIR)/html-single.xsl + $(XSLTPROC) $(XSLPARAMS) $(XSLDIR)/html-single.xsl $< > $@ + +%.txt: %.xml $(XSLDIR)/text.xsl + $(XSLTPROC) $(XSLPARAMS) $(XSLDIR)/text.xsl $< > $@.html + links -dump $@.html | perl -pe 's/[\r\0]//g' > $@ + rm -f $@.html + +%.txt.gz: %.txt + gzip -ncf9 $< > $@ diff --git a/xsl/common.xsl b/xsl/common.xsl new file mode 100644 index 0000000..cfa406d --- /dev/null +++ b/xsl/common.xsl @@ -0,0 +1,15 @@ +<?xml version="1.0" encoding="utf-8"?> + +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + <xsl:param name="header.navigation">1</xsl:param> + <xsl:param name="footer.navigation">1</xsl:param> + + <xsl:param name="generate.consistent.ids">1</xsl:param> + + <xsl:param name="section.autolabel">1</xsl:param> + <xsl:param name="section.label.includes.component.label">1</xsl:param> + + <xsl:param name="make.valid.html">1</xsl:param> + +</xsl:stylesheet> diff --git a/xsl/html-chunk.xsl b/xsl/html-chunk.xsl new file mode 100644 index 0000000..eb29f5d --- /dev/null +++ b/xsl/html-chunk.xsl @@ -0,0 +1,16 @@ +<?xml version="1.0" encoding="utf-8"?> + +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/xhtml/chunk.xsl"/> + <xsl:include href="common.xsl"/> + + <xsl:output method="html" encoding="UTF-8"/> + + <xsl:param name="use.id.as.filename">1</xsl:param> + + <xsl:param name="chunk.section.depth">0</xsl:param> + <xsl:param name="chunker.output.indent">yes</xsl:param> + <xsl:param name="chunker.output.encoding" select="'utf-8'"/> + +</xsl:stylesheet> diff --git a/xsl/html-single.xsl b/xsl/html-single.xsl new file mode 100644 index 0000000..14e711e --- /dev/null +++ b/xsl/html-single.xsl @@ -0,0 +1,10 @@ +<?xml version="1.0" encoding="utf-8"?> + +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl"/> + <xsl:include href="common.xsl"/> + + <xsl:output method="html" encoding="UTF-8" indent="yes"/> + +</xsl:stylesheet> diff --git a/xsl/text.xsl b/xsl/text.xsl new file mode 100644 index 0000000..46d8c6f --- /dev/null +++ b/xsl/text.xsl @@ -0,0 +1,10 @@ +<?xml version="1.0" encoding="utf-8"?> + +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + <xsl:import href="http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl"/> + <xsl:include href="common.xsl"/> + + <xsl:param name="preferred.mediaobject.role">text</xsl:param> + +</xsl:stylesheet> -- 2.12.2.564.g063fe858b8
From 94eeb7ff9d6402992d9ea6713be30d01c240c9ab Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Tue, 10 Jan 2017 09:29:45 +0100 Subject: [PATCH 5/8] Add new DebianDoc-SGML to DocBook conversion script This is a conversion script, that should automate the whole process so that it can hopefully be repeated at any later point and is able handle modifications to the documents sources. --- tools/sgml2docbook | 116 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100755 tools/sgml2docbook diff --git a/tools/sgml2docbook b/tools/sgml2docbook new file mode 100755 index 0000000..2273d32 --- /dev/null +++ b/tools/sgml2docbook @@ -0,0 +1,116 @@ +#!/bin/sh + +set -e +set -x + +cleanup=yes +tidyxml=no + +files="menu-policy.sgml perl-policy.sgml upgrading-checklist.sgml policy.sgml" + +export LC_ALL=C + +# Generate fake entity files to be able to process the SGML and XML. +( + echo '<!entity version "@@@VERSION@@@">' + echo '<!entity date "@@@DATE@@@">' +) >version.ent +( + echo '<?xml version="1.0" encoding="utf-8"?>' + echo '<!ENTITY version "@@@VERSION@@@">' + echo '<!ENTITY date "@@@DATE@@@">' +) >version.xml + +for sgml in $files; do + base=$(basename $sgml .sgml) + + # Remove invalid entities, encode other entities, and wrap + sed -e 's/ *© *//g' \ + -e 's/, &date;//g' \ + -e 's/&\([a-z]*\);/@@@\1@@@/g' <$sgml >$base.new.sgml + + # Do the conversion. + debiandoc2dbk -1 -l en $base.new.sgml + + # Recode, and fixup markup. + iconv -f iso-8859-15 -t utf-8 <$base.new.dbk | + sed -e 's/@@@\([a-z]*\)@@@/\&\1;/g' \ + -e 's/&version;/@@@version@@@/g' \ + -e 's/<!-- put date -->/@@@date@@@/' \ + -e '/<!-- Include entity/d' \ + -e 's/<!-- \(<!ENTITY .*\)"version.ent"\(>.*\) *-->/\1"version.xml"\2/g' \ + -e 's:<personname>\([^ ]*\) *\([^ ]*\)</personname>:<personname><firstname>\1</firstname><surname>\2</surname></personname>:g' \ + -e 's:<personname>\(.*The Debian Policy mailing List.*\)</personname>:<othername>\1</othername>:gi' \ + -e 's/ *$//g' \ + >$base.xml + + if [ "$base" = "policy" ]; then + # Fixup IDs, and insert entity. + sed -i \ + -e '/^]>/i<!-- current Debian changes file format -->\ +<!ENTITY changesversion "1.8">' \ + -e 's:<literal>"</literal>:&<!-- balance " -->:' \ + -e 's/\(id\|linkend\)="\(pkg-alternatives\|pkg-binarypkg\|pkg-conffiles\|pkg-controlfields\|pkg-diversions\|pkg-scope\|pkg-sourcepkg\)"/\1="ap-\2"/g' \ + -e 's/\(id\|linkend\)="\(archive\|binary\|controlfields\|customized-programs\|docs\|files\|maintainerscripts\|opersys\|relationships\|scope\|sharedlibs\|source\)"/\1="ch-\2"/' \ + -e '/\(id\|linkend\)="\(ap-\|ch-\|s-\|s[0-9]\)[^"]*"/!s/\(id\|linkend\)="\([^"]*\)"/\1="s-\2"/g' \ + $base.xml + elif [ "$base" = "perl-policy" ]; then + # Fixup IDs. + sed -i \ + -e 's/\(id\|linkend\)="perl6"/\1="ap-perl6"/' \ + -e 's/\(id\|linkend\)="module-packages"/\1="ch-module_packages"/' \ + -e 's/\(id\|linkend\)="\(embed\|perl\|programs\|site\|files\)"/\1="ch-\2"/' \ + -e '/\(id\|linkend\)="\(ap-\|ch-\|ch[0-9]\|s-\|s[0-9]\)[^"]*"/!s/\(id\|linkend\)="\([^"]*\)"/\1="s-\2"/g' \ + $base.xml + elif [ "$base" = "upgrading-checklist" ]; then + # Fixup IDs. + sed -i -e 's/id="\([0-9]\)/id="s-\1/g' \ + $base.xml + fi + + case "$tidyxml" in + xmllint) + xmllint --encode utf-8 --format --nonet --postvalid $base.xml \ + | sponge $base.xml + ;; + pandoc) + # Converts from book to an article, does not preserve entity declarations, + # nice output. + pandoc --normalize --chapters -s -f docbook -t docbook $base.xml \ + | sponge $base.xml + ;; + tidy) + # Messes up spacing, and joins tags and words. + tidy -raw -wrap 76 --indent yes --quiet yes \ + -xml --input-xml yes --output-xml yes \ + --preserve-entities yes --vertical-space yes --add-xml-space yes \ + -i -m $base.xml + ;; + esac + + # Restore entities. + sed -i -e 's/@@@\([a-z]*\)@@@/\&\1;/g' $base.xml + + # Test generating single file. + xsltproc --nonet --output "$base-1.html" \ + xsl/html-single.xsl $base.xml + + # Test generating chunked files. + mkdir -p $base.html + xsltproc --nonet --stringparam base.dir $base.html/ \ + xsl/html-chunk.xsl $base.xml + + # Update git tree. + git add $base.xml + git rm $base.sgml + + # Cleanup temporary files. + if [ $cleanup = yes ]; then + rm -f $base.new.* + fi +done + +# Cleanup temporary files. +if [ $cleanup = yes ]; then + rm -f version.ent version.xml +fi -- 2.12.2.564.g063fe858b8
From 84181ecf089551f211ee62a9a1115eeefd1cd05f Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Mon, 9 Jan 2017 05:22:59 +0100 Subject: [PATCH 7/8] Update build system for DocBook conversion Switch to use xsltproc and dblatex for the converted sources. Stop using tidy(1) because it produces bigger files, and the output generated by xsltproc is pretty clean and compliant already. Remove all build dependencies not used at all or not used directly by our build system, even if they end up being pulled anyway by our direct dependencies. --- .gitignore | 2 +- Makefile | 45 ++++++++++++++++++++++++++++----------------- README.md | 10 +++++----- debian/control | 7 +------ debian/rules | 38 +++++++++++++++++--------------------- xsl/dblatex.xsl | 9 +++++++++ 6 files changed, 61 insertions(+), 50 deletions(-) create mode 100644 xsl/dblatex.xsl diff --git a/.gitignore b/.gitignore index 66b5940..66c32a5 100644 --- a/.gitignore +++ b/.gitignore @@ -17,7 +17,7 @@ /stamp-build /stamp-binary /upgrading-checklist.html/ -/version.ent +/version.xml *-1.html *.html.tar.gz *.pdf diff --git a/Makefile b/Makefile index 288c4ae..2df0ac0 100644 --- a/Makefile +++ b/Makefile @@ -1,8 +1,13 @@ include debian/rules -policy.sgml: version.ent -menu-policy.sgml: version.ent -perl-policy.sgml: version.ent +policy.xml: version.xml +menu-policy.xml: version.xml +perl-policy.xml: version.xml +upgrading-checklist.xml: version.xml + +XSLTPROC = xsltproc --nonet --xinclude +XMLLINT = xmllint --nonet --noout --postvalid --xinclude +DBLATEX = dblatex -p xsl/dblatex.xsl $(MDWN_FILES:=.txt): %.txt: %.md cat $^ > $@ @@ -11,34 +16,40 @@ $(MDWN_FILES:=.txt): %.txt: %.md $(MDWN_FILES:=.html): %.html: %.md $(MDWN) $< > $@ +upgrading-checklist-1.html \ +upgrading-checklist.html/index.html: XSLPARAMS = --stringparam generate.toc '' + %.validate: % - onsgmls -wall -gues $< + $(XMLLINT) $< -%.html/index.html: %.sgml - LANG=C debiandoc2html $< +%.html/index.html: %.xml xsl/html-chunk.xsl + mkdir -p $(@D) + $(XSLTPROC) $(XSLPARAMS) \ + --stringparam base.dir $(@D)/ \ + xsl/html-chunk.xsl $< -%-1.html: %.sgml - LANG=C debiandoc2html -1 -b $*-1d $< && \ - mv $*-1d.html/index.html $*-1.html && \ - rmdir $*-1d.html +%-1.html: %.xml xsl/html-single.xsl + $(XSLTPROC) $(XSLPARAMS) xsl/html-single.xsl $< > $@ %.html.tar.gz: %.html/index.html GZIP=-n9 tar -czf $(<:/index.html=.tar.gz) $(<:/index.html=) -$(SGML_FILES:=.txt): %.txt: %.sgml - LANG=C debiandoc2text $< +$(XML_FILES:=.txt): %.txt: %.xml + $(XSLTPROC) $(XSLPARAMS) xsl/text.xsl $< > $@.html + links -dump $@.html | perl -pe 's/[\r\0]//g' > $@ + rm -f $@.html %.txt.gz: %.txt gzip -ncf9 $< > $@ -%.ps: %.sgml - LANG=C debiandoc2latexps $< +%.ps: %.xml + $(DBLATEX) --ps $< %.ps.gz: %.ps gzip -ncf9 $< > $@ -%.pdf: %.sgml - LANG=C debiandoc2latexpdf $< +%.pdf: %.xml + $(DBLATEX) --pdf $< %.pdf.gz: %.pdf gzip -ncf9 $< > $@ @@ -70,7 +81,7 @@ distclean: rm -f $(filter-out $(leavealone),$(wildcard *.txt *.txt.gz *.html.tar.gz *.pdf *.ps)) rm -f *.lout* lout.li *.sasp* *.tex *.aux *.toc *.idx *.log *.out *.dvi *.tpt rm -f `find . -name "*~" -o -name "*.bak" -o -name ".#*" -o -name core` - rm -f version.ent + rm -f version.xml rm -f *.rej *.orig # if a rule bombs out, delete the target diff --git a/README.md b/README.md index 76a236b..68085e7 100644 --- a/README.md +++ b/README.md @@ -192,14 +192,14 @@ proposed wording, is: <number> is the bug number in the BTS and <user>is a designator of the Policy team member who is shepherding the bug. + Commit wording changes in that branch until consensus is - achieved. Do not modify debian/changelog or upgrading-checklist.html + achieved. Do not modify debian/changelog or upgrading-checklist.xml during this phase. Use the BTS to track who proposed the wording and who seconded it. + git pull master to make sure you have the latest version. + Once the change has been approved by enough people, git merge the branch into master immediately after making the final commit adding the changelog entry to minimize conflicts. -+ add the debian/changelog and upgrading-checklist.html changes, and ++ add the debian/changelog and upgrading-checklist.xml changes, and commit to master. + Push master out so other people may merge in their own bug branches without conflicts. @@ -233,8 +233,8 @@ The Git commands used for this workflow are: git checkout master git merge bug12345-rra - # edit debian/changelog and upgrading-checklist.html - git add debian/changelog upgrading-checklist.html + # edit debian/changelog and upgrading-checklist.xml + git add debian/changelog upgrading-checklist.xml git commit git push origin master git branch -d bug12345-rra @@ -278,7 +278,7 @@ assuming that you haven't packed the refs in your repository. For a final Policy release, change UNRELEASED to unstable in debian/changelog and update the timestamp to match the final release time (dch -r may be helpful for this), update the release date in -upgrading-checklist.html, update Standards-Version in debian/control, +upgrading-checklist.xml, update Standards-Version in debian/control, and commit that change. Then do the final release build and make sure that it builds and installs. diff --git a/debian/control b/debian/control index 0f59405..87259d0 100644 --- a/debian/control +++ b/debian/control @@ -8,17 +8,12 @@ Uploaders: Section: doc Priority: optional Build-Depends: - debiandoc-sgml (>= 1.1.47), + dblatex, docbook-xml, docbook-xsl, - ghostscript, libtext-multimarkdown-perl, libxml2-utils, links | elinks, - opensp, - texlive, - texlive-latex-extra, - tidy, xsltproc, Standards-Version: 3.9.9 Vcs-Browser: https://anonscm.debian.org/git/dbnpolicy/policy.git diff --git a/debian/rules b/debian/rules index d21ef57..33c06ab 100755 --- a/debian/rules +++ b/debian/rules @@ -16,11 +16,12 @@ TMPTOP := $(SRCTOP)/debian/tmp DOCDIR := $(TMPTOP)/usr/share/doc/$(package) LIBDIR := $(TMPTOP)/usr/share/doc-base -# SGML source files in the top-level directory. We do some common actions -# with each of these: validate, build text, HTML, and one-page HTML output. -SGML_FILES := policy menu-policy perl-policy upgrading-checklist +# DocBook source files in the top-level directory. We do some common actions +# with each of these: build text, HTML, and one-page HTML output. +XML_FILES := policy menu-policy perl-policy upgrading-checklist -XML_VERSION := copyright-format/version.xml debconf_spec/include/version.xml +XML_VERSION := copyright-format/version.xml debconf_spec/include/version.xml \ + version.xml # Markdown source files in the top-level directory. We generate text and # HTML versions from these. @@ -38,8 +39,8 @@ FHS_FILES := fhs-2.3.html fhs-2.3.ps.gz fhs-2.3.txt.gz fhs-2.3.pdf.gz # A list of the simple Policy files that we include in the documentation # directory of the generated package. The directories of HTML output are # handled separately. -POLICY_FILES := $(SGML_FILES:=.txt.gz) \ - $(SGML_FILES:=-1.html) \ +POLICY_FILES := $(XML_FILES:=.txt.gz) \ + $(XML_FILES:=-1.html) \ virtual-package-names-list.txt \ copyright-format/copyright-format-1.0.html \ copyright-format/copyright-format-1.0.txt.gz \ @@ -56,9 +57,9 @@ POLICY_FILES := $(SGML_FILES:=.txt.gz) \ # are individual generated files to remove. DIRS_TO_CLEAN are entire # directories to remove. STAMPS_TO_CLEAN := stamp-binary stamp-build -DIRS_TO_CLEAN := $(SGML_FILES:=.html) debian/tmp fhs -FILES_TO_CLEAN := $(SGML_FILES:=.txt) $(SGML_FILES:=.txt.gz) \ - $(SGML_FILES:=.html.tar.gz) $(SGML_FILES:=-1.html) \ +DIRS_TO_CLEAN := $(XML_FILES:=.html) debian/tmp fhs +FILES_TO_CLEAN := $(XML_FILES:=.txt) $(XML_FILES:=.txt.gz) \ + $(XML_FILES:=.html.tar.gz) $(XML_FILES:=-1.html) \ $(MDWN_FILES:=.html) $(MDWN_FILES:=.txt) \ policy.pdf.gz policy.ps.gz \ policy.pdf policy.ps policy.tpt policy.txt \ @@ -75,9 +76,8 @@ mkdir := install -d -o root -g root -m 755 all build build-indep: stamp-build build-arch: stamp-build: $(MDWN_FILES:=.md) \ - $(SGML_FILES:=.sgml) \ + $(XML_FILES:=.xml) \ $(XML_VERSION) \ - version.ent \ copyright-format/copyright-format-1.0.xml \ autopkgtest/autopkgtest.md \ autopkgtest/version.txt \ @@ -85,10 +85,10 @@ stamp-build: $(MDWN_FILES:=.md) \ debconf_spec/include/priorities.xml \ debconf_spec/include/statuscodes.xml \ debconf_spec/include/types.xml - $(MAKE) $(SGML_FILES:=.sgml.validate) \ - $(SGML_FILES:=.html.tar.gz) \ - $(SGML_FILES:=-1.html) \ - $(SGML_FILES:=.txt.gz) \ + $(MAKE) $(XML_FILES:=.xml.validate) \ + $(XML_FILES:=.html.tar.gz) \ + $(XML_FILES:=-1.html) \ + $(XML_FILES:=.txt.gz) \ policy.ps.gz policy.pdf.gz $(MAKE) $(MDWN_FILES:=.html) \ $(MDWN_FILES:=.txt) @@ -100,11 +100,7 @@ stamp-build: $(MDWN_FILES:=.md) \ # Create the version files for inclusion in the various documents. We want # to put the Policy version and date in each document, even if they # separately have their own versions. -configure: version.ent $(XML_VERSION) autopkgtest/version.txt -version.ent: debian/changelog - rm -f $@ - echo "<!entity version \"$(version)\">" >> $@ - echo "<!entity date \"$(date)\">" >> $@ +configure: $(XML_VERSION) autopkgtest/version.txt $(XML_VERSION): debian/changelog rm -f $@ echo '<?xml version="1.0" encoding="utf-8"?>' > $@ @@ -149,7 +145,7 @@ stamp-binary: stamp-build # # Install generated HTML directories. # - @set -ex; for file in $(SGML_FILES); do \ + @set -ex; for file in $(XML_FILES); do \ tar -C $(DOCDIR) -zxf $$file.html.tar.gz; \ done # diff --git a/xsl/dblatex.xsl b/xsl/dblatex.xsl new file mode 100644 index 0000000..5824ba8 --- /dev/null +++ b/xsl/dblatex.xsl @@ -0,0 +1,9 @@ +<?xml version="1.0" encoding="utf-8"?> + +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + <xsl:include href="common.xsl"/> + + <xsl:param name="latex.output.revhistory">0</xsl:param> + +</xsl:stylesheet> -- 2.12.2.564.g063fe858b8
From d3467a2fcf207daec472c27015184daa5f4594dd Mon Sep 17 00:00:00 2001 From: Guillem Jover <guil...@debian.org> Date: Sun, 26 Mar 2017 22:52:20 +0200 Subject: [PATCH 8/8] Make the upgrading-checklist a proper appendix of the policy manual This integrates the upgrading checklist properly into the policy manual, so that there are now navigation references and it appears in the TOC. As a side effect we get rid of the split HTML output because it would contain just a single index.html file anyway, but keep compatibility files for the one-page HTML and text outputs. --- .gitignore | 1 - Makefile | 16 ++-------------- debian/rules | 19 +++++++++++-------- policy.xml | 2 ++ upgrading-checklist.xml | 37 ++++++------------------------------- 5 files changed, 21 insertions(+), 54 deletions(-) diff --git a/.gitignore b/.gitignore index 66c32a5..a200169 100644 --- a/.gitignore +++ b/.gitignore @@ -16,7 +16,6 @@ /policy.html/ /stamp-build /stamp-binary -/upgrading-checklist.html/ /version.xml *-1.html *.html.tar.gz diff --git a/Makefile b/Makefile index 2df0ac0..cd4b124 100644 --- a/Makefile +++ b/Makefile @@ -3,7 +3,6 @@ include debian/rules policy.xml: version.xml menu-policy.xml: version.xml perl-policy.xml: version.xml -upgrading-checklist.xml: version.xml XSLTPROC = xsltproc --nonet --xinclude XMLLINT = xmllint --nonet --noout --postvalid --xinclude @@ -16,8 +15,7 @@ $(MDWN_FILES:=.txt): %.txt: %.md $(MDWN_FILES:=.html): %.html: %.md $(MDWN) $< > $@ -upgrading-checklist-1.html \ -upgrading-checklist.html/index.html: XSLPARAMS = --stringparam generate.toc '' +upgrading-checklist-1.html: XSLPARAMS = --stringparam generate.toc '' %.validate: % $(XMLLINT) $< @@ -34,7 +32,7 @@ upgrading-checklist.html/index.html: XSLPARAMS = --stringparam generate.toc '' %.html.tar.gz: %.html/index.html GZIP=-n9 tar -czf $(<:/index.html=.tar.gz) $(<:/index.html=) -$(XML_FILES:=.txt): %.txt: %.xml +$(XML_FILES:=.txt) $(XML_SPLIT_FILES:=.txt): %.txt: %.xml $(XSLTPROC) $(XSLPARAMS) xsl/text.xsl $< > $@.html links -dump $@.html | perl -pe 's/[\r\0]//g' > $@ rm -f $@.html @@ -54,16 +52,6 @@ $(XML_FILES:=.txt): %.txt: %.xml %.pdf.gz: %.pdf gzip -ncf9 $< > $@ -# This is a temporary hack to fold the upgrading-checklist into the Policy -# HTML directory so that it can be deployed alongside Policy on -# www.debian.org in a way that lets the cross-document links work properly. -# The correct solution is to make upgrading-checklist an appendix of Policy, -# which will probably be done as part of a general conversion to DocBook. -policy.html.tar.gz:: policy.html/upgrading-checklist.html -policy.html/upgrading-checklist.html: upgrading-checklist-1.html \ - policy.html/index.html - cp -p $< $@ - # convenience aliases :) html: policy.html/index.html html-1: policy-1.html diff --git a/debian/rules b/debian/rules index 33c06ab..a47ab60 100755 --- a/debian/rules +++ b/debian/rules @@ -18,7 +18,11 @@ LIBDIR := $(TMPTOP)/usr/share/doc-base # DocBook source files in the top-level directory. We do some common actions # with each of these: build text, HTML, and one-page HTML output. -XML_FILES := policy menu-policy perl-policy upgrading-checklist +XML_FILES := policy menu-policy perl-policy + +# The upgrading-checklist used to be a document of its own, which was merged +# with the conversion to DocBook. Keep backwards compatibility files. +XML_SPLIT_FILES := upgrading-checklist XML_VERSION := copyright-format/version.xml debconf_spec/include/version.xml \ version.xml @@ -41,6 +45,8 @@ FHS_FILES := fhs-2.3.html fhs-2.3.ps.gz fhs-2.3.txt.gz fhs-2.3.pdf.gz # handled separately. POLICY_FILES := $(XML_FILES:=.txt.gz) \ $(XML_FILES:=-1.html) \ + $(XML_SPLIT_FILES:=.txt.gz) \ + $(XML_SPLIT_FILES:=-1.html) \ virtual-package-names-list.txt \ copyright-format/copyright-format-1.0.html \ copyright-format/copyright-format-1.0.txt.gz \ @@ -60,6 +66,8 @@ STAMPS_TO_CLEAN := stamp-binary stamp-build DIRS_TO_CLEAN := $(XML_FILES:=.html) debian/tmp fhs FILES_TO_CLEAN := $(XML_FILES:=.txt) $(XML_FILES:=.txt.gz) \ $(XML_FILES:=.html.tar.gz) $(XML_FILES:=-1.html) \ + $(XML_SPLIT_FILES:=.txt) $(XML_SPLIT_FILES:=.txt.gz) \ + $(XML_SPLIT_FILES:=-1.html) \ $(MDWN_FILES:=.html) $(MDWN_FILES:=.txt) \ policy.pdf.gz policy.ps.gz \ policy.pdf policy.ps policy.tpt policy.txt \ @@ -89,6 +97,8 @@ stamp-build: $(MDWN_FILES:=.md) \ $(XML_FILES:=.html.tar.gz) \ $(XML_FILES:=-1.html) \ $(XML_FILES:=.txt.gz) \ + $(XML_SPLIT_FILES:=-1.html) \ + $(XML_SPLIT_FILES:=.txt.gz) \ policy.ps.gz policy.pdf.gz $(MAKE) $(MDWN_FILES:=.html) \ $(MDWN_FILES:=.txt) @@ -149,13 +159,6 @@ stamp-binary: stamp-build tar -C $(DOCDIR) -zxf $$file.html.tar.gz; \ done # -# Make upgrading-checklist-1.html a symlink to the copy in the policy.html -# directory. This is temporary until it becomes a proper appendix. -# - rm -f $(DOCDIR)/upgrading-checklist-1.html - ln -s policy.html/upgrading-checklist.html \ - $(DOCDIR)/upgrading-checklist-1.html -# # Install doc-base files. # @set -ex; for file in $(DESC_FILES); do \ diff --git a/policy.xml b/policy.xml index f901692..5848af1 100644 --- a/policy.xml +++ b/policy.xml @@ -10767,6 +10767,8 @@ it well. </para> </appendix> +<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" + href="upgrading-checklist.xml" /> </book> diff --git a/upgrading-checklist.xml b/upgrading-checklist.xml index f0dbd17..0241315 100644 --- a/upgrading-checklist.xml +++ b/upgrading-checklist.xml @@ -1,33 +1,13 @@ <?xml version='1.0' encoding='utf-8'?> <!-- -*- DocBook -*- --> -<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" +<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ -<!ENTITY % versiondata SYSTEM "version.xml"> %versiondata; ]> -<book lang="en"> +<appendix id="upgrading-checklist"><title>Upgrading checklist</title> -<title>Policy checklist for upgrading your packages</title> - -<bookinfo> - -<authorgroup> -<author><personname><firstname>Bill</firstname><surname>Allombert</surname></personname><email>ballo...@debian.org</email></author> -<author><personname><firstname>Josip</firstname><surname>Rodin</surname></personname></author> -<author><personname><firstname>Julian</firstname><surname>Gilbey</surname></personname></author> -<author><personname><firstname>Russ</firstname><surname>Allbery</surname></personname><email>r...@debian.org</email></author> -<author><personname><firstname>Manoj</firstname><surname>Srivastava</surname></personname><email>sriva...@debian.org</email></author> - -</authorgroup> -<releaseinfo></releaseinfo> - -<pubdate>&date;</pubdate> - - -</bookinfo> - - -<chapter id="ch1"><title>About the checklist</title> +<abstract> +<title>About the checklist</title> <para> The checklist below has been created to simplify the upgrading process of old packages. Note that this list is not "official"; it simply gives an indication @@ -49,9 +29,8 @@ and then check out the Policy Manual for details. If you are upgrading from Policy version < 2.5.0, it may be easier to check through the whole of policy instead of picking your way through this list. </para> -</chapter> +</abstract> -<chapter id="ch2"><title>The checklist</title> <section id="s-3.9.9.0"><title>Version 3.9.9.0</title> <para> Not Released Yet @@ -3471,8 +3450,4 @@ Upstream changelog must be installed too </itemizedlist> </section> -</chapter> - - -</book> - +</appendix> -- 2.12.2.564.g063fe858b8