slachiewicz opened a new issue, #139:
URL: https://github.com/apache/maven-doxia-converter/issues/139
Site documentation across the Maven repositories is still largely written in
APT, a format only Doxia understands and which few contributors can read or
edit comfortably. Markdown is already supported by `maven-site-plugin` out of
the box, several repositories have been partly migrated already, and
`doxia-converter` can do most of the mechanical work.
This issue tracks converting the remaining APT site documents to Markdown
across the Maven repositories, the fixes to `doxia-converter` and Doxia that
the conversion turned up, and the repair of pages that were migrated earlier
and are rendering incorrectly today.
## Scope
Counting only real site documents (under `src/site/`, excluding `src/it` and
`src/test` fixtures, which are inputs to the APT parser's own tests and must
stay APT):
| | files |
|---|---|
| APT site documents at the start | 729 |
| converted so far | 151 |
| remaining, in scope | 458 across 29 repositories |
| remaining, out of scope | 120 |
Out of scope: the archived SVN sandbox, `codehaus-plexus.github.io` (a
different community), and the Maven 3 maintenance branches, which are probably
not worth churning.
Two facts make the mechanical part cheaper than it looks:
* `maven-site-plugin` already declares `doxia-module-markdown`, so no POM
change is needed to add a Markdown page.
* The site renderer strips `.vm` generically, so `foo.apt.vm` becomes
`foo.md.vm` and Velocity keeps working. Of 442 `.apt.vm` files only two use
Velocity control directives; the rest only interpolate properties.
## Traps found so far
Each of these was found by building the site before and after and comparing
the generated HTML, not by reading the source.
**Velocity eats Markdown headings.** `##` starts a Markdown ATX heading and
also a Velocity line comment, so in any `*.md.vm` file every heading below
level one is deleted before Doxia sees the document. No error, no warning, the
line is simply gone. This already affects published pages that were migrated
earlier (see below), and 222 of the remaining `.apt.vm` files have subsections.
**A literal reference comes back to life.** APT writes `$\{foo\}` to show
`${foo}` as text. The parser unescapes it, so the converted `*.md.vm` holds a
live reference and Velocity resolves it. `maven-shared-reporting-exec` rendered
`[org.apache.maven.model.ReportPlugin@c754401]` where the page meant to
document `${project.reporting.plugins}`. Nine files were affected.
**Macro parameters truncated.**
`%{snippet|file=${project.build.directory}/x.txt}` lost everything after the
first closing brace, so several snippets on a page collapsed onto the same
wrong target.
**Link destinations with spaces.** APT allows a space in an anchor, Markdown
does not: `{{{#Identity Mapper}Identity Mapper}}` converted to a link that
resolves nowhere. Worth noting the APT rendering was already wrong here — it
emitted `href="#Identity Mapper"` pointing at `id="Identity_Mapper"` — so the
conversion fixes a pre-existing broken link.
**Over-escaping.** Converted text was full of `\(`, `\)`, `1\.6`, `"`
and `'`, which are not special in Markdown and make the source unpleasant
to edit.
## Changes to the tooling
*doxia-converter*
- [ ] Keep Velocity constructs intact when converting a `*.vm` source.
References, directive lines and block comments are hidden from the parser and
put back afterwards, which also fixes `${project.name}` used as a section title
coming out as `# $<a id="project.name"></a>project.name`.
- [ ] Warn when conversion turns literal text into a live Velocity reference.
- [ ] Keep a masked reference looking like a URL, so a destination opening
with one is not mangled into an anchor.
*doxia*
- [ ] Let APT macro blocks end at the last brace on the line.
- [ ] Stop over-escaping text in the Markdown sink.
- [ ] Encode link destinations that contain a space.
*doxia-sitetools*
- [ ] Keep Markdown headings from being read as Velocity comments. Lines
that would be swallowed are wrapped in `#[[ ... ]]#` before evaluation; a
document that needs no shielding still goes through the existing code path
unchanged.
## Pages migrated earlier that render incorrectly today
Every subsection heading on these pages is missing from the published site:
- [ ] `maven-build-cache-extension` — `index.md.vm`
- [ ] `maven-checkstyle-plugin` — `history.md.vm`
- [ ] `maven-scripting-plugin` — `jsr223-script-engines.md.vm`
- [ ] `maven-release-plugin` — `migrate.md.vm`, `usage.md.vm`,
`usage/prepare-release.md.vm`
- [ ] `maven-scm-plugin` — `examples/scm-advance-features.md.vm`
These are worth fixing in the pages as well as in the renderer, since a
repository only picks up the renderer fix once Doxia is released and the plugin
bumps its site plugin.
## Conversion progress
Every entry below was verified by building the site before and after and
comparing the visible text and link targets of every generated page. The
remaining differences are `<b>`/`<i>` becoming `<strong>`/`<em>` and quotes in
prose picking up the Markdown module's typographic substitution.
**Done — one page each**
- [ ] maven-shared-utils, maven-shared-jar, maven-script-interpreter,
maven-reporting-impl, maven-reporting-exec, maven-mapping, maven-jarsigner,
maven-dependency-tree, maven-dependency-analyzer
- [ ] maven-compiler-plugin (`usage`), maven-dist-tool, maven-indexer
(`indexer-cli`), maven-apache-parent (`docs`)
- [ ] plexus-compiler, plexus-interpolation
**Done — whole repository**
- [ ] maven-wrapper, maven-apache-resources, maven-parent, plexus-io, modello
- [ ] maven-clean-plugin, maven-install-plugin, maven-verifier-plugin
- [ ] maven-acr-plugin, maven-ejb-plugin, maven-jar-plugin,
maven-jmod-plugin, maven-rar-plugin, maven-shade-plugin, maven-source-plugin,
maven-war-plugin
- [ ] maven-jxr, maven-doap-plugin, maven-jdeps-plugin,
maven-project-info-reports-plugin
- [ ] maven-artifact-plugin, maven-help-plugin, maven-jarsigner-plugin,
maven-jdeprscan-plugin, maven-remote-resources-plugin,
maven-scm-publish-plugin, maven-stage-plugin, maven-toolchains-plugin
- [ ] maven-file-management, maven-filtering, maven-shared-resources
**Remaining, largest first**
- [ ] maven-enforcer 47, maven core 38, maven-surefire 35, maven-ear-plugin
25, maven-archetype 22, maven-archetypes 21, maven-javadoc-plugin 19,
maven-invoker-plugin 17, maven-assembly-plugin 16, maven-dependency-plugin 15,
maven-pmd-plugin 14, maven-changes-plugin 13, maven-wagon 12,
maven-plugin-tools 11, maven-resources-plugin 11, maven-compiler-plugin 11,
maven-checkstyle-plugin 10, maven-changelog-plugin 10, maven-antrun-plugin 9,
and the remainder with fewer than 9 each
## How each page is converted
1. `git mv` the file so the rename is a separate reviewable step.
2. Convert with `doxia-converter` to get a draft.
3. Drop the YAML front matter it adds and fold the licence header into one
block comment.
4. If the page only used Velocity for `${project.name}`, spell the title out
and drop the `.vm`, which also avoids the heading problem entirely.
5. If it keeps a reference, use setext underlines for level two headings and
`#[[ ... ]]#` for anything deeper. A setext underline needs a blank line before
its title or the title is swallowed by the preceding block.
6. Escape references that APT rendered literally.
7. Build the site and compare every generated page against the pre-migration
output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
