Documentation is hard. It's difficult to write, difficult to find and always out of date. That's why we implemented our in-tree documentation system that underpins firefox-source-docs.mozilla.org. The documentation lives next to the code that it documents, so it can be updated within the same commit that makes the underlying changes. Unfortunately, this system has never been a pleasure to work with. Docs are slow to build, RST is foreign to most and the moz.build syntax to integrate them is confusing. Not to mention some serious bugs around how files are uploaded and stored in S3.
I'm happy to announce many great new features for firefox-source-docs, courtesy of our GSoC student Shivam (aka :championshuttler). You can see a blog post he wrote on his experience here <https://medium.com/@shivams2799/adventures-in-gsoc-649434368314>. Below is a list of all the work and awesome new features he added to our documentation infrastructure. Please note that bug 1576912 <https://bugzilla.mozilla.org/show_bug.cgi?id=1576912> tracks saving this information into the docs themselves. *Server Synchronization* Previously, the doc system had a surprisingly major flaw. Docs were only ever uploaded to S3, never deleted. This meant that even if a source file no longer exists (e.g it was removed or renamed), the related page would continue to live on the server indefinitely. External links to the page would continue to function, the users of the link being none-the-wiser. Yikes! We now compare all the files that exist on the server against the list of source files in `mozilla-central`. Any files on the server that no longer exist in `mozilla-central` are removed. *Redirects* We now have the ability to define redirects in-tree! This will allow us to refactor and move docs around to our hearts content without needing to worry about stale external URLs. To set up a redirect simply add a line to this file: https://searchfox.org/mozilla-central/source/tools/docs/redirects.yml Any request starting with the prefix on the left, will be rewritten to the prefix on the right by the server. So for example a request to `/testing/marionette/marionette/index.html` will be re-written to `/testing/marionette/index.html`. Amazon's API only supports prefix redirects, so anything more complex isn't supported. *Nested Doc Trees* This has been implemented for awhile and wasn't part of the GSoC project, but due to a lack of redirects it didn't make sense to advertise too broadly. Up until recently, we've tended to add all doc folders to the root index. In other words each subsection is visible at the top level, creating a rather long and <https://firefox-source-docs.mozilla.org/> incoherent <https://firefox-source-docs.mozilla.org/> list of pages to the left. This feature essentially means we can now group related docs together under common "landing pages". This will allow us to refactor the docs into a structure that makes more sense. For example we could have a landing page for docs describing Gecko's internals, and another one for docs describing developer workflows in `mozilla-central`. Mark Banner has already started to organize some of the docs under some of these landing pages. See this commit <https://hg.mozilla.org/mozilla-central/rev/f0eedecf8f2d> for a rough example. To clarify a few things: 1. The path specified in `SPHINX_TREES` *does not* need to correspond to a path in `mozilla-central`. For example, I could register my docs using `SPHINX_TREES["/foo"] = "docs"`, which would make that doc tree accessible at `firefox-source-docs.mozilla.org/foo`. 2. Any subtrees that are nested under another index will automatically be hidden from the main index. This means you should make sure to link to any subtrees from somewhere in the landing page. So given my earlier doc tree at `/foo`, if I now created a subtree and registered it using `SPHINX_TREES["/foo/bar"] = "docs"`, those docs would *not* show up in the main index. 3. The relation between subtrees and their parents does not necessarily have any bearing with their relation on the file system. For example, a doc tree that lives under `/devtools` can be nested under an index that lives under `/browser`. I realize this explanation is about as clear as mud, so feel free to reach out to me if you need a hand. Also keep an eye on bug 1576912 <https://bugzilla.mozilla.org/show_bug.cgi?id=1576912> which will provide a better explanation of how registration works. *ReStructuredText Linter* RST isn't the easiest of markup languages, but it's powerful and what `Sphinx` (the library used to build our docs) uses, so we're stuck with it. But at least we now have a linter which will catch basic problems in `.rst` files early. Be sure to run `./mach lint -o` to test your outgoing changes before submitting to review! *Mermaid Integration* Mermaid <https://mermaidjs.github.io/#/> is a tool that lets you generate flow charts, sequence diagrams, gantt charts, class diagrams and vcs graphs from a simple markup language. This means you can embed these directly in the source rather than creating an image with some external tool and then checking it into the tree. To add a diagram, simply put something like this into your page: .. mermaid:: > > graph TD; > A-->B; > A-->C; > B-->D; > C-->D; > See mermaid's official docs <https://mermaidjs.github.io/#/> for more details on the syntax. *Performance, Warnings and Lint Issues* Shivam also made many smaller changes, including modest perf improvements to the build, fixing some of the warnings generated from `Sphinx` and enabling the `rst` linter across the tree by fixing (almost) all pre-existing issues. *Final Words* Our in-tree documentation could be *a lot* better than it currently is. If you are interested in helping to re-organize and improve them, please reach out (especially if you have technical writing skills)! We now have many of the tools that we were previously lacking. I'd also like to thank Shivam for all the great work he put into this project! As always, let me know if you have any questions or concerns. Cheers, Andrew _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
