[
https://issues.apache.org/jira/browse/LUCENE-9321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097899#comment-17097899
]
Uwe Schindler edited comment on LUCENE-9321 at 5/2/20, 10:31 AM:
-----------------------------------------------------------------
Hi,
as said on the PR, there is actually two (or better three) different ways how
to consume the javadocs and each of those have different requirements with
inter-module links:
# One consumer is using the javadocs from the Maven JAR files (e.g., in the
IDE)
# The other consumers are using the website where the javadocs are all at one
place. They not only do this for releases, but also for snapshot builds by
Jenkins (see below, this makes the current "absolute" links not working they
way how it is setup).
# Somebody downloads the tar.gz file of Lucene and wants to browse the
javadocs from there. Actually I do this all the time when I am validating a
release (because that's the only way to do this, as the javadocs are not yet
deployed on central web page).
The consumer #1 is perfectly fine with current setup. For us it's easy to
package. The only things that is currenty borken is the way how the absolute
links are generated: They are hardcoded!!! This cannot be like that. We have
nightly snapshot builds on Jenkins where we producce snapshots where all
Javadocs go into nowhere. In the ANT build this is handled by making the
"Documentation base URL" configurable for Lucene/Solr: Instead of hardcoding
{{[https://lucene.apache.org/lucene/a_b_c]}} the Jenkins server sets a property
on the ANT invocation. By that all links which are absolute are correct. A
release manager can also set this, but there's currently automatism in ANT: If
the version does not end in "-SNAPSHOT", the links are generated using the
absolute links using the version number. We have version.properties for that.
This is how Jenkins (Solr 8.x Job) is setup, the same should be possible for
Gradle (just define "base URL path" with 2 properties):
!screenshot-1.png|width=753,height=316!
This allows to browse the full documentation here:
[https://builds.apache.org/view/L/view/Lucene/job/Solr-Artifacts-8.x/javadoc/]
(including valid absolute links also cross-project to Lucene). All Snapshot
artifacts deployed on snapshots.apache.org (including ZIP files) have those
links inside. This makes it easy for the user to browse and also somebody using
the artifacts in his IDE (think about Elasticsearch or any other projects using
snapshot artifacts from ASF). They are perfectly fine, it's now also better
than before!
No comes user #3: He downloads the targz/zip file and wants to browser Javadocs
or the development member who votes for a release. He wants to show the
javadocs. Unfortunately he can't as all links are dead (the Javadocs are not
yet published). Also somebody who downloaded the tar.gz file wants to dive
through the documentation with *relative* links. With just copying or
symlinking all Javadocs to some central folder, this isn't satisfied.
User #2 is somehow inbetween, but I tend to make him identical to user #3. I
don't like it to publish HTML pages on lucene.apache.org with absolute links to
lucene.apache.org. We recently changed to HTTPS, so for similar cases all links
in historic Javadocs would need to be rewritten. Thanks to redirects it still
works, but there can be man-in-the-middle problems. I wanted to download the
whole SVN repository in the near future and let run a {{sed}} through it to fix
all old links. This is major work. If links are all relative, you don't have
that problem.
bq. Other linting tasks in ant's "documentation-lint", ecjLint and
checkMissingDocs work fine with per-project javadoc folder.
They work, because documentation-lint does not check everything. The linter
does not follow absolute links, so it can't verify. It just passes. It's OK to
check that all links in the module are correct, but it can't check the full
documentation. So before a release "documentation-lint" must also be ran on the
tol level. This is a requirement for the release. But for this to work, the
links must be relative.
*Now comes my proposal:*
- I tend to leave the per-project javadocs as is, they should be used to build
maven artifacts. This makes IDE users happy and I hope also Dawid. The only
thing is to allow to configure the lucene and solr specific "base" url for
absolute links. This allows to make snapshot artifacts on Jenkins correctly.
Maybe also copy the "heuristic" from Ant to generate links based on "-SNAPSHOT"
or not.
- For the website and .tar.gz release (so packaging) the release manager should
run the whole javadocs a second time (we should *not* copy them). For this
second run for packaging purposes, we change the Javadocs output directory to
the top-level one (as proposed by Tomoko). In addition the absolute links
should be relative. This can easily be done using java.net.URI class. Just
build the absolute URI for the current project and the absolute URI for the
"link" statement on the javadocs command line. Then call URI#relativize and use
the relative URI as parameter to Javadocs. Make sure to use URI class for that,
don't do it by hand!!!! Don't use Path class, as we may have whitespace in
links. The result as embeeded in HTML pages is an URI, not a file system path
(also with forward slashes)! So the relative links are relative URIs, not
relative Paths!
IMHO, I'd make 2 tasks per project "renderJavadocs" (the current one, just with
configureable base URI) for per-module docs. Then there should be a second one
"renderGlobalJavadocs" per module, which uses the techniques described before
to regenerate. The ":lucene:documentation" task depends on those
renderGlobalJavadocs. There should also be a "documentation-lint" top-level.
The per-project ones should be renamed to "javadocs-lint".
was (Author: thetaphi):
Hi,
as said on the PR, there is actually two (or better three) different ways how
to consume the javadocs and each of those have different requirements with
inter-module links:
# One consumer is using the javadocs from the Maven JAR files (e.g., in the
IDE)
# The other consumers are using the website where the javadocs are all at one
place. They not only do this for releases, but also for snapshot builds by
Jenkins (see below, this makes the current "absolute" links not working they
way how it is setup).
# Somebody downloads the tar.gz file of Lucene and wants to browse the
javadocs from there. Actually I do this all the time when I am validating a
release (because that's the only way to do this, as the javadocs are not yet
deployed on central web page).
The consumer #1 is perfectly fine with current setup. For us it's easy to
package. The only things that is currenty borken is the way how the absolute
links are generated: They are hardcoded!!! This cannot be like that. We have
nightly snapshot builds on Jenkins where we producce snapshots where all
Javadocs go into nowhere. In the ANT build this is handled by making the
"Documentation base URL" configurable for Lucene/Solr: Instead of hardcoding
{{[https://lucene.apache.org/lucene/a_b_c]}} the Jenkins server sets a property
on the ANT invocation. By that all links which are absolute are correct. A
release manager can also set this, but there's currently automatism in ANT: If
the version does not end in "-SNAPSHOT", the links are generated using the
absolute links using the version number. We have version.properties for that.
This is how Jenkins (Solr 8.x Job) is setup, the same should be possible for
Gradle (just define "base URL path" with 2 properties):
!screenshot-1.png|width=753,height=316!
This allows to browse the full documentation here:
[https://builds.apache.org/view/L/view/Lucene/job/Solr-Artifacts-8.x/javadoc/]
(including valid absolute links also cross-project to Lucene). All Snapshot
artifacts deployed on snapshots.apache.org (including ZIP files) have those
links inside. This makes it easy for the user to browse and also somebody using
the artifacts in his IDE (think about Elasticsearch or any other projects using
snapshot artifacts from ASF). They are perfectly fine, it's now also better
than before!
No comes user #3: He downloads the targz/zip file and wants to browser Javadocs
or the development member who votes for a release. He wants to show the
javadocs. Unfortunately he can't as all links are dead (the Javadocs are not
yet published). Also somebody who downloaded the tar.gz file wants to dive
through the documentation with *relative* links. With just copying or
symlinking all Javadocs to some central folder, this isn't satisfied.
User #2 is somehow inbetween, but I tend to make him identical to user #3. I
don't like it to publish HTML pages on lucene.apache.org with absolute links to
lucene.apache.org. We recently changed to HTTPS, so for similar cases all links
in historic Javadocs would need to be rewritten. Thanks to redirects it still
works, but there can be man-in-the-middle problems. I wanted to download the
whole SVN repository in the near future and let run a {{sed}} through it to fix
all old links. This is major work. If links are all relative, you don't have
that problem.
bq. Other linting tasks in ant's "documentation-lint", ecjLint and
checkMissingDocs work fine with per-project javadoc folder.
They work, because documentation-lint does not check everything. The linter
does not follow absolute links, so it can't verify. It just passes. It's OK to
check that all links in the module are correct, but it can't check the full
documentation. So before a release "documentation-lint" must also be ran on the
tol level. This is a requirement for the release. But for this to work, the
links must be relative.
*Now comes my proposal:*
- I tend to leave the per-project javadocs as is, they should be used to build
maven artifacts. This makes IDE users happy and I hope also Dawid. The only
thing is to allow to configure the lucene and solr specific "base" url for
absolute links. This allows to make snapshot artifacts on Jenkins correctly.
Maybe also copy the "heuristic" from Ant to generate links based on "-SNAPSHOT"
or not.
- For the website and .tar.gz release (so packaging) the release manager should
run the whole javadocs a second time (we should *not* copy them). For this
second run for packaging purposes, we change the Javadocs output directory to
the top-level one (as proposed by Tomoko). In addition the absolute links
should be relative. This can easily be done using java.net.URI class. Just
build the absolute URI for the current project and the absolute URI for the
"link" statement on the javadocs command line. Then call URI#relativize and use
the relative URI as parameter to Javadocs. Make sure to use URI class for that,
don't do it by hand!!!! Don't use Path class, as we may have whitespace in
links. The result as embeeded in HTML pages is an URI, not a file system path
(also with forward slashes)! So the relative links are relative URIs, not
relative Paths!
IMHO, I'd make 2 tasks per project "renderJavadocs" (the current one, just with
configureable base URI) for per-module docs. Then there should be a second one
"renderGlobalJavadocs" per module, which uses the techniques described before
to regenerate. The ":lucene:documentation" task depends on those
renderGlobalJavadocs.
> Port documentation task to gradle
> ---------------------------------
>
> Key: LUCENE-9321
> URL: https://issues.apache.org/jira/browse/LUCENE-9321
> Project: Lucene - Core
> Issue Type: Sub-task
> Components: general/build
> Reporter: Tomoko Uchida
> Assignee: Tomoko Uchida
> Priority: Major
> Attachments: screenshot-1.png
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> This is a placeholder issue for porting ant "documentation" task to gradle.
> The generated documents should be able to be published on lucene.apache.org
> web site on "as-is" basis.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
