This is an automated email from the ASF dual-hosted git repository.
mcvsubbu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new ea2d621 Added documentation regarding design docs (#4002)
ea2d621 is described below
commit ea2d6215421cf422a1740bac97fdf44e001439b9
Author: Subbu Subramaniam <mcvsu...@users.noreply.github.com>
AuthorDate: Fri Mar 22 10:18:19 2019 -0700
Added documentation regarding design docs (#4002)
Added documentation pointing to design docs.
Also fixed some warnings thrown during compilation
---
docs/code_modules.rst | 5 +--
docs/contribution_guidelines.rst | 75 ++++++++++++++++++++++++----------------
docs/pinot_hadoop.rst | 3 +-
3 files changed, 51 insertions(+), 32 deletions(-)
diff --git a/docs/code_modules.rst b/docs/code_modules.rst
index 33e5a20..cf439cb 100644
--- a/docs/code_modules.rst
+++ b/docs/code_modules.rst
@@ -141,13 +141,14 @@ Auxiliary modules
In addition to the core modules described above, Pinot code provides the
following modules:
* ``pinot-tools``: This module is a collection of many tools useful for
setting up Pinot cluster, creating/updating segments.
-It also houses the Pinot quick start guide code.
+ It also houses the Pinot quick start guide code.
* ``pinot-perf``: This module has a collection of benchmark test code used to
evaluate design options.
* ``pinot-client-api``: This module houses the Java client API. See
:ref:`java-client` for more info.
* ``pinot-integration-tests``: This module holds integration tests that test
functionality across multiple classes or components.
+
These tests typically do not rely on mocking and provide more end to end
coverage for code.
.. _extension-modules:
@@ -157,4 +158,4 @@ Extension modules
``pinot-hadoop-filesystem`` and ``pinot-azure-filesystem`` are module added to
support extensions to Pinot filesystem.
The functionality is broken down into modules of their own to avoid polluting
the common modules with additional large libraries.
These libraries bring in transitive dependencies of their own that can cause
classpath conflicts at runtime. We would like to
-avoid this for the common usage of Pinot as much as possible.
\ No newline at end of file
+avoid this for the common usage of Pinot as much as possible.
diff --git a/docs/contribution_guidelines.rst b/docs/contribution_guidelines.rst
index e980158..f4a4b47 100644
--- a/docs/contribution_guidelines.rst
+++ b/docs/contribution_guidelines.rst
@@ -25,12 +25,26 @@ Contribution Guidelines
Before you begin to contribute, make sure you have reviewed :ref:`dev-setup`
and :ref:`code-modules` sections and that you have created your own fork of the
pinot source code.
+Create a design document
+------------------------
+If your change is relatively minor, you can skip this step. If you are
+adding new major feature, we suggest that you add a design document
+and solicit comments from the community before submitting any code.
+
+`Here <https://cwiki.apache.org/confluence/display/PINOT/Design+Documents>`_
+is a list of current design documents.
+
+
Create an issue for the change
------------------------------
Create a Pinot issue for the change you would like to make. Provide
information on why the change is needed and how you
plan to address it. Use the conversations on the issue as a way to validate
assumptions and the right way to proceed.
Be sure to review sections on :ref:`compatibility-changes` and
:ref:`external-libs`.
+If you have a design document, please refer to the design documents
+in your Issue. You may even want to create multiple
+issues depending on the extent of your change.
+
Once you are clear about what you want to do, proceed with the next steps
listed below.
Create a branch for your change
@@ -61,8 +75,8 @@ Code documentation
Please ensure your code is adequately documented. Some things to consider for
documentation:
* Always include class level java docs.
-At the top class level, we are looking for information about what
functionality is provided by the class,
-what state is maintained by the class, whether there are
concurrency/thread-safety concerns and any exceptional behavior that the class
might exhibit.
+ At the top class level, we are looking for information about what
functionality is provided by the class,
+ what state is maintained by the class, whether there are
concurrency/thread-safety concerns and any exceptional behavior that the class
might exhibit.
* Document public methods and their parameters.
@@ -101,8 +115,8 @@ Be cautious about pulling in external dependencies. You
will need to consider mu
* Is the external library maintained by an active community of contributors?
* What are the licensing terms for the library. For more information about
handling licenses, see :ref:`handling-licenses`.
* Are you adding the library to :ref:`pinot-foundation` modules? This will
affect the rest of the Pinot code base.
-If the new library pulls in a lot of transitive dependencies, then we might
encounter unexpected issues with multiple classes in the classpath.
-These issues are hard to catch with tests as the order of loading the
libraries at runtime matters. If you absolutely need the support, consider
adding it via extension modules, see :ref:`extension-modules`.
+ If the new library pulls in a lot of transitive dependencies, then we might
encounter unexpected issues with multiple classes in the classpath.
+ These issues are hard to catch with tests as the order of loading the
libraries at runtime matters. If you absolutely need the support, consider
adding it via extension modules, see :ref:`extension-modules`.
Testing your changes
^^^^^^^^^^^^^^^^^^^^
@@ -141,13 +155,13 @@ All source code files should have license headers. To
automatically add the head
.. note::
-If you checkin third-party code or files, please make sure you review Apache
guidelines:
+ If you checkin third-party code or files, please make sure you review Apache
guidelines:
-* `Licences that can be included
<https://www.apache.org/legal/resolved.html#what-can-we-include-in-an-asf-project-category-a>`_
+ * `Licences that can be included
<https://www.apache.org/legal/resolved.html#what-can-we-include-in-an-asf-project-category-a>`_
-* `Licences that may be included
<https://www.apache.org/legal/resolved.html#what-can-we-maybe-include-in-an-asf-project-category-b>`_
+ * `Licences that may be included
<https://www.apache.org/legal/resolved.html#what-can-we-maybe-include-in-an-asf-project-category-b>`_
-* `Licenses that should not be included
<https://www.apache.org/legal/resolved.html#what-can-we-not-include-in-an-asf-project-category-x>`_
+ * `Licenses that should not be included
<https://www.apache.org/legal/resolved.html#what-can-we-not-include-in-an-asf-project-category-x>`_
Once you determine the code you are pulling in adhere to the guidelines above,
go ahead pull the changes in.
Do not add license headers for them. Follow these instructions to ensure we
are compliant with Apache Licensing process:
@@ -159,12 +173,12 @@ Do not add license headers for them. Follow these
instructions to ensure we are
If attention to the licensing terms in not paid early on, they will be caught
much later in the process, when we prepare to make a new release.
Updating code at that time to work with the right libraries at that time might
require bigger refactoring changes and delay the release process.
-Creating a PR
--------------
+Creating a Pull Request (PR)
+----------------------------
* Verifying code-style
-Run the following command to verify the code-style before posting a PR
+ Run the following command to verify the code-style before posting a PR
.. code-block:: none
@@ -172,14 +186,14 @@ Run the following command to verify the code-style before
posting a PR
* Run tests
-Before you create a review request for the changes, make sure you have run the
corresponding unit tests for your changes.
-You can run individual tests via the IDE or via maven command-line. Finally
run all tests locally by running ``mvn clean install -Pbin-dist``.
+ Before you create a review request for the changes, make sure you have run
the corresponding unit tests for your changes.
+ You can run individual tests via the IDE or via maven command-line. Finally
run all tests locally by running ``mvn clean install -Pbin-dist``.
-For changes that are related to performance issues or race conditions, it is
hard to write reliable tests, so we recommend running manual stress tests to
validate the changes. You ``MUST`` note the manual tests done in the PR
description.
+ For changes that are related to performance issues or race conditions, it is
hard to write reliable tests, so we recommend running manual stress tests to
validate the changes. You ``MUST`` note the manual tests done in the PR
description.
* Push changes and create a PR for review
-Commit your changes with a meaningful commit message.
+ Commit your changes with a meaningful commit message.
.. code-block:: none
@@ -187,24 +201,25 @@ Commit your changes with a meaningful commit message.
$ git commit -m "Meaningful oneliner for the change"
$ git push origin <your issue branch>
-After this, create a PullRequest in `github
<https://github.com/apache/incubator-pinot/pulls>`_. Include the following
information in the description:
+ After this, create a PullRequest in `github
<https://github.com/apache/incubator-pinot/pulls>`_. Include the following
information in the description:
+
+ * The changes that are included in the PR.
-* The changes that are included in the PR.
+ * Design document, if any.
-* Information on any implementation choices that were made.
+ * Information on any implementation choices that were made.
-* Evidence of sufficient testing. You ``MUST`` indicate the tests done, either
manually or automated.
+ * Evidence of sufficient testing. You ``MUST`` indicate the tests done,
either manually or automated.
-Once the PR is created, the code base is compiled and all tests are run via
``travis``. Make sure you followup on any issues flagged by travis and address
them.
-If you see test failures that are intermittent, ``please`` create an issue to
track them.
+ Once the PR is created, the code base is compiled and all tests are run via
``travis``. Make sure you followup on any issues flagged by travis and address
them.
+ If you see test failures that are intermittent, ``please`` create an issue
to track them.
-Once the ``travis`` run is clear, request reviews from atleast 2 committers on
the project and be sure to gently to followup on the issue with the reviewers.
+ Once the ``travis`` run is clear, request reviews from atleast 2 committers
on the project and be sure to gently to followup on the issue with the
reviewers.
-* Addressing comments and Rebasing
-Once you receive comments on github on your changes, be sure to respond to
them on github and address the concerns.
-If any discussions happen offline for the changes in question, make sure to
capture the outcome of the discussion, so others can follow along as well.
+* Once you receive comments on github on your changes, be sure to respond to
them on github and address the concerns.
+ If any discussions happen offline for the changes in question, make sure to
capture the outcome of the discussion, so others can follow along as well.
-It is possible that while your change is being reviewed, other changes were
made to the master branch. Be sure to pull rebase your change on the new
changes thus:
+ It is possible that while your change is being reviewed, other changes were
made to the master branch. Be sure to pull rebase your change on the new
changes thus:
.. code-block:: none
@@ -217,12 +232,14 @@ It is possible that while your change is being reviewed,
other changes were made
$ git checkout <your issue branch>
$ git rebase master
-At this time, if rebase flags any conflicts, resolve the conflicts and follow
the instructions provided by the rebase command.
+ At this time, if rebase flags any conflicts, resolve the conflicts and
follow the instructions provided by the rebase command.
-Run additional tests/validations for the new changes and update the PR by
pushing your changes:
+ Run additional tests/validations for the new changes and update the PR by
pushing your changes:
.. code-block:: none
$ git push origin <your issue branch>
-* Once your change is merged, check to see if any documentation needs to be
updated. If so, create a PR for documentation.
+* When you have addressed all comments and have an approved PR, one of the
committers can merge your PR.
+
+* After your change is merged, check to see if any documentation needs to be
updated. If so, create a PR for documentation.
diff --git a/docs/pinot_hadoop.rst b/docs/pinot_hadoop.rst
index 2211f06..fc3573b 100644
--- a/docs/pinot_hadoop.rst
+++ b/docs/pinot_hadoop.rst
@@ -205,6 +205,7 @@ To consume in realtime, we simply need to create a table
that uses the same sche
consume from, using a table definition such as this one:
.. code-block:: none
+
{
"tableName":"flights",
"segmentsConfig" : {
@@ -260,4 +261,4 @@ We can then query the table and see the events stream in:
{"traceInfo":{},"numDocsScanned":17,"aggregationResults":[{"function":"count_star","value":"17"}],"timeUsedMs":27,"segmentStatistics":[],"exceptions":[],"totalDocs":17}
-Repeating the query multiple times should show the events slowly being
streamed into the table.
\ No newline at end of file
+Repeating the query multiple times should show the events slowly being
streamed into the table.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
