mcvsubbu commented on a change in pull request #4284: Put managing Pinot under in_production URL: https://github.com/apache/incubator-pinot/pull/4284#discussion_r291257422
########## File path: docs/in_production.rst ########## @@ -46,17 +46,81 @@ The ordering is as follows: Managing Pinot ~~~~~~~~~~~~~~ -Creating tables ---------------- +There are two ways to manage Pinot cluster, i.e. using Pinot management console and ``pinot-admin.sh`` script. -Updating tables ---------------- +Pinot Management Console +------------------------ -Uploading data +There is a REST API which allows management of tables, tenants, segments and schemas. It can be accessed by going to +``http://[controller_host]/help`` which offers a web UI to do these tasks, as well as document the REST API. The below +is the screenshot of the console. + + .. figure:: img/pinot-console.png + +For example, to list all the schemas within Pinot cluster: + + .. figure:: img/list-schemas.png + +To rebalance segments of a table across servers: + + .. figure:: img/rebalance-table.png + +pinot-admin.sh -------------- -Configuring realtime data ingestion ------------------------------------ +``pinot-admin.sh`` is another way of managing Pinot cluster. This script can be generated by running +``mvn install package -DskipTests -Pbin-dist`` in the directory in which you checked out Pinot. + +For example, to create a pinot segment: + +.. code-block:: none + + $ ./pinot-distribution/target/apache-pinot-incubating-0.1.0-SNAPSHOT-bin/apache-pinot-incubating-0.1.0-SNAPSHOT-bin/bin/pinot-admin.sh CreateSegment -dataDir /Users/host1/Desktop/test/ -format CSV -outDir /Users/host1/Desktop/test2/ -tableName baseballStats -segmentName baseballStats_data -overwrite -schemaFile ./pinot-distribution/target/apache-pinot-incubating-0.1.0-SNAPSHOT-bin/apache-pinot-incubating-0.1.0-SNAPSHOT-bin/sample_data/baseballStats_schema.json + Executing command: CreateSegment -generatorConfigFile null -dataDir /Users/host1/Desktop/test/ -format CSV -outDir /Users/host1/Desktop/test2/ -overwrite true -tableName baseballStats -segmentName baseballStats_data -timeColumnName null -schemaFile ./pinot-distribution/target/apache-pinot-incubating-0.1.0-SNAPSHOT-bin/apache-pinot-incubating-0.1.0-SNAPSHOT-bin/sample_data/baseballStats_schema.json -readerConfigFile null -enableStarTreeIndex false -starTreeIndexSpecFile null -hllSize 9 -hllColumns null -hllSuffix _hll -numThreads 1 + Accepted files: [/Users/host1/Desktop/test/baseballStats_data.csv] + Finished building StatsCollector! + Collected stats for 97889 documents + Created dictionary for INT column: homeRuns with cardinality: 67, range: 0 to 73 + Created dictionary for INT column: playerStint with cardinality: 5, range: 1 to 5 + Created dictionary for INT column: groundedIntoDoublePlays with cardinality: 35, range: 0 to 36 + Created dictionary for INT column: numberOfGames with cardinality: 165, range: 1 to 165 + Created dictionary for INT column: AtBatting with cardinality: 699, range: 0 to 716 + Created dictionary for INT column: stolenBases with cardinality: 114, range: 0 to 138 + Created dictionary for INT column: tripples with cardinality: 32, range: 0 to 36 + Created dictionary for INT column: hitsByPitch with cardinality: 41, range: 0 to 51 + Created dictionary for STRING column: teamID with cardinality: 149, max length in bytes: 3, range: ALT to WSU + Created dictionary for INT column: numberOfGamesAsBatter with cardinality: 166, range: 0 to 165 + Created dictionary for INT column: strikeouts with cardinality: 199, range: 0 to 223 + Created dictionary for INT column: sacrificeFlies with cardinality: 20, range: 0 to 19 + Created dictionary for INT column: caughtStealing with cardinality: 36, range: 0 to 42 + Created dictionary for INT column: baseOnBalls with cardinality: 154, range: 0 to 232 + Created dictionary for STRING column: playerName with cardinality: 11976, max length in bytes: 43, range: to Zoilo Casanova + Created dictionary for INT column: doules with cardinality: 64, range: 0 to 67 + Created dictionary for STRING column: league with cardinality: 7, max length in bytes: 2, range: AA to UA + Created dictionary for INT column: yearID with cardinality: 143, range: 1871 to 2013 + Created dictionary for INT column: hits with cardinality: 250, range: 0 to 262 + Created dictionary for INT column: runsBattedIn with cardinality: 175, range: 0 to 191 + Created dictionary for INT column: G_old with cardinality: 166, range: 0 to 165 + Created dictionary for INT column: sacrificeHits with cardinality: 54, range: 0 to 67 + Created dictionary for INT column: intentionalWalks with cardinality: 45, range: 0 to 120 + Created dictionary for INT column: runs with cardinality: 167, range: 0 to 192 + Created dictionary for STRING column: playerID with cardinality: 18107, max length in bytes: 9, range: aardsda01 to zwilldu01 + Start building IndexCreator! + Finished records indexing in IndexCreator! + Finished segment seal! + Converting segment: /Users/host1/Desktop/test2/baseballStats_data_0 to v3 format + v3 segment location for segment: baseballStats_data_0 is /Users/host1/Desktop/test2/baseballStats_data_0/v3 + Deleting files in v1 segment directory: /Users/host1/Desktop/test2/baseballStats_data_0 + Driver, record read time : 369 + Driver, stats collector time : 0 + Driver, indexing time : 373 + +To query a table: Review comment: ```suggestion Here is an example of executing a query on a Pinot table: ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
