Author: billie
Date: Mon Mar 17 21:10:11 2014
New Revision: 1578577
URL: http://svn.apache.org/r1578577
Log:
ACCUMULO-2467 updated manuals and examples for 1.4 and 1.5
Added:
accumulo/site/trunk/content/1.4/user_manual/Development_Clients.mdtext
(with props)
Modified:
accumulo/site/trunk/content/1.4/accumulo_user_manual.pdf
accumulo/site/trunk/content/1.4/examples/bulkIngest.mdtext
accumulo/site/trunk/content/1.4/examples/dirlist.mdtext
accumulo/site/trunk/content/1.4/examples/filedata.mdtext
accumulo/site/trunk/content/1.4/examples/mapred.mdtext
accumulo/site/trunk/content/1.4/user_manual/Administration.mdtext
accumulo/site/trunk/content/1.4/user_manual/Contents.mdtext
accumulo/site/trunk/content/1.4/user_manual/Table_Configuration.mdtext
accumulo/site/trunk/content/1.4/user_manual/Writing_Accumulo_Clients.mdtext
accumulo/site/trunk/content/1.4/user_manual/accumulo_user_manual.mdtext
accumulo/site/trunk/content/1.4/user_manual/index.mdtext
accumulo/site/trunk/content/1.5/accumulo_user_manual.html
accumulo/site/trunk/content/1.5/examples/filedata.mdtext
accumulo/site/trunk/content/1.5/examples/index.mdtext
Modified: accumulo/site/trunk/content/1.4/accumulo_user_manual.pdf
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/accumulo_user_manual.pdf?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
Binary files - no diff available.
Modified: accumulo/site/trunk/content/1.4/examples/bulkIngest.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/examples/bulkIngest.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/examples/bulkIngest.mdtext (original)
+++ accumulo/site/trunk/content/1.4/examples/bulkIngest.mdtext Mon Mar 17
21:10:11 2014
@@ -28,7 +28,7 @@ accumulo instance name, and a comma-sepa
$ ./bin/accumulo
org.apache.accumulo.examples.simple.mapreduce.bulk.SetupTable instance
zookeepers username password test_bulk row_00000333 row_00000666
$ ./bin/accumulo
org.apache.accumulo.examples.simple.mapreduce.bulk.GenerateTestData 0 1000
bulk/test_1.txt
- $ ./bin/tool.sh lib/examples-simple-*[^c].jar
org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample instance
zookeepers username password test_bulk bulk tmp/bulkWork
+ $ ./bin/tool.sh lib/examples-simple-*[^cs].jar
org.apache.accumulo.examples.simple.mapreduce.bulk.BulkIngestExample instance
zookeepers username password test_bulk bulk tmp/bulkWork
$ ./bin/accumulo
org.apache.accumulo.examples.simple.mapreduce.bulk.VerifyIngest instance
zookeepers username password test_bulk 0 1000
For a high level discussion of bulk ingest, see the docs dir.
Modified: accumulo/site/trunk/content/1.4/examples/dirlist.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/examples/dirlist.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/examples/dirlist.mdtext (original)
+++ accumulo/site/trunk/content/1.4/examples/dirlist.mdtext Mon Mar 17 21:10:11
2014
@@ -37,14 +37,18 @@ This may take some time if there are lar
Note that running this example will create tables dirTable, indexTable, and
dataTable in Accumulo that you should delete when you have completed the
example.
If you modify a file or add new files in the directory ingested (e.g.
/local/username/workspace), you can run Ingest again to add new information
into the Accumulo tables.
-To browse the data ingested, use Viewer.java. Be sure to give the "username"
user the authorizations to see the data (in this case, run "setauths -u
username -s exampleVis" in the shell, and use the string "exampleVis" as the
"auths" in command lines below).
+To browse the data ingested, use Viewer.java. Be sure to give the "username"
user the authorizations to see the data (in this case, run
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.Viewer
instance zookeepers username password dirTable dataTable auths
/local/username/workspace
+ $ ./bin/accumulo shell -u root -e 'setauths -u username -s exampleVis'
+
+then run the Viewer:
+
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.Viewer
instance zookeepers username password dirTable dataTable exampleVis
/local/username/workspace
To list the contents of specific directories, use QueryUtil.java.
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil
instance zookeepers username password dirTable auths /local/username
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil
instance zookeepers username password dirTable auths /local/username/workspace
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil
instance zookeepers username password dirTable exampleVis /local/username
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil
instance zookeepers username password dirTable exampleVis
/local/username/workspace
To perform searches on file or directory names, also use QueryUtil.java.
Search terms must contain no more than one wild card and cannot contain "/".
*Note* these queries run on the _indexTable_ table instead of the dirTable
table.
Modified: accumulo/site/trunk/content/1.4/examples/filedata.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/examples/filedata.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/examples/filedata.mdtext (original)
+++ accumulo/site/trunk/content/1.4/examples/filedata.mdtext Mon Mar 17
21:10:11 2014
@@ -23,8 +23,8 @@ The example has the following classes:
* ChunkCombiner - An Iterator that dedupes file data and sets their
visibilities to a combined visibility based on current references to the file
data.
* ChunkInputFormat - An Accumulo InputFormat that provides keys containing
file info (List<Entry<Key,Value>>) and values with an InputStream over the file
(ChunkInputStream).
* ChunkInputStream - An input stream over file data stored in Accumulo.
- * FileDataIngest - Takes a list of files and archives them into Accumulo
keyed on the SHA1 hashes of the files.
- * FileDataQuery - Retrieves file data based on the SHA1 hash of the file.
(Used by the dirlist.Viewer.)
+ * FileDataIngest - Takes a list of files and archives them into Accumulo
keyed on hashes of the files.
+ * FileDataQuery - Retrieves file data based on the hash of the file. (Used by
the dirlist.Viewer.)
* KeyUtil - A utility for creating and parsing null-byte separated strings
into/from Text objects.
* VisibilityCombiner - A utility for merging visibilities into the form
(VIS1)|(VIS2)|...
@@ -40,7 +40,7 @@ Open the accumulo shell and look at the
Run the CharacterHistogram MapReduce to add some information about the file.
- $ bin/tool.sh lib/examples-simple*[^c].jar
org.apache.accumulo.examples.simple.filedata.CharacterHistogram instance
zookeepers username password dataTable exampleVis exampleVis
+ $ bin/tool.sh lib/examples-simple*[^cs].jar
org.apache.accumulo.examples.simple.filedata.CharacterHistogram instance
zookeepers username password dataTable exampleVis exampleVis
Scan again to see the histogram stored in the 'info' column family.
Modified: accumulo/site/trunk/content/1.4/examples/mapred.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/examples/mapred.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/examples/mapred.mdtext (original)
+++ accumulo/site/trunk/content/1.4/examples/mapred.mdtext Mon Mar 17 21:10:11
2014
@@ -50,7 +50,7 @@ for the column family count.
After creating the table, run the word count map reduce job.
- $ bin/tool.sh lib/examples-simple*[^c].jar
org.apache.accumulo.examples.simple.mapreduce.WordCount instance zookeepers
/user/username/wc wordCount -u username -p password
+ $ bin/tool.sh lib/examples-simple*[^cs].jar
org.apache.accumulo.examples.simple.mapreduce.WordCount instance zookeepers
/user/username/wc wordCount -u username -p password
11/02/07 18:20:11 INFO input.FileInputFormat: Total input paths to process
: 1
11/02/07 18:20:12 INFO mapred.JobClient: Running job: job_201102071740_0003
Modified: accumulo/site/trunk/content/1.4/user_manual/Administration.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/Administration.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/Administration.mdtext (original)
+++ accumulo/site/trunk/content/1.4/user_manual/Administration.mdtext Mon Mar
17 21:10:11 2014
@@ -142,6 +142,30 @@ To verify that Accumulo is running, chec
To shutdown cleanly, run bin/stop-all.sh and the master will orchestrate the
shutdown of all the tablet servers. Shutdown waits for all minor compactions to
finish, so it may take some time for particular configurations.
+### <a id=Adding_a_Node></a> Adding a Node
+
+Update your $ACCUMULO_HOME/conf/slaves (or $ACCUMULO_CONF_DIR/slaves) file to
account for the addition.
+
+
+ $ACCUMULO_HOME/bin/accumulo admin start <host(s)> {<host> ...}
+
+
+Alternatively, you can ssh to each of the hosts you want to add and run
$ACCUMULO_HOME/bin/start-here.sh.
+
+Make sure the host in question has the new configuration, or else the tablet
server won't start; at a minimum this needs to be on the host(s) being added,
but in practice it's good to ensure consistent configuration across all nodes.
+
+### <a id=Decomissioning_a_Node></a> Decomissioning a Node
+
+If you need to take a node out of operation, you can trigger a graceful
shutdown of a tablet server. Accumulo will automatically rebalance the tablets
across the available tablet servers.
+
+
+ $ACCUMULO_HOME/bin/accumulo admin stop <host(s)> {<host> ...}
+
+
+Alternatively, you can ssh to each of the hosts you want to remove and run
$ACCUMULO_HOME/bin/stop-here.sh.
+
+Be sure to update your $ACCUMULO_HOME/conf/slaves (or
$ACCUMULO_CONF_DIR/slaves) file to account for the removal of these hosts. Bear
in mind that the monitor will not re-read the slaves file automatically, so it
will report the decomissioned servers as down; it's recommended that you
restart the monitor so that the node list is up to date.
+
## <a id=Monitoring></a> Monitoring
The Accumulo Master provides an interface for monitoring the status and health
of Accumulo components. This interface can be accessed by pointing a web
browser to
Modified: accumulo/site/trunk/content/1.4/user_manual/Contents.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/Contents.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/Contents.mdtext (original)
+++ accumulo/site/trunk/content/1.4/user_manual/Contents.mdtext Mon Mar 17
21:10:11 2014
@@ -65,107 +65,124 @@ Notice: Licensed to the Apache Softwa
* [Isolated Scanner][30]
* [BatchScanner][31]
+ * [Proxy][32]
+
+ * [Prequisites][33]
+ * [Configuration][34]
+ * [Running the Proxy Server][35]
+ * [Creating a Proxy Client][36]
+ * [Using a Proxy Client][37]
+
+
+
+* [Development Clients][38]
+
+ * [Mock Accumulo][39]
+ * [Mini Accumulo Cluster][40]
+
-* [Table Configuration][32]
+* [Table Configuration][41]
- * [Locality Groups][33]
+ * [Locality Groups][42]
- * [Managing Locality Groups via the Shell][34]
- * [Managing Locality Groups via the Client API][35]
+ * [Managing Locality Groups via the Shell][43]
+ * [Managing Locality Groups via the Client API][44]
- * [Constraints][36]
- * [Bloom Filters][37]
- * [Iterators][38]
+ * [Constraints][45]
+ * [Bloom Filters][46]
+ * [Iterators][47]
- * [Setting Iterators via the Shell][39]
- * [Setting Iterators Programmatically][40]
- * [Versioning Iterators and Timestamps][41]
- * [Filters][42]
- * [Combiners][43]
+ * [Setting Iterators via the Shell][48]
+ * [Setting Iterators Programmatically][49]
+ * [Versioning Iterators and Timestamps][50]
+ * [Filters][51]
+ * [Combiners][52]
- * [Block Cache][44]
- * [Compaction][45]
- * [Pre-splitting tables][46]
- * [Merging tablets][47]
- * [Delete Range][48]
- * [Cloning Tables][49]
+ * [Block Cache][53]
+ * [Compaction][54]
+ * [Pre-splitting tables][55]
+ * [Merging tablets][56]
+ * [Delete Range][57]
+ * [Cloning Tables][58]
-* [Table Design][50]
+* [Table Design][59]
- * [Basic Table][51]
- * [RowID Design][52]
- * [Indexing][53]
- * [Entity-Attribute and Graph Tables][54]
- * [Document-Partitioned Indexing][55]
+ * [Basic Table][60]
+ * [RowID Design][61]
+ * [Indexing][62]
+ * [Entity-Attribute and Graph Tables][63]
+ * [Document-Partitioned Indexing][64]
-* [High-Speed Ingest][56]
+* [High-Speed Ingest][65]
- * [Pre-Splitting New Tables][57]
- * [Multiple Ingester Clients][58]
- * [Bulk Ingest][59]
- * [Logical Time for Bulk Ingest][60]
- * [MapReduce Ingest][61]
+ * [Pre-Splitting New Tables][66]
+ * [Multiple Ingester Clients][67]
+ * [Bulk Ingest][68]
+ * [Logical Time for Bulk Ingest][69]
+ * [MapReduce Ingest][70]
-* [Analytics][62]
+* [Analytics][71]
- * [MapReduce][63]
+ * [MapReduce][72]
- * [Mapper and Reducer classes][64]
- * [AccumuloInputFormat options][65]
- * [AccumuloOutputFormat options][66]
+ * [Mapper and Reducer classes][73]
+ * [AccumuloInputFormat options][74]
+ * [AccumuloOutputFormat options][75]
- * [Combiners][67]
+ * [Combiners][76]
- * [Feature Vectors][68]
+ * [Feature Vectors][77]
- * [Statistical Modeling][69]
+ * [Statistical Modeling][78]
-* [Security][70]
+* [Security][79]
- * [Security Label Expressions][71]
- * [Security Label Expression Syntax][72]
- * [Authorization][73]
- * [User Authorizations][74]
- * [Secure Authorizations Handling][75]
- * [Query Services Layer][76]
+ * [Security Label Expressions][80]
+ * [Security Label Expression Syntax][81]
+ * [Authorization][82]
+ * [User Authorizations][83]
+ * [Secure Authorizations Handling][84]
+ * [Query Services Layer][85]
-* [Administration][77]
+* [Administration][86]
- * [Hardware][78]
- * [Network][79]
- * [Installation][80]
- * [Dependencies][81]
- * [Configuration][82]
+ * [Hardware][87]
+ * [Network][88]
+ * [Installation][89]
+ * [Dependencies][90]
+ * [Configuration][91]
- * [Edit conf/accumulo-env.sh][83]
- * [Cluster Specification][84]
- * [Accumulo Settings][85]
- * [Deploy Configuration][86]
+ * [Edit conf/accumulo-env.sh][92]
+ * [Cluster Specification][93]
+ * [Accumulo Settings][94]
+ * [Deploy Configuration][95]
- * [Initialization][87]
- * [Running][88]
+ * [Initialization][96]
+ * [Running][97]
- * [Starting Accumulo][89]
- * [Stopping Accumulo][90]
+ * [Starting Accumulo][98]
+ * [Stopping Accumulo][99]
+ * [Adding a Node][100]
+ * [Decomissioning a Node][101]
- * [Monitoring][91]
- * [Logging][92]
- * [Recovery][93]
+ * [Monitoring][102]
+ * [Logging][103]
+ * [Recovery][104]
-* [Shell Commands][94]
+* [Shell Commands][105]
@@ -200,67 +217,78 @@ Notice: Licensed to the Apache Softwa
[29]: Writing_Accumulo_Clients.html#Scanner
[30]: Writing_Accumulo_Clients.html#Isolated_Scanner
[31]: Writing_Accumulo_Clients.html#BatchScanner
- [32]: Table_Configuration.html
- [33]: Table_Configuration.html#Locality_Groups
- [34]: Table_Configuration.html#Managing_Locality_Groups_via_the_Shell
- [35]: Table_Configuration.html#Managing_Locality_Groups_via_the_Client_API
- [36]: Table_Configuration.html#Constraints
- [37]: Table_Configuration.html#Bloom_Filters
- [38]: Table_Configuration.html#Iterators
- [39]: Table_Configuration.html#Setting_Iterators_via_the_Shell
- [40]: Table_Configuration.html#Setting_Iterators_Programmatically
- [41]: Table_Configuration.html#Versioning_Iterators_and_Timestamps
- [42]: Table_Configuration.html#Filters
- [43]: Table_Configuration.html#Combiners
- [44]: Table_Configuration.html#Block_Cache
- [45]: Table_Configuration.html#Compaction
- [46]: Table_Configuration.html#Pre-splitting_tables
- [47]: Table_Configuration.html#Merging_tablets
- [48]: Table_Configuration.html#Delete_Range
- [49]: Table_Configuration.html#Cloning_Tables
- [50]: Table_Design.html
- [51]: Table_Design.html#Basic_Table
- [52]: Table_Design.html#RowID_Design
- [53]: Table_Design.html#Indexing
- [54]: Table_Design.html#Entity-Attribute_and_Graph_Tables
- [55]: Table_Design.html#Document-Partitioned_Indexing
- [56]: High_Speed_Ingest.html
- [57]: High_Speed_Ingest.html#Pre-Splitting_New_Tables
- [58]: High_Speed_Ingest.html#Multiple_Ingester_Clients
- [59]: High_Speed_Ingest.html#Bulk_Ingest
- [60]: High_Speed_Ingest.html#Logical_Time_for_Bulk_Ingest
- [61]: High_Speed_Ingest.html#MapReduce_Ingest
- [62]: Analytics.html
- [63]: Analytics.html#MapReduce
- [64]: Analytics.html#Mapper_and_Reducer_classes
- [65]: Analytics.html#AccumuloInputFormat_options
- [66]: Analytics.html#AccumuloOutputFormat_options
- [67]: Analytics.html#Combiners
- [68]: Analytics.html#Feature_Vectors
- [69]: Analytics.html#Statistical_Modeling
- [70]: Security.html
- [71]: Security.html#Security_Label_Expressions
- [72]: Security.html#Security_Label_Expression_Syntax
- [73]: Security.html#Authorization
- [74]: Security.html#User_Authorizations
- [75]: Security.html#Secure_Authorizations_Handling
- [76]: Security.html#Query_Services_Layer
- [77]: Administration.html
- [78]: Administration.html#Hardware
- [79]: Administration.html#Network
- [80]: Administration.html#Installation
- [81]: Administration.html#Dependencies
- [82]: Administration.html#Configuration
- [83]: Administration.html#Edit_conf/accumulo-env.sh
- [84]: Administration.html#Cluster_Specification
- [85]: Administration.html#Accumulo_Settings
- [86]: Administration.html#Deploy_Configuration
- [87]: Administration.html#Initialization
- [88]: Administration.html#Running
- [89]: Administration.html#Starting_Accumulo
- [90]: Administration.html#Stopping_Accumulo
- [91]: Administration.html#Monitoring
- [92]: Administration.html#Logging
- [93]: Administration.html#Recovery
- [94]: Shell_Commands.html
+ [32]: Writing_Accumulo_Clients.html#Proxy
+ [33]: Writing_Accumulo_Clients.html#Prequisites
+ [34]: Writing_Accumulo_Clients.html#Configuration
+ [35]: Writing_Accumulo_Clients.html#Running_the_Proxy_Server
+ [36]: Writing_Accumulo_Clients.html#Creating_a_Proxy_Client
+ [37]: Writing_Accumulo_Clients.html#Using_a_Proxy_Client
+ [38]: Development_Clients.html
+ [39]: Development_Clients.html#Mock_Accumulo
+ [40]: Development_Clients.html#Mini_Accumulo_Cluster
+ [41]: Table_Configuration.html
+ [42]: Table_Configuration.html#Locality_Groups
+ [43]: Table_Configuration.html#Managing_Locality_Groups_via_the_Shell
+ [44]: Table_Configuration.html#Managing_Locality_Groups_via_the_Client_API
+ [45]: Table_Configuration.html#Constraints
+ [46]: Table_Configuration.html#Bloom_Filters
+ [47]: Table_Configuration.html#Iterators
+ [48]: Table_Configuration.html#Setting_Iterators_via_the_Shell
+ [49]: Table_Configuration.html#Setting_Iterators_Programmatically
+ [50]: Table_Configuration.html#Versioning_Iterators_and_Timestamps
+ [51]: Table_Configuration.html#Filters
+ [52]: Table_Configuration.html#Combiners
+ [53]: Table_Configuration.html#Block_Cache
+ [54]: Table_Configuration.html#Compaction
+ [55]: Table_Configuration.html#Pre-splitting_tables
+ [56]: Table_Configuration.html#Merging_tablets
+ [57]: Table_Configuration.html#Delete_Range
+ [58]: Table_Configuration.html#Cloning_Tables
+ [59]: Table_Design.html
+ [60]: Table_Design.html#Basic_Table
+ [61]: Table_Design.html#RowID_Design
+ [62]: Table_Design.html#Indexing
+ [63]: Table_Design.html#Entity-Attribute_and_Graph_Tables
+ [64]: Table_Design.html#Document-Partitioned_Indexing
+ [65]: High_Speed_Ingest.html
+ [66]: High_Speed_Ingest.html#Pre-Splitting_New_Tables
+ [67]: High_Speed_Ingest.html#Multiple_Ingester_Clients
+ [68]: High_Speed_Ingest.html#Bulk_Ingest
+ [69]: High_Speed_Ingest.html#Logical_Time_for_Bulk_Ingest
+ [70]: High_Speed_Ingest.html#MapReduce_Ingest
+ [71]: Analytics.html
+ [72]: Analytics.html#MapReduce
+ [73]: Analytics.html#Mapper_and_Reducer_classes
+ [74]: Analytics.html#AccumuloInputFormat_options
+ [75]: Analytics.html#AccumuloOutputFormat_options
+ [76]: Analytics.html#Combiners
+ [77]: Analytics.html#Feature_Vectors
+ [78]: Analytics.html#Statistical_Modeling
+ [79]: Security.html
+ [80]: Security.html#Security_Label_Expressions
+ [81]: Security.html#Security_Label_Expression_Syntax
+ [82]: Security.html#Authorization
+ [83]: Security.html#User_Authorizations
+ [84]: Security.html#Secure_Authorizations_Handling
+ [85]: Security.html#Query_Services_Layer
+ [86]: Administration.html
+ [87]: Administration.html#Hardware
+ [88]: Administration.html#Network
+ [89]: Administration.html#Installation
+ [90]: Administration.html#Dependencies
+ [91]: Administration.html#Configuration
+ [92]: Administration.html#Edit_conf/accumulo-env.sh
+ [93]: Administration.html#Cluster_Specification
+ [94]: Administration.html#Accumulo_Settings
+ [95]: Administration.html#Deploy_Configuration
+ [96]: Administration.html#Initialization
+ [97]: Administration.html#Running
+ [98]: Administration.html#Starting_Accumulo
+ [99]: Administration.html#Stopping_Accumulo
+ [100]: Administration.html#Adding_a_Node
+ [101]: Administration.html#Decomissioning_a_Node
+ [102]: Administration.html#Monitoring
+ [103]: Administration.html#Logging
+ [104]: Administration.html#Recovery
+ [105]: Shell_Commands.html
Added: accumulo/site/trunk/content/1.4/user_manual/Development_Clients.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/Development_Clients.mdtext?rev=1578577&view=auto
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/Development_Clients.mdtext
(added)
+++ accumulo/site/trunk/content/1.4/user_manual/Development_Clients.mdtext Mon
Mar 17 21:10:11 2014
@@ -0,0 +1,119 @@
+Title: Apache Accumulo User Manual: Development Clients
+Notice: Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+ .
+ http://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+** Next:** [Table Configuration][2] ** Up:** [Apache Accumulo User Manual
Version 1.4][4] ** Previous:** [Writing Accumulo Clients][6] **
[Contents][8]**
+
+<a id=CHILD_LINKS></a>**Subsections**
+
+* [Mock Accumulo][9]
+* [Mini Accumulo Cluster][10]
+
+* * *
+
+## <a id=Development_Clients></a> Development Clients
+
+Normally, Accumulo consists of lots of moving parts. Even a stand-alone
version of Accumulo requires Hadoop, Zookeeper, the Accumulo master, a tablet
server, etc. If you want to write a unit test that uses Accumulo, you need a
lot of infrastructure in place before your test can run.
+
+## <a id=Mock_Accumulo></a> Mock Accumulo
+
+Mock Accumulo supplies mock implementations for much of the client API. It
presently does not enforce users, logins, permissions, etc. It does support
Iterators and Combiners. Note that MockAccumulo holds all data in memory, and
will not retain any data or settings between runs.
+
+While normal interaction with the Accumulo client looks like this:
+
+
+ Instance instance = new ZooKeeperInstance(...);
+ Connector conn = instance.getConnector(user, passwd);
+
+
+To interact with the MockAccumulo, just replace the ZooKeeperInstance with
MockInstance:
+
+
+ Instance instance = new MockInstance();
+
+
+In fact, you can use the "-fake" option to the Accumulo shell and interact
with MockAccumulo:
+
+
+ $ ./bin/accumulo shell --fake -u root -p ''
+
+ Shell - Apache Accumulo Interactive Shell
+ -
+ - version: 1.4.4
+ - instance name: mock-instance
+ - instance id: mock-instance-id
+ -
+ - type 'help' for a list of available commands
+ -
+ root@mock-instance> createtable test
+ root@mock-instance test> insert row1 cf cq value
+ root@mock-instance test> insert row2 cf cq value2
+ root@mock-instance test> insert row3 cf cq value3
+ root@mock-instance test> scan
+ row1 cf:cq [] value
+ row2 cf:cq [] value2
+ row3 cf:cq [] value3
+ root@mock-instance test> scan -b row2 -e row2
+ row2 cf:cq [] value2
+ root@mock-instance test>
+
+
+When testing Map Reduce jobs, you can also set the Mock Accumulo on the
AccumuloInputFormat and AccumuloOutputFormat classes:
+
+
+ // ... set up job configuration
+ AccumuloInputFormat.setMockInstance(job, "mockInstance");
+ AccumuloOutputFormat.setMockInstance(job, "mockInstance");
+
+
+## <a id=Mini_Accumulo_Cluster></a> Mini Accumulo Cluster
+
+While the Mock Accumulo provides a lightweight implementation of the client
API for unit testing, it is often necessary to write more realistic end-to-end
integration tests that take advantage of the entire ecosystem. The Mini
Accumulo Cluster makes this possible by configuring and starting Zookeeper,
initializing Accumulo, and starting the Master as well as some Tablet Servers.
It runs against the local filesystem instead of having to start up HDFS.
+
+To start it up, you will need to supply an empty directory and a root password
as arguments:
+
+
+ File tempDirectory = // JUnit and Guava supply mechanisms for creating
temp directories
+ MiniAccumuloCluster accumulo = new MiniAccumuloCluster(tempDirectory,
"password");
+ accumulo.start();
+
+
+Once we have our mini cluster running, we will want to interact with the
Accumulo client API:
+
+
+ Instance instance = new ZooKeeperInstance(accumulo.getInstanceName(),
accumulo.getZooKeepers());
+ Connector conn = instance.getConnector("root", "password");
+
+
+Upon completion of our development code, we will want to shutdown our
MiniAccumuloCluster:
+
+
+ accumulo.stop()
+ // delete your temporary folder
+
+
+* * *
+
+** Next:** [Table Configuration][2] ** Up:** [Apache Accumulo User Manual
Version 1.4][4] ** Previous:** [Writing Accumulo Clients][6] **
[Contents][8]**
+
+ [2]: Table_Configuration.html
+ [4]: accumulo_user_manual.html
+ [6]: Writing_Accumulo_Clients.html
+ [8]: Contents.html
+ [9]: Development_Clients.html#Mock_Accumulo
+ [10]: Development_Clients.html#Mini_Accumulo_Cluster
+
Propchange:
accumulo/site/trunk/content/1.4/user_manual/Development_Clients.mdtext
------------------------------------------------------------------------------
svn:eol-style = native
Modified: accumulo/site/trunk/content/1.4/user_manual/Table_Configuration.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/Table_Configuration.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/Table_Configuration.mdtext
(original)
+++ accumulo/site/trunk/content/1.4/user_manual/Table_Configuration.mdtext Mon
Mar 17 21:10:11 2014
@@ -16,7 +16,7 @@ Notice: Licensed to the Apache Softwa
specific language governing permissions and limitations
under the License.
-** Next:** [Table Design][2] ** Up:** [Apache Accumulo User Manual Version
1.4][4] ** Previous:** [Writing Accumulo Clients][6] ** [Contents][8]**
+** Next:** [Table Design][2] ** Up:** [Apache Accumulo User Manual Version
1.4][4] ** Previous:** [Development Clients][6] ** [Contents][8]**
<a id=CHILD_LINKS></a>**Subsections**
@@ -145,8 +145,8 @@ Iterators provide a modular mechanism fo
scanner.addIterator(new IteratorSetting(
15, // priority
- "com.company.MyIterator", // class name
- "myiter" // name this iterator
+ "myiter", // name this iterator
+ "com.company.MyIterator" // class name
));
@@ -470,11 +470,11 @@ The du command in the shell shows how mu
* * *
-** Next:** [Table Design][2] ** Up:** [Apache Accumulo User Manual Version
1.4][4] ** Previous:** [Writing Accumulo Clients][6] ** [Contents][8]**
+** Next:** [Table Design][2] ** Up:** [Apache Accumulo User Manual Version
1.4][4] ** Previous:** [Development Clients][6] ** [Contents][8]**
[2]: Table_Design.html
[4]: accumulo_user_manual.html
- [6]: Writing_Accumulo_Clients.html
+ [6]: Development_Clients.html
[8]: Contents.html
[9]: Table_Configuration.html#Locality_Groups
[10]: Table_Configuration.html#Constraints
Modified:
accumulo/site/trunk/content/1.4/user_manual/Writing_Accumulo_Clients.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/Writing_Accumulo_Clients.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/Writing_Accumulo_Clients.mdtext
(original)
+++ accumulo/site/trunk/content/1.4/user_manual/Writing_Accumulo_Clients.mdtext
Mon Mar 17 21:10:11 2014
@@ -16,7 +16,7 @@ Notice: Licensed to the Apache Softwa
specific language governing permissions and limitations
under the License.
-** Next:** [Table Configuration][2] ** Up:** [Apache Accumulo User Manual
Version 1.4][4] ** Previous:** [Accumulo Shell][6] ** [Contents][8]**
+** Next:** [Development Clients][2] ** Up:** [Apache Accumulo User Manual
Version 1.4][4] ** Previous:** [Accumulo Shell][6] ** [Contents][8]**
<a id=CHILD_LINKS></a>**Subsections**
@@ -24,6 +24,7 @@ Notice: Licensed to the Apache Softwa
* [Connecting][10]
* [Writing Data][11]
* [Reading Data][12]
+* [Proxy][13]
* * *
@@ -162,11 +163,105 @@ The BatchScanner is configured similarly
An example of the BatchScanner can be found at
accumulo/docs/examples/README.batch
+## <a id=Proxy></a> Proxy
+
+The proxy API allows the interaction with Accumulo with languages other than
Java. A proxy server is provided in the codebase and a client can further be
generated.
+
+### <a id=Prequisites></a> Prequisites
+
+The proxy server can live on any node in which the basic client API would
work. That means it must be able to communicate with the Master, ZooKeepers,
NameNode, and the Data nodes. A proxy client only needs the ability to
communicate with the proxy server.
+
+### <a id=Configuration></a> Configuration
+
+The configuration options for the proxy server live inside of a properties
file. At the very least, you need to supply the following properties:
+
+
+ protocolFactory=org.apache.thrift.protocol.TCompactProtocol$Factory
+ tokenClass=org.apache.accumulo.core.client.security.tokens.PasswordToken
+ port=42424
+ instance=test
+ zookeepers=localhost:2181
+
+
+You can find a sample configuration file in your distribution:
+
+
+ $ACCUMULO_HOME/proxy/proxy.properties.
+
+
+This sample configuration file further demonstrates an ability to back the
proxy server by MockAccumulo or the MiniAccumuloCluster.
+
+### <a id=Running_the_Proxy_Server></a> Running the Proxy Server
+
+After the properties file holding the configuration is created, the proxy
server can be started using the following command in the Accumulo distribution
(assuming your properties file is named config.properties):
+
+
+ $ACCUMULO_HOME/bin/accumulo proxy -p config.properties
+
+
+### <a id=Creating_a_Proxy_Client></a> Creating a Proxy Client
+
+Aside from installing the Thrift compiler, you will also need the
language-specific library for Thrift installed to generate client code in that
language. Typically, your operating system's package manager will be able to
automatically install these for you in an expected location such as
/usr/lib/python/site-packages/thrift.
+
+You can find the thrift file for generating the client:
+
+
+ $ACCUMULO_HOME/proxy/proxy.thrift.
+
+
+After a client is generated, the port specified in the configuration
properties above will be used to connect to the server.
+
+### <a id=Using_a_Proxy_Client></a> Using a Proxy Client
+
+The following examples have been written in Java and the method signatures may
be slightly different depending on the language specified when generating
client with the Thrift compiler. After initiating a connection to the Proxy
(see Apache Thrift's documentation for examples of connecting to a Thrift
service), the methods on the proxy client will be available. The first thing to
do is log in:
+
+
+ Map password = new HashMap<String,String>();
+ password.put("password", "secret");
+ ByteBuffer token = client.login("root", password);
+
+
+Once logged in, the token returned will be used for most subsequent calls to
the client. Let's create a table, add some data, scan the table, and delete it.
+
+First, create a table.
+
+
+ client.createTable(token, "myTable", true, TimeType.MILLIS);
+
+
+Next, add some data:
+
+
+ // first, create a writer on the server
+ String writer = client.createWriter(token, "myTable", new WriterOptions());
+
+ // build column updates
+ Map<ByteBuffer, List<ColumnUpdate> cells> cellsToUpdate = //...
+
+ // send updates to the server
+ client.updateAndFlush(writer, "myTable", cellsToUpdate);
+
+ client.closeWriter(writer);
+
+
+Scan for the data and batch the return of the results on the server:
+
+
+ String scanner = client.createScanner(token, "myTable", new ScanOptions());
+ ScanResult results = client.nextK(scanner, 100);
+
+ for(KeyValue keyValue : results.getResultsIterator()) {
+ // do something with results
+ }
+
+ client.closeScanner(scanner);
+
+
* * *
-** Next:** [Table Configuration][2] ** Up:** [Apache Accumulo User Manual
Version 1.4][4] ** Previous:** [Accumulo Shell][6] ** [Contents][8]**
+** Next:** [Development Clients][2] ** Up:** [Apache Accumulo User Manual
Version 1.4][4] ** Previous:** [Accumulo Shell][6] ** [Contents][8]**
- [2]: Table_Configuration.html
+ [2]: Development_Clients.html
[4]: accumulo_user_manual.html
[6]: Accumulo_Shell.html
[8]: Contents.html
@@ -174,4 +269,5 @@ accumulo/docs/examples/README.batch
[10]: Writing_Accumulo_Clients.html#Connecting
[11]: Writing_Accumulo_Clients.html#Writing_Data
[12]: Writing_Accumulo_Clients.html#Reading_Data
+ [13]: Writing_Accumulo_Clients.html#Proxy
Modified:
accumulo/site/trunk/content/1.4/user_manual/accumulo_user_manual.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/accumulo_user_manual.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/accumulo_user_manual.mdtext
(original)
+++ accumulo/site/trunk/content/1.4/user_manual/accumulo_user_manual.mdtext Mon
Mar 17 21:10:11 2014
@@ -35,13 +35,14 @@ Version 1.4
* [Accumulo Design][7]
* [Accumulo Shell][8]
* [Writing Accumulo Clients][9]
-* [Table Configuration][10]
-* [Table Design][11]
-* [High-Speed Ingest][12]
-* [Analytics][13]
-* [Security][14]
-* [Administration][15]
-* [Shell Commands][16]
+* [Development Clients][10]
+* [Table Configuration][11]
+* [Table Design][12]
+* [High-Speed Ingest][13]
+* [Analytics][14]
+* [Security][15]
+* [Administration][16]
+* [Shell Commands][17]
@@ -53,11 +54,12 @@ Version 1.4
[7]: Accumulo_Design.html
[8]: Accumulo_Shell.html
[9]: Writing_Accumulo_Clients.html
- [10]: Table_Configuration.html
- [11]: Table_Design.html
- [12]: High_Speed_Ingest.html
- [13]: Analytics.html
- [14]: Security.html
- [15]: Administration.html
- [16]: Shell_Commands.html
+ [10]: Development_Clients.html
+ [11]: Table_Configuration.html
+ [12]: Table_Design.html
+ [13]: High_Speed_Ingest.html
+ [14]: Analytics.html
+ [15]: Security.html
+ [16]: Administration.html
+ [17]: Shell_Commands.html
Modified: accumulo/site/trunk/content/1.4/user_manual/index.mdtext
URL:
http://svn.apache.org/viewvc/accumulo/site/trunk/content/1.4/user_manual/index.mdtext?rev=1578577&r1=1578576&r2=1578577&view=diff
==============================================================================
--- accumulo/site/trunk/content/1.4/user_manual/index.mdtext (original)
+++ accumulo/site/trunk/content/1.4/user_manual/index.mdtext Mon Mar 17
21:10:11 2014
@@ -35,13 +35,14 @@ Version 1.4
* [Accumulo Design][7]
* [Accumulo Shell][8]
* [Writing Accumulo Clients][9]
-* [Table Configuration][10]
-* [Table Design][11]
-* [High-Speed Ingest][12]
-* [Analytics][13]
-* [Security][14]
-* [Administration][15]
-* [Shell Commands][16]
+* [Development Clients][10]
+* [Table Configuration][11]
+* [Table Design][12]
+* [High-Speed Ingest][13]
+* [Analytics][14]
+* [Security][15]
+* [Administration][16]
+* [Shell Commands][17]
@@ -53,11 +54,12 @@ Version 1.4
[7]: Accumulo_Design.html
[8]: Accumulo_Shell.html
[9]: Writing_Accumulo_Clients.html
- [10]: Table_Configuration.html
- [11]: Table_Design.html
- [12]: High_Speed_Ingest.html
- [13]: Analytics.html
- [14]: Security.html
- [15]: Administration.html
- [16]: Shell_Commands.html
+ [10]: Development_Clients.html
+ [11]: Table_Configuration.html
+ [12]: Table_Design.html
+ [13]: High_Speed_Ingest.html
+ [14]: Analytics.html
+ [15]: Security.html
+ [16]: Administration.html
+ [17]: Shell_Commands.html
