Jekyll build from gh-pages:af8b0e5 update 1.7 and 1.8 user manuals with recent changes
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/fc21741f Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/fc21741f Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/fc21741f Branch: refs/heads/asf-site Commit: fc21741fa75d4d2fe2b56b01ad58d6ce1e4bd4aa Parents: a26a875 Author: Sean Busbey <bus...@cloudera.com> Authored: Fri Oct 7 01:18:09 2016 -0500 Committer: Sean Busbey <bus...@cloudera.com> Committed: Fri Oct 7 01:42:53 2016 -0500 ---------------------------------------------------------------------- 1.7/accumulo_user_manual.html | 229 +++++++++++++++++++++++++++++++------ 1.8/accumulo_user_manual.html | 137 +++++++++++++++++++++- feed.xml | 4 +- 3 files changed, 329 insertions(+), 41 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/accumulo/blob/fc21741f/1.7/accumulo_user_manual.html ---------------------------------------------------------------------- diff --git a/1.7/accumulo_user_manual.html b/1.7/accumulo_user_manual.html index 9f34918..a4f4213 100644 --- a/1.7/accumulo_user_manual.html +++ b/1.7/accumulo_user_manual.html @@ -6,7 +6,7 @@ <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="generator" content="Asciidoctor 1.5.2"> <meta name="author" content="Apache Accumulo Project"> -<title>Apache Accumulo User Manual Version 1.7</title> +<title>Apache Accumulo® User Manual Version 1.7</title> <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400";> <style> /* Asciidoctor default stylesheet | MIT License | http://asciidoctor.org */ @@ -415,7 +415,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b </head> <body class="book toc2 toc-left"> <div id="header"> -<h1>Apache Accumulo User Manual Version 1.7</h1> +<h1>Apache Accumulo® User Manual Version 1.7</h1> <div class="details"> <span id="author" class="author">Apache Accumulo Project</span><br> <span id="email" class="email"><a href="mailto:d...@accumulo.apache.org";>d...@accumulo.apache.org</a></span><br> @@ -691,7 +691,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <li><a href="#_generate_principal_and_keytab">Generate Principal and Keytab</a></li> <li><a href="#_server_configuration_2">Server Configuration</a></li> <li><a href="#_kerberosauthenticator">KerberosAuthenticator</a></li> -<li><a href="#_accumulo_initialization">Accumulo Initialization</a></li> +<li><a href="#_administrative_user">Administrative User</a></li> <li><a href="#_verifying_secure_access">Verifying secure access</a></li> <li><a href="#_impersonation">Impersonation</a></li> <li><a href="#_delegation_tokens_2">Delegation Tokens</a></li> @@ -701,6 +701,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <ul class="sectlevel4"> <li><a href="#_create_client_principal">Create client principal</a></li> <li><a href="#_configuration_3">Configuration</a></li> +<li><a href="#_verifying_administrative_access">Verifying Administrative Access</a></li> <li><a href="#_delegationtokens_with_mapreduce">DelegationTokens with MapReduce</a></li> </ul> </li> @@ -874,6 +875,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <li><a href="#_general_kerberos_principal">general.kerberos.principal</a></li> <li><a href="#_general_kerberos_renewal_period">general.kerberos.renewal.period</a></li> <li><a href="#_general_legacy_metrics">general.legacy.metrics</a></li> +<li><a href="#_general_max_scanner_retry_period">general.max.scanner.retry.period</a></li> <li><a href="#_general_rpc_timeout">general.rpc.timeout</a></li> <li><a href="#_general_security_credential_provider_paths">general.security.credential.provider.paths</a></li> <li><a href="#_general_server_message_size_max">general.server.message.size.max</a></li> @@ -954,6 +956,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <li><a href="#_tserver_wal_replication">tserver.wal.replication</a></li> <li><a href="#_tserver_wal_sync">tserver.wal.sync</a></li> <li><a href="#_tserver_wal_sync_method">tserver.wal.sync.method</a></li> +<li><a href="#_tserver_walog_max_age">tserver.walog.max.age</a></li> <li><a href="#_tserver_walog_max_size">tserver.walog.max.size</a></li> <li><a href="#_tserver_walog_maximum_wait_duration">tserver.walog.maximum.wait.duration</a></li> <li><a href="#_tserver_walog_tolerated_creation_failures">tserver.walog.tolerated.creation.failures</a></li> @@ -976,6 +979,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <li><a href="#_gc_threads_delete">gc.threads.delete</a></li> <li><a href="#_gc_trace_percent">gc.trace.percent</a></li> <li><a href="#_gc_trash_ignore">gc.trash.ignore</a></li> +<li><a href="#_gc_wal_dead_server_wait">gc.wal.dead.server.wait</a></li> </ul> </li> <li><a href="#MONITOR_PREFIX">A.3.10. monitor.*</a> @@ -1577,7 +1581,7 @@ Connector conn = inst.getConnector("user", new PasswordToken("passwd"));</code>< </div> </div> <div class="paragraph"> -<p>The PasswordToken is the most common implementation of an \texttt{AuthenticationToken}. +<p>The PasswordToken is the most common implementation of an <code>AuthenticationToken</code>. This general interface allow authentication as an Accumulo user to come from a variety of sources or means. The CredentialProviderToken leverages the Hadoop CredentialProviders (new in Hadoop 2.6).</p> @@ -4705,11 +4709,11 @@ cluster, this is a table ID. In this example, we want to enable replication on <code>my_table</code> and configure our peer <code>accumulo_peer</code> as a target, sending the data to the table with an ID of <code>2</code> in <code>accumulo_peer</code>.</p> </div> -<div class="paragraph"> -<p>\begingroup\fontsize{8pt}{8pt}\selectfont\begin{verbatim} -root@accumulo_primary> config -t my_table -s table.replication=true -root@accumulo_primary> config -t my_table -s table.replication.target.acccumulo_peer=2 -\end{verbatim}\endgroup</p> +<div class="listingblock"> +<div class="content"> +<pre>root@accumulo_primary> config -t my_table -s table.replication=true +root@accumulo_primary> config -t my_table -s table.replication.target.accumulo_peer=2</pre> +</div> </div> <div class="paragraph"> <p>To replicate a single table on the primary to multiple peers, the second command @@ -5468,6 +5472,11 @@ numerous guidelines already exist on the subject of configuring Hadoop and ZooKe use with Kerberos and won’t be covered here. It is assumed that you have functional Hadoop and ZooKeeper already installed.</p> </div> +<div class="paragraph"> +<p>Note that on an existing cluster the server side changes will require a full cluster shutdown and restart. You should +wait to restart the TraceServers until after you’ve completed the rest of the cluster set up and provisioned +a trace user with appropriate permissions.</p> +</div> <div class="sect3"> <h4 id="_servers">15.4.1. Servers</h4> <div class="paragraph"> @@ -5591,6 +5600,12 @@ keytab/principal to serialize traces. Like non-Kerberized instances, the table m to the trace.user. The same <code>_HOST</code> replacement is performed on this value, substituted the FQDN for <code>_HOST</code>.</p></td> </tr> <tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">trace.token.property.keytab</p></td> +<td class="tableblock halign-left valign-top"></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">You can optionally specify the path to a keytab file for the principal given in the <code>trace.user</code> property. If you don’t +set this path, it will default to the value given in <code>general.kerberos.principal</code>.</p></td> +</tr> +<tr> <td class="tableblock halign-left valign-top"><p class="tableblock">general.delegation.token.lifetime</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">7d</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">The length of time that the server-side secret used to create delegation tokens is valid. After a server-side secret @@ -5645,7 +5660,7 @@ to granting Authorizations and Permissions to new users.</p> </div> </div> <div class="sect4"> -<h5 id="_accumulo_initialization">Accumulo Initialization</h5> +<h5 id="_administrative_user">Administrative User</h5> <div class="paragraph"> <p>Out of the box (without Kerberos enabled), Accumulo has a single user with administrative permissions "root". This users is used to "bootstrap" other users, creating less-privileged users for applications using @@ -5659,6 +5674,40 @@ enabled, Accumulo will prompt for the name of a user to grant the same permissio user would normally have. The name of the Accumulo user to grant administrative permissions to can also be given by the <code>-u</code> or <code>--user</code> options.</p> </div> +<div class="paragraph"> +<p>If you are enabling Kerberos on an existing cluster, you will need to reinitialize the security system in +order to replace the existing "root" user with one that can be used with Kerberos. These steps should be +completed after you have done the previously described configuration changes and will require access to +a complete <code>accumulo-site.xml</code>, including the instance secret. Note that this process will delete all +existing users in the system; you will need to reassign user permissions based on Kerberos principals.</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Ensure Accumulo is not running.</p> +</li> +<li> +<p>Given the path to a <code>accumulo-site.xml</code> with the instance secret, run the security reset tool. If you are +prompted for a password you can just hit return, since it won’t be used.</p> +</li> +</ol> +</div> +<div class="listingblock"> +<div class="content"> +<pre>$ ACCUMULO_CONF_DIR=/path/to/server/conf/ accumulo init --reset-security +Running against secured HDFS +Principal (user) to grant administrative privileges to : acculumo_ad...@example.com +Enter initial password for accumulo_ad...@example.com (this may not be applicable for your security setup): +Confirm initial password for accumulo_ad...@example.com:</pre> +</div> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Start the Accumulo cluster</p> +</li> +</ol> +</div> </div> <div class="sect4"> <h5 id="_verifying_secure_access">Verifying secure access</h5> @@ -5800,7 +5849,7 @@ Default principal: u...@example.com Valid starting Expires Service principal 01/07/2015 11:56:35 01/08/2015 11:56:35 krbtgt/example....@example.com - renew until 01/14/2015 11:56:35</pre> + renew until 01/14/2015 11:56:35</pre> </div> </div> </div> @@ -5808,7 +5857,7 @@ Valid starting Expires Service principal <h5 id="_configuration_3">Configuration</h5> <div class="paragraph"> <p>The second thing clients need to do is to set up their client configuration file. By -default, this file is stored in <code>~/.accumulo/conf</code>, <code>$ACCUMULO_CONF_DIR/client.conf</code> or +default, this file is stored in <code>~/.accumulo/config</code>, <code>$ACCUMULO_CONF_DIR/client.conf</code> or <code>$ACCUMULO_HOME/conf/client.conf</code>. Accumulo utilities also allow you to provide your own copy of this file in any location using the <code>--config-file</code> command line option.</p> </div> @@ -5821,16 +5870,59 @@ copy of this file in any location using the <code>--config-file</code> command l <p><code>instance.rpc.sasl.enabled</code>=<em>true</em></p> </li> <li> +<p><code>rpc.sasl.qop</code>=<em>auth</em></p> +</li> +<li> <p><code>kerberos.server.primary</code>=<em>accumulo</em></p> </li> </ul> </div> <div class="paragraph"> -<p>The second and third properties <strong>must</strong> match the configuration of the accumulo servers; this is +<p>Each of these properties <strong>must</strong> match the configuration of the accumulo servers; this is required to set up the SASL transport.</p> </div> </div> <div class="sect4"> +<h5 id="_verifying_administrative_access">Verifying Administrative Access</h5> +<div class="paragraph"> +<p>At this point you should have enough configured on the server and client side to interact with +the system. You should verify that the administrative user you chose earlier can successfully +interact with the sytem.</p> +</div> +<div class="paragraph"> +<p>While this example logs in via <code>kinit</code> with a password, any login method that caches Kerberos tickets +should work.</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>$ kinit accumulo_ad...@example.com +Password for accumulo_ad...@example.com: ****************************** +$ accumulo shell + +Shell - Apache Accumulo Interactive Shell +- +- version: 1.7.2 +- instance name: MYACCUMULO +- instance id: 483b9038-889f-4b2d-b72b-dfa2bb5dbd07 +- +- type 'help' for a list of available commands +- +accumulo_ad...@example.com@MYACCUMULO> userpermissions +System permissions: System.GRANT, System.CREATE_TABLE, System.DROP_TABLE, System.ALTER_TABLE, System.CREATE_USER, System.DROP_USER, System.ALTER_USER, System.SYSTEM, System.CREATE_NAMESPACE, System.DROP_NAMESPACE, System.ALTER_NAMESPACE, System.OBTAIN_DELEGATION_TOKEN + +Namespace permissions (accumulo): Namespace.READ, Namespace.ALTER_TABLE + +Table permissions (accumulo.metadata): Table.READ, Table.ALTER_TABLE +Table permissions (accumulo.replication): Table.READ +Table permissions (accumulo.root): Table.READ, Table.ALTER_TABLE + +accumulo_ad...@example.com@MYACCUMULO> quit +$ kdestroy +$</pre> +</div> +</div> +</div> +<div class="sect4"> <h5 id="_delegationtokens_with_mapreduce">DelegationTokens with MapReduce</h5> <div class="paragraph"> <p>To use DelegationTokens in a custom MapReduce job, the call to <code>setConnectorInfo()</code> method @@ -5919,7 +6011,7 @@ Default principal: u...@example.com Valid starting Expires Service principal 01/07/2015 11:56:35 01/08/2015 11:56:35 krbtgt/example....@example.com - renew until 01/14/2015 11:56:35 + renew until 01/14/2015 11:56:35 $ export KRB5CCNAME=/tmp/krb5cc_123 $ echo $KRB5CCNAME /tmp/krb5cc_123</pre> @@ -6031,7 +6123,45 @@ servers are not configured to listen on the address denoted by their FQDN.</p> </div> <div class="paragraph"> <p>The values in the Accumulo "hosts" files (In <code>$ACCUMULO_CONF_DIR</code>: <code>masters</code>, <code>monitors</code>, <code>slaves</code>, <code>tracers</code>, -and <code>gc</code>) should match the instance componentof the Kerberos server principal (e.g. <code>host</code> in <code>accumulo/host\@EXAMPLE.COM</code>).</p> +and <code>gc</code>) should match the instance componentof the Kerberos server principal (e.g. <code>host</code> in <code>accumulo/h...@example.com</code>).</p> +</div> +<div class="paragraph"> +<p><strong>Q</strong>: After configuring my system for Kerberos, server processes come up normally and I can interact with the system. However, +when I attempt to use the "Recent Traces" page on the Monitor UI I get a stacktrace similar to:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre> java.lang.AssertionError: AuthenticationToken should not be null + at org.apache.accumulo.monitor.servlets.trace.Basic.getScanner(Basic.java:139) + at org.apache.accumulo.monitor.servlets.trace.Summary.pageBody(Summary.java:164) + at org.apache.accumulo.monitor.servlets.BasicServlet.doGet(BasicServlet.java:63) + at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) + at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) + at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) + at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551) + at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) + at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:568) + at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221) + at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) + at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478) + at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) + at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) + at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) + at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) + at org.eclipse.jetty.server.Server.handle(Server.java:462) + at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) + at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232) + at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) + at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) + at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) + at java.lang.Thread.run(Thread.java:745)</pre> +</div> +</div> +<div class="paragraph"> +<p><strong>A</strong>: This indicates that the Monitor has not been able to successfully log in a client-side user to read from the <code>trace</code> table. Accumulo allows the TraceServer to rely on the property <code>general.kerberos.keytab</code> as a fallback when logging in the trace user if the <code>trace.token.property.keytab</code> property isn’t defined. Some earlier versions of Accumulo did not do this same fallback for the Monitor’s use of the trace user. The end result is that if you configure <code>general.kerberos.keytab</code> and not <code>trace.token.property.keytab</code> you will end up with a system that properly logs trace information but can’t view it.</p> +</div> +<div class="paragraph"> +<p>Ensure you have set <code>trace.token.property.keytab</code> to point to a keytab for the principal defined in <code>trace.user</code> in the <code>accumulo-site.xml</code> file for the Monitor, since that should work in all versions of Accumulo.</p> </div> </div> </div> @@ -6151,18 +6281,20 @@ same default ports) on the same hardware.</p> <div class="sect2"> <h3 id="_installation">16.3. Installation</h3> <div class="paragraph"> -<p>Choose a directory for the Accumulo installation. This directory will be referenced -by the environment variable <code>$ACCUMULO_HOME</code>. Run the following:</p> +<p>Download a binary distribution of Accumulo and install it to a directory on a disk with +sufficient space:</p> </div> <div class="literalblock"> <div class="content"> -<pre>$ tar xzf accumulo-1.6.0-bin.tar.gz # unpack to subdirectory -$ mv accumulo-1.6.0 $ACCUMULO_HOME # move to desired location</pre> +<pre>cd <install directory> +tar xzf accumulo-X.Y.Z-bin.tar.gz # Replace 'X.Y.Z' with your Accumulo version +cd accumulo-X.Y.Z</pre> </div> </div> <div class="paragraph"> -<p>Repeat this step at each machine within the cluster. Usually all machines have the -same <code>$ACCUMULO_HOME</code>.</p> +<p>Repeat this step on each machine in your cluster. Typically, the same <code><install directory></code> +is chosen for all machines in the cluster. When you configure Accumulo, the <code>$ACCUMULO_HOME</code> +environment variable should be set to <code>/path/to/<install directory>/accumulo-X.Y.Z</code>.</p> </div> </div> <div class="sect2"> @@ -6286,9 +6418,10 @@ also locate the native maps shared library by setting <code>LD_LIBRARY_PATH</cod <h5 id="_native_maps_configuration">Native Maps Configuration</h5> <div class="paragraph"> <p>As mentioned, Accumulo will use the native libraries if they are found in the expected -location and if it is not configured to ignore them. Using the native maps over JVM -Maps nets a noticable improvement in ingest rates; however, certain configuration -variables are important to modify when increasing the size of the native map.</p> +location and <code>tserver.memory.maps.native.enabled</code> is set to <code>true</code> (which is the default). +Using the native maps over JVM Maps nets a noticable improvement in ingest rates; however, +certain configuration variables are important to modify when increasing the size of the +native map.</p> </div> <div class="paragraph"> <p>To adjust the size of the native map, increase the value of <code>tserver.memory.maps.max</code>. @@ -6448,7 +6581,7 @@ when the Configuration object for accumulo-site.xml is accessed.</p> <div class="paragraph"> <p>One of the implementations provided in Hadoop-2.6.0 is a Java KeyStore CredentialProvider. Each entry in the KeyStore is the Accumulo Property key name. For example, to store the -\texttt{instance.secret}, the following command can be used:</p> +<code>instance.secret</code>, the following command can be used:</p> </div> <div class="literalblock"> <div class="content"> @@ -6590,13 +6723,8 @@ take some time for particular configurations.</p> <div class="paragraph"> <p>Update your <code>$ACCUMULO_HOME/conf/slaves</code> (or <code>$ACCUMULO_CONF_DIR/slaves</code>) file to account for the addition.</p> </div> -<div class="literalblock"> -<div class="content"> -<pre>$ACCUMULO_HOME/bin/accumulo admin start <host(s)> {<host> ...}</pre> -</div> -</div> <div class="paragraph"> -<p>Alternatively, you can ssh to each of the hosts you want to add and run:</p> +<p>Next, ssh to each of the hosts you want to add and run:</p> </div> <div class="literalblock"> <div class="content"> @@ -9090,7 +9218,7 @@ default | table.failures.ignore ..................... | false</pre> <div class="sect4"> <h5 id="_instance_secret">instance.secret</h5> <div class="paragraph"> -<p>A secret unique to a given instance that all servers must know in order to communicate with one another. Change it before initialization. To change it later use ./bin/accumulo accumulo.server.util.ChangeSecret [oldpasswd] [newpasswd], and then update conf/accumulo-site.xml everywhere.</p> +<p>A secret unique to a given instance that all servers must know in order to communicate with one another.It should be changed prior to the initialization of Accumulo. To change it after Accumulo has been initialized, use the ChangeSecret tool and then update conf/accumulo-site.xml everywhere. Before using the ChangeSecret tool, make sure Accumulo is not running and you are logged in as the user that controls Accumulo files in HDFS. To use the ChangeSecret tool, run the command: ./bin/accumulo org.apache.accumulo.server.util.ChangeSecret</p> </div> <div class="paragraph"> <p><em>Type:</em> STRING<br> @@ -9305,6 +9433,17 @@ $HADOOP_PREFIX/share/hadoop/yarn/lib/jersey.*.jar, </div> </div> <div class="sect4"> +<h5 id="_general_max_scanner_retry_period">general.max.scanner.retry.period</h5> +<div class="paragraph"> +<p>The maximum amount of time that a Scanner should wait before retrying a failed RPC</p> +</div> +<div class="paragraph"> +<p><em>Type:</em> TIMEDURATION<br> +<em>Zookeeper Mutable:</em> no<br> +<em>Default Value:</em> <code>5s</code></p> +</div> +</div> +<div class="sect4"> <h5 id="_general_rpc_timeout">general.rpc.timeout</h5> <div class="paragraph"> <p>Time to wait on I/O for simple, short RPC calls</p> @@ -10109,6 +10248,17 @@ $HADOOP_PREFIX/share/hadoop/yarn/lib/jersey.*.jar, </div> </div> <div class="sect4"> +<h5 id="_tserver_walog_max_age">tserver.walog.max.age</h5> +<div class="paragraph"> +<p>The maximum age for each write-ahead log.</p> +</div> +<div class="paragraph"> +<p><em>Type:</em> TIMEDURATION<br> +<em>Zookeeper Mutable:</em> yes<br> +<em>Default Value:</em> <code>24h</code></p> +</div> +</div> +<div class="sect4"> <h5 id="_tserver_walog_max_size">tserver.walog.max.size</h5> <div class="paragraph"> <p>The maximum size for each write-ahead log. See comment for property tserver.memory.maps.max</p> @@ -10269,6 +10419,17 @@ $HADOOP_PREFIX/share/hadoop/yarn/lib/jersey.*.jar, <em>Default Value:</em> <code>false</code></p> </div> </div> +<div class="sect4"> +<h5 id="_gc_wal_dead_server_wait">gc.wal.dead.server.wait</h5> +<div class="paragraph"> +<p>Time to wait after a tserver is first seen as dead before removing associated WAL files</p> +</div> +<div class="paragraph"> +<p><em>Type:</em> TIMEDURATION<br> +<em>Zookeeper Mutable:</em> yes<br> +<em>Default Value:</em> <code>1h</code></p> +</div> +</div> </div> <div class="sect3"> <h4 id="MONITOR_PREFIX">A.3.10. monitor.*</h4> @@ -11284,8 +11445,8 @@ An example is <em>java.lang.String</em>, rather than <em>String</em></p> </div> <div id="footer"> <div id="footer-text"> -Last updated 2016-02-22 16:32:20 EST +Last updated 2016-10-07 00:54:42 -05:00 </div> </div> </body> -</html> +</html> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/accumulo/blob/fc21741f/1.8/accumulo_user_manual.html ---------------------------------------------------------------------- diff --git a/1.8/accumulo_user_manual.html b/1.8/accumulo_user_manual.html index 28522f6..c05a537 100644 --- a/1.8/accumulo_user_manual.html +++ b/1.8/accumulo_user_manual.html @@ -700,7 +700,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <li><a href="#_generate_principal_and_keytab">Generate Principal and Keytab</a></li> <li><a href="#_server_configuration_2">Server Configuration</a></li> <li><a href="#_kerberosauthenticator">KerberosAuthenticator</a></li> -<li><a href="#_accumulo_initialization">Accumulo Initialization</a></li> +<li><a href="#_administrative_user">Administrative User</a></li> <li><a href="#_verifying_secure_access">Verifying secure access</a></li> <li><a href="#_impersonation">Impersonation</a></li> <li><a href="#_delegation_tokens_2">Delegation Tokens</a></li> @@ -710,6 +710,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b <ul class="sectlevel4"> <li><a href="#_create_client_principal">Create client principal</a></li> <li><a href="#_configuration_3">Configuration</a></li> +<li><a href="#_verifying_administrative_access">Verifying Administrative Access</a></li> <li><a href="#_delegationtokens_with_mapreduce">DelegationTokens with MapReduce</a></li> </ul> </li> @@ -5686,6 +5687,11 @@ numerous guidelines already exist on the subject of configuring Hadoop and ZooKe use with Kerberos and won’t be covered here. It is assumed that you have functional Hadoop and ZooKeeper already installed.</p> </div> +<div class="paragraph"> +<p>Note that on an existing cluster the server side changes will require a full cluster shutdown and restart. You should +wait to restart the TraceServers until after you’ve completed the rest of the cluster set up and provisioned +a trace user with appropriate permissions.</p> +</div> <div class="sect3"> <h4 id="_servers">16.4.1. Servers</h4> <div class="paragraph"> @@ -5809,6 +5815,12 @@ keytab/principal to serialize traces. Like non-Kerberized instances, the table m to the trace.user. The same <code>_HOST</code> replacement is performed on this value, substituted the FQDN for <code>_HOST</code>.</p></td> </tr> <tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">trace.token.property.keytab</p></td> +<td class="tableblock halign-left valign-top"></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">You can optionally specify the path to a keytab file for the principal given in the <code>trace.user</code> property. If you don’t +set this path, it will default to the value given in <code>general.kerberos.principal</code>.</p></td> +</tr> +<tr> <td class="tableblock halign-left valign-top"><p class="tableblock">general.delegation.token.lifetime</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">7d</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">The length of time that the server-side secret used to create delegation tokens is valid. After a server-side secret @@ -5863,7 +5875,7 @@ to granting Authorizations and Permissions to new users.</p> </div> </div> <div class="sect4"> -<h5 id="_accumulo_initialization">Accumulo Initialization</h5> +<h5 id="_administrative_user">Administrative User</h5> <div class="paragraph"> <p>Out of the box (without Kerberos enabled), Accumulo has a single user with administrative permissions "root". This users is used to "bootstrap" other users, creating less-privileged users for applications using @@ -5877,6 +5889,40 @@ enabled, Accumulo will prompt for the name of a user to grant the same permissio user would normally have. The name of the Accumulo user to grant administrative permissions to can also be given by the <code>-u</code> or <code>--user</code> options.</p> </div> +<div class="paragraph"> +<p>If you are enabling Kerberos on an existing cluster, you will need to reinitialize the security system in +order to replace the existing "root" user with one that can be used with Kerberos. These steps should be +completed after you have done the previously described configuration changes and will require access to +a complete <code>accumulo-site.xml</code>, including the instance secret. Note that this process will delete all +existing users in the system; you will need to reassign user permissions based on Kerberos principals.</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Ensure Accumulo is not running.</p> +</li> +<li> +<p>Given the path to a <code>accumulo-site.xml</code> with the instance secret, run the security reset tool. If you are +prompted for a password you can just hit return, since it won’t be used.</p> +</li> +</ol> +</div> +<div class="listingblock"> +<div class="content"> +<pre>$ ACCUMULO_CONF_DIR=/path/to/server/conf/ accumulo init --reset-security +Running against secured HDFS +Principal (user) to grant administrative privileges to : acculumo_ad...@example.com +Enter initial password for accumulo_ad...@example.com (this may not be applicable for your security setup): +Confirm initial password for accumulo_ad...@example.com:</pre> +</div> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Start the Accumulo cluster</p> +</li> +</ol> +</div> </div> <div class="sect4"> <h5 id="_verifying_secure_access">Verifying secure access</h5> @@ -6026,7 +6072,7 @@ Valid starting Expires Service principal <h5 id="_configuration_3">Configuration</h5> <div class="paragraph"> <p>The second thing clients need to do is to set up their client configuration file. By -default, this file is stored in <code>~/.accumulo/conf</code>, <code>$ACCUMULO_CONF_DIR/client.conf</code> or +default, this file is stored in <code>~/.accumulo/config</code>, <code>$ACCUMULO_CONF_DIR/client.conf</code> or <code>$ACCUMULO_HOME/conf/client.conf</code>. Accumulo utilities also allow you to provide your own copy of this file in any location using the <code>--config-file</code> command line option.</p> </div> @@ -6039,16 +6085,59 @@ copy of this file in any location using the <code>--config-file</code> command l <p><code>instance.rpc.sasl.enabled</code>=<em>true</em></p> </li> <li> +<p><code>rpc.sasl.qop</code>=<em>auth</em></p> +</li> +<li> <p><code>kerberos.server.primary</code>=<em>accumulo</em></p> </li> </ul> </div> <div class="paragraph"> -<p>The second and third properties <strong>must</strong> match the configuration of the accumulo servers; this is +<p>Each of these properties <strong>must</strong> match the configuration of the accumulo servers; this is required to set up the SASL transport.</p> </div> </div> <div class="sect4"> +<h5 id="_verifying_administrative_access">Verifying Administrative Access</h5> +<div class="paragraph"> +<p>At this point you should have enough configured on the server and client side to interact with +the system. You should verify that the administrative user you chose earlier can successfully +interact with the sytem.</p> +</div> +<div class="paragraph"> +<p>While this example logs in via <code>kinit</code> with a password, any login method that caches Kerberos tickets +should work.</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>$ kinit accumulo_ad...@example.com +Password for accumulo_ad...@example.com: ****************************** +$ accumulo shell + +Shell - Apache Accumulo Interactive Shell +- +- version: 1.7.2 +- instance name: MYACCUMULO +- instance id: 483b9038-889f-4b2d-b72b-dfa2bb5dbd07 +- +- type 'help' for a list of available commands +- +accumulo_ad...@example.com@MYACCUMULO> userpermissions +System permissions: System.GRANT, System.CREATE_TABLE, System.DROP_TABLE, System.ALTER_TABLE, System.CREATE_USER, System.DROP_USER, System.ALTER_USER, System.SYSTEM, System.CREATE_NAMESPACE, System.DROP_NAMESPACE, System.ALTER_NAMESPACE, System.OBTAIN_DELEGATION_TOKEN + +Namespace permissions (accumulo): Namespace.READ, Namespace.ALTER_TABLE + +Table permissions (accumulo.metadata): Table.READ, Table.ALTER_TABLE +Table permissions (accumulo.replication): Table.READ +Table permissions (accumulo.root): Table.READ, Table.ALTER_TABLE + +accumulo_ad...@example.com@MYACCUMULO> quit +$ kdestroy +$</pre> +</div> +</div> +</div> +<div class="sect4"> <h5 id="_delegationtokens_with_mapreduce">DelegationTokens with MapReduce</h5> <div class="paragraph"> <p>To use DelegationTokens in a custom MapReduce job, the call to <code>setConnectorInfo()</code> method @@ -6251,6 +6340,44 @@ servers are not configured to listen on the address denoted by their FQDN.</p> <p>The values in the Accumulo "hosts" files (In <code>$ACCUMULO_CONF_DIR</code>: <code>masters</code>, <code>monitors</code>, <code>slaves</code>, <code>tracers</code>, and <code>gc</code>) should match the instance componentof the Kerberos server principal (e.g. <code>host</code> in <code>accumulo/h...@example.com</code>).</p> </div> +<div class="paragraph"> +<p><strong>Q</strong>: After configuring my system for Kerberos, server processes come up normally and I can interact with the system. However, +when I attempt to use the "Recent Traces" page on the Monitor UI I get a stacktrace similar to:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre> java.lang.AssertionError: AuthenticationToken should not be null + at org.apache.accumulo.monitor.servlets.trace.Basic.getScanner(Basic.java:139) + at org.apache.accumulo.monitor.servlets.trace.Summary.pageBody(Summary.java:164) + at org.apache.accumulo.monitor.servlets.BasicServlet.doGet(BasicServlet.java:63) + at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) + at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) + at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) + at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551) + at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) + at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:568) + at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221) + at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) + at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478) + at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) + at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) + at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) + at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) + at org.eclipse.jetty.server.Server.handle(Server.java:462) + at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) + at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232) + at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) + at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) + at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) + at java.lang.Thread.run(Thread.java:745)</pre> +</div> +</div> +<div class="paragraph"> +<p><strong>A</strong>: This indicates that the Monitor has not been able to successfully log in a client-side user to read from the <code>trace</code> table. Accumulo allows the TraceServer to rely on the property <code>general.kerberos.keytab</code> as a fallback when logging in the trace user if the <code>trace.token.property.keytab</code> property isn’t defined. Some earlier versions of Accumulo did not do this same fallback for the Monitor’s use of the trace user. The end result is that if you configure <code>general.kerberos.keytab</code> and not <code>trace.token.property.keytab</code> you will end up with a system that properly logs trace information but can’t view it.</p> +</div> +<div class="paragraph"> +<p>Ensure you have set <code>trace.token.property.keytab</code> to point to a keytab for the principal defined in <code>trace.user</code> in the <code>accumulo-site.xml</code> file for the Monitor, since that should work in all versions of Accumulo.</p> +</div> </div> </div> </div> @@ -11832,7 +11959,7 @@ An example is <em>java.lang.String</em>, rather than <em>String</em></p> </div> <div id="footer"> <div id="footer-text"> -Last updated 2016-09-29 16:41:28 -04:00 +Last updated 2016-10-07 01:14:08 -05:00 </div> </div> </body> http://git-wip-us.apache.org/repos/asf/accumulo/blob/fc21741f/feed.xml ---------------------------------------------------------------------- diff --git a/feed.xml b/feed.xml index 7cbc7e4..fd6e1a4 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ </description> <link>https://accumulo.apache.org/</link> <atom:link href="https://accumulo.apache.org/feed.xml"; rel="self" type="application/rss+xml"/> - <pubDate>Thu, 29 Sep 2016 17:21:10 -0400</pubDate> - <lastBuildDate>Thu, 29 Sep 2016 17:21:10 -0400</lastBuildDate> + <pubDate>Fri, 07 Oct 2016 01:42:42 -0500</pubDate> + <lastBuildDate>Fri, 07 Oct 2016 01:42:42 -0500</lastBuildDate> <generator>Jekyll v3.2.1</generator> </channel>
