This is an automated email from the ASF dual-hosted git repository.
pinal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/atlas.git
The following commit(s) were added to refs/heads/master by this push:
new 84365f5cf ATLAS-4941: Atlas fails to startup locally after compression
type has changed to SNAPPY
84365f5cf is described below
commit 84365f5cfc271f3a4e8ad430aee8eb0faf674c30
Author: Pinal Shah <pinal.s...@freestoneinfotech.com>
AuthorDate: Wed Dec 18 15:04:22 2024 +0700
ATLAS-4941: Atlas fails to startup locally after compression type has
changed to SNAPPY
Signed-off-by: Pinal Shah <pinal.s...@freestoneinfotech.com>
---
docs/src/documents/Hook/HookCouchbase.md | 2 +-
.../src/documents/Setup/InstallationInstruction.md | 29 +++++++++++++++++++++-
.../audit/HBaseBasedAuditRepository.java | 2 +-
3 files changed, 30 insertions(+), 3 deletions(-)
diff --git a/docs/src/documents/Hook/HookCouchbase.md
b/docs/src/documents/Hook/HookCouchbase.md
index 53b9e9cb3..5f3e62f1f 100644
--- a/docs/src/documents/Hook/HookCouchbase.md
+++ b/docs/src/documents/Hook/HookCouchbase.md
@@ -32,7 +32,7 @@ The bridge uses environment variables for configuration.
| CB_PASSWORD | Couchbase Cluster password
|
"password" |
| CB_ENABLE_TLS | Use TLS
|
false |
| CB_BUCKET | Couchbase bucket to monitor
|
"default" |
-| CB_COLLECTIONS | Comma-separated list of collections to monitor with
each collection listed as <scope>.<collection> |
|
+| CB_COLLECTIONS | Comma-separated list of collections to monitor with
each collection listed as {scope}.{collection} |
|
| DCP_PORT | DCP port to use
|
11210 |
| DCP_FIELD_THRESHOLD | A threshold that indicates in what percentage of
analyzed messages per collection a field must appear before it is sent to
Atlas | 0 |
| DCP_SAMPLE_RATIO | Percentage of DCP messages to be analyzed in form of
a short between 0 and 1. |
1 |
diff --git a/docs/src/documents/Setup/InstallationInstruction.md
b/docs/src/documents/Setup/InstallationInstruction.md
index de432e034..7476a6c03 100644
--- a/docs/src/documents/Setup/InstallationInstruction.md
+++ b/docs/src/documents/Setup/InstallationInstruction.md
@@ -147,6 +147,14 @@ Apache HBase tables used by Apache Atlas can be set using
the following configur
atlas.audit.hbase.tablename=apache_atlas_entity_audit`}
</SyntaxHighlighter>
+#### Configuring Apache Hbase Compression Algorithm
+
+By default, Atlas uses GZ (Gzip) as Hbase compression. Other compression
algorithm can be configured using below configuration:
+
+<SyntaxHighlighter wrapLines={true} language="powershell" style={theme.dark}>
+{`atlas.graph.storage.hbase.compression-algorithm=SNAPPY`}
+</SyntaxHighlighter>
+
#### Configuring Apache Solr as the indexing backend for the Graph Repository
By default, Apache Atlas uses JanusGraph as the graph repository and is the
only graph repository implementation available currently. For configuring
JanusGraph to work with Apache Solr, please follow the instructions below
@@ -236,7 +244,7 @@ By default, Apache Atlas uses
[JanusGraph](https://janusgraph.org/) as the graph
<SyntaxHighlighter wrapLines={true} language="powershell" style={theme.dark}>
{`atlas.graph.index.search.backend=elasticsearch
-atlas.graph.index.search.hostname=<the hostname(s) of the Elasticsearch master
nodes comma separated>
+atlas.graph.index.search.hostname={the hostname(s) of the Elasticsearch master
nodes comma separated}
atlas.graph.index.search.elasticsearch.client-only=true`}
</SyntaxHighlighter>
@@ -362,3 +370,22 @@ run setup again.
If the setup failed due to Apache HBase schema setup errors, it may be
necessary to repair Apache HBase schema. If no
data has been stored, one can also disable and drop the Apache HBase tables
used by Apache Atlas and run setup again.
+
+#### Startup issues when SNAPPY compression is configured
+If you encounter issues where the Atlas server doesn't start after setting
SNAPPY as the compression algorithm, it could be due to missing or incompatible
libraries required for SNAPPY compression.
+
+Solution: Ensure Snappy compression libraries are available.
+If not already installed, download and configure the Snappy library on your
system.
+- Download and Install Snappy: The library can be found on platforms like
GitHub or official distribution repositories for your operating system.
+ <SyntaxHighlighter wrapLines={true} language="powershell" style={theme.dark}>
+ {`On Linux:
+ sudo apt-get install libsnappy-dev`}
+ </SyntaxHighlighter>
+- Download and extract Hadoop according to the hbase version: [Hadoop
3.2.4](https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz)
+- Update the HBase Configuration:
+ Add below lines to the file `hbase-env.sh`:
+ <SyntaxHighlighter wrapLines={true} language="powershell" style={theme.dark}>
+ {`export HADOOP_HOME={path-to-hadoop}
+ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native`}
+ </SyntaxHighlighter>
+- Restart Atlas Server
diff --git
a/repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
b/repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
index e0ecacc6d..d2446a1b7 100644
---
a/repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
+++
b/repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
@@ -848,7 +848,7 @@ public class HBaseBasedAuditRepository extends
AbstractStorageBasedAuditReposito
String tableNameStr = atlasConf.getString(CONFIG_TABLE_NAME,
DEFAULT_TABLE_NAME);
tableName = TableName.valueOf(tableNameStr);
- compressionType = atlasConf.getString(CONFIG_COMPRESSION_ALGORITHM,
atlasConf.getString(HBASE_STORE_COMPRESSION_PROPERTY, "SNAPPY"));
+ compressionType = atlasConf.getString(CONFIG_COMPRESSION_ALGORITHM,
atlasConf.getString(HBASE_STORE_COMPRESSION_PROPERTY, "GZ"));
try {
connection = createConnection(hbaseConf);
