This is an automated email from the ASF dual-hosted git repository.
pan3793 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 1a6a8e3c47 [ZEPPELIN-6420] Drop Alluxio Interpreter
1a6a8e3c47 is described below
commit 1a6a8e3c4734a605c04476803b08a38e97a5cccb
Author: Cheng Pan <[email protected]>
AuthorDate: Mon May 25 19:13:10 2026 +0800
[ZEPPELIN-6420] Drop Alluxio Interpreter
### What is this PR for?
[Alluxio Open Source](https://github.com/Alluxio/alluxio) stops
development, other projects like Trino, and public cloud vendors like Ali Cloud
also remove Alluxio support.
### What type of PR is it?
Breaking Change.
### What is the Jira issue?
ZEPPELIN-6420
### How should this be tested?
Pass GHA.
### Screenshots (if appropriate)
### Questions:
* Does the license files need to update? Yes, updated.
* Is there breaking changes for older versions? Yes, it removes a feature.
* Does this needs documentation? Docs are updated too.
Closes #5256 from pan3793/ZEPPELIN-6420.
Signed-off-by: Cheng Pan <[email protected]>
---
.github/dependabot.yml | 11 -
.github/workflows/core.yml | 2 +-
.github/workflows/frontend.yml | 2 +-
AGENTS.md | 1 -
alluxio/pom.xml | 110 ---------
.../zeppelin/alluxio/AlluxioInterpreter.java | 265 ---------------------
.../src/main/resources/interpreter-setting.json | 27 ---
.../zeppelin/alluxio/AlluxioInterpreterTest.java | 232 ------------------
conf/interpreter-list | 1 -
dev/create_release.sh | 2 +-
docs/_includes/themes/zeppelin/_navigation.html | 1 -
docs/index.md | 1 -
docs/interpreter/alluxio.md | 250 -------------------
docs/interpreter/hdfs.md | 4 +-
docs/usage/interpreter/installation.md | 5 -
pom.xml | 1 -
zeppelin-distribution/src/bin_license/LICENSE | 4 -
17 files changed, 5 insertions(+), 914 deletions(-)
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
index 58425fdcbe..0a475fa0e0 100644
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -56,17 +56,6 @@ updates:
patterns:
- "*"
- - package-ecosystem: "maven"
- directory: "/alluxio"
- schedule:
- interval: "weekly"
- open-pull-requests-limit: 0
- groups:
- alluxio-security-updates:
- applies-to: security-updates
- patterns:
- - "*"
-
- package-ecosystem: "maven"
directory: "/bigquery"
schedule:
diff --git a/.github/workflows/core.yml b/.github/workflows/core.yml
index db7344d6ad..e4fa6d60ab 100644
--- a/.github/workflows/core.yml
+++ b/.github/workflows/core.yml
@@ -93,7 +93,7 @@ jobs:
matrix:
java: [ 11 ]
env:
- INTERPRETERS:
'hbase,jdbc,file,flink-cmd,cassandra,elasticsearch,bigquery,alluxio,livy,groovy,java,neo4j,sparql,mongodb,influxdb,shell'
+ INTERPRETERS:
'hbase,jdbc,file,flink-cmd,cassandra,elasticsearch,bigquery,livy,groovy,java,neo4j,sparql,mongodb,influxdb,shell'
steps:
- name: Checkout
uses: actions/checkout@v4
diff --git a/.github/workflows/frontend.yml b/.github/workflows/frontend.yml
index f019d213c6..8f6bc4880e 100644
--- a/.github/workflows/frontend.yml
+++ b/.github/workflows/frontend.yml
@@ -23,7 +23,7 @@ env:
SPARK_PRINT_LAUNCH_COMMAND: "true"
SPARK_LOCAL_IP: 127.0.0.1
ZEPPELIN_LOCAL_IP: 127.0.0.1
- INTERPRETERS:
'!hbase,!jdbc,!file,!flink,!cassandra,!elasticsearch,!bigquery,!alluxio,!livy,!groovy,!java,!neo4j,!sparql,!mongodb'
+ INTERPRETERS:
'!hbase,!jdbc,!file,!flink,!cassandra,!elasticsearch,!bigquery,!livy,!groovy,!java,!neo4j,!sparql,!mongodb'
ZEPPELIN_E2E_TEST_NOTEBOOK_DIR: '/tmp/zeppelin-e2e-notebooks'
permissions:
diff --git a/AGENTS.md b/AGENTS.md
index 085f8213db..4c78d608d0 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -168,7 +168,6 @@ Each interpreter is an independent Maven module inheriting
from `zeppelin-interp
| `sparql/` | SPARQL queries |
| `influxdb/` | InfluxDB |
| `file/` | HDFS/local file browser |
-| `alluxio/` | Alluxio file system |
### Plugin Modules (`zeppelin-plugins/`)
diff --git a/alluxio/pom.xml b/alluxio/pom.xml
deleted file mode 100644
index ff51bb4e1c..0000000000
--- a/alluxio/pom.xml
+++ /dev/null
@@ -1,110 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd";>
- <modelVersion>4.0.0</modelVersion>
-
- <parent>
- <artifactId>zeppelin-interpreter-parent</artifactId>
- <groupId>org.apache.zeppelin</groupId>
- <version>0.13.0-SNAPSHOT</version>
- <relativePath>../zeppelin-interpreter-parent/pom.xml</relativePath>
- </parent>
-
- <artifactId>zeppelin-alluxio</artifactId>
- <packaging>jar</packaging>
- <name>Zeppelin: Alluxio interpreter</name>
-
- <properties>
- <alluxio.version>2.9.0</alluxio.version>
- <interpreter.name>alluxio</interpreter.name>
- </properties>
-
- <dependencies>
- <dependency>
- <groupId>org.alluxio</groupId>
- <artifactId>alluxio-shell</artifactId>
- <version>${alluxio.version}</version>
- </dependency>
-
- <!-- TEST -->
- <dependency>
- <groupId>com.google.guava</groupId>
- <artifactId>guava</artifactId>
- <version>32.0.0-jre</version>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>com.google.protobuf</groupId>
- <artifactId>protobuf-java</artifactId>
- <version>3.25.5</version>
- <scope>test</scope>
- </dependency>
-
- <dependency>
- <groupId>org.mockito</groupId>
- <artifactId>mockito-core</artifactId>
- <scope>test</scope>
- </dependency>
-
- <dependency>
- <groupId>org.alluxio</groupId>
- <artifactId>alluxio-minicluster</artifactId>
- <version>${alluxio.version}</version>
- <scope>test</scope>
- <exclusions>
- <exclusion>
- <groupId>org.apache.hadoop</groupId>
- <artifactId>hadoop-client</artifactId>
- </exclusion>
- </exclusions>
- </dependency>
-
- <dependency>
- <groupId>org.apache.hadoop</groupId>
- <artifactId>hadoop-client-api</artifactId>
- <scope>test</scope>
- </dependency>
-
- <dependency>
- <groupId>org.apache.hadoop</groupId>
- <artifactId>hadoop-client-runtime</artifactId>
- <scope>test</scope>
- </dependency>
- </dependencies>
-
- <build>
- <plugins>
- <plugin>
- <artifactId>maven-enforcer-plugin</artifactId>
- </plugin>
- <plugin>
- <artifactId>maven-resources-plugin</artifactId>
- </plugin>
- <plugin>
- <artifactId>maven-shade-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-checkstyle-plugin</artifactId>
- </plugin>
- </plugins>
- </build>
-
-</project>
diff --git
a/alluxio/src/main/java/org/apache/zeppelin/alluxio/AlluxioInterpreter.java
b/alluxio/src/main/java/org/apache/zeppelin/alluxio/AlluxioInterpreter.java
deleted file mode 100644
index 5478926ec9..0000000000
--- a/alluxio/src/main/java/org/apache/zeppelin/alluxio/AlluxioInterpreter.java
+++ /dev/null
@@ -1,265 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.zeppelin.alluxio;
-
-import alluxio.cli.fs.FileSystemShell;
-import alluxio.conf.Configuration;
-import alluxio.conf.AlluxioConfiguration;
-import alluxio.conf.PropertyKey;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.ByteArrayOutputStream;
-import java.io.IOException;
-import java.io.PrintStream;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Properties;
-import java.util.stream.Stream;
-
-
-import org.apache.zeppelin.completer.CompletionType;
-import org.apache.zeppelin.interpreter.Interpreter;
-import org.apache.zeppelin.interpreter.InterpreterContext;
-import org.apache.zeppelin.interpreter.InterpreterResult;
-import org.apache.zeppelin.interpreter.InterpreterResult.Code;
-import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion;
-
-/**
- * Alluxio interpreter for Zeppelin.
- */
-public class AlluxioInterpreter extends Interpreter {
-
- private static final Logger LOGGER =
LoggerFactory.getLogger(AlluxioInterpreter.class);
-
- protected static final String ALLUXIO_MASTER_HOSTNAME =
"alluxio.master.hostname";
- protected static final String ALLUXIO_MASTER_PORT = "alluxio.master.port";
-
- private FileSystemShell fs;
-
- private int totalCommands = 0;
- private int completedCommands = 0;
-
- private final String alluxioMasterHostname;
- private final String alluxioMasterPort;
-
- protected final List<String> keywords = Arrays.asList("cat", "chgrp",
- "chmod", "chown", "copyFromLocal", "copyToLocal", "count",
- "createLineage", "deleteLineage", "du", "fileInfo", "free",
- "getCapacityBytes", "getUsedBytes", "listLineages", "load",
- "loadMetadata", "location", "ls", "mkdir", "mount", "mv",
- "persist", "pin", "report", "rm", "setTtl", "tail", "touch",
- "unmount", "unpin", "unsetTtl");
-
- public AlluxioInterpreter(Properties property) {
- super(property);
-
- alluxioMasterHostname = property.getProperty(ALLUXIO_MASTER_HOSTNAME);
- alluxioMasterPort = property.getProperty(ALLUXIO_MASTER_PORT);
- }
-
- private Stream<String> filteredProperties(String prefix) {
- return properties.stringPropertyNames().stream().filter(
- propertyKey -> propertyKey.startsWith(prefix)
- );
- }
-
- @Override
- public void open() {
- LOGGER.info("Starting Alluxio shell to connect to " +
alluxioMasterHostname +
- " on port " + alluxioMasterPort);
- // Setting the extra parameters being set in the interpreter config
starting with alluxio
- filteredProperties("alluxio.").forEach(x -> System.setProperty(x,
properties.getProperty(x)));
-
- System.setProperty(PropertyKey.USER_RPC_RETRY_MAX_DURATION.getName(),
"5s");
-
- AlluxioConfiguration conf = Configuration.global();
- // Reduce the RPC retry max duration to fall earlier for CLIs
- fs = new FileSystemShell(conf);
- }
-
- @Override
- public void close() {
- LOGGER.info("Closing Alluxio shell");
- try {
- fs.close();
- } catch (IOException e) {
- LOGGER.error("Cannot close connection", e);
- }
- }
-
- @Override
- public InterpreterResult interpret(String st, InterpreterContext context) {
- String[] lines = splitAndRemoveEmpty(st, "\n");
- return interpret(lines, context);
- }
-
- private InterpreterResult interpret(String[] commands, InterpreterContext
context) {
- boolean isSuccess = true;
- totalCommands = commands.length;
- completedCommands = 0;
-
- ByteArrayOutputStream baos = new ByteArrayOutputStream();
- PrintStream ps = new PrintStream(baos);
- PrintStream old = System.out;
-
- System.setOut(ps);
-
- for (String command : commands) {
- int commandResult = 1;
- String[] args = splitAndRemoveEmpty(command, " ");
- if (args.length > 0 && args[0].equals("help")) {
- System.out.println(getCommandList());
- } else {
- commandResult = fs.run(args);
- }
- if (commandResult != 0) {
- isSuccess = false;
- break;
- } else {
- completedCommands += 1;
- }
- System.out.println();
- }
-
- System.out.flush();
- System.setOut(old);
-
- if (isSuccess) {
- return new InterpreterResult(Code.SUCCESS, baos.toString());
- } else {
- return new InterpreterResult(Code.ERROR, baos.toString());
- }
- }
-
- private String[] splitAndRemoveEmpty(String st, String splitSeparator) {
- String[] voices = st.split(splitSeparator);
- ArrayList<String> result = new ArrayList<>();
- for (String voice : voices) {
- if (!voice.trim().isEmpty()) {
- result.add(voice);
- }
- }
- return result.toArray(new String[result.size()]);
- }
-
- private String[] splitAndRemoveEmpty(String[] sts, String splitSeparator) {
- ArrayList<String> result = new ArrayList<>();
- for (String st : sts) {
- result.addAll(Arrays.asList(splitAndRemoveEmpty(st, splitSeparator)));
- }
- return result.toArray(new String[result.size()]);
- }
-
- @Override
- public void cancel(InterpreterContext context) { }
-
- @Override
- public FormType getFormType() {
- return FormType.NATIVE;
- }
-
- @Override
- public int getProgress(InterpreterContext context) {
- return completedCommands * 100 / totalCommands;
- }
-
- @Override
- public List<InterpreterCompletion> completion(String buf, int cursor,
- InterpreterContext interpreterContext) {
- String[] words = splitAndRemoveEmpty(splitAndRemoveEmpty(buf, "\n"), " ");
- String lastWord = "";
- if (words.length > 0) {
- lastWord = words[ words.length - 1 ];
- }
-
- List<InterpreterCompletion> voices = new LinkedList<>();
- for (String command : keywords) {
- if (command.startsWith(lastWord)) {
- voices.add(new InterpreterCompletion(command, command,
CompletionType.command.name()));
- }
- }
- return voices;
- }
-
- private String getCommandList() {
- StringBuilder sb = new StringBuilder();
- sb.append("Commands list:");
- sb.append("\n\t[help] - List all available commands.");
- sb.append("\n\t[cat <path>] - Prints the file's contents to the console.");
- sb.append("\n\t[chgrp [-R] <group> <path>] - Changes the group of a file
or directory " +
- "specified by args. Specify -R to change the group recursively.");
- sb.append("\n\t[chmod -R <mode> <path>] - Changes the permission of a file
or directory " +
- "specified by args. Specify -R to change the permission
recursively.");
- sb.append("\n\t[chown -R <owner> <path>] - Changes the owner of a file or
directory " +
- "specified by args. Specify -R to change the owner recursively.");
- sb.append("\n\t[copyFromLocal <src> <remoteDst>] - Copies a file or a
directory from " +
- "local filesystem to Alluxio filesystem.");
- sb.append("\n\t[copyToLocal <src> <localDst>] - Copies a file or a
directory from the " +
- "Alluxio filesystem to the local filesystem.");
- sb.append("\n\t[count <path>] - Displays the number of files and
directories matching " +
- "the specified prefix.");
- sb.append("\n\t[createLineage <inputFile1,...> <outputFile1,...> " +
- "[<cmd_arg1> <cmd_arg2> ...]] - Creates a lineage.");
- sb.append("\n\t[deleteLineage <lineageId> <cascade(true|false)>] - Deletes
a lineage. If " +
- "cascade is specified as true, dependent lineages will also be
deleted.");
- sb.append("\n\t[du <path>] - Displays the size of the specified file or
directory.");
- sb.append("\n\t[fileInfo <path>] - Displays all block info for the
specified file.");
- sb.append("\n\t[free <file path|folder path>] - Removes the file or
directory(recursively) " +
- "from Alluxio memory space.");
- sb.append("\n\t[getCapacityBytes] - Gets the capacity of the Alluxio file
system.");
- sb.append("\n\t[getUsedBytes] - Gets number of bytes used in the Alluxio
file system.");
- sb.append("\n\t[listLineages] - Lists all lineages.");
- sb.append("\n\t[load <path>] - Loads a file or directory in Alluxio space,
makes it " +
- "resident in memory.");
- sb.append("\n\t[loadMetadata <path>] - Loads metadata for the given
Alluxio path from the " +
- "under file system.");
- sb.append("\n\t[location <path>] - Displays the list of hosts storing the
specified file.");
- sb.append("\n\t[ls [-R] <path>] - Displays information for all files and
directories " +
- "directly under the specified path. Specify -R to display files
and " +
- "directories recursively.");
- sb.append("\n\t[mkdir <path1> [path2] ... [pathn]] - Creates the specified
directories, " +
- "including any parent directories that are required.");
- sb.append("\n\t[mount <alluxioPath> <ufsURI>] - Mounts a UFS path onto an
Alluxio path.");
- sb.append("\n\t[mv <src> <dst>] - Renames a file or directory.");
- sb.append("\n\t[persist <alluxioPath>] - Persists a file or directory
currently stored " +
- "only in Alluxio to the UnderFileSystem.");
- sb.append("\n\t[pin <path>] - Pins the given file or directory in memory
(works " +
- "recursively for directories). Pinned files are never evicted from
memory, unless " +
- "TTL is set.");
- sb.append("\n\t[report <path>] - Reports to the master that a file is
lost.");
- sb.append("\n\t[rm [-R] <path>] - Removes the specified file. Specify -R
to remove file or " +
- "directory recursively.");
- sb.append("\n\t[setTtl <path> <time to live(in milliseconds)>] - Sets a
new TTL value for " +
- "the file at path.");
- sb.append("\n\t[tail <path>] - Prints the file's last 1KB of contents to
the console.");
- sb.append("\n\t[touch <path>] - Creates a 0 byte file. The file will be
written to the " +
- "under file system.");
- sb.append("\n\t[unmount <alluxioPath>] - Unmounts an Alluxio path.");
- sb.append("\n\t[unpin <path>] - Unpins the given file or folder from
memory " +
- "(works recursively for a directory).");
- sb.append("\n\\t[unsetTtl <path>] - Unsets the TTL value for the given
path.");
- sb.append("\n\t[unpin <path>] - Unpin the given file to allow Alluxio to
evict " +
- "this file again. If the given path is a directory, it recursively
unpins " +
- "all files contained and any new files created within this
directory.");
- return sb.toString();
- }
-}
diff --git a/alluxio/src/main/resources/interpreter-setting.json
b/alluxio/src/main/resources/interpreter-setting.json
deleted file mode 100644
index 7a8cd62d51..0000000000
--- a/alluxio/src/main/resources/interpreter-setting.json
+++ /dev/null
@@ -1,27 +0,0 @@
-[
- {
- "group": "alluxio",
- "name": "alluxio",
- "className": "org.apache.zeppelin.alluxio.AlluxioInterpreter",
- "properties": {
- "alluxio.master.hostname": {
- "envName": "ALLUXIO_MASTER_HOSTNAME",
- "propertyName": "alluxio.master.hostname",
- "defaultValue": "localhost",
- "description": "Alluxio master hostname",
- "type": "string"
- },
- "alluxio.master.port": {
- "envName": "ALLUXIO_MASTER_PORT",
- "propertyName": "alluxio.master.port",
- "defaultValue": "19998",
- "description": "Alluxio master port",
- "type": "number"
- }
- },
- "editor": {
- "editOnDblClick": false,
- "completionSupport": true
- }
- }
-]
diff --git
a/alluxio/src/test/java/org/apache/zeppelin/alluxio/AlluxioInterpreterTest.java
b/alluxio/src/test/java/org/apache/zeppelin/alluxio/AlluxioInterpreterTest.java
deleted file mode 100644
index 00405113a2..0000000000
---
a/alluxio/src/test/java/org/apache/zeppelin/alluxio/AlluxioInterpreterTest.java
+++ /dev/null
@@ -1,232 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.zeppelin.alluxio;
-
-
-import alluxio.conf.Configuration;
-import alluxio.grpc.WritePType;
-import alluxio.client.file.FileSystemTestUtils;
-import alluxio.master.LocalAlluxioCluster;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Properties;
-
-import alluxio.AlluxioURI;
-import alluxio.client.file.FileSystem;
-import alluxio.client.file.URIStatus;
-import alluxio.exception.AlluxioException;
-import alluxio.util.io.BufferUtils;
-import alluxio.util.io.PathUtils;
-
-import org.apache.zeppelin.completer.CompletionType;
-import org.apache.zeppelin.interpreter.InterpreterResult;
-import org.apache.zeppelin.interpreter.InterpreterResult.Code;
-import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion;
-import org.junit.jupiter.api.AfterEach;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Test;
-
-import static alluxio.cli.fs.command.CountCommand.COUNT_FORMAT;
-import static org.junit.jupiter.api.Assertions.assertArrayEquals;
-import static org.junit.jupiter.api.Assertions.assertEquals;
-import static org.junit.jupiter.api.Assertions.assertNotEquals;
-import static org.junit.jupiter.api.Assertions.assertTrue;
-
-public class AlluxioInterpreterTest {
- private AlluxioInterpreter alluxioInterpreter;
- private LocalAlluxioCluster mLocalAlluxioCluster = null;
- private FileSystem fs = null;
-
- @AfterEach
- final void after() throws Exception {
- if (alluxioInterpreter != null) {
- alluxioInterpreter.close();
- }
-
- mLocalAlluxioCluster.stop();
- }
-
- @BeforeEach
- final void before() throws Exception {
- mLocalAlluxioCluster = new LocalAlluxioCluster(1, false);
- mLocalAlluxioCluster.initConfiguration("alluxio-test");
- Configuration.global().validate();
- mLocalAlluxioCluster.start();
-
- fs = mLocalAlluxioCluster.getClient();
-
- final Properties props = new Properties();
- props.put(AlluxioInterpreter.ALLUXIO_MASTER_HOSTNAME,
mLocalAlluxioCluster.getHostname());
- props.put(AlluxioInterpreter.ALLUXIO_MASTER_PORT,
mLocalAlluxioCluster.getMasterRpcPort() + "");
- alluxioInterpreter = new AlluxioInterpreter(props);
- alluxioInterpreter.open();
- }
-
- @Test
- void testCompletion() {
- List<InterpreterCompletion> expectedResultOne = Arrays.asList(
- new InterpreterCompletion("cat", "cat", CompletionType.command.name()),
- new InterpreterCompletion("chgrp", "chgrp",
CompletionType.command.name()),
- new InterpreterCompletion("chmod", "chmod",
CompletionType.command.name()),
- new InterpreterCompletion("chown", "chown",
CompletionType.command.name()),
- new InterpreterCompletion("copyFromLocal", "copyFromLocal",
CompletionType.command.name()),
- new InterpreterCompletion("copyToLocal", "copyToLocal",
CompletionType.command.name()),
- new InterpreterCompletion("count", "count",
CompletionType.command.name()),
- new InterpreterCompletion("createLineage", "createLineage",
CompletionType.command.name()));
- List<InterpreterCompletion> expectedResultTwo = Arrays.asList(
- new InterpreterCompletion("copyFromLocal", "copyFromLocal",
- CompletionType.command.name()),
- new InterpreterCompletion("copyToLocal", "copyToLocal",
- CompletionType.command.name()),
- new InterpreterCompletion("count", "count",
CompletionType.command.name()));
- List<InterpreterCompletion> expectedResultThree = Arrays.asList(
- new InterpreterCompletion("copyFromLocal", "copyFromLocal",
- CompletionType.command.name()),
- new InterpreterCompletion("copyToLocal", "copyToLocal",
- CompletionType.command.name()));
- List<InterpreterCompletion> expectedResultNone = new ArrayList<>();
-
- List<InterpreterCompletion> resultOne = alluxioInterpreter.completion("c",
0, null);
- List<InterpreterCompletion> resultTwo =
alluxioInterpreter.completion("co", 0, null);
- List<InterpreterCompletion> resultThree =
alluxioInterpreter.completion("copy", 0, null);
- List<InterpreterCompletion> resultNotMatch =
alluxioInterpreter.completion("notMatch", 0, null);
- List<InterpreterCompletion> resultAll = alluxioInterpreter.completion("",
0, null);
-
- assertEquals(expectedResultOne, resultOne);
- assertEquals(expectedResultTwo, resultTwo);
- assertEquals(expectedResultThree, resultThree);
- assertEquals(expectedResultNone, resultNotMatch);
-
- List<String> allCompletionList = new ArrayList<>();
- for (InterpreterCompletion ic : resultAll) {
- allCompletionList.add(ic.getName());
- }
- assertEquals(alluxioInterpreter.keywords, allCompletionList);
- }
-
- @Test
- void catTest() throws IOException {
- FileSystemTestUtils.createByteFile(fs, "/testFile", WritePType.MUST_CACHE,
10, 10);
- InterpreterResult output = alluxioInterpreter.interpret("cat /testFile",
null);
-
- byte[] expected = BufferUtils.getIncreasingByteArray(10);
-
- assertEquals(Code.SUCCESS, output.code());
- assertArrayEquals(expected,
- output.message().get(0).getData().substring(0,
- output.message().get(0).getData().length() -
1).getBytes());
- }
-
- @Test
- void loadFileTest() throws IOException, AlluxioException {
- FileSystemTestUtils.createByteFile(fs, "/testFile",
WritePType.CACHE_THROUGH, 10, 10);
-
- int memPercentage = fs.getStatus(new
AlluxioURI("/testFile")).getInMemoryPercentage();
- assertNotEquals(0, memPercentage);
-
- alluxioInterpreter.interpret("load /testFile", null);
-
- memPercentage = fs.getStatus(new
AlluxioURI("/testFile")).getInMemoryPercentage();
- assertEquals(100, memPercentage);
- }
-
- @Test
- void copyToLocalTest() throws IOException {
- FileSystemTestUtils.createByteFile(fs, "/testFile", WritePType.MUST_CACHE,
10, 10);
-
- InterpreterResult output = alluxioInterpreter.interpret("copyToLocal
/testFile " +
- mLocalAlluxioCluster.getAlluxioHome() + "/testFile", null);
-
- assertEquals(
- "Copied /testFile to file://" +
mLocalAlluxioCluster.getAlluxioHome() + "/testFile\n\n",
- output.message().get(0).getData());
- fileReadTest("/testFile", 10);
- }
-
- @Test
- void countTest() throws IOException {
- FileSystemTestUtils.createByteFile(fs, "/testRoot/testFileA",
- WritePType.MUST_CACHE, 10);
- FileSystemTestUtils.createByteFile(fs, "/testRoot/testDir/testFileB",
- WritePType.MUST_CACHE, 20);
- FileSystemTestUtils.createByteFile(fs, "/testRoot/testFileB",
- WritePType.MUST_CACHE, 30);
-
- InterpreterResult output = alluxioInterpreter.interpret("count /testRoot",
null);
-
- String expected = "";
- expected += String.format(COUNT_FORMAT, "File Count", "Folder Count",
"Folder Size");
- expected += String.format(COUNT_FORMAT, 3, 1, 60);
- expected += "\n";
- assertEquals(expected, output.message().get(0).getData());
-
- InterpreterResult output2 = alluxioInterpreter.interpret("count -h
/testRoot", null);
- String expected2 = "";
- expected2 += String.format(COUNT_FORMAT, "File Count", "Folder Count",
"Folder Size");
- expected2 += String.format(COUNT_FORMAT, 3, 1, "60B");
- expected2 += "\n";
- assertEquals(expected2, output2.message().get(0).getData());
- }
-
- @Test
- void lsTest() throws IOException, AlluxioException {
- URIStatus[] files = new URIStatus[3];
-
- FileSystemTestUtils.createByteFile(fs, "/testRoot/testFileA",
- WritePType.MUST_CACHE, 10, 10);
- FileSystemTestUtils.createByteFile(fs, "/testRoot/testDir/testFileB",
- WritePType.MUST_CACHE, 20, 20);
- FileSystemTestUtils.createByteFile(fs, "/testRoot/testFileC",
- WritePType.THROUGH, 30, 30);
-
- files[0] = fs.getStatus(new AlluxioURI("/testRoot/testFileA"));
- files[1] = fs.getStatus(new AlluxioURI("/testRoot/testDir"));
- files[2] = fs.getStatus(new AlluxioURI("/testRoot/testFileC"));
-
- InterpreterResult output = alluxioInterpreter.interpret("ls /testRoot",
null);
-
- assertEquals(Code.SUCCESS, output.code());
- }
-
- @Test
- void mkdirTest() throws IOException, AlluxioException {
- String qualifiedPath =
- "alluxio://" + mLocalAlluxioCluster.getHostname() + ":"
- + mLocalAlluxioCluster.getMasterRpcPort() +
"/root/testFile1";
- InterpreterResult output = alluxioInterpreter.interpret("mkdir " +
qualifiedPath, null);
- boolean existsDir = fs.exists(new AlluxioURI("/root/testFile1"));
- assertEquals(
- "Successfully created directory " + qualifiedPath + "\n\n",
- output.message().get(0).getData());
- assertTrue(existsDir);
- }
-
- private void fileReadTest(String fileName, int size) throws IOException {
- File testFile = new
File(PathUtils.concatPath(mLocalAlluxioCluster.getAlluxioHome(), fileName));
- FileInputStream fis = new FileInputStream(testFile);
- byte[] read = new byte[size];
- fis.read(read);
- fis.close();
- assertTrue(BufferUtils.equalIncreasingByteArray(size, read));
- }
-}
diff --git a/conf/interpreter-list b/conf/interpreter-list
index 792b6d6858..c6e0305f60 100644
--- a/conf/interpreter-list
+++ b/conf/interpreter-list
@@ -17,7 +17,6 @@
#
# [name] [maven artifact] [description]
-alluxio org.apache.zeppelin:zeppelin-alluxio:0.13.0-SNAPSHOT
Alluxio interpreter
angular org.apache.zeppelin:zeppelin-angular:0.13.0-SNAPSHOT
HTML and AngularJS view rendering
bigquery org.apache.zeppelin:zeppelin-bigquery:0.13.0-SNAPSHOT
BigQuery interpreter
cassandra org.apache.zeppelin:zeppelin-cassandra:0.13.0-SNAPSHOT
Cassandra interpreter
diff --git a/dev/create_release.sh b/dev/create_release.sh
index b5529169f6..0622497b9e 100755
--- a/dev/create_release.sh
+++ b/dev/create_release.sh
@@ -97,7 +97,7 @@ function make_binary_release() {
git_clone
make_source_package
-make_binary_release netinst "-Pweb-classic -pl
!hbase,!jdbc,!file,!flink,!cassandra,!elasticsearch,!bigquery,!alluxio,!livy,!groovy,!java,!neo4j,!sparql,!mongodb,!shell
-am"
+make_binary_release netinst "-Pweb-classic -pl
!hbase,!jdbc,!file,!flink,!cassandra,!elasticsearch,!bigquery,!livy,!groovy,!java,!neo4j,!sparql,!mongodb,!shell
-am"
make_binary_release all "-Pweb-classic -pl !shell"
diff --git a/docs/_includes/themes/zeppelin/_navigation.html
b/docs/_includes/themes/zeppelin/_navigation.html
index 9305359fdf..a82c1c3682 100644
--- a/docs/_includes/themes/zeppelin/_navigation.html
+++ b/docs/_includes/themes/zeppelin/_navigation.html
@@ -139,7 +139,6 @@
<li><a
href="{{BASE_PATH}}/interpreter/python.html">Python</a></li>
<li><a href="{{BASE_PATH}}/interpreter/r.html">R</a></li>
<li role="separator" class="divider"></li>
- <li><a
href="{{BASE_PATH}}/interpreter/alluxio.html">Alluxio</a></li>
<li><a
href="{{BASE_PATH}}/interpreter/bigquery.html">BigQuery</a></li>
<li><a
href="{{BASE_PATH}}/interpreter/cassandra.html">Cassandra</a></li>
<li><a
href="{{BASE_PATH}}/interpreter/elasticsearch.html">Elasticsearch</a></li>
diff --git a/docs/index.md b/docs/index.md
index 75d362ca86..7f9f8ada49 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -134,7 +134,6 @@ limitations under the License.
* [How to Contribute
(website)](./development/contribution/how_to_contribute_website.html)
#### Available Interpreters
- * [Alluxio](./interpreter/alluxio.html)
* [BigQuery](./interpreter/bigquery.html)
* [Cassandra](./interpreter/cassandra.html)
* [Elasticsearch](./interpreter/elasticsearch.html)
diff --git a/docs/interpreter/alluxio.md b/docs/interpreter/alluxio.md
deleted file mode 100644
index 119f545ff2..0000000000
--- a/docs/interpreter/alluxio.md
+++ /dev/null
@@ -1,250 +0,0 @@
----
-layout: page
-title: "Alluxio Interpreter for Apache Zeppelin"
-description: "Alluxio is a memory-centric distributed storage system enabling
reliable data sharing at memory-speed across cluster frameworks."
-group: interpreter
----
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-{% include JB/setup %}
-
-# Alluxio Interpreter for Apache Zeppelin
-
-<div id="toc"></div>
-
-## Overview
-[Alluxio](http://alluxio.org/) is a memory-centric distributed storage system
enabling reliable data sharing at memory-speed across cluster frameworks.
-
-## Configuration
-<table class="table-configuration">
- <tr>
- <th>Name</th>
- <th>Class</th>
- <th>Description</th>
- </tr>
- <tr>
- <td>alluxio.master.hostname</td>
- <td>localhost</td>
- <td>Alluxio master hostname</td>
- </tr>
- <tr>
- <td>alluxio.master.port</td>
- <td>19998</td>
- <td>Alluxio master port</td>
- </tr>
-</table>
-
-## Enabling Alluxio Interpreter
-In a notebook, to enable the **Alluxio** interpreter, click on the **Gear**
icon and select **Alluxio**.
-
-## Using the Alluxio Interpreter
-In a paragraph, use `%alluxio` to select the **Alluxio** interpreter and then
input all commands.
-
-```bash
-%alluxio
-help
-```
-
-> **Tip :** Use ( Ctrl + . ) for autocompletion.
-
-## Interpreter Commands
-The **Alluxio** interpreter accepts the following commands.
-
-<center>
- <table class="table-configuration">
- <tr>
- <th>Operation</th>
- <th>Syntax</th>
- <th>Description</th>
- </tr>
- <tr>
- <td>cat</td>
- <td>cat "path"</td>
- <td>Print the content of the file to the console.</td>
- </tr>
- <tr>
- <td>chgrp</td>
- <td>chgrp "group" "path"</td>
- <td>Change the group of the directory or file.</td>
- </tr>
- <tr>
- <td>chmod</td>
- <td>chmod "permission" "path"</td>
- <td>Change the permission of the directory or file.</td>
- </tr>
- <tr>
- <td>chown</td>
- <td>chown "owner" "path"</td>
- <td>Change the owner of the directory or file.</td>
- </tr>
- <tr>
- <td>copyFromLocal</td>
- <td>copyFromLocal "source path" "remote path"</td>
- <td>Copy the specified file specified by "source path" to the path
specified by "remote path".
- This command will fail if "remote path" already exists.</td>
- </tr>
- <tr>
- <td>copyToLocal</td>
- <td>copyToLocal "remote path" "local path"</td>
- <td>Copy the specified file from the path specified by "remote path" to
a local destination.</td>
- </tr>
- <tr>
- <td>count</td>
- <td>count "path"</td>
- <td>Display the number of folders and files matching the specified
prefix in "path".</td>
- </tr>
- <tr>
- <td>du</td>
- <td>du "path"</td>
- <td>Display the size of a file or a directory specified by the input
path.</td>
- </tr>
- <tr>
- <td>fileInfo</td>
- <td>fileInfo "path"</td>
- <td>Print the information of the blocks of a specified file.</td>
- </tr>
- <tr>
- <td>free</td>
- <td>free "path"</td>
- <td>Free a file or all files under a directory from Alluxio. If the
file/directory is also
- in under storage, it will still be available there.</td>
- </tr>
- <tr>
- <td>getCapacityBytes</td>
- <td>getCapacityBytes</td>
- <td>Get the capacity of the AlluxioFS.</td>
- </tr>
- <tr>
- <td>getUsedBytes</td>
- <td>getUsedBytes</td>
- <td>Get number of bytes used in the AlluxioFS.</td>
- </tr>
- <tr>
- <td>load</td>
- <td>load "path"</td>
- <td>Load the data of a file or a directory from under storage into
Alluxio.</td>
- </tr>
- <tr>
- <td>loadMetadata</td>
- <td>loadMetadata "path"</td>
- <td>Load the metadata of a file or a directory from under storage into
Alluxio.</td>
- </tr>
- <tr>
- <td>location</td>
- <td>location "path"</td>
- <td>Display a list of hosts that have the file data.</td>
- </tr>
- <tr>
- <td>ls</td>
- <td>ls "path"</td>
- <td>List all the files and directories directly under the given path
with information such as
- size.</td>
- </tr>
- <tr>
- <td>mkdir</td>
- <td>mkdir "path1" ... "pathn"</td>
- <td>Create directory(ies) under the given paths, along with any
necessary parent directories.
- Multiple paths separated by spaces or tabs. This command will fail if
any of the given paths
- already exist.</td>
- </tr>
- <tr>
- <td>mount</td>
- <td>mount "path" "uri"</td>
- <td>Mount the underlying file system path "uri" into the Alluxio
namespace as "path". The "path"
- is assumed not to exist and is created by the operation. No data or
metadata is loaded from under
- storage into Alluxio. After a path is mounted, operations on objects
under the mounted path are
- mirror to the mounted under storage.</td>
- </tr>
- <tr>
- <td>mv</td>
- <td>mv "source" "destination"</td>
- <td>Move a file or directory specified by "source" to a new location
"destination". This command
- will fail if "destination" already exists.</td>
- </tr>
- <tr>
- <td>persist</td>
- <td>persist "path"</td>
- <td>Persist a file or directory currently stored only in Alluxio to the
underlying file system.</td>
- </tr>
- <tr>
- <td>pin</td>
- <td>pin "path"</td>
- <td>Pin the given file to avoid evicting it from memory. If the given
path is a directory, it
- recursively pins all the files contained and any new files created
within this directory.</td>
- </tr>
- <tr>
- <td>report</td>
- <td>report "path"</td>
- <td>Report to the master that a file is lost.</td>
- </tr>
- <tr>
- <td>rm</td>
- <td>rm "path"</td>
- <td>Remove a file. This command will fail if the given path is a
directory rather than a file.</td>
- </tr>
- <tr>
- <td>setTtl</td>
- <td>setTtl "time"</td>
- <td>Set the TTL (time to live) in milliseconds to a file.</td>
- </tr>
- <tr>
- <td>tail</td>
- <td>tail "path"</td>
- <td>Print the last 1KB of the specified file to the console.</td>
- </tr>
- <tr>
- <td>touch</td>
- <td>touch "path"</td>
- <td>Create a 0-byte file at the specified location.</td>
- </tr>
- <tr>
- <td>unmount</td>
- <td>unmount "path"</td>
- <td>Unmount the underlying file system path mounted in the Alluxio
namespace as "path". Alluxio
- objects under "path" are removed from Alluxio, but they still exist in
the previously mounted
- under storage.</td>
- </tr>
- <tr>
- <td>unpin</td>
- <td>unpin "path"</td>
- <td>Unpin the given file to allow Alluxio to evict this file again. If
the given path is a
- directory, it recursively unpins all files contained and any new files
created within this
- directory.</td>
- </tr>
- <tr>
- <td>unsetTtl</td>
- <td>unsetTtl</td>
- <td>Remove the TTL (time to live) setting from a file.</td>
- </tr>
- </table>
-</center>
-
-## How to test it's working
-Be sure to have configured correctly the Alluxio interpreter, then open a new
paragraph and type one of the above commands.
-
-Below a simple example to show how to interact with Alluxio interpreter.
-Following steps are performed:
-
-* using sh interpreter a new text file is created on local machine
-* using Alluxio interpreter:
- * is listed the content of the afs (Alluxio File System) root
- * the file previously created is copied to afs
- * is listed again the content of the afs root to check the existence of the
new copied file
- * is showed the content of the copied file (using the tail command)
- * the file previously copied to afs is copied to local machine
-* using sh interpreter it's checked the existence of the new file copied from
Alluxio and its content is showed
-
-<center>
- 
-</center>
diff --git a/docs/interpreter/hdfs.md b/docs/interpreter/hdfs.md
index d7b7bf885d..bec3785aad 100644
--- a/docs/interpreter/hdfs.md
+++ b/docs/interpreter/hdfs.md
@@ -1,7 +1,7 @@
---
layout: page
title: "HDFS File System Interpreter for Apache Zeppelin"
-description: "Hadoop File System is a distributed, fault tolerant file system
part of the hadoop project and is often used as storage for distributed
processing engines like Hadoop MapReduce and Apache Spark or underlying file
systems like Alluxio."
+description: "Hadoop File System is a distributed, fault tolerant file system
part of the hadoop project and is often used as storage for distributed
processing engines like Hadoop MapReduce and Apache Spark or underlying file
systems like S3."
group: interpreter
---
<!--
@@ -24,7 +24,7 @@ limitations under the License.
<div id="toc"></div>
## Overview
-[Hadoop File System](http://hadoop.apache.org/) is a distributed, fault
tolerant file system part of the hadoop project and is often used as storage
for distributed processing engines like [Hadoop
MapReduce](http://hadoop.apache.org/) and [Apache
Spark](http://spark.apache.org/) or underlying file systems like
[Alluxio](http://www.alluxio.org/).
+[Hadoop File System](http://hadoop.apache.org/) is a distributed, fault
tolerant file system part of the hadoop project and is often used as storage
for distributed processing engines like [Hadoop
MapReduce](http://hadoop.apache.org/) and [Apache
Spark](http://spark.apache.org/) or underlying file systems like
[S3](https://aws.amazon.com/s3/).
## Configuration
<table class="table-configuration">
diff --git a/docs/usage/interpreter/installation.md
b/docs/usage/interpreter/installation.md
index f3e980d5e8..31ed91d1af 100644
--- a/docs/usage/interpreter/installation.md
+++ b/docs/usage/interpreter/installation.md
@@ -82,11 +82,6 @@ You can also find the below community managed interpreter
list in `conf/interpre
<th>Maven Artifact</th>
<th>Description</th>
</tr>
- <tr>
- <td>alluxio</td>
- <td>org.apache.zeppelin:zeppelin-alluxio:0.13.0-SNAPSHOT</td>
- <td>Alluxio interpreter</td>
- </tr>
<tr>
<td>angular</td>
<td>org.apache.zeppelin:zeppelin-angular:0.13.0-SNAPSHOT</td>
diff --git a/pom.xml b/pom.xml
index 8fd194cd48..8eb5254405 100644
--- a/pom.xml
+++ b/pom.xml
@@ -77,7 +77,6 @@
<module>cassandra</module>
<module>elasticsearch</module>
<module>bigquery</module>
- <module>alluxio</module>
<module>neo4j</module>
<module>java</module>
<module>sparql</module>
diff --git a/zeppelin-distribution/src/bin_license/LICENSE
b/zeppelin-distribution/src/bin_license/LICENSE
index e82c18f43b..3c6939d403 100644
--- a/zeppelin-distribution/src/bin_license/LICENSE
+++ b/zeppelin-distribution/src/bin_license/LICENSE
@@ -103,10 +103,6 @@ The following components are provided under Apache License.
(Apache 2.0) Shiro Web (org.apache.shiro:shiro-web:1.8.0 -
https://shiro.apache.org)
(Apache 2.0) SnakeYAML (org.yaml:snakeyaml:1.15 - http://www.snakeyaml.org)
(Apache 2.0) Protocol Buffers (com.google.protobuf:protobuf-java:2.5.0 -
https://github.com/google/protobuf/releases)
- (Apache 2.0) Alluxio Shell (org.alluxio:alluxio-shell:1.0.0 -
http://alluxio.org)
- (Apache 2.0) Alluxio Servers (org.alluxio:alluxio-core-server:1.0.0 -
http://alluxio.org)
- (Apache 2.0) Alluxio Minicluster (org.alluxio:alluxio-minicluster:1.0.0 -
http://alluxio.org)
- (Apache 2.0) Alluxio Underfs Local
(org.alluxio:alluxio-underfs-local:1.0.0 - http://alluxio.org)
(Apache 2.0) Microsoft Azure Storage Library for Java
(com.microsoft.azure:azure-storage:4.0.0 -
https://github.com/Azure/azure-storage-java)
(Apache 2.0) stream (com.clearspring.analytics:stream:2.7.0) -
https://github.com/addthis/stream-lib/blob/v2.7.0/LICENSE.txt
(Apache 2.0) io.dropwizard.metrics:3.1.2 -
https://github.com/dropwizard/metrics/blob/v3.1.2/LICENSE
