Repository: zeppelin
Updated Branches:
refs/heads/master 336df5617 -> f35d5de7d
[ZEPPELIN-2069] Helium Package Configuration
### What is this PR for?
Supporting helium package configurations. I attached screenshots.
#### Implementation details.
In case of spell, spell developer can create config spec in their
`package.json` and it will be part of `helium.json` which is consumed by
Zeppelin.
```
"config": {
"repeat": {
"type": "number",
"description": "How many times to repeat",
"defaultValue": 1
}
},
```
1. Persists conf per `package namepackage version` since each version can
require different configs even if they are the same package.
2. Saves key-value config only. Since config spec (e.g `type`, `desc`,
`defaultValue`) can be provided. So it's not efficient save both of them.
3. Extracts config related functions to `helium.service.js` since it can be
used not only in `helium.controller.js` for view but also should be used in
`paragraph.controller.js`, `result.controller.js` for executing spell.
### What type of PR is it?
[Feature]
### Todos
* [x] - create config view in `/helium`
* [x] - persist config per `packageversion`
* [x] - pass config to spell
### What is the Jira issue?
[ZEPPELIN-2069](https://issues.apache.org/jira/browse/ZEPPELIN-2069)
### How should this be tested?
- Build with examples `mvn clean package -Phelium-dev -Pexamples -DskipTests;`
- Open `/helium` page
- Update the `echo-spell` config
- Execute the spell like the screenshot below. (you don't need to refresh the
page, since executing spell will fetch config from server)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1am...@gmail.com>
Closes #1982 from 1ambda/ZEPPELIN-2069/helium-package-configuration and
squashes the following commits:
dbc4f10 [1ambda] fix: Add getAllPackageInfoWithoutRefresh
ce5f8c0 [1ambda] fix: Remove version 'local'
696f7f8 [1ambda] fix: DON'T serialize version field in HeliumPackage
e599ab9 [1ambda] feat: Close spell config panel after saving
c9b0145 [1ambda] feat: Make spell execution transactional
d9e87a8 [1ambda] refactor: Create API call for config
453016b [1ambda] fix: configExists
e6d5181 [1ambda] fix: Lint error
33a2bd8 [1ambda] refactor: HeliumService
f31bf3c [1ambda] feat: Add disabled class to cfg button while fetching
76d50ca [1ambda] fix: Use artifact as key of config
729c5ba [1ambda] fix: Remove digest from para ctrl
4d3c2c7 [1ambda] feat: Add config to framework, examples
70ebe29 [1ambda] feat: Pass confs to spell interpret()
115191e [1ambda] refactor: Extact spell related code to helium
3aa6c54 [1ambda] feat: Support helium conf in frontend
dea2929 [1ambda] chore: Add conf to example spells
6910e97 [1ambda] feat: Support config for helium pkg in backend
0a0c565 [1ambda] feat: Support config, version field for helium pkg
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/f35d5de7
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/f35d5de7
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/f35d5de7
Branch: refs/heads/master
Commit: f35d5de7d61810fc6e70edd9dc779d7b498a4978
Parents: 336df56
Author: 1ambda <1am...@gmail.com>
Authored: Mon Feb 27 17:39:42 2017 +0900
Committer: Lee moon soo <m...@apache.org>
Committed: Thu Mar 2 13:06:54 2017 +0900
----------------------------------------------------------------------
.../zeppelin-example-spell-echo/index.js | 25 ++-
.../zeppelin-example-spell-echo.json | 7 +
.../zeppelin-example-spell-translator/index.js | 16 +-
.../zeppelin-example-spell-translator.json | 7 +
.../apache/zeppelin/helium/HeliumPackage.java | 8 +-
.../zeppelin/helium/HeliumPackageTest.java | 39 +++-
.../org/apache/zeppelin/rest/HeliumRestApi.java | 126 ++++++++++++-
.../apache/zeppelin/server/ZeppelinServer.java | 4 +-
.../apache/zeppelin/socket/NotebookServer.java | 27 ++-
zeppelin-web/src/app/helium/helium.config.js | 100 +++++++++++
.../src/app/helium/helium.controller.js | 139 +++++++-------
zeppelin-web/src/app/helium/helium.css | 21 +++
zeppelin-web/src/app/helium/helium.html | 80 +++++++--
zeppelin-web/src/app/helium/index.js | 19 ++
.../notebook/paragraph/paragraph.controller.js | 104 ++++++++---
.../paragraph/result/result.controller.js | 69 +++----
zeppelin-web/src/app/spell/spell-base.js | 3 +-
.../src/components/helium/helium-conf.js | 96 ++++++++++
.../src/components/helium/helium-package.js | 47 +++++
.../src/components/helium/helium.service.js | 180 +++++++++++++++++--
.../websocketEvents/websocketEvents.factory.js | 2 +
.../websocketEvents/websocketMsg.service.js | 2 +-
zeppelin-web/src/index.js | 4 +-
.../java/org/apache/zeppelin/helium/Helium.java | 125 +++++++++++--
.../org/apache/zeppelin/helium/HeliumConf.java | 37 +++-
.../org/apache/zeppelin/helium/HeliumTest.java | 6 +-
26 files changed, 1093 insertions(+), 200 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-examples/zeppelin-example-spell-echo/index.js
----------------------------------------------------------------------
diff --git a/zeppelin-examples/zeppelin-example-spell-echo/index.js
b/zeppelin-examples/zeppelin-example-spell-echo/index.js
index 955178e..5b379e1 100644
--- a/zeppelin-examples/zeppelin-example-spell-echo/index.js
+++ b/zeppelin-examples/zeppelin-example-spell-echo/index.js
@@ -26,7 +26,28 @@ export default class EchoSpell extends SpellBase {
super("%echo");
}
- interpret(paragraphText) {
- return new SpellResult(paragraphText);
+ /**
+ * Consumes text and return `SpellResult`.
+ *
+ * @param paragraphText {string} which doesn't include magic
+ * @param config {Object}
+ * @return {SpellResult}
+ */
+ interpret(paragraphText, config) {
+ let repeat = 1;
+
+ try {
+ repeat = parseFloat(config.repeat);
+ } catch (error) {
+ /** ignore, use default value */
+ }
+
+ let repeated = "";
+
+ for (let i = 0; i < repeat; i++) {
+ repeated += `${paragraphText}\n`;
+ }
+
+ return new SpellResult(repeated);
}
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json
----------------------------------------------------------------------
diff --git
a/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json
b/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json
index f267b97..fe1d06e 100644
---
a/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json
+++
b/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json
@@ -21,6 +21,13 @@
"artifact" : "./zeppelin-examples/zeppelin-example-spell-echo",
"license" : "Apache-2.0",
"icon" : "<i class='fa fa-repeat'></i>",
+ "config": {
+ "repeat": {
+ "type": "number",
+ "description": "How many times to repeat",
+ "defaultValue": 1
+ }
+ },
"spell": {
"magic": "%echo",
"usage": "%echo <TEXT>"
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-examples/zeppelin-example-spell-translator/index.js
----------------------------------------------------------------------
diff --git a/zeppelin-examples/zeppelin-example-spell-translator/index.js
b/zeppelin-examples/zeppelin-example-spell-translator/index.js
index 834e707..284ce3b 100644
--- a/zeppelin-examples/zeppelin-example-spell-translator/index.js
+++ b/zeppelin-examples/zeppelin-example-spell-translator/index.js
@@ -28,11 +28,18 @@ export default class TranslatorSpell extends SpellBase {
super("%translator");
}
- interpret(paragraphText) {
+ /**
+ * Consumes text and return `SpellResult`.
+ *
+ * @param paragraphText {string} which doesn't include magic
+ * @param config {Object}
+ * @return {SpellResult}
+ */
+ interpret(paragraphText, config) {
const parsed = this.parseConfig(paragraphText);
+ const auth = config['access-token'];
const source = parsed.source;
const target = parsed.target;
- const auth = parsed.auth;
const text = parsed.text;
/**
@@ -49,7 +56,7 @@ export default class TranslatorSpell extends SpellBase {
}
parseConfig(text) {
- const pattern = /^\s*(\S+)-(\S+)\s*(\S+)([\S\s]*)/g;
+ const pattern = /^\s*(\S+)-(\S+)\s*([\S\s]*)/g;
const match = pattern.exec(text);
if (!match) {
@@ -59,8 +66,7 @@ export default class TranslatorSpell extends SpellBase {
return {
source: match[1],
target: match[2],
- auth: match[3],
- text: match[4],
+ text: match[3],
}
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json
----------------------------------------------------------------------
diff --git
a/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json
b/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json
index 8f99783..965e90c 100644
---
a/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json
+++
b/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json
@@ -21,6 +21,13 @@
"artifact" : "./zeppelin-examples/zeppelin-example-spell-translator",
"license" : "Apache-2.0",
"icon" : "<i class='fa fa-globe '></i>",
+ "config": {
+ "access-token": {
+ "type": "string",
+ "description": "access token for Google Translation API",
+ "defaultValue": "EXAMPLE-TOKEN"
+ }
+ },
"spell": {
"magic": "%translator",
"usage": "%translator <source>-<target> <access-key> <TEXT>"
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java
----------------------------------------------------------------------
diff --git
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java
index e8e6b7c..62c4bcf 100644
---
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java
+++
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java
@@ -18,6 +18,8 @@ package org.apache.zeppelin.helium;
import org.apache.zeppelin.annotation.Experimental;
+import java.util.Map;
+
/**
* Helium package definition
*/
@@ -33,7 +35,8 @@ public class HeliumPackage {
private String license;
private String icon;
- public SpellPackageInfo spell;
+ private SpellPackageInfo spell;
+ private Map<String, Object> config;
public HeliumPackage(HeliumType type,
String name,
@@ -100,6 +103,7 @@ public class HeliumPackage {
public String getLicense() {
return license;
}
+
public String getIcon() {
return icon;
}
@@ -107,4 +111,6 @@ public class HeliumPackage {
public SpellPackageInfo getSpellInfo() {
return spell;
}
+
+ public Map<String, Object> getConfig() { return config; }
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java
----------------------------------------------------------------------
diff --git
a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java
b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java
index aadae41..a697fbd 100644
---
a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java
+++
b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java
@@ -20,6 +20,8 @@ package org.apache.zeppelin.helium;
import com.google.gson.Gson;
import org.junit.Test;
+import java.util.Map;
+
import static org.junit.Assert.*;
public class HeliumPackageTest {
@@ -28,7 +30,7 @@ public class HeliumPackageTest {
@Test
public void parseSpellPackageInfo() {
- String exampleSpell = "{\n" +
+ String examplePackage = "{\n" +
" \"type\" : \"SPELL\",\n" +
" \"name\" : \"echo-spell\",\n" +
" \"description\" : \"'%echo' - return just what receive
(example)\",\n" +
@@ -41,8 +43,41 @@ public class HeliumPackageTest {
" }\n" +
"}";
- HeliumPackage p = gson.fromJson(exampleSpell, HeliumPackage.class);
+ HeliumPackage p = gson.fromJson(examplePackage, HeliumPackage.class);
assertEquals(p.getSpellInfo().getMagic(), "%echo");
assertEquals(p.getSpellInfo().getUsage(), "%echo <TEXT>");
}
+
+ @Test
+ public void parseConfig() {
+ String examplePackage = "{\n" +
+ " \"type\" : \"SPELL\",\n" +
+ " \"name\" : \"translator-spell\",\n" +
+ " \"description\" : \"Translate langauges using Google API
(examaple)\",\n" +
+ " \"artifact\" :
\"./zeppelin-examples/zeppelin-example-spell-translator\",\n" +
+ " \"license\" : \"Apache-2.0\",\n" +
+ " \"icon\" : \"<i class='fa fa-globe '></i>\",\n" +
+ " \"config\": {\n" +
+ " \"access-token\": {\n" +
+ " \"type\": \"string\",\n" +
+ " \"description\": \"access token for Google Translation
API\",\n" +
+ " \"defaultValue\": \"EXAMPLE-TOKEN\"\n" +
+ " }\n" +
+ " },\n" +
+ " \"spell\": {\n" +
+ " \"magic\": \"%translator\",\n" +
+ " \"usage\": \"%translator <source>-<target> <access-key>
<TEXT>\"\n" +
+ " }\n" +
+ "}";
+
+ HeliumPackage p = gson.fromJson(examplePackage, HeliumPackage.class);
+ Map<String, Object> config = p.getConfig();
+ Map<String, Object> accessToken = (Map<String, Object>)
config.get("access-token");
+
+ assertEquals((String) accessToken.get("type"),"string");
+ assertEquals((String) accessToken.get("description"),
+ "access token for Google Translation API");
+ assertEquals((String) accessToken.get("defaultValue"),
+ "EXAMPLE-TOKEN");
+ }
}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java
----------------------------------------------------------------------
diff --git
a/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java
b/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java
index c318be5..9234cc5 100644
--- a/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java
+++ b/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java
@@ -18,8 +18,10 @@
package org.apache.zeppelin.rest;
import com.google.gson.Gson;
+import com.google.gson.JsonParseException;
import com.google.gson.reflect.TypeToken;
import org.apache.commons.io.FileUtils;
+import org.apache.commons.lang3.StringUtils;
import org.apache.zeppelin.helium.Helium;
import org.apache.zeppelin.helium.HeliumPackage;
import org.apache.zeppelin.notebook.Note;
@@ -34,6 +36,7 @@ import javax.ws.rs.core.Response;
import java.io.File;
import java.io.IOException;
import java.util.List;
+import java.util.Map;
/**
* Helium Rest Api
@@ -56,13 +59,34 @@ public class HeliumRestApi {
}
/**
- * Get all packages
- * @return
+ * Get all package infos
*/
@GET
- @Path("all")
- public Response getAll() {
- return new JsonResponse(Response.Status.OK, "",
helium.getAllPackageInfo()).build();
+ @Path("package")
+ public Response getAllPackageInfo() {
+ return new JsonResponse(
+ Response.Status.OK, "", helium.getAllPackageInfo()).build();
+ }
+
+ /**
+ * Get single package info
+ */
+ @GET
+ @Path("package/{packageName}")
+ public Response getSinglePackageInfo(@PathParam("packageName") String
packageName) {
+ if (StringUtils.isEmpty(packageName)) {
+ return new JsonResponse(
+ Response.Status.BAD_REQUEST,
+ "Can't get package info for empty name").build();
+ }
+
+ try {
+ return new JsonResponse(
+ Response.Status.OK, "",
helium.getSinglePackageInfo(packageName)).build();
+ } catch (RuntimeException e) {
+ logger.error(e.getMessage(), e);
+ return new JsonResponse(Response.Status.INTERNAL_SERVER_ERROR,
e.getMessage()).build();
+ }
}
@GET
@@ -165,6 +189,98 @@ public class HeliumRestApi {
return new JsonResponse(Response.Status.OK, order).build();
}
+ @GET
+ @Path("spell/config/{packageName}")
+ public Response getSpellConfigUsingMagic(@PathParam("packageName") String
packageName) {
+ if (StringUtils.isEmpty(packageName)) {
+ return new JsonResponse(Response.Status.BAD_REQUEST,
+ "packageName is empty" ).build();
+ }
+
+ try {
+ Map<String, Map<String, Object>> config =
+ helium.getSpellConfig(packageName);
+
+ if (config == null) {
+ return new JsonResponse(Response.Status.BAD_REQUEST,
+ "Failed to find enabled package for " + packageName).build();
+ }
+
+ return new JsonResponse(Response.Status.OK, config).build();
+ } catch (RuntimeException e) {
+ logger.error(e.getMessage(), e);
+ return new JsonResponse(Response.Status.INTERNAL_SERVER_ERROR,
e.getMessage()).build();
+ }
+ }
+
+ @GET
+ @Path("config")
+ public Response getAllPackageConfigs() {
+ try {
+ Map<String, Map<String, Object>> config = helium.getAllPackageConfig();
+ return new JsonResponse(Response.Status.OK, config).build();
+ } catch (RuntimeException e) {
+ logger.error(e.getMessage(), e);
+ return new JsonResponse(Response.Status.INTERNAL_SERVER_ERROR,
e.getMessage()).build();
+ }
+ }
+
+ @GET
+ @Path("config/{packageName}/{artifact}")
+ public Response getPackageConfig(@PathParam("packageName") String
packageName,
+ @PathParam("artifact") String artifact) {
+ if (StringUtils.isEmpty(packageName) || StringUtils.isEmpty(artifact)) {
+ return new JsonResponse(Response.Status.BAD_REQUEST,
+ "package name or artifact is empty"
+ ).build();
+ }
+
+ try {
+ Map<String, Map<String, Object>> config =
+ helium.getPackageConfig(packageName, artifact);
+
+ if (config == null) {
+ return new JsonResponse(Response.Status.BAD_REQUEST,
+ "Failed to find package for " + artifact).build();
+ }
+
+ return new JsonResponse(Response.Status.OK, config).build();
+ } catch (RuntimeException e) {
+ logger.error(e.getMessage(), e);
+ return new JsonResponse(Response.Status.INTERNAL_SERVER_ERROR,
e.getMessage()).build();
+ }
+ }
+
+ @POST
+ @Path("config/{packageName}/{artifact}")
+ public Response updatePackageConfig(@PathParam("packageName") String
packageName,
+ @PathParam("artifact") String artifact,
+ String rawConfig) {
+
+ if (StringUtils.isEmpty(packageName) || StringUtils.isEmpty(artifact)) {
+ return new JsonResponse(Response.Status.BAD_REQUEST,
+ "package name or artifact is empty"
+ ).build();
+ }
+
+ Map<String, Object> packageConfig = null;
+
+ try {
+ packageConfig = gson.fromJson(
+ rawConfig, new TypeToken<Map<String, Object>>(){}.getType());
+ helium.updatePackageConfig(artifact, packageConfig);
+ } catch (JsonParseException e) {
+ logger.error(e.getMessage(), e);
+ return new JsonResponse(Response.Status.BAD_REQUEST,
+ e.getMessage()).build();
+ } catch (IOException | RuntimeException e) {
+ return new JsonResponse(Response.Status.INTERNAL_SERVER_ERROR,
+ e.getMessage()).build();
+ }
+
+ return new JsonResponse(Response.Status.OK, packageConfig).build();
+ }
+
@POST
@Path("order/visualization")
public Response getVisualizationPackageOrder(String orderedPackageNameList) {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
----------------------------------------------------------------------
diff --git
a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
index 6036ce4..768a154 100644
---
a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
+++
b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
@@ -126,7 +126,7 @@ public class ZeppelinServer extends Application {
new File(conf.getRelativeDir("zeppelin-web/src/app/spell")));
}
- this.helium = new Helium(
+ ZeppelinServer.helium = new Helium(
conf.getHeliumConfPath(),
conf.getHeliumRegistry(),
new File(conf.getRelativeDir(ConfVars.ZEPPELIN_DEP_LOCALREPO),
@@ -177,7 +177,7 @@ public class ZeppelinServer extends Application {
// Web UI
final WebAppContext webApp = setupWebAppContext(contexts, conf);
- // REST api
+ // Create `ZeppelinServer` using reflection and setup REST Api
setupRestApiContextHandler(webApp, conf);
// Notebook server
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
----------------------------------------------------------------------
diff --git
a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
index 6b4c12d..ee88375 100644
---
a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
+++
b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
@@ -2281,7 +2281,6 @@ public class NotebookServer extends WebSocketServlet
resp.put("editor", notebook().getInterpreterSettingManager().
getEditorSetting(interpreter, user, noteId, replName));
conn.send(serializeMessage(resp));
- return;
}
private void getInterpreterSettings(NotebookSocket conn, AuthenticationInfo
subject)
@@ -2324,11 +2323,31 @@ public class NotebookServer extends WebSocketServlet
.equals(WatcherSecurityKey.getKey()));
}
+ /**
+ * Send websocket message to all connections regardless of notebook id
+ */
+ private void broadcastToAllConnections(String serialized) {
+ broadcastToAllConnectionsExcept(null, serialized);
+ }
+
+ private void broadcastToAllConnectionsExcept(NotebookSocket exclude, String
serialized) {
+ synchronized (connectedSockets) {
+ for (NotebookSocket conn: connectedSockets) {
+ if (exclude != null && exclude.equals(conn)) {
+ continue;
+ }
+
+ try {
+ conn.send(serialized);
+ } catch (IOException e) {
+ LOG.error("Cannot broadcast message to watcher", e);
+ }
+ }
+ }
+ }
+
private void broadcastToWatchers(String noteId, String subject, Message
message) {
synchronized (watcherSockets) {
- if (watcherSockets.isEmpty()) {
- return;
- }
for (NotebookSocket watcher : watcherSockets) {
try {
watcher.send(
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/helium/helium.config.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/helium/helium.config.js
b/zeppelin-web/src/app/helium/helium.config.js
new file mode 100644
index 0000000..e21fe19
--- /dev/null
+++ b/zeppelin-web/src/app/helium/helium.config.js
@@ -0,0 +1,100 @@
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+export const HeliumConfFieldType = {
+ NUMBER: 'number',
+ JSON: 'json',
+ STRING: 'string',
+};
+
+/**
+ * @param persisted <Object> including `type`, `description`, `defaultValue`
for each conf key
+ * @param spec <Object> including `value` for each conf key
+ */
+export function mergePersistedConfWithSpec(persisted, spec) {
+ const confs = [];
+
+ for(let name in spec) {
+ const specField = spec[name];
+ const persistedValue = persisted[name];
+
+ const value = (persistedValue) ? persistedValue : specField.defaultValue;
+ const merged = {
+ name: name, type: specField.type, description: specField.description,
+ value: value, defaultValue: specField.defaultValue,
+ };
+
+ confs.push(merged);
+ }
+
+ return confs;
+}
+
+export function createPackageConf(defaultPackages, persistedPackacgeConfs) {
+ let packageConfs = {};
+
+ for (let name in defaultPackages) {
+ const pkgInfo = defaultPackages[name];
+
+ const configSpec = pkgInfo.pkg.config;
+ if (!configSpec) { continue; }
+
+ const version = pkgInfo.pkg.version;
+ if (!version) { continue; }
+
+ let config = {};
+ if (persistedPackacgeConfs[name] && persistedPackacgeConfs[name][version])
{
+ config = persistedPackacgeConfs[name][version];
+ }
+
+ const confs = mergePersistedConfWithSpec(config, configSpec);
+ packageConfs[name] = confs;
+ }
+
+ return packageConfs;
+}
+
+export function parseConfigValue(type, stringified) {
+ let value = stringified;
+
+ try {
+ if (HeliumConfFieldType.NUMBER === type) {
+ value = parseFloat(stringified);
+ } else if (HeliumConfFieldType.JSON === type) {
+ value = JSON.parse(stringified);
+ }
+ } catch(error) {
+ // return just the stringified one
+ console.error(`Failed to parse conf type ${type}, value ${value}`);
+ }
+
+ return value;
+}
+
+/**
+ * create persistable config object
+ */
+export function createPersistableConfig(currentConf) {
+ // persist key-value only
+ // since other info (e.g type, desc) can be provided by default config
+ const filtered = currentConf.reduce((acc, c) => {
+ let value = parseConfigValue(c.type, c.value);
+ acc[c.name] = value;
+ return acc;
+ }, {});
+
+ return filtered;
+}
+
+
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/helium/helium.controller.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/helium/helium.controller.js
b/zeppelin-web/src/app/helium/helium.controller.js
index fafa4ec..5a19ea0 100644
--- a/zeppelin-web/src/app/helium/helium.controller.js
+++ b/zeppelin-web/src/app/helium/helium.controller.js
@@ -14,61 +14,36 @@
import { HeliumType, } from '../../components/helium/helium-type';
-angular.module('zeppelinWebApp').controller('HeliumCtrl', HeliumCtrl);
-
-function HeliumCtrl($scope, $rootScope, $sce, baseUrlSrv, ngToast,
heliumService) {
+export default function HeliumCtrl($scope, $rootScope, $sce,
+ baseUrlSrv, ngToast, heliumService) {
'ngInject';
- $scope.packageInfos = {};
- $scope.defaultVersions = {};
+ $scope.pkgSearchResults = {};
+ $scope.defaultPackages = {};
$scope.showVersions = {};
$scope.bundleOrder = [];
$scope.bundleOrderChanged = false;
+ $scope.defaultPackageConfigs = {}; // { pkgName, [{name, type, desc, value,
defaultValue}] }
+
+ function init() {
+ // get all package info and set config
+ heliumService.getAllPackageInfoAndDefaultPackages()
+ .then(({ pkgSearchResults, defaultPackages }) => {
+ $scope.pkgSearchResults = pkgSearchResults;
+ $scope.defaultPackages = defaultPackages;
+ return heliumService.getAllPackageConfigs()
+ })
+ .then(defaultPackageConfigs => {
+ $scope.defaultPackageConfigs = defaultPackageConfigs;
+ });
- var buildDefaultVersionListToDisplay = function(packageInfos) {
- var defaultVersions = {};
- // show enabled version if any version of package is enabled
- for (var name in packageInfos) {
- var pkgs = packageInfos[name];
- for (var pkgIdx in pkgs) {
- var pkg = pkgs[pkgIdx];
- pkg.pkg.icon = $sce.trustAsHtml(pkg.pkg.icon);
- if (pkg.enabled) {
- defaultVersions[name] = pkg;
- pkgs.splice(pkgIdx, 1);
- break;
- }
- }
-
- // show first available version if package is not enabled
- if (!defaultVersions[name]) {
- defaultVersions[name] = pkgs[0];
- pkgs.splice(0, 1);
- }
- }
- $scope.defaultVersions = defaultVersions;
- };
-
- var getAllPackageInfo = function() {
- heliumService.getAllPackageInfo().
- success(function(data, status) {
- $scope.packageInfos = data.body;
- buildDefaultVersionListToDisplay($scope.packageInfos);
- }).
- error(function(data, status) {
- console.log('Can not load package info %o %o', status, data);
- });
- };
-
- var getBundleOrder = function() {
- heliumService.getVisualizationPackageOrder().
- success(function(data, status) {
- $scope.bundleOrder = data.body;
- }).
- error(function(data, status) {
- console.log('Can not get bundle order %o %o', status, data);
- });
- };
+ // 2. get vis package order
+ heliumService.getVisualizationPackageOrder()
+ .then(visPackageOrder => {
+ $scope.bundleOrder = visPackageOrder;
+ $scope.bundleOrderChanged = false;
+ });
+ }
$scope.bundleOrderListeners = {
accept: function(sourceItemHandleScope, destSortableScope) {return true;},
@@ -78,14 +53,6 @@ function HeliumCtrl($scope, $rootScope, $sce, baseUrlSrv,
ngToast, heliumService
}
};
- var init = function() {
- getAllPackageInfo();
- getBundleOrder();
- $scope.bundleOrderChanged = false;
- };
-
- init();
-
$scope.saveBundleOrder = function() {
var confirm = BootstrapDialog.confirm({
closable: false,
@@ -115,24 +82,24 @@ function HeliumCtrl($scope, $rootScope, $sce, baseUrlSrv,
ngToast, heliumService
}
}
});
- }
+ };
var getLicense = function(name, artifact) {
- var pkg = _.filter($scope.defaultVersions[name], function(p) {
+ var filteredPkgSearchResults = _.filter($scope.defaultPackages[name],
function(p) {
return p.artifact === artifact;
});
var license;
- if (pkg.length === 0) {
- pkg = _.filter($scope.packageInfos[name], function(p) {
+ if (filteredPkgSearchResults.length === 0) {
+ filteredPkgSearchResults = _.filter($scope.pkgSearchResults[name],
function(p) {
return p.pkg.artifact === artifact;
});
- if (pkg.length > 0) {
- license = pkg[0].pkg.license;
+ if (filteredPkgSearchResults.length > 0) {
+ license = filteredPkgSearchResults[0].pkg.license;
}
} else {
- license = pkg[0].license;
+ license = filteredPkgSearchResults[0].license;
}
if (!license) {
@@ -226,4 +193,48 @@ function HeliumCtrl($scope, $rootScope, $sce, baseUrlSrv,
ngToast, heliumService
return (pkg.type === HeliumType.SPELL || pkg.type ===
HeliumType.VISUALIZATION) &&
!$scope.isLocalPackage(pkgSearchResult);
};
+
+ $scope.configExists = function(pkgSearchResult) {
+ // helium package config is persisted per version
+ return pkgSearchResult.pkg.config && pkgSearchResult.pkg.artifact;
+ };
+
+ $scope.configOpened = function(pkgSearchResult) {
+ return pkgSearchResult.configOpened && !pkgSearchResult.configFetching;
+ };
+
+ $scope.getConfigButtonClass = function(pkgSearchResult) {
+ return (pkgSearchResult.configOpened && pkgSearchResult.configFetching) ?
+ 'disabled' : '';
+ }
+
+ $scope.toggleConfigButton = function(pkgSearchResult) {
+ if (pkgSearchResult.configOpened) {
+ pkgSearchResult.configOpened = false;
+ return;
+ }
+
+ const pkg = pkgSearchResult.pkg;
+ const pkgName = pkg.name;
+ pkgSearchResult.configFetching = true;
+ pkgSearchResult.configOpened = true;
+
+ heliumService.getSinglePackageConfigs(pkg)
+ .then(confs => {
+ $scope.defaultPackageConfigs[pkgName] = confs;
+ pkgSearchResult.configFetching = false;
+ });
+ };
+
+ $scope.saveConfig = function(pkgSearchResult) {
+ const pkgName = pkgSearchResult.pkg.name;
+ const currentConf = $scope.defaultPackageConfigs[pkgName];
+
+ heliumService.saveConfig(pkgSearchResult.pkg, currentConf, () => {
+ // close after config is saved
+ pkgSearchResult.configOpened = false;
+ });
+ };
+
+ init();
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/helium/helium.css
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/helium/helium.css
b/zeppelin-web/src/app/helium/helium.css
index c8643a4..23e8c19 100644
--- a/zeppelin-web/src/app/helium/helium.css
+++ b/zeppelin-web/src/app/helium/helium.css
@@ -79,6 +79,10 @@
width: 500px;
}
+.spellConfigButton {
+ background-color: #FEFEFE;
+}
+
.heliumPackageList .heliumPackageDisabledArtifact {
color:gray;
}
@@ -132,4 +136,21 @@
color: #636363;
}
+.heliumConfig {
+ margin-top: 30px;
+ margin-bottom: 10px;
+}
+.heliumConfigTable {
+ margin-top: 15px;
+ vertical-align:middle;
+ margin-bottom: 15px;
+}
+
+.heliumConfigValueInput {
+
+}
+
+.heliumConfigValueText {
+ vertical-align: top;
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/helium/helium.html
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/helium/helium.html
b/zeppelin-web/src/app/helium/helium.html
index f67797c..7718666 100644
--- a/zeppelin-web/src/app/helium/helium.html
+++ b/zeppelin-web/src/app/helium/helium.html
@@ -29,7 +29,7 @@ limitations under the License.
<div class="btn-group" data-ng-repeat="pkgName in bundleOrder"
as-sortable-item>
<div class="btn btn-default btn-sm"
- ng-bind-html='defaultVersions[pkgName].pkg.icon'
+ ng-bind-html='defaultPackages[pkgName].pkg.icon'
as-sortable-item-handle>
</div>
</div>
@@ -45,32 +45,38 @@ limitations under the License.
<div class="box width-full heliumPackageContainer">
<div class="row heliumPackageList"
- ng-repeat="(pkgName, pkgInfo) in defaultVersions">
+ ng-repeat="(pkgName, pkgSearchResult) in defaultPackages">
+
<div class="col-md-12">
<div class="heliumPackageHead">
<div class="heliumPackageIcon"
- ng-bind-html=pkgInfo.pkg.icon></div>
+ ng-bind-html=pkgSearchResult.pkg.icon></div>
<div class="heliumPackageName">
- <span ng-if="hasNpmLink(pkgInfo)">
+ <span ng-if="hasNpmLink(pkgSearchResult)">
<a target="_blank"
href="https://www.npmjs.com/package/{{pkgName}}";>{{pkgName}}</a>
</span>
- <span ng-if="!hasNpmLink(pkgInfo)" ng-class="{'heliumLocalPackage':
isLocalPackage(pkgInfo)}">
+ <span ng-if="!hasNpmLink(pkgSearchResult)"
ng-class="{'heliumLocalPackage': isLocalPackage(pkgSearchResult)}">
{{pkgName}}
</span>
- <span class="heliumType">{{pkgInfo.pkg.type}}</span>
+ <span class="heliumType">{{pkgSearchResult.pkg.type}}</span>
</div>
- <div ng-show="!pkgInfo.enabled"
- ng-click="enable(pkgName, pkgInfo.pkg.artifact)"
+ <div ng-show="!pkgSearchResult.enabled"
+ ng-click="enable(pkgName, pkgSearchResult.pkg.artifact)"
class="btn btn-success btn-xs"
style="float:right">Enable</div>
- <div ng-show="pkgInfo.enabled"
+ <div ng-show="pkgSearchResult.enabled"
ng-click="disable(pkgName)"
class="btn btn-info btn-xs"
style="float:right">Disable</div>
+ <div ng-show="configExists(pkgSearchResult)"
+ ng-click="toggleConfigButton(pkgSearchResult)"
+ ng-class="getConfigButtonClass(pkgSearchResult)"
+ class="btn btn-default btn-xs spellConfigButton"
+ style="float:right; margin-right:5px;">Config</div>
</div>
- <div ng-class="{heliumPackageDisabledArtifact: !pkgInfo.enabled,
heliumPackageEnabledArtifact: pkgInfo.enabled}">
- {{pkgInfo.pkg.artifact}}
- <span ng-show="packageInfos[pkgName].length > 0"
+ <div ng-class="{heliumPackageDisabledArtifact: !pkgSearchResult.enabled,
heliumPackageEnabledArtifact: pkgSearchResult.enabled}">
+ {{pkgSearchResult.pkg.artifact}}
+ <span ng-show="pkgSearchResults[pkgName].length > 0"
ng-click="toggleVersions(pkgName)">
versions
</span>
@@ -78,28 +84,64 @@ limitations under the License.
<ul class="heliumPackageVersions"
ng-show="showVersions[pkgName]">
<li class="heliumPackageDisabledArtifact"
- ng-repeat="pkg in packageInfos[pkgName]">
- {{pkg.pkg.artifact}} -
- <span ng-click="enable(pkgName, pkg.pkg.artifact)"
+ ng-repeat="pkgSearchResult in pkgSearchResults[pkgName]">
+ {{pkgSearchResult.pkg.artifact}} -
+ <span ng-click="enable(pkgName, pkgSearchResult.pkg.artifact)"
style="margin-left:3px;cursor:pointer;text-decoration:
underline;color:#3071a9">
enable
</span>
</li>
</ul>
<div class="heliumPackageDescription">
- {{pkgInfo.pkg.description}}
+ {{pkgSearchResult.pkg.description}}
</div>
- <div ng-if="pkgInfo.pkg.type === 'SPELL' && pkgInfo.pkg.spell"
+ <div ng-if="pkgSearchResult.pkg.type === 'SPELL' &&
pkgSearchResult.pkg.spell"
class="spellInfo">
<div>
<span class="spellInfoDesc">MAGIC</span>
- <span class="spellInfoValue">{{pkgInfo.pkg.spell.magic}} </span>
+ <span class="spellInfoValue">{{pkgSearchResult.pkg.spell.magic}}
</span>
</div>
<div>
<span class="spellInfoDesc">USAGE</span>
- <pre class="spellUsage">{{pkgInfo.pkg.spell.usage}} </pre>
+ <pre class="spellUsage">{{pkgSearchResult.pkg.spell.usage}} </pre>
+ </div>
+ </div>
+
+ <!--start: config-->
+ <div class="heliumConfig" ng-if="configOpened(pkgSearchResult)">
+ <h5>Configuration</h5>
+ <table class="heliumConfigTable table table-striped">
+ <tr>
+ <th>Name</th>
+ <th>Type</th>
+ <th>Description</th>
+ <th>Value</th>
+ </tr>
+ <tr>
+ </tr>
+ <tr data-ng-repeat="cfg in
defaultPackageConfigs[pkgSearchResult.pkg.name]">
+ <td style="vertical-align: middle;">{{cfg.name}}</td>
+ <td style="vertical-align: middle;">{{cfg.type}}</td>
+ <td style="vertical-align: middle;">{{cfg.description}}</td>
+ <td>
+ <div class="input-group">
+ <input type="text" class="form-control" style="border-radius:
5px;"
+ data-ng-model="cfg.value"
placeholder="{{cfg.defaultValue}}" />
+ </div>
+ </td>
+ </tr>
+ </table>
+
+ <div>
+ <button class="btn btn-primary"
+ ng-click="saveConfig(pkgSearchResult)">Save</button>
+ <button class="btn btn-default"
+ ng-click="toggleConfigButton(pkgSearchResult)">Close</button>
</div>
</div>
+ <!--end: config-->
+
</div>
+
</div>
</div>
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/helium/index.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/helium/index.js
b/zeppelin-web/src/app/helium/index.js
new file mode 100644
index 0000000..632969e
--- /dev/null
+++ b/zeppelin-web/src/app/helium/index.js
@@ -0,0 +1,19 @@
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import HeliumController from './helium.controller';
+
+angular.module('zeppelinWebApp')
+ .controller('HeliumCtrl', HeliumController);
+
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
index 72fe0d7..358ea92 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
@@ -32,6 +32,12 @@ function ParagraphCtrl($scope, $rootScope, $route, $window,
$routeParams, $locat
$scope.originalText = '';
$scope.editor = null;
+ // transactional info for spell execution
+ $scope.spellTransaction = {
+ totalResultCount: 0, renderedResultCount: 0,
+ propagated: false, resultsMsg: [], paragraphText: '',
+ };
+
var editorSetting = {};
// flag that is used to set editor setting on paste percent sign
var pastePercentSign = false;
@@ -227,7 +233,6 @@ function ParagraphCtrl($scope, $rootScope, $route, $window,
$routeParams, $locat
$scope.paragraph.status = 'ERROR';
$scope.paragraph.errorMessage = errorMessage;
console.error('Failed to execute interpret() in spell\n', error);
- if (digestRequired) { $scope.$digest(); }
if (!propagated) {
$scope.propagateSpellResult(
@@ -237,9 +242,49 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
}
};
- $scope.runParagraphUsingSpell = function(spell, paragraphText,
+ $scope.prepareSpellTransaction = function(resultsMsg, propagated,
paragraphText) {
+ $scope.spellTransaction.totalResultCount = resultsMsg.length;
+ $scope.spellTransaction.renderedResultCount = 0;
+ $scope.spellTransaction.propagated = propagated;
+ $scope.spellTransaction.resultsMsg = resultsMsg;
+ $scope.spellTransaction.paragraphText = paragraphText;
+ };
+
+ /**
+ * - update spell transaction count and
+ * - check transaction is finished based on the result count
+ * @returns {boolean}
+ */
+ $scope.increaseSpellTransactionResultCount = function() {
+ $scope.spellTransaction.renderedResultCount += 1;
+
+ const total = $scope.spellTransaction.totalResultCount;
+ const current = $scope.spellTransaction.renderedResultCount;
+ return total === current;
+ };
+
+ $scope.cleanupSpellTransaction = function() {
+ const status = 'FINISHED';
+ $scope.paragraph.status = status;
+ $scope.paragraph.results.code = status;
+
+ const propagated = $scope.spellTransaction.propagated;
+ const resultsMsg = $scope.spellTransaction.resultsMsg;
+ const paragraphText = $scope.spellTransaction.paragraphText;
+
+ if (!propagated) {
+ const propagable = SpellResult.createPropagable(resultsMsg);
+ $scope.propagateSpellResult(
+ $scope.paragraph.id, $scope.paragraph.title,
+ paragraphText, propagable, status, '',
+ $scope.paragraph.config, $scope.paragraph.settings.params);
+ }
+ };
+
+ $scope.runParagraphUsingSpell = function(paragraphText,
magic, digestRequired, propagated) {
$scope.paragraph.results = {};
+ $scope.paragraph.status = 'PENDING';
$scope.paragraph.errorMessage = '';
if (digestRequired) { $scope.$digest(); }
@@ -248,30 +293,20 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
const splited = paragraphText.split(magic);
// remove leading spaces
const textWithoutMagic = splited[1].replace(/^\s+/g, '');
- const spellResult = spell.interpret(textWithoutMagic);
- const parsed = spellResult.getAllParsedDataWithTypes(
- heliumService.getAllSpells(), magic, textWithoutMagic);
// handle actual result message in promise
- parsed.then(resultsMsg => {
- const status = 'FINISHED';
- $scope.paragraph.status = status;
- $scope.paragraph.results.code = status;
- $scope.paragraph.results.msg = resultsMsg;
- $scope.paragraph.config.tableHide = false;
- if (digestRequired) { $scope.$digest(); }
-
- if (!propagated) {
- const propagable = SpellResult.createPropagable(resultsMsg);
- $scope.propagateSpellResult(
- $scope.paragraph.id, $scope.paragraph.title,
- paragraphText, propagable, status, '',
- $scope.paragraph.config, $scope.paragraph.settings.params);
- }
- }).catch(error => {
- $scope.handleSpellError(paragraphText, error,
- digestRequired, propagated);
- });
+ heliumService.executeSpell(magic, textWithoutMagic)
+ .then(resultsMsg => {
+ $scope.prepareSpellTransaction(resultsMsg, propagated,
paragraphText);
+
+ $scope.paragraph.results.msg = resultsMsg;
+ $scope.paragraph.config.tableHide = false;
+
+ })
+ .catch(error => {
+ $scope.handleSpellError(paragraphText, error,
+ digestRequired, propagated);
+ });
} catch (error) {
$scope.handleSpellError(paragraphText, error,
digestRequired, propagated);
@@ -309,11 +344,9 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
}
const magic = SpellResult.extractMagic(paragraphText);
- const spell = heliumService.getSpellByMagic(magic);
- if (spell) {
- $scope.runParagraphUsingSpell(
- spell, paragraphText, magic, digestRequired, propagated);
+ if (heliumService.getSpellByMagic(magic)) {
+ $scope.runParagraphUsingSpell(paragraphText, magic, digestRequired,
propagated);
} else {
$scope.runParagraphUsingBackendInterpreter(paragraphText);
}
@@ -1157,6 +1190,8 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
}
};
+ /** $scope.$on */
+
$scope.$on('runParagraphUsingSpell', function(event, data) {
const oldPara = $scope.paragraph;
let newPara = data.paragraph;
@@ -1341,4 +1376,17 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
$scope.$on('closeTable', function(event) {
$scope.closeTable($scope.paragraph);
});
+
+ $scope.$on('resultRendered', function(event, paragraphId) {
+ if ($scope.paragraph.id !== paragraphId) {
+ return;
+ }
+
+ /** increase spell result count and return if not finished */
+ if (!$scope.increaseSpellTransactionResultCount()) {
+ return;
+ }
+
+ $scope.cleanupSpellTransaction();
+ });
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
----------------------------------------------------------------------
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index f308070..374f0d8 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -281,6 +281,10 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
} else {
console.error(`Unknown Display Type: ${type}`);
}
+
+ // send message to parent that this result is rendered
+ const paragraphId = $scope.$parent.paragraph.id;
+ $scope.$emit('resultRendered', paragraphId);
};
const renderResult = function(type, refresh) {
@@ -296,12 +300,7 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
renderApp(`p${appState.id}`, appState);
} else {
if (!DefaultDisplayType[type]) {
- const spell = heliumService.getSpellByMagic(type);
- if (!spell) {
- console.error(`Can't execute spell due to unknown display type:
${type}`);
- return;
- }
- $scope.renderCustomDisplay(type, data, spell);
+ $scope.renderCustomDisplay(type, data);
} else {
const targetElemId = $scope.createDisplayDOMId(`p${$scope.id}`, type);
$scope.renderDefaultDisplay(targetElemId, type, data, refresh);
@@ -316,38 +315,40 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
/**
* Render multiple sub results for custom display
*/
- $scope.renderCustomDisplay = function(type, data, spell) {
+ $scope.renderCustomDisplay = function(type, data) {
// get result from intp
-
- const spellResult = spell.interpret(data.trim());
- const parsed = spellResult.getAllParsedDataWithTypes(
- heliumService.getAllSpells());
+ if (!heliumService.getSpellByMagic(type)) {
+ console.error(`Can't execute spell due to unknown display type:
${type}`);
+ return;
+ }
// custom display result can include multiple subset results
- parsed.then(dataWithTypes => {
- const containerDOMId = `p${$scope.id}_custom`;
- const afterLoaded = () => {
- const containerDOM = angular.element(`#${containerDOMId}`);
- // Spell.interpret() can create multiple outputs
- for(let i = 0; i < dataWithTypes.length; i++) {
- const dt = dataWithTypes[i];
- const data = dt.data;
- const type = dt.type;
-
- // prepare each DOM to be filled
- const subResultDOMId =
$scope.createDisplayDOMId(`p${$scope.id}_custom_${i}`, type);
- const subResultDOM = document.createElement('div');
- containerDOM.append(subResultDOM);
- subResultDOM.setAttribute('id', subResultDOMId);
-
- $scope.renderDefaultDisplay(subResultDOMId, type, data, true);
- }
- };
+ heliumService.executeSpellAsDisplaySystem(type, data)
+ .then(dataWithTypes => {
+ const containerDOMId = `p${$scope.id}_custom`;
+ const afterLoaded = () => {
+ const containerDOM = angular.element(`#${containerDOMId}`);
+ // Spell.interpret() can create multiple outputs
+ for(let i = 0; i < dataWithTypes.length; i++) {
+ const dt = dataWithTypes[i];
+ const data = dt.data;
+ const type = dt.type;
+
+ // prepare each DOM to be filled
+ const subResultDOMId =
$scope.createDisplayDOMId(`p${$scope.id}_custom_${i}`, type);
+ const subResultDOM = document.createElement('div');
+ containerDOM.append(subResultDOM);
+ subResultDOM.setAttribute('id', subResultDOMId);
+
+ $scope.renderDefaultDisplay(subResultDOMId, type, data, true);
+ }
+ };
- retryUntilElemIsLoaded(containerDOMId, afterLoaded);
- }).catch(error => {
- console.error(`Failed to render custom display: ${$scope.type}\n` +
error);
- });
+ retryUntilElemIsLoaded(containerDOMId, afterLoaded);
+ })
+ .catch(error => {
+ console.error(`Failed to render custom display: ${$scope.type}\n` +
error);
+ });
};
/**
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/app/spell/spell-base.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/app/spell/spell-base.js
b/zeppelin-web/src/app/spell/spell-base.js
index 85c85e5..6dcb576 100644
--- a/zeppelin-web/src/app/spell/spell-base.js
+++ b/zeppelin-web/src/app/spell/spell-base.js
@@ -31,9 +31,10 @@ export class SpellBase {
* Consumes text and return `SpellResult`.
*
* @param paragraphText {string} which doesn't include magic
+ * @param config {Object}
* @return {SpellResult}
*/
- interpret(paragraphText) {
+ interpret(paragraphText, config) {
throw new Error('SpellBase.interpret() should be overrided');
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/components/helium/helium-conf.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/components/helium/helium-conf.js
b/zeppelin-web/src/components/helium/helium-conf.js
new file mode 100644
index 0000000..a93f6f0
--- /dev/null
+++ b/zeppelin-web/src/components/helium/helium-conf.js
@@ -0,0 +1,96 @@
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+export const HeliumConfFieldType = {
+ NUMBER: 'number',
+ JSON: 'json',
+ STRING: 'string',
+};
+
+/**
+ * @param persisted <Object> including `type`, `description`, `defaultValue`
for each conf key
+ * @param spec <Object> including `value` for each conf key
+ */
+export function mergePersistedConfWithSpec(persisted, spec) {
+ const confs = [];
+
+ for(let name in spec) {
+ const specField = spec[name];
+ const persistedValue = persisted[name];
+
+ const value = (persistedValue) ? persistedValue : specField.defaultValue;
+ const merged = {
+ name: name, type: specField.type, description: specField.description,
+ value: value, defaultValue: specField.defaultValue,
+ };
+
+ confs.push(merged);
+ }
+
+ return confs;
+}
+
+export function createAllPackageConfigs(defaultPackages, persistedConfs) {
+ let packageConfs = {};
+
+ for (let name in defaultPackages) {
+ const pkgSearchResult = defaultPackages[name];
+
+ const spec = pkgSearchResult.pkg.config;
+ if (!spec) { continue; }
+
+ const artifact = pkgSearchResult.pkg.artifact;
+ if (!artifact) { continue; }
+
+ let persistedConf = {};
+ if (persistedConfs[artifact]) {
+ persistedConf = persistedConfs[artifact];
+ }
+
+ const confs = mergePersistedConfWithSpec(persistedConf, spec);
+ packageConfs[name] = confs;
+ }
+
+ return packageConfs;
+}
+
+export function parseConfigValue(type, stringified) {
+ let value = stringified;
+
+ try {
+ if (HeliumConfFieldType.NUMBER === type) {
+ value = parseFloat(stringified);
+ } else if (HeliumConfFieldType.JSON === type) {
+ value = JSON.parse(stringified);
+ }
+ } catch(error) {
+ // return just the stringified one
+ console.error(`Failed to parse conf type ${type}, value ${value}`);
+ }
+
+ return value;
+}
+
+/**
+ * persist key-value only
+ * since other info (e.g type, desc) can be provided by default config
+ */
+export function createPersistableConfig(currentConfs) {
+ const filtered = currentConfs.reduce((acc, c) => {
+ acc[c.name] = parseConfigValue(c.type, c.value);
+ return acc;
+ }, {});
+
+ return filtered;
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/components/helium/helium-package.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/components/helium/helium-package.js
b/zeppelin-web/src/components/helium/helium-package.js
new file mode 100644
index 0000000..8192a6a
--- /dev/null
+++ b/zeppelin-web/src/components/helium/helium-package.js
@@ -0,0 +1,47 @@
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+export function createDefaultPackage(pkgSearchResult, sce) {
+ for (let pkgIdx in pkgSearchResult) {
+ const pkg = pkgSearchResult[pkgIdx];
+ pkg.pkg.icon = sce.trustAsHtml(pkg.pkg.icon);
+ if (pkg.enabled) {
+ pkgSearchResult.splice(pkgIdx, 1);
+ return pkg;
+ }
+ }
+
+ // show first available version if package is not enabled
+ const result = pkgSearchResult[0];
+ pkgSearchResult.splice(0, 1);
+ return result;
+}
+
+/**
+ * create default packages based on `enabled` field and `latest` version.
+ *
+ * @param pkgSearchResults
+ * @param sce angular `$sce` object
+ * @returns {Object} including {name, pkgInfo}
+ */
+export function createDefaultPackages(pkgSearchResults, sce) {
+ const defaultPackages = {};
+ // show enabled version if any version of package is enabled
+ for (let name in pkgSearchResults) {
+ const pkgSearchResult = pkgSearchResults[name];
+ defaultPackages[name] = createDefaultPackage(pkgSearchResult, sce)
+ }
+
+ return defaultPackages;
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/components/helium/helium.service.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/components/helium/helium.service.js
b/zeppelin-web/src/components/helium/helium.service.js
index ee7bddc..8b3e6a3 100644
--- a/zeppelin-web/src/components/helium/helium.service.js
+++ b/zeppelin-web/src/components/helium/helium.service.js
@@ -13,21 +13,32 @@
*/
import { HeliumType, } from './helium-type';
+import {
+ createAllPackageConfigs,
+ createPersistableConfig,
+ mergePersistedConfWithSpec,
+} from './helium-conf';
+import {
+ createDefaultPackages,
+} from './helium-package';
angular.module('zeppelinWebApp').service('heliumService', heliumService);
-function heliumService($http, baseUrlSrv, ngToast) {
+export default function heliumService($http, $sce, baseUrlSrv) {
'ngInject';
var url = baseUrlSrv.getRestApiBase() + '/helium/bundle/load';
if (process.env.HELIUM_BUNDLE_DEV) {
url = url + '?refresh=true';
}
+
+ let visualizationBundles = [];
// name `heliumBundles` should be same as
`HelumBundleFactory.HELIUM_BUNDLES_VAR`
- var heliumBundles = [];
+ let heliumBundles = [];
// map for `{ magic: interpreter }`
let spellPerMagic = {};
- let visualizationBundles = [];
+ // map for `{ magic: package-name }`
+ let pkgNamePerMagic = {}
// load should be promise
this.load = $http.get(url).success(function(response) {
@@ -39,7 +50,9 @@ function heliumService($http, baseUrlSrv, ngToast) {
heliumBundles.map(b => {
if (b.type === HeliumType.SPELL) {
const spell = new b.class(); // eslint-disable-line new-cap
+ const pkgName = b.id;
spellPerMagic[spell.getMagic()] = spell;
+ pkgNamePerMagic[spell.getMagic()] = pkgName;
} else if (b.type === HeliumType.VISUALIZATION) {
visualizationBundles.push(b);
}
@@ -57,29 +70,54 @@ function heliumService($http, baseUrlSrv, ngToast) {
return spellPerMagic[magic];
};
- /**
- * @returns {Object} map for `{ magic : spell }`
- */
- this.getAllSpells = function() {
- return spellPerMagic;
+ this.executeSpell = function(magic, textWithoutMagic) {
+ const promisedConf = this.getSinglePackageConfigUsingMagic(magic)
+ .then(confs => createPersistableConfig(confs));
+
+ return promisedConf.then(conf => {
+ const spell = this.getSpellByMagic(magic);
+ const spellResult = spell.interpret(textWithoutMagic, conf);
+ const parsed = spellResult.getAllParsedDataWithTypes(
+ spellPerMagic, magic, textWithoutMagic);
+
+ return parsed;
+ });
+ };
+
+ this.executeSpellAsDisplaySystem = function(magic, textWithoutMagic) {
+ const promisedConf = this.getSinglePackageConfigUsingMagic(magic)
+ .then(confs => createPersistableConfig(confs));
+
+ return promisedConf.then(conf => {
+ const spell = this.getSpellByMagic(magic);
+ const spellResult = spell.interpret(textWithoutMagic.trim(), conf);
+ const parsed = spellResult.getAllParsedDataWithTypes(spellPerMagic);
+
+ return parsed;
+ });
};
this.getVisualizationBundles = function() {
return visualizationBundles;
};
+ /**
+ * @returns {Promise} which returns bundleOrder
+ */
this.getVisualizationPackageOrder = function() {
- return $http.get(baseUrlSrv.getRestApiBase() +
'/helium/order/visualization');
+ return $http.get(baseUrlSrv.getRestApiBase() +
'/helium/order/visualization')
+ .then(function(response, status) {
+ return response.data.body;
+ })
+ .catch(function(error) {
+ console.error('Can not get bundle order', error);
+ });
};
this.setVisualizationPackageOrder = function(list) {
return $http.post(baseUrlSrv.getRestApiBase() +
'/helium/order/visualization', list);
};
- this.getAllPackageInfo = function() {
- return $http.get(baseUrlSrv.getRestApiBase() + '/helium/all');
- };
-
this.enable = function(name, artifact) {
return $http.post(baseUrlSrv.getRestApiBase() + '/helium/enable/' + name,
artifact);
};
@@ -87,4 +125,120 @@ function heliumService($http, baseUrlSrv, ngToast) {
this.disable = function(name) {
return $http.post(baseUrlSrv.getRestApiBase() + '/helium/disable/' + name);
};
+
+ this.saveConfig = function(pkg , defaultPackageConfig,
closeConfigPanelCallback) {
+ // in case of local package, it will include `/`
+ const pkgArtifact = encodeURIComponent(pkg.artifact);
+ const pkgName = pkg.name;
+ const filtered = createPersistableConfig(defaultPackageConfig);
+
+ if (!pkgName || !pkgArtifact|| !filtered) {
+ console.error(
+ `Can't save config for helium package '${pkgArtifact}'`, filtered);
+ return;
+ }
+
+ const url =
`${baseUrlSrv.getRestApiBase()}/helium/config/${pkgName}/${pkgArtifact}`;
+ return $http.post(url, filtered)
+ .then(() => {
+ if (closeConfigPanelCallback) { closeConfigPanelCallback(); }
+ }).catch((error) => {
+ console.error(`Failed to save config for ${pkgArtifact}`, error);
+ });
+ };
+
+ /**
+ * @returns {Promise<Object>} which including {name, Array<package info for
artifact>}
+ */
+ this.getAllPackageInfo = function() {
+ return $http.get(`${baseUrlSrv.getRestApiBase()}/helium/package`)
+ .then(function(response, status) {
+ return response.data.body;
+ })
+ .catch(function(error) {
+ console.error('Failed to get all package infos', error);
+ });
+ };
+
+ this.getDefaultPackages = function() {
+ return this.getAllPackageInfo()
+ .then(pkgSearchResults => {
+ return createDefaultPackages(pkgSearchResults, $sce);
+ });
+ };
+
+ this.getAllPackageInfoAndDefaultPackages = function() {
+ return this.getAllPackageInfo()
+ .then(pkgSearchResults => {
+ return {
+ pkgSearchResults: pkgSearchResults,
+ defaultPackages: createDefaultPackages(pkgSearchResults, $sce),
+ };
+ });
+ };
+
+ /**
+ * get all package configs.
+ * @return { Promise<{name, Array<Object>}> }
+ */
+ this.getAllPackageConfigs = function() {
+ const promisedDefaultPackages = this.getDefaultPackages();
+ const promisedPersistedConfs =
+ $http.get(`${baseUrlSrv.getRestApiBase()}/helium/config`)
+ .then(function(response, status) {
+ return response.data.body;
+ });
+
+ return Promise.all([promisedDefaultPackages, promisedPersistedConfs])
+ .then(values => {
+ const defaultPackages = values[0];
+ const persistedConfs = values[1];
+
+ return createAllPackageConfigs(defaultPackages, persistedConfs);
+ })
+ .catch(function(error) {
+ console.error('Failed to get all package configs', error);
+ });
+ };
+
+ /**
+ * get the package config which is persisted in server.
+ * @return { Promise<Array<Object>> }
+ */
+ this.getSinglePackageConfigs = function(pkg) {
+ const pkgName = pkg.name;
+ // in case of local package, it will include `/`
+ const pkgArtifact = encodeURIComponent(pkg.artifact);
+
+ if (!pkgName || !pkgArtifact) {
+ console.error('Failed to fetch config for\n', pkg);
+ return Promise.resolve([]);
+ }
+
+ const confUrl =
`${baseUrlSrv.getRestApiBase()}/helium/config/${pkgName}/${pkgArtifact}`;
+ const promisedConf = $http.get(confUrl)
+ .then(function(response, status) {
+ return response.data.body;
+ });
+
+ return promisedConf.then(({confSpec, confPersisted}) => {
+ const merged = mergePersistedConfWithSpec(confPersisted, confSpec)
+ return merged;
+ });
+ };
+
+ this.getSinglePackageConfigUsingMagic = function(magic) {
+ const pkgName = pkgNamePerMagic[magic];
+
+ const confUrl =
`${baseUrlSrv.getRestApiBase()}/helium/spell/config/${pkgName}`;
+ const promisedConf = $http.get(confUrl)
+ .then(function(response, status) {
+ return response.data.body;
+ });
+
+ return promisedConf.then(({confSpec, confPersisted}) => {
+ const merged = mergePersistedConfWithSpec(confPersisted, confSpec)
+ return merged;
+ });
+ }
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
----------------------------------------------------------------------
diff --git
a/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
b/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
index 9c6e585..bcc0fd3 100644
--- a/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
+++ b/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
@@ -168,6 +168,8 @@ function websocketEvents($rootScope, $websocket, $location,
baseUrlSrv) {
$rootScope.$broadcast('setNoteRevisionResult', data);
} else if (op === 'PARAS_INFO') {
$rootScope.$broadcast('updateParaInfos', data);
+ } else {
+ console.error(`unknown websocket op: ${op}`);
}
});
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
----------------------------------------------------------------------
diff --git
a/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
b/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
index 8d6863d..0ac4d58 100644
--- a/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
+++ b/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
@@ -334,7 +334,7 @@ function websocketMsgSrv($rootScope, websocketEvents) {
getInterpreterSettings: function() {
websocketEvents.sendNewEvent({op: 'GET_INTERPRETER_SETTINGS'});
- }
+ },
};
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-web/src/index.js
----------------------------------------------------------------------
diff --git a/zeppelin-web/src/index.js b/zeppelin-web/src/index.js
index 191d2e3..820d2d6 100644
--- a/zeppelin-web/src/index.js
+++ b/zeppelin-web/src/index.js
@@ -18,7 +18,6 @@ import './app/home/home.controller.js';
import './app/handsontable/handsonHelper.js';
import './app/notebook/notebook.controller.js';
-/** start: global variable `zeppelin` related files */
import './app/tabledata/tabledata.js';
import './app/tabledata/transformation.js';
import './app/tabledata/pivot.js';
@@ -32,7 +31,6 @@ import
'./app/visualization/builtins/visualization-piechart.js';
import './app/visualization/builtins/visualization-areachart.js';
import './app/visualization/builtins/visualization-linechart.js';
import './app/visualization/builtins/visualization-scatterchart.js';
-/** end: global variable `zeppelin` related files */
import './app/jobmanager/jobmanager.controller.js';
import './app/jobmanager/jobs/job.controller.js';
@@ -45,7 +43,7 @@ import './app/notebook/paragraph/paragraph.controller.js';
import './app/notebook/paragraph/result/result.controller.js';
import './app/search/result-list.controller.js';
import './app/notebookRepos/notebookRepos.controller.js';
-import './app/helium/helium.controller.js';
+import './app/helium';
import './components/arrayOrderingSrv/arrayOrdering.service.js';
import './components/clipboard/clipboard.controller.js';
import './components/navbar/navbar.controller.js';
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
----------------------------------------------------------------------
diff --git
a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
index 918e9aa..b4d5a79 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
@@ -19,6 +19,7 @@ package org.apache.zeppelin.helium;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.apache.commons.io.FileUtils;
+import org.apache.commons.lang3.StringUtils;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.resource.DistributedResourcePool;
@@ -144,7 +145,7 @@ public class Helium {
}
private void clearNotExistsPackages() {
- Map<String, List<HeliumPackageSearchResult>> all =
getAllPackageInfo(false);
+ Map<String, List<HeliumPackageSearchResult>> all =
getAllPackageInfoWithoutRefresh();
// clear visualization display order
List<String> packageOrder = heliumConf.getBundleDisplayOrder();
@@ -165,11 +166,20 @@ public class Helium {
}
}
+ public Map<String, List<HeliumPackageSearchResult>>
getAllPackageInfoWithoutRefresh() {
+ return getAllPackageInfo(false, null);
+ }
+
public Map<String, List<HeliumPackageSearchResult>> getAllPackageInfo() {
- return getAllPackageInfo(true);
+ return getAllPackageInfo(true, null);
}
- public Map<String, List<HeliumPackageSearchResult>>
getAllPackageInfo(boolean refresh) {
+ /**
+ * @param refresh
+ * @param packageName
+ */
+ public Map<String, List<HeliumPackageSearchResult>>
getAllPackageInfo(boolean refresh,
+ String
packageName) {
Map<String, String> enabledPackageInfo = heliumConf.getEnabledPackages();
synchronized (registry) {
@@ -179,6 +189,12 @@ public class Helium {
try {
for (HeliumPackage pkg : r.getAll()) {
String name = pkg.getName();
+
+ if (!StringUtils.isEmpty(packageName) &&
+ !name.equals(packageName)) {
+ continue;
+ }
+
String artifact = enabledPackageInfo.get(name);
boolean enabled = (artifact != null &&
artifact.equals(pkg.getArtifact()));
@@ -192,8 +208,12 @@ public class Helium {
}
}
} else {
-
for (String name : allPackages.keySet()) {
+ if (!StringUtils.isEmpty(packageName) &&
+ !name.equals(packageName)) {
+ continue;
+ }
+
List<HeliumPackageSearchResult> pkgs = allPackages.get(name);
String artifact = enabledPackageInfo.get(name);
LinkedList<HeliumPackageSearchResult> newResults =
@@ -222,11 +242,34 @@ public class Helium {
}
}
- public HeliumPackageSearchResult getPackageInfo(String name, String
artifact) {
- Map<String, List<HeliumPackageSearchResult>> infos =
getAllPackageInfo(false);
- List<HeliumPackageSearchResult> packages = infos.get(name);
+ public List<HeliumPackageSearchResult> getSinglePackageInfo(String
packageName) {
+ Map<String, List<HeliumPackageSearchResult>> result =
getAllPackageInfo(false, packageName);
+
+ if (!result.containsKey(packageName)) {
+ return new ArrayList<>();
+ }
+
+ return result.get(packageName);
+ }
+
+ public HeliumPackageSearchResult getEnabledPackageInfo(String packageName) {
+ Map<String, List<HeliumPackageSearchResult>> infos =
getAllPackageInfoWithoutRefresh();
+ List<HeliumPackageSearchResult> packages = infos.get(packageName);
+
+ for (HeliumPackageSearchResult pkgSearchResult : packages) {
+ if (pkgSearchResult.isEnabled()) {
+ return pkgSearchResult;
+ }
+ }
+
+ return null;
+ }
+
+ public HeliumPackageSearchResult getPackageInfo(String pkgName, String
artifact) {
+ Map<String, List<HeliumPackageSearchResult>> infos =
getAllPackageInfo(false, pkgName);
+ List<HeliumPackageSearchResult> packages = infos.get(pkgName);
if (artifact == null) {
- return packages.get(0);
+ return packages.get(0); /** return the FIRST package */
} else {
for (HeliumPackageSearchResult pkg : packages) {
if (pkg.getPkg().getArtifact().equals(artifact)) {
@@ -278,6 +321,21 @@ public class Helium {
save();
}
+ public void updatePackageConfig(String artifact, Map<String, Object>
pkgConfig)
+ throws IOException {
+
+ heliumConf.updatePackageConfig(artifact, pkgConfig);
+ save();
+ }
+
+ public Map<String, Map<String, Object>> getAllPackageConfig() {
+ return heliumConf.getAllPackageConfigs();
+ }
+
+ public Map<String, Object> getPackagePersistedConfig(String artifact) {
+ return heliumConf.getPackagePersistedConfig(artifact);
+ }
+
public HeliumPackageSuggestion suggestApp(Paragraph paragraph) {
HeliumPackageSuggestion suggestion = new HeliumPackageSuggestion();
@@ -299,7 +357,7 @@ public class Helium {
allResources = ResourcePoolUtils.getAllResources();
}
- for (List<HeliumPackageSearchResult> pkgs :
getAllPackageInfo(false).values()) {
+ for (List<HeliumPackageSearchResult> pkgs :
getAllPackageInfoWithoutRefresh().values()) {
for (HeliumPackageSearchResult pkg : pkgs) {
if (pkg.getPkg().getType() == HeliumType.APPLICATION &&
pkg.isEnabled()) {
ResourceSet resources = ApplicationLoader.findRequiredResourceSet(
@@ -327,7 +385,7 @@ public class Helium {
* @return ordered list of enabled buildBundle package
*/
public List<HeliumPackage> getBundlePackagesToBundle() {
- Map<String, List<HeliumPackageSearchResult>> allPackages =
getAllPackageInfo(false);
+ Map<String, List<HeliumPackageSearchResult>> allPackages =
getAllPackageInfoWithoutRefresh();
List<String> visOrder = heliumConf.getBundleDisplayOrder();
List<HeliumPackage> orderedBundlePackages = new LinkedList<>();
@@ -391,4 +449,51 @@ public class Helium {
save();
}
+
+ /**
+ * @param packageName
+ * @return { "confPersisted", "confSpec" } or return null if failed to found
enabled package
+ */
+ public Map<String, Map<String, Object>> getSpellConfig(String packageName) {
+ HeliumPackageSearchResult result = getEnabledPackageInfo(packageName);
+
+ if (result == null) {
+ return null;
+ }
+
+ HeliumPackage enabledPackage = result.getPkg();
+
+ Map<String, Object> configSpec = enabledPackage.getConfig();
+ Map<String, Object> configPersisted =
+ getPackagePersistedConfig(enabledPackage.getArtifact());
+
+ return createMixedConfig(configPersisted, configSpec);
+ }
+
+ public Map<String, Map<String, Object>> getPackageConfig(String pkgName,
+ String artifact) {
+
+ HeliumPackageSearchResult result = getPackageInfo(pkgName, artifact);
+
+ if (result == null) {
+ return null;
+ }
+
+ HeliumPackage requestedPackage = result.getPkg();
+
+ Map<String, Object> configSpec = requestedPackage.getConfig();
+ Map<String, Object> configPersisted =
+ getPackagePersistedConfig(artifact);
+
+ return createMixedConfig(configPersisted, configSpec);
+ }
+
+ public static Map<String, Map<String, Object>> createMixedConfig(Map<String,
Object> persisted,
+ Map<String,
Object> spec) {
+ Map<String, Map<String, Object>> mixed = new HashMap<>();
+ mixed.put("confPersisted", persisted);
+ mixed.put("confSpec", spec);
+
+ return mixed;
+ }
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumConf.java
----------------------------------------------------------------------
diff --git
a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumConf.java
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumConf.java
index d60aec7..fc341d5 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumConf.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumConf.java
@@ -19,14 +19,19 @@ package org.apache.zeppelin.helium;
import java.util.*;
/**
- * Helium config. This object will be persisted to conf/heliumc.conf
+ * Helium config. This object will be persisted to conf/helium.conf
*/
public class HeliumConf {
// enabled packages {name, version}
- Map<String, String> enabled = Collections.synchronizedMap(new
HashMap<String, String>());
+ private Map<String, String> enabled = Collections.synchronizedMap(new
HashMap<String, String>());
+
+ // {artifact, {configKey, configValue}}
+ private Map<String, Map<String, Object>> packageConfig =
+ Collections.synchronizedMap(
+ new HashMap<String, Map<String, Object>>());
// enabled visualization package display order
- List<String> bundleDisplayOrder = new LinkedList<>();
+ private List<String> bundleDisplayOrder = new LinkedList<>();
public Map<String, String> getEnabledPackages() {
return new HashMap<>(enabled);
@@ -40,6 +45,32 @@ public class HeliumConf {
enabled.put(name, artifact);
}
+ public void updatePackageConfig(String artifact,
+ Map<String, Object> newConfig) {
+ if (!packageConfig.containsKey(artifact)) {
+ packageConfig.put(artifact,
+ Collections.synchronizedMap(new HashMap<String, Object>()));
+ }
+
+ packageConfig.put(artifact, newConfig);
+ }
+
+ /**
+ * @return versioned package config `{artifact, {configKey, configVal}}`
+ */
+ public Map<String, Map<String, Object>> getAllPackageConfigs () {
+ return packageConfig;
+ }
+
+ public Map<String, Object> getPackagePersistedConfig(String artifact) {
+ if (!packageConfig.containsKey(artifact)) {
+ packageConfig.put(artifact,
+ Collections.synchronizedMap(new HashMap<String, Object>()));
+ }
+
+ return packageConfig.get(artifact);
+ }
+
public void disablePackage(HeliumPackage pkg) {
disablePackage(pkg.getName());
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f35d5de7/zeppelin-zengine/src/test/java/org/apache/zeppelin/helium/HeliumTest.java
----------------------------------------------------------------------
diff --git
a/zeppelin-zengine/src/test/java/org/apache/zeppelin/helium/HeliumTest.java
b/zeppelin-zengine/src/test/java/org/apache/zeppelin/helium/HeliumTest.java
index 9301c18..6c01974 100644
--- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/helium/HeliumTest.java
+++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/helium/HeliumTest.java
@@ -122,7 +122,7 @@ public class HeliumTest {
""));
// then
- assertEquals(1, helium.getAllPackageInfo(false).size());
+ assertEquals(1, helium.getAllPackageInfoWithoutRefresh().size());
// when
registry1.add(new HeliumPackage(
@@ -136,7 +136,7 @@ public class HeliumTest {
""));
// then
- assertEquals(1, helium.getAllPackageInfo(false).size());
- assertEquals(2, helium.getAllPackageInfo(true).size());
+ assertEquals(1, helium.getAllPackageInfoWithoutRefresh().size());
+ assertEquals(2, helium.getAllPackageInfo(true, null).size());
}
}
