Repository: zeppelin Updated Branches: refs/heads/master 9dc9c7512 -> 7e2a1b5d4
[ZEPPELIN-699] Add new synchronous paragraph run REST API ### What is this PR for? Right now, when calling the REST API `http://<ip>:<port>/api/notebook/job/<note_id>/<paragraph_id>` Zeppelin always returns **OK** as shown by this source code: https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java#L477 This ticket will update the behavior so that Zeppelin also return the result of the paragraph execution ### What type of PR is it? [Improvement] ### Todos * [ ] - Code Review * [ ] - Simple Test ### Is there a relevant Jira issue? **[ZEPPELIN-699]** ### How should this be tested? * `git fetch origin pull/746/head:ParagraphExecutionRESTAPI` * `git checkout ParagraphExecutionRESTAPI` * `mvn clean package -DskipTests` * `bin/zeppelin-daemon.sh restart` * Create a new note * In the first paragraph, put the following code ```scala %sh echo "Current time = "`date +"%T" ``` * Retrieve the current note id in the URL * Retrieve the current paragraph id * Use a REST Client like **[POSTman]** to create a HTTP POST query `http://<ip>:<port>/api/notebook/run/<note_id>/<paragraph_id>` * You should receive something similar as follow for answer ``` { "status": "OK", "body": { "code": "SUCCESS", "type": "TEXT", "msg": "Current time = 16:14:18\n" } } ``` ### Screenshots (if appropriate)  API Documentation update **Existing asynchronous API**  **New synchronous API**  ### Questions: * Does the licenses files need update? --> **No** * Is there breaking changes for older versions? --> **No** * Does this needs documentation? --> **Yes** [ZEPPELIN-699]: https://issues.apache.org/jira/browse/ZEPPELIN-699 [POSTman]: https://www.getpostman.com/ Author: DuyHai DOAN <doanduy...@gmail.com> Closes #746 from doanduyhai/ZEPPELIN-699 and squashes the following commits: fb0570c [DuyHai DOAN] [ZEPPELIN-699] Update Notebook REST API documentation 8367acf [DuyHai DOAN] [ZEPPELIN-699] Add new synchronous paragraph run REST API Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/7e2a1b5d Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/7e2a1b5d Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/7e2a1b5d Branch: refs/heads/master Commit: 7e2a1b5d4f134489428a24c7c5502f189224b506 Parents: 9dc9c75 Author: DuyHai DOAN <doanduy...@gmail.com> Authored: Thu Jun 2 16:24:38 2016 +0200 Committer: Felix Cheung <felixche...@apache.org> Committed: Wed Aug 31 23:56:39 2016 -0700 ---------------------------------------------------------------------- docs/rest-api/rest-notebook.md | 55 ++++++++++++- .../apache/zeppelin/rest/NotebookRestApi.java | 81 ++++++++++++++++---- .../java/org/apache/zeppelin/notebook/Note.java | 26 ++++++- 3 files changed, 145 insertions(+), 17 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/7e2a1b5d/docs/rest-api/rest-notebook.md ---------------------------------------------------------------------- diff --git a/docs/rest-api/rest-notebook.md b/docs/rest-api/rest-notebook.md index c7e17ea..ed39a4f 100644 --- a/docs/rest-api/rest-notebook.md +++ b/docs/rest-api/rest-notebook.md @@ -450,12 +450,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple </table> <br/> -### Run a paragraph +### Run a paragraph asynchronously <table class="table-configuration"> <col width="200"> <tr> <td>Description</td> - <td>This ```POST``` method runs the paragraph by given notebook and paragraph id. + <td>This ```POST``` method runs the paragraph asynchronously by given notebook and paragraph id. This API always return SUCCESS even if the execution of the paragraph fails later because the API is asynchronous </td> </tr> <tr> @@ -488,6 +488,56 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple </table> <br/> +### Run a paragraph synchronously + <table class="table-configuration"> + <col width="200"> + <tr> + <td>Description</td> + <td> This ```POST``` method runs the paragraph synchronously by given notebook and paragraph id. This API can return SUCCESS or ERROR depending on the outcome of the paragraph execution + </td> + </tr> + <tr> + <td>URL</td> + <td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td> + </tr> + <tr> + <td>Success code</td> + <td>200</td> + </tr> + <tr> + <td> Fail code</td> + <td> 500 </td> + </tr> + <tr> + <td> sample JSON input (optional, only needed when if you want to update dynamic form's value) </td> + <td><pre> +{ + "name": "name of new notebook", + "params": { + "formLabel1": "value1", + "formLabel2": "value2" + } +}</pre></td> + </tr> + <tr> + <td> sample JSON response </td> + <td><pre>{"status": "OK"}</pre></td> + </tr> + <tr> + <td> sample JSON error </td> + <td><pre> +{ + "status": "INTERNAL\_SERVER\_ERROR", + "body": { + "code": "ERROR", + "type": "TEXT", + "msg": "bash: -c: line 0: unexpected EOF while looking for matching ``'\nbash: -c: line 1: syntax error: unexpected end of file\nExitValue: 2" + } +}</pre></td> + </tr> + </table> + +<br/> ### Stop a paragraph <table class="table-configuration"> <col width="200"> @@ -922,4 +972,3 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple </tr> </tr> </table> - \ No newline at end of file http://git-wip-us.apache.org/repos/asf/zeppelin/blob/7e2a1b5d/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java b/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java index db0cbec..17ec74f 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java @@ -570,7 +570,7 @@ public class NotebookRestApi { } /** - * Run paragraph job REST API + * Run asynchronously paragraph job REST API * * @param message - JSON with params if user wants to update dynamic form's value * null, empty string, empty json if user doesn't want to update @@ -583,7 +583,7 @@ public class NotebookRestApi { public Response runParagraph(@PathParam("notebookId") String notebookId, @PathParam("paragraphId") String paragraphId, String message) throws IOException, IllegalArgumentException { - LOG.info("run paragraph job {} {} {}", notebookId, paragraphId, message); + LOG.info("run paragraph job asynchronously {} {} {}", notebookId, paragraphId, message); Note note = notebook.getNote(notebookId); if (note == null) { @@ -596,22 +596,60 @@ public class NotebookRestApi { } // handle params if presented - if (!StringUtils.isEmpty(message)) { - RunParagraphWithParametersRequest request = - gson.fromJson(message, RunParagraphWithParametersRequest.class); - Map<String, Object> paramsForUpdating = request.getParams(); - if (paramsForUpdating != null) { - paragraph.settings.getParams().putAll(paramsForUpdating); - AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal()); - note.setLastReplName(paragraph.getId()); - note.persist(subject); - } - } + handleParagraphParams(message, note, paragraph); note.run(paragraph.getId()); return new JsonResponse<>(Status.OK).build(); } +/** + * Run synchronously a paragraph REST API + * + * @param noteId - noteId + * @param paragraphId - paragraphId + * @param message - JSON with params if user wants to update dynamic form's value + * null, empty string, empty json if user doesn't want to update + * + * @return JSON with status.OK + * @throws IOException, IllegalArgumentException + */ + @POST + @Path("run/{notebookId}/{paragraphId}") + @ZeppelinApi + public Response runParagraphSynchronously(@PathParam("notebookId") String noteId, + @PathParam("paragraphId") String paragraphId, + String message) throws + IOException, IllegalArgumentException { + LOG.info("run paragraph synchronously {} {} {}", noteId, paragraphId, message); + + Note note = notebook.getNote(noteId); + if (note == null) { + return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build(); + } + + Paragraph paragraph = note.getParagraph(paragraphId); + if (paragraph == null) { + return new JsonResponse<>(Status.NOT_FOUND, "paragraph not found.").build(); + } + + // handle params if presented + handleParagraphParams(message, note, paragraph); + + if (paragraph.getListener() == null) { + note.initializeJobListenerForParagraph(paragraph); + } + + paragraph.run(); + + final InterpreterResult result = paragraph.getResult(); + + if (result.code() == InterpreterResult.Code.SUCCESS) { + return new JsonResponse<>(Status.OK, result).build(); + } else { + return new JsonResponse<>(Status.INTERNAL_SERVER_ERROR, result).build(); + } + } + /** * Stop(delete) paragraph job REST API * @@ -803,4 +841,21 @@ public class NotebookRestApi { return new JsonResponse<>(Status.OK, notebooksFound).build(); } + + private void handleParagraphParams(String message, Note note, Paragraph paragraph) + throws IOException { + // handle params if presented + if (!StringUtils.isEmpty(message)) { + RunParagraphWithParametersRequest request = + gson.fromJson(message, RunParagraphWithParametersRequest.class); + Map<String, Object> paramsForUpdating = request.getParams(); + if (paramsForUpdating != null) { + paragraph.settings.getParams().putAll(paramsForUpdating); + AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal()); + note.setLastReplName(paragraph.getId()); + note.persist(subject); + } + } + } + } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/7e2a1b5d/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java index 2b89524..0b2b4c6 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java @@ -17,6 +17,8 @@ package org.apache.zeppelin.notebook; +import static java.lang.String.format; + import java.io.IOException; import java.io.Serializable; import java.util.HashMap; @@ -168,6 +170,28 @@ public class Note implements Serializable, ParagraphJobListener { } } + public void initializeJobListenerForParagraph(Paragraph paragraph) { + final Note paragraphNote = paragraph.getNote(); + if (paragraphNote.getId().equals(this.getId())) { + throw new IllegalArgumentException(format("The paragraph %s from note %s " + + "does not belong to note %s", paragraph.getId(), paragraphNote.getId(), + this.getId())); + } + + boolean foundParagraph = false; + for (Paragraph ownParagraph : paragraphs) { + if (paragraph.getId().equals(ownParagraph.getId())) { + paragraph.setListener(this.jobListenerFactory.getParagraphJobListener(this)); + foundParagraph = true; + } + } + + if (!foundParagraph) { + throw new IllegalArgumentException(format("Cannot find paragraph %s " + + "from note %s", paragraph.getId(), paragraphNote.getId())); + } + } + void setJobListenerFactory(JobListenerFactory jobListenerFactory) { this.jobListenerFactory = jobListenerFactory; } @@ -480,7 +504,7 @@ public class Note implements Serializable, ParagraphJobListener { logger.debug("New paragraph: {}", pText); p.setEffectiveText(pText); } else { - String intpExceptionMsg = String.format("%s", + String intpExceptionMsg = format("%s", p.getJobName() + "'s Interpreter " + requiredReplName + " not found"
