sonatype-lift[bot] commented on code in PR #2002: URL: https://github.com/apache/zookeeper/pull/2002#discussion_r1226302893
########## zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java: ########## @@ -0,0 +1,134 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zookeeper.server; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; +import java.util.Set; +import org.apache.jute.BinaryInputArchive; +import org.apache.jute.InputArchive; +import org.apache.yetus.audience.InterfaceAudience; +import org.apache.zookeeper.server.persistence.FileSnap; +import org.apache.zookeeper.server.persistence.SnapStream; + +/** + * Recursively processes a snapshot file collecting child node count and summarizes the data size + * below each node. + * "starting_node" defines the node where the recursion starts + * "max_depth" defines the depth where the tool still writes to the output. + * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will + * only contain the starting node's and it's children's stats, 2 ads another level and so on. + * This ONLY affects the level of details displayed, NOT the calculation. + */ [email protected] public class SnapshotRecursiveSummary { + + /** + * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth + * + */ + public static void main(String[] args) throws Exception { + if (args.length != 3) { + System.err.println(getUsage()); + System.exit(2); + } + int maxDepth = 0; + try { + maxDepth = Integer.parseInt(args[2]); + } catch (NumberFormatException e) { + System.err.println(getUsage()); + System.exit(2); + } + + new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth); + } + + public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException { + File snapshotFile = new File(snapshotFileName); + try (InputStream is = SnapStream.getInputStream(snapshotFile)) { + InputArchive ia = BinaryInputArchive.getArchive(is); + + DataTree dataTree = new DataTree(); + Map<Long, Integer> sessions = new HashMap<>(); + FileSnap.deserialize(dataTree, sessions, ia); Review Comment: <picture><img alt="14% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/14/display.svg";></picture> <b>*RESOURCE_LEAK:</b>* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 68 is not released after line 72. ââ <b>5 similar findings have been found in this PR</b> <details><summary>đ Expand here to view all instances of this finding</summary><br/> <div align=\"center\"> | **File Path** | **Line Number** | | ------------- | ------------- | | zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocket.java | [143](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocket.java#L143) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java | [102](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java#L102) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/NIOServerCnxn.java | [440](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/NIOServerCnxn.java#L440) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/NIOServerCnxn.java | [439](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/NIOServerCnxn.java#L439) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotComparer.java | [268](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotComparer.java#L268) | <p><a href="https://lift.sonatype.com/results/github.com/apache/zookeeper/01H2QBJ7D0JVSBB9367JMDYSEK?t=Infer|RESOURCE_LEAK" target="_blank">Visit the Lift Web Console</a> to find more details in your report.</p></div></details> --- <details><summary>âšī¸ Expand to see all <b>@sonatype-lift</b> commands</summary> You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings. | **Command** | **Usage** | | ------------- | ------------- | | `@sonatype-lift ignore` | Leave out the above finding from this PR | | `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR | | `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file | **Note:** When talking to LiftBot, you need to **refresh** the page to see its response. <sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details> ########## zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java: ########## @@ -0,0 +1,134 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zookeeper.server; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; +import java.util.Set; +import org.apache.jute.BinaryInputArchive; +import org.apache.jute.InputArchive; +import org.apache.yetus.audience.InterfaceAudience; +import org.apache.zookeeper.server.persistence.FileSnap; +import org.apache.zookeeper.server.persistence.SnapStream; + +/** + * Recursively processes a snapshot file collecting child node count and summarizes the data size + * below each node. + * "starting_node" defines the node where the recursion starts + * "max_depth" defines the depth where the tool still writes to the output. + * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will + * only contain the starting node's and it's children's stats, 2 ads another level and so on. + * This ONLY affects the level of details displayed, NOT the calculation. + */ [email protected] public class SnapshotRecursiveSummary { + + /** + * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth + * + */ + public static void main(String[] args) throws Exception { + if (args.length != 3) { + System.err.println(getUsage()); + System.exit(2); + } + int maxDepth = 0; + try { + maxDepth = Integer.parseInt(args[2]); + } catch (NumberFormatException e) { + System.err.println(getUsage()); + System.exit(2); + } + + new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth); + } + + public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException { + File snapshotFile = new File(snapshotFileName); Review Comment: <picture><img alt="9% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/9/display.svg";></picture> <b>*[PATH_TRAVERSAL_IN](https://find-sec-bugs.github.io/bugs.htm#PATH_TRAVERSAL_IN):</b>* This API (java/io/File.<init>(Ljava/lang/String;)V) reads a file whose location might be specified by user input --- <details><summary>âšī¸ Expand to see all <b>@sonatype-lift</b> commands</summary> You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings. | **Command** | **Usage** | | ------------- | ------------- | | `@sonatype-lift ignore` | Leave out the above finding from this PR | | `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR | | `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file | **Note:** When talking to LiftBot, you need to **refresh** the page to see its response. <sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details> ########## zookeeper-server/src/main/java/org/apache/zookeeper/server/admin/JettyAdminServer.java: ########## @@ -253,21 +259,88 @@ protected void doGet( // Extract keyword arguments to command from request parameters @SuppressWarnings("unchecked") Map<String, String[]> parameterMap = request.getParameterMap(); - Map<String, String> kwargs = new HashMap<String, String>(); + Map<String, String> kwargs = new HashMap<>(); for (Map.Entry<String, String[]> entry : parameterMap.entrySet()) { kwargs.put(entry.getKey(), entry.getValue()[0]); } + final String authInfo = request.getHeader(HttpHeader.AUTHORIZATION.asString()); // Run the command - CommandResponse cmdResponse = Commands.runCommand(cmd, zkServer, kwargs); + final CommandResponse cmdResponse = Commands.runGetCommand(cmd, zkServer, kwargs, authInfo, request); + response.setStatus(cmdResponse.getStatusCode()); - // Format and print the output of the command - CommandOutputter outputter = new JsonOutputter(); - response.setStatus(HttpServletResponse.SC_OK); + final Map<String, String> headers = cmdResponse.getHeaders(); + for (final Map.Entry<String, String> header : headers.entrySet()) { + response.addHeader(header.getKey(), header.getValue()); + } + final String clientIP = IPAuthenticationProvider.getClientIPAddress(request); + if (cmdResponse.getInputStream() == null) { + // Format and print the output of the command + CommandOutputter outputter = new JsonOutputter(clientIP); + response.setContentType(outputter.getContentType()); + outputter.output(cmdResponse, response.getWriter()); + } else { + // Stream out the output of the command + CommandOutputter outputter = new StreamOutputter(clientIP); + response.setContentType(outputter.getContentType()); + outputter.output(cmdResponse, response.getOutputStream()); + } + } + + /** + * Serves HTTP POST requests. It reads request payload as raw data. + * It's up to each command to process the payload accordingly. + * For example, RestoreCommand uses the payload InputStream directly + * to read snapshot data. + */ + @Override + protected void doPost(final HttpServletRequest request, + final HttpServletResponse response) throws ServletException, IOException { + final String cmdName = extractCommandNameFromURL(request, response); + if (cmdName != null) { + final String authInfo = request.getHeader(HttpHeader.AUTHORIZATION.asString()); + final CommandResponse cmdResponse = Commands.runPostCommand(cmdName, zkServer, request.getInputStream(), authInfo, request); + final String clientIP = IPAuthenticationProvider.getClientIPAddress(request); + sendJSONResponse(response, cmdResponse, clientIP); + } + } + + /** + * Extracts the command name from URL if it exists otherwise null + */ + private String extractCommandNameFromURL(final HttpServletRequest request, + final HttpServletResponse response) throws IOException { + String cmd = request.getPathInfo(); + if (cmd == null || cmd.equals("/")) { + printCommandLinks(response); + return null; + } + // Strip leading "/" + return cmd.substring(1); + } + + /** + * Prints the list of URLs to each registered command as response. + */ + private void printCommandLinks(final HttpServletResponse response) throws IOException { + for (final String link : commandLinks()) { + response.getWriter().println(link); Review Comment: <picture><img alt="18% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/18/display.svg";></picture> <b>*[XSS_SERVLET](https://find-sec-bugs.github.io/bugs.htm#XSS_SERVLET):</b>* This use of java/io/PrintWriter.println(Ljava/lang/String;)V could be vulnerable to XSS in the Servlet --- <details><summary>âšī¸ Expand to see all <b>@sonatype-lift</b> commands</summary> You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings. | **Command** | **Usage** | | ------------- | ------------- | | `@sonatype-lift ignore` | Leave out the above finding from this PR | | `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR | | `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file | **Note:** When talking to LiftBot, you need to **refresh** the page to see its response. <sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details> ########## zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java: ########## @@ -772,16 +765,16 @@ public Stat setACL(String path, List<ACL> acl, int version) throws KeeperExcepti } } - public List<ACL> getACL(String path, Stat stat) throws KeeperException.NoNodeException { + public List<ACL> getACL(String path, Stat stat) throws NoNodeException { DataNode n = nodes.get(path); if (n == null) { - throw new KeeperException.NoNodeException(); + throw new NoNodeException(); } synchronized (n) { if (stat != null) { n.copyStat(stat); } - return new ArrayList<ACL>(aclCache.convertLong(n.acl)); + return new ArrayList<>(aclCache.convertLong(n.acl)); Review Comment: <picture><img alt="7% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/7/display.svg";></picture> <b>*THREAD_SAFETY_VIOLATION:</b>* Read/Write race. Non-private method `DataTree.getACL(...)` indirectly reads with synchronization from container `this.aclCache.longKeyMap` via call to `Map.get(...)`. Potentially races with unsynchronized write in method `DataTree.deserialize(...)`. Reporting because this access may occur on a background thread. ââ <b>8 similar findings have been found in this PR</b> <details><summary>đ Expand here to view all instances of this finding</summary><br/> <div align=\"center\"> | **File Path** | **Line Number** | | ------------- | ------------- | | zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java | [1515](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java#L1515) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java | [317](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java#L317) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java | [326](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/PrepRequestProcessor.java#L326) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java | [1488](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java#L1488) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java | [331](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/ZooKeeperServer.java#L331) | | zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNetty.java | [247](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxnSocketNetty.java#L247) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/ZKDatabase.java | [636](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/ZKDatabase.java#L636) | | zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java | [460](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java#L460) | <p><a href="https://lift.sonatype.com/results/github.com/apache/zookeeper/01H2QBJ7D0JVSBB9367JMDYSEK?t=Infer|THREAD_SAFETY_VIOLATION" target="_blank">Visit the Lift Web Console</a> to find more details in your report.</p></div></details> --- <details><summary>âšī¸ Expand to see all <b>@sonatype-lift</b> commands</summary> You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings. | **Command** | **Usage** | | ------------- | ------------- | | `@sonatype-lift ignore` | Leave out the above finding from this PR | | `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR | | `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file | **Note:** When talking to LiftBot, you need to **refresh** the page to see its response. <sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details> ########## tools/ci/test-connectivity.py: ########## @@ -0,0 +1,48 @@ +#!/usr/bin/env python3 + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import argparse +import subprocess + +from pathlib import Path + +class Server(): + def __init__(self, binpath): + self.binpath = binpath + def __enter__(self): + subprocess.run([f'{self.binpath}', 'start'], check=True) Review Comment: <picture><img alt="6% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/6/display.svg";></picture> <b>*[B603](https://bandit.readthedocs.io/en/1.7.4/plugins/b603_subprocess_without_shell_equals_true.html):</b>* subprocess call - check for execution of untrusted input. ââ <b>2 similar findings have been found in this PR</b> <details><summary>đ Expand here to view all instances of this finding</summary><br/> <div align=\"center\"> | **File Path** | **Line Number** | | ------------- | ------------- | | tools/ci/test-connectivity.py | [31](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/tools/ci/test-connectivity.py#L31) | | tools/ci/test-connectivity.py | [48](https://github.com/apache/zookeeper/blob/e18effa279d8787adf91e796f2fdc4a27c958f1e/tools/ci/test-connectivity.py#L48) | <p><a href="https://lift.sonatype.com/results/github.com/apache/zookeeper/01H2QBJ7D0JVSBB9367JMDYSEK?t=Bandit|B603" target="_blank">Visit the Lift Web Console</a> to find more details in your report.</p></div></details> --- <details><summary>âšī¸ Expand to see all <b>@sonatype-lift</b> commands</summary> You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings. | **Command** | **Usage** | | ------------- | ------------- | | `@sonatype-lift ignore` | Leave out the above finding from this PR | | `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR | | `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file | **Note:** When talking to LiftBot, you need to **refresh** the page to see its response. <sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details> ########## tools/ci/test-connectivity.py: ########## @@ -0,0 +1,48 @@ +#!/usr/bin/env python3 + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import argparse +import subprocess + +from pathlib import Path + +class Server(): Review Comment: <picture><img alt="8% of developers fix this issue" src="https://lift.sonatype.com/api/commentimage/fixrate/8/display.svg";></picture> <b>*E302:</b>* expected 2 blank lines, found 1 --- <details><summary>âšī¸ Expand to see all <b>@sonatype-lift</b> commands</summary> You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings. | **Command** | **Usage** | | ------------- | ------------- | | `@sonatype-lift ignore` | Leave out the above finding from this PR | | `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR | | `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file | **Note:** When talking to LiftBot, you need to **refresh** the page to see its response. <sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://lift.sonatype.com/api/commentimage/fixrate/14/display.svg"](https://lift.sonatype.com/api/commentimage/fixrate/14/display.svg"){kind=link}
![https://lift.sonatype.com/api/commentimage/fixrate/9/display.svg"](https://lift.sonatype.com/api/commentimage/fixrate/9/display.svg"){kind=link}
![https://lift.sonatype.com/api/commentimage/fixrate/18/display.svg"](https://lift.sonatype.com/api/commentimage/fixrate/18/display.svg"){kind=link}
![https://lift.sonatype.com/api/commentimage/fixrate/7/display.svg"](https://lift.sonatype.com/api/commentimage/fixrate/7/display.svg"){kind=link}
![https://lift.sonatype.com/api/commentimage/fixrate/6/display.svg"](https://lift.sonatype.com/api/commentimage/fixrate/6/display.svg"){kind=link}
![https://lift.sonatype.com/api/commentimage/fixrate/8/display.svg"](https://lift.sonatype.com/api/commentimage/fixrate/8/display.svg"){kind=link}