Hi Alexander, I didn’t do extensive HDFS testing that I didn’t encounter those JIRAs in the Confluence page. From a smoketest standpoint it did seem to work.
Allan From: Alexander Batyrshin <[email protected]> Date: Saturday, October 5, 2019 at 6:39 PM To: Anu Engineer <[email protected]> Cc: Allan Espinosa <[email protected]>, <[email protected]> Subject: Re: Kubernetes integration for HDFS ENV JAVA_HOME /usr/lib/jvm/jre-11-openjdk Is Hadoop OK with this version? As far i can see - this version is not compatible https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions -- Alexander Batyrshin aka bash Biomechanical Artificial Sabotage Humanoid On Fri, 20 Sep 2019 at 08:22, Anu Engineer <[email protected]> wrote: That is very interesting. A sub-project of Hadoop called Ozone tests regularly with Kubernetes. If you would like to contribute your work to Hadoop, you can send us a pull request and we can maintain this code as a sample under Ozone K8s samples directory. An advantage with that approach is that this will get regularly tested. Please note that K8s support is very early / experimental under Ozone at this point of time. -- Anu On Thu, Sep 19, 2019 at 7:30 PM Allan Espinosa <[email protected]> wrote: Hello folks, I just wanted to share a PoC of running HDFS on Kubernetes that I hacked on over the summer. https://github.com/aespinosa/cloud-hadoop The above repo introduces a "hdfs kube_namenode" command where the namenode formats itself or runs "-bootstrapStandby" depending on the state of provisioning an HDFS cluster. The state information is stored inside Kubernetes as metadata annotations to the manifest .yml of running HDFS. With this, it is simpler to bring up HA sandbox HDFS clusters without the need for externally orchestrating namenode and zkfc formatting. Hope others find this useful. Regards, Allan --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
