That is very interesting. A sub-project of Hadoop called Ozone tests regularly with Kubernetes. If you would like to contribute your work to Hadoop, you can send us a pull request and we can maintain this code as a sample under Ozone K8s samples directory. An advantage with that approach is that this will get regularly tested. Please note that K8s support is very early / experimental under Ozone at this point of time.
-- Anu On Thu, Sep 19, 2019 at 7:30 PM Allan Espinosa <[email protected]> wrote: > Hello folks, > > I just wanted to share a PoC of running HDFS on Kubernetes that I hacked > on over the summer. > > https://github.com/aespinosa/cloud-hadoop > > The above repo introduces a "hdfs kube_namenode" command where the > namenode formats itself or runs "-bootstrapStandby" depending on the state > of provisioning an HDFS cluster. The state information is stored inside > Kubernetes as metadata annotations to the manifest .yml of running HDFS. > > With this, it is simpler to bring up HA sandbox HDFS clusters without the > need for externally orchestrating namenode and zkfc formatting. > > Hope others find this useful. > > Regards, > Allan > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
