Hello Hema, HDFS Team, I shall explain the use case based on which we have posted the queries below. Our product contains one of the activities which uses HDFS client driver to distribute files to HDFS server. HDFS server is installed separately & is not part of our product packaging. We are planning to move HDFS Client Driver to latest stable version of HDFS 3.3.3 in our future release of product.
When our customer upgrades to our latest version of product, this will imply that HDFS Client Driver shall be version of HDFS 3.3.3. Here arises the use case where there is a possibility of HDFS Client being on latest version 3.3.3 v/s HDFS Server being on older version of 2.7.3. Customer may eventually upgrade HDFS Server to latest stable version 3.3.3 as well but there is a possibility that HDFS Server upgrade shall not be done simultaneously. So my query is as below: 1. Use Case1: HDFS Client Driver on latest stable version of HDFS 3.3.3 while HDFS Server is on older version 2.7.3: 1. Can you please let us know if there are any HDFS Client level API changes in 3.3.3 version from older version of 2.7.3 which are not backward compatible. 2. Suppose we move HDFS Client to version 3.3.3 and HDFS Server remains on older version 2.7.3. Is there any functionality present in older HDFS Client version 2.7.3 which shall not work if we upgrade HDFS Client to version 3.3.3 with HDFS Server still on older release 2.7.3? 1. I have gone through this Hadoop documentation link<https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_Protocols> (Apache Hadoop 3.3.3 – Apache Hadoop Compatibility<https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_Protocols>) where there is mention of “forward compatible across minor releases within a major version“. However I am not able to find mention of compatibility across Major versions. * Could you please point me to compatibility matrix and/or statement which can be used to deduce the compatibility of the new client driver with old released Hadoop servers & vice-versa Regards, Deepti Sharma PMP® & ITIL From: Hema Kumar <[email protected]> Sent: Wednesday, August 3, 2022 10:01 AM To: Deepti Sharma S <[email protected]> Cc: [email protected]; Deepak Nangia <[email protected]>; [email protected] Subject: Re: Backward Compatibility for HDFS 2.7.3 Hi Deepti, I have a question regarding the use case, What is the benefit you are getting from updating only client? -- Hema Kumar On Tue, Aug 2, 2022 at 11:04 PM Deepti Sharma S <[email protected]<mailto:[email protected]>> wrote: Hello Team, We are currently using Hadoop HDFS version 2.7.3 at HDFS Client Driver & HDFS Server. We have following queries: 1. We are planning to move HDFS Client Driver to latest stable version of HDFS 3.3.3. * Can you please let us know if there are any HDFS Client level API changes in 3.3.3 version from older version of 2.7.3 which are not backward compatible. * Suppose we move HDFS Client to version 3.3.3 and HDFS Server remains on older version 2.7.3. Is there any functionality present in older HDFS Client version 2.7.3 which shall not work if we upgrade HDFS Client to version 3.3.3 with HDFS Server still on older release 2.7.3? 1. I have gone through this Hadoop documentation link<https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_Protocols> where there is mention of “forward compatible across minor releases within a major version“. However I am not able to find mention of compatibility across Major versions. * Could you please point me to compatibility matrix and/or statement which can be used to deduce the compatibility of the new client driver with old released Hadoop servers & vice-versa. Regards, Deepti Sharma PMP® & ITIL
