Proposed Pull Request Change

📄 Document Links

View on GitHub View on Microsoft Learn

--- title: Open-source components and versions - Azure HDInsight 4.0 description: Learn about the open-source components and versions in Azure HDInsight 4.0. ms.service: azure-hdinsight ms.topic: conceptual author: apurbasroy ms.author: apsinhar ms.reviewer: nijelsf ms.date: 04/11/2024 --- # HDInsight 4.0 component versions In this article, you learn about the open-source components and versions in Azure HDInsight 4.0. ## Open-source components available with HDInsight version 4.0 The Open-source component versions associated with HDInsight 4.0 are present in the following table. | Component | HDInsight 4.0 | |------------------------|---------------| | Apache Hadoop and YARN | 3.1.1 | | Apache Tez | 0.9.1 | | Apache Pig | 0.16.1 | | Apache Hive | 3.1.2 | | Apache Ranger | 1.1.0 | | Apache HBase | 2.1.6 | | Apache Sqoop | 1.5.0 | | Apache Oozie | 4.3.1 | | Apache Zookeeper | 3.4.6 | | Apache Phoenix | 5 | | Apache Spark | 2.4.4 | | Apache Livy | 0.5 | | Apache Kafka | 2.1.1 | | Apache Ambari | 2.7.0 | | Apache Zeppelin | 0.8.0 | This table lists certain HDInsight 4.0 cluster types that have retired or will be retiring soon. | Cluster Type | Framework version | Support expiration date | Retirement date | |---------------------------------|-------------------|------------------------------|-----------------| | HDInsight 4.0 Spark | 2.3 | June 30, 2020 | June 30, 2020 | | HDInsight 4.0 Kafka | 1.1 | Dec 31, 2020 | Dec 31, 2020 | | HDInsight 4.0 Kafka | 2.1.0 | Sep 30, 2022 | Oct 1, 2022 | ## Apache Spark 2.4 to Spark 3.x Migration Guides Spark 2.4 to Spark 3.x Migration Guides see [here](https://spark.apache.org/docs/latest/migration-guide.html). ## Next steps - [Cluster setup for Apache Hadoop, Spark, and more on HDInsight](hdinsight-hadoop-provision-linux-clusters.md) - [Enterprise Security Package](./enterprise-security-package.md) - [Work in Apache Hadoop on HDInsight from a Windows PC](hdinsight-hadoop-windows-tools.md)