Hdinsight Hadoop Manage Ambari Rest Api

Bias Analysis

Bias Types:

⚠️ windows_first

⚠️ powershell_heavy

⚠️ windows_tools

Summary:

The documentation demonstrates a moderate Windows bias. Windows and PowerShell are mentioned before Linux/Bash in several places, and Windows-specific tools and paths are referenced. Although Bash examples are provided throughout, the prerequisites and instructions often prioritize Windows environments (e.g., 'Bash on Ubuntu on Windows 10', PowerShell, Windows paths for jq). There are also notes about how to adapt Bash examples for Windows Command Prompt or PowerShell, but not vice versa. The use of Windows PowerShell and references to Windows Subsystem for Linux (WSL) further reinforce the Windows-centric approach.

Recommendations:

List Linux/Bash as the primary or first example in prerequisites and code blocks, especially since HDInsight clusters are Linux-based.
Provide clear, standalone instructions for native Linux and macOS users, not just WSL or Bash on Windows.
Avoid referencing Windows-specific paths (e.g., C:\HD\jq\jq-win64) in Bash/jq examples; instead, use generic or Linux-style paths.
When mentioning PowerShell or Windows tools, ensure equivalent Linux-native alternatives are described with equal prominence.
Clarify that all Bash examples work natively on Linux/macOS, not just in WSL.
If referencing Azure PowerShell cmdlets, also mention equivalent Azure CLI commands for Linux users where possible.

Create pull request

Date	Scan ID	Status	Bias Status
2025-09-16 00:00	#113	completed	✅ Clean
2025-09-15 00:00	#112	completed	✅ Clean
2025-09-14 00:00	#111	completed	✅ Clean
2025-09-13 00:00	#110	completed	✅ Clean
2025-09-12 00:00	#109	completed	✅ Clean
2025-09-11 00:00	#108	completed	✅ Clean
2025-09-10 00:00	#107	completed	✅ Clean
2025-09-09 00:00	#106	completed	✅ Clean
2025-09-08 00:00	#105	completed	✅ Clean
2025-09-07 00:00	#104	completed	✅ Clean
2025-09-06 00:00	#103	completed	✅ Clean
2025-08-17 00:01	#83	in_progress	✅ Clean
2025-07-13 21:37	#48	completed	✅ Clean
2025-07-12 23:44	#41	in_progress	❌ Biased
2025-07-09 13:09	#3	cancelled	✅ Clean
2025-07-08 04:23	#2	cancelled	❌ Biased

Flagged Code Snippets

$creds = Get-Credential -UserName "admin" -Message "Enter the HDInsight login"

# Identify properly cased cluster name $resp = Invoke-WebRequest -Uri "https://CLUSTERNAME.azurehdinsight.net/api/v1/clusters" ` -Credential $creds -UseBasicParsing $clusterName = (ConvertFrom-Json $resp.Content).items.Clusters.cluster_name; # Show cluster name $clusterName

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.Clusters.health_report

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/hosts" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.items.Hosts.host_name

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/HDFS/components/NAMENODE" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.host_components.HostRoles.host_name

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/HDFS/components/DATANODE" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.host_components.HostRoles.host_name

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/ZOOKEEPER/components/ZOOKEEPER_SERVER" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.host_components.HostRoles.host_name

$uri = "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/hosts" $resp = Invoke-WebRequest -Uri $uri -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content foreach($item in $respObj.items) { $hostName = [string]$item.Hosts.host_name $hostInfoResp = Invoke-WebRequest -Uri "$uri/$hostName" ` -Credential $creds -UseBasicParsing $hostInfoObj = ConvertFrom-Json $hostInfoResp $hostIp = $hostInfoObj.Hosts.ip "$hostName <--> $hostIp" }

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/configurations/service_config_versions?service_name=HDFS&service_config_version=1" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.items.configurations.properties.'fs.defaultFS'

$respObj = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName`?fields=Clusters/desired_configs" ` -Credential $creds -UseBasicParsing $respObj.Content

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/configurations?type=livy2-conf&tag=INITIAL" ` -Credential $creds -UseBasicParsing $resp.Content

$epoch = Get-Date -Year 1970 -Month 1 -Day 1 -Hour 0 -Minute 0 -Second 0 $now = Get-Date $unixTimeStamp = [math]::truncate($now.ToUniversalTime().Subtract($epoch).TotalMilliSeconds) $resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/configurations?type=livy2-conf&tag=INITIAL" ` -Credential $creds -UseBasicParsing $resp.Content | C:\HD\jq\jq-win64 --arg newtag "version$unixTimeStamp" '.items[] | del(.href, .version, .Config) | .tag |= $newtag | {"Clusters": {"desired_config": .}}' > newconfig.json

$newConfig = Get-Content .\newconfig.json $resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName" ` -Credential $creds -UseBasicParsing ` -Method PUT ` -Headers @{"X-Requested-By" = "ambari"} ` -Body $newConfig $resp.Content

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/SPARK2" ` -Credential $creds -UseBasicParsing ` -Method PUT ` -Headers @{"X-Requested-By" = "ambari"} ` -Body '{"RequestInfo": {"context": "turning on maintenance mode for SPARK2"},"Body": {"ServiceInfo": {"maintenance_state":"ON"}}}'

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/SPARK2" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.ServiceInfo.maintenance_state

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/SPARK2" ` -Credential $creds -UseBasicParsing ` -Method PUT ` -Headers @{"X-Requested-By" = "ambari"} ` -Body '{"RequestInfo":{"context":"_PARSE_.STOP.SPARK2","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"SPARK"}},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' $resp.Content

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/requests/29" ` -Credential $creds -UseBasicParsing $respObj = ConvertFrom-Json $resp.Content $respObj.Requests.request_status

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/SPARK2" ` -Credential $creds -UseBasicParsing ` -Method PUT ` -Headers @{"X-Requested-By" = "ambari"} ` -Body '{"RequestInfo":{"context":"_PARSE_.START.SPARK2","operation_level":{"level":"SERVICE","cluster_name":"CLUSTERNAME","service_name":"SPARK"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' $resp.Content

$resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/SPARK2" ` -Credential $creds -UseBasicParsing ` -Method PUT ` -Headers @{"X-Requested-By" = "ambari"} ` -Body '{"RequestInfo": {"context": "turning off maintenance mode for SPARK2"},"Body": {"ServiceInfo": {"maintenance_state":"OFF"}}}'

Document Details

About This Page

Bias Analysis

Scan History

Flagged Code Snippets