Document Details

Apache Hadoop Use Mapreduce Curl
Home / Scan #41 / Apache Hadoop Use Mapreduce Curl
Sad Tux - Windows bias detected
This page contains Windows bias

About This Page

This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
GitHub View on GitHub 📚 View on Microsoft Learn

Bias Analysis

Detected Bias Types
windows_first
powershell_heavy
windows_tools
missing_linux_example
Summary
The documentation demonstrates a Windows bias by providing command-line examples that use Windows-specific syntax (e.g., 'set' for environment variables, caret '^' for line continuation, and Windows paths for jq), and by listing PowerShell as a primary method. There are no explicit Linux/bash shell examples, and the curl examples are tailored for Windows command prompt rather than bash. The use of Windows tools and patterns is prevalent, while Linux equivalents are not shown.
Recommendations
  • Add explicit Linux/bash shell examples for all curl commands, using bash syntax for environment variables (e.g., 'export'), line continuation (backslash '\'), and typical Linux paths for jq (e.g., '/usr/bin/jq').
  • Present both Windows and Linux examples side-by-side or clearly label which environment each example is for.
  • Avoid using Windows-specific tools or paths in generic curl examples; provide cross-platform alternatives.
  • In the prerequisites, clarify that curl/jq usage is cross-platform and provide installation guidance for both Windows and Linux.
  • Ensure that Linux/bash examples are given equal prominence to Windows/PowerShell examples.
GitHub Create Pull Request

Scan History

Date Scan Status Result
2026-01-14 00:00 #250 in_progress Biased Biased
2026-01-13 00:00 #246 completed Biased Biased
2026-01-11 00:00 #240 completed Biased Biased
2026-01-10 00:00 #237 completed Biased Biased
2026-01-09 00:34 #234 completed Biased Biased
2026-01-08 00:53 #231 completed Biased Biased
2026-01-06 18:15 #225 cancelled Clean Clean
2025-09-06 00:00 #103 completed Clean Clean
2025-08-17 00:01 #83 cancelled Clean Clean
2025-07-13 21:37 #48 completed Biased Biased
2025-07-12 23:44 #41 cancelled Biased Biased

Flagged Code Snippets

    set CLUSTERNAME=
    set PASSWORD=
    
    $resp = Invoke-WebRequest -Uri "https://$clustername.azurehdinsight.net/templeton/v1/status" `
        -Credential $creds `
        -UseBasicParsing
    $resp.Content
    
    curl -u admin:%PASSWORD% -d user.name=admin ^
    -d jar=/example/jars/hadoop-mapreduce-examples.jar ^
    -d class=wordcount -d arg=/example/data/gutenberg/davinci.txt -d arg=/example/data/output ^
    https://%CLUSTERNAME%.azurehdinsight.net/templeton/v1/mapreduce/jar | ^
    C:\HDI\jq-win64.exe .id
    
    set JOBID=job_1415651640909_0026

    curl -G -u admin:%PASSWORD% -d user.name=admin https://%CLUSTERNAME%.azurehdinsight.net/templeton/v1/jobs/%JOBID% | ^
    C:\HDI\jq-win64.exe .status.state
    
    $clusterName="CLUSTERNAME"
    $creds = Get-Credential -UserName admin -Message "Enter the cluster login password"
    
    $reqParams = @{}
    $reqParams."user.name" = "admin"
    $reqParams.jar = "/example/jars/hadoop-mapreduce-examples.jar"
    $reqParams.class = "wordcount"
    $reqParams.arg = @()
    $reqParams.arg += "/example/data/gutenberg/davinci.txt"
    $reqparams.arg += "/example/data/output"
    $resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/templeton/v1/mapreduce/jar" `
       -Credential $creds `
       -Body $reqParams `
       -Method POST `
       -UseBasicParsing
    $jobID = (ConvertFrom-Json $resp.Content).id
    $jobID
    
    $reqParams=@{"user.name"="admin"}
    $resp = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/templeton/v1/jobs/$jobID" `
       -Credential $creds `
       -Body $reqParams `
       -UseBasicParsing

    # ConvertFrom-JSON can't handle duplicate names with different case
    # So change one to prevent the error
    $fixDup=$resp.Content.Replace("jobID","job_ID")
    (ConvertFrom-Json $fixDup).status.state