Document Details
Transform Data Using Hadoop Streaming
❌
This page contains Windows bias
About This Page
This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
Bias Analysis
Bias Types:
⚠️
missing_linux_example
⚠️
windows_tools
Summary:
The documentation provides only Windows-centric examples for Hadoop Streaming activities, notably referencing '.exe' executables (e.g., MyMapper.exe, MyReducer.exe) as mapper and reducer programs. There are no examples or mentions of Linux/Unix-style scripts (such as Python, Bash, or shell scripts), which are more common in Hadoop environments. This could mislead users into thinking only Windows executables are supported, omitting the typical Linux usage patterns.
Recommendations:
- Add examples using Linux/Unix-style scripts (e.g., 'MyMapper.py', 'MyReducer.sh') as mapper and reducer programs in the JSON sample.
- Explicitly state that both Windows executables and Linux scripts are supported, and clarify any platform-specific requirements.
- Provide guidance or links on how to create and upload Linux-compatible scripts for Hadoop Streaming jobs.
- Ensure that documentation does not imply a Windows-only workflow by balancing references to both Windows and Linux tools and patterns.
Create pull request
Scan History
| Date | Scan ID | Status | Bias Status |
|---|---|---|---|
| 2025-07-12 23:44 | #41 | in_progress | ❌ Biased |
| 2025-07-12 00:58 | #8 | cancelled | ✅ Clean |
| 2025-07-10 05:06 | #7 | processing | ✅ Clean |
Flagged Code Snippets
{
"name": "Streaming Activity",
"description": "Description",
"type": "HDInsightStreaming",
"linkedServiceName": {
"referenceName": "MyHDInsightLinkedService",
"type": "LinkedServiceReference"
},
"typeProperties": {
"mapper": "MyMapper.exe",
"reducer": "MyReducer.exe",
"combiner": "MyCombiner.exe",
"fileLinkedService": {
"referenceName": "MyAzureStorageLinkedService",
"type": "LinkedServiceReference"
},
"filePaths": [
"<containername>/example/apps/MyMapper.exe",
"<containername>/example/apps/MyReducer.exe",
"<containername>/example/apps/MyCombiner.exe"
],
"input": "wasb://<containername>@<accountname>.blob.core.windows.net/example/input/MapperInput.txt",
"output": "wasb://<containername>@<accountname>.blob.core.windows.net/example/output/ReducerOutput.txt",
"commandEnvironment": [
"CmdEnvVarName=CmdEnvVarValue"
],
"getDebugInfo": "Failure",
"arguments": [
"SampleHadoopJobArgument1"
],
"defines": {
"param1": "param1Value"
}
}
}