Document Details
Transform Data Using Hadoop Streaming
❌
This page contains Windows bias
About This Page
This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
Bias Analysis
Bias Types:
⚠️
missing_linux_example
⚠️
windows_tools
Summary:
The documentation page provides only Windows-centric examples for Hadoop Streaming activities, specifically referencing '.exe' executables (e.g., 'MyMapper.exe', 'MyReducer.exe') in both the JSON sample and file paths. There are no examples or mentions of Linux/Unix-style scripts (e.g., Python, Bash, or shell scripts), which are more common in Hadoop Streaming scenarios. The documentation implicitly assumes Windows tools and omits Linux equivalents.
Recommendations:
- Provide parallel examples using Linux/Unix-style scripts (e.g., 'MyMapper.py', 'MyReducer.sh') in the JSON sample and file paths.
- Explicitly mention that both Windows executables and Linux scripts are supported, and clarify any platform-specific considerations.
- Include a section or note on how to prepare and upload Linux-based mapper and reducer scripts to Azure Storage.
- Balance the documentation by showing both Windows and Linux usage patterns, especially in code and configuration samples.
Create pull request
Scan History
| Date | Scan ID | Status | Bias Status |
|---|---|---|---|
| 2025-08-17 00:01 | #83 | in_progress | ✅ Clean |
| 2025-07-13 21:37 | #48 | completed | ❌ Biased |
| 2025-07-09 13:09 | #3 | cancelled | ✅ Clean |
| 2025-07-08 04:23 | #2 | cancelled | ❌ Biased |
Flagged Code Snippets
{
"name": "Streaming Activity",
"description": "Description",
"type": "HDInsightStreaming",
"linkedServiceName": {
"referenceName": "MyHDInsightLinkedService",
"type": "LinkedServiceReference"
},
"typeProperties": {
"mapper": "MyMapper.exe",
"reducer": "MyReducer.exe",
"combiner": "MyCombiner.exe",
"fileLinkedService": {
"referenceName": "MyAzureStorageLinkedService",
"type": "LinkedServiceReference"
},
"filePaths": [
"<containername>/example/apps/MyMapper.exe",
"<containername>/example/apps/MyReducer.exe",
"<containername>/example/apps/MyCombiner.exe"
],
"input": "wasb://<containername>@<accountname>.blob.core.windows.net/example/input/MapperInput.txt",
"output": "wasb://<containername>@<accountname>.blob.core.windows.net/example/output/ReducerOutput.txt",
"commandEnvironment": [
"CmdEnvVarName=CmdEnvVarValue"
],
"getDebugInfo": "Failure",
"arguments": [
"SampleHadoopJobArgument1"
],
"defines": {
"param1": "param1Value"
}
}
}