Document Details
Tutorial Transform Data Spark Powershell
❌
This page contains Windows bias
About This Page
This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
Bias Analysis
Bias Types:
⚠️
powershell_heavy
⚠️
windows_tools
⚠️
missing_linux_example
⚠️
windows_first
Summary:
The documentation demonstrates a strong bias toward Windows and PowerShell. All command-line examples are provided exclusively in PowerShell, with no mention of Bash, Azure CLI, or cross-platform alternatives. The tutorial assumes the use of Azure PowerShell throughout, and there are no instructions or guidance for users on Linux or macOS. The prerequisite section only references Azure PowerShell, and the workflow is tightly coupled to Windows-centric tooling and patterns.
Recommendations:
- Provide equivalent Azure CLI (az) commands alongside PowerShell for all deployment and management steps, as Azure CLI is cross-platform and widely used on Linux and macOS.
- Explicitly mention that the tutorial can be followed on Linux/macOS, and provide guidance for setting up the environment on those platforms.
- Add Bash shell script examples for key steps, such as uploading files to Azure Storage (using az storage blob upload or similar), and deploying resources.
- In the prerequisites, include instructions for installing Azure CLI and/or using the Azure Portal as alternatives to PowerShell.
- Where PowerShell is referenced, clarify that it is available on all platforms, but also offer alternative command-line approaches.
- Consider restructuring the tutorial to present cross-platform or platform-neutral approaches first, or at least in parallel with Windows-specific instructions.
Create pull request
Scan History
| Date | Scan ID | Status | Bias Status |
|---|---|---|---|
| 2025-07-12 23:44 | #41 | in_progress | ❌ Biased |
| 2025-07-12 00:58 | #8 | cancelled | ✅ Clean |
| 2025-07-10 05:06 | #7 | processing | ✅ Clean |
Flagged Code Snippets
$pipelineName = "MySparkOnDemandPipeline" # Name of the pipeline
Connect-AzAccount
Get-AzSubscription
Select-AzSubscription -SubscriptionId "<SubscriptionId>"
$df = Set-AzDataFactoryV2 -Location EastUS -Name $dataFactoryName -ResourceGroupName $resourceGroupName
$df
Set-AzDataFactoryV2LinkedService -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name "MyStorageLinkedService" -File "MyStorageLinkedService.json"
Set-AzDataFactoryV2LinkedService -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name "MyOnDemandSparkLinkedService" -File "MyOnDemandSparkLinkedService.json"
Set-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name $pipelineName -File "MySparkOnDemandPipeline.json"
$runId = Invoke-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineName $pipelineName
$resourceGroupName = "ADFTutorialResourceGroup"
$dataFactoryName = "MyDataFactory09102017"
New-AzResourceGroup -Name $resourceGroupName -Location "East Us"
while ($True) {
$result = Get-AzDataFactoryV2ActivityRun -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineRunId $runId -RunStartedAfter (Get-Date).AddMinutes(-30) -RunStartedBefore (Get-Date).AddMinutes(30)
if(!$result) {
Write-Host "Waiting for pipeline to start..." -foregroundcolor "Yellow"
}
elseif (($result | Where-Object { $_.Status -eq "InProgress" } | Measure-Object).count -ne 0) {
Write-Host "Pipeline run status: In Progress" -foregroundcolor "Yellow"
}
else {
Write-Host "Pipeline '"$pipelineName"' run finished. Result:" -foregroundcolor "Yellow"
$result
break
}
($result | Format-List | Out-String)
Start-Sleep -Seconds 15
}
Write-Host "Activity `Output` section:" -foregroundcolor "Yellow"
$result.Output -join "`r`n"
Write-Host "Activity `Error` section:" -foregroundcolor "Yellow"
$result.Error -join "`r`n"