Document Details
Tutorial Transform Data Spark Powershell
❌
This page contains Windows bias
About This Page
This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
Bias Analysis
Bias Types:
⚠️
powershell_heavy
⚠️
windows_first
⚠️
missing_linux_example
⚠️
windows_tools
Summary:
The documentation is heavily biased towards Windows and PowerShell usage. All command-line instructions are provided exclusively using Azure PowerShell, with no mention of Azure CLI, Bash, or cross-platform alternatives. The prerequisite section specifically requires Azure PowerShell, and all deployment and monitoring steps are shown only with PowerShell scripts. There are no examples or guidance for Linux or macOS users, nor is there mention of how to perform these tasks using non-Windows tools.
Recommendations:
- Add equivalent instructions using Azure CLI, which is cross-platform and works natively on Linux, macOS, and Windows.
- Provide Bash shell script examples for Linux/macOS users alongside PowerShell examples.
- In the prerequisites, mention both Azure PowerShell and Azure CLI as supported options, with links to installation guides for each.
- Clearly indicate in each step whether the instructions are for Windows/PowerShell or for Linux/Bash, and provide both where possible.
- Consider including a table or section comparing the commands in PowerShell and Bash/Azure CLI to help users choose the best fit for their environment.
- Review the use of terms like 'launch PowerShell' and replace with more inclusive language such as 'open your terminal or command prompt'.
Create pull request
Scan History
| Date | Scan ID | Status | Bias Status |
|---|---|---|---|
| 2025-09-08 00:00 | #105 | completed | ✅ Clean |
| 2025-08-17 00:01 | #83 | in_progress | ✅ Clean |
| 2025-07-13 21:37 | #48 | completed | ✅ Clean |
| 2025-07-09 13:09 | #3 | cancelled | ✅ Clean |
| 2025-07-08 04:23 | #2 | cancelled | ❌ Biased |
Flagged Code Snippets
$df
$runId = Invoke-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineName $pipelineName
$pipelineName = "MySparkOnDemandPipeline" # Name of the pipeline
Connect-AzAccount
Get-AzSubscription
Select-AzSubscription -SubscriptionId "<SubscriptionId>"
New-AzResourceGroup -Name $resourceGroupName -Location "East Us"
$df = Set-AzDataFactoryV2 -Location EastUS -Name $dataFactoryName -ResourceGroupName $resourceGroupName
Set-AzDataFactoryV2LinkedService -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name "MyStorageLinkedService" -File "MyStorageLinkedService.json"
Set-AzDataFactoryV2LinkedService -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name "MyOnDemandSparkLinkedService" -File "MyOnDemandSparkLinkedService.json"
Set-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name $pipelineName -File "MySparkOnDemandPipeline.json"
while ($True) {
$result = Get-AzDataFactoryV2ActivityRun -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineRunId $runId -RunStartedAfter (Get-Date).AddMinutes(-30) -RunStartedBefore (Get-Date).AddMinutes(30)
if(!$result) {
Write-Host "Waiting for pipeline to start..." -foregroundcolor "Yellow"
}
elseif (($result | Where-Object { $_.Status -eq "InProgress" } | Measure-Object).count -ne 0) {
Write-Host "Pipeline run status: In Progress" -foregroundcolor "Yellow"
}
else {
Write-Host "Pipeline '"$pipelineName"' run finished. Result:" -foregroundcolor "Yellow"
$result
break
}
($result | Format-List | Out-String)
Start-Sleep -Seconds 15
}
Write-Host "Activity `Output` section:" -foregroundcolor "Yellow"
$result.Output -join "`r`n"
Write-Host "Activity `Error` section:" -foregroundcolor "Yellow"
$result.Error -join "`r`n"
$resourceGroupName = "ADFTutorialResourceGroup"
$dataFactoryName = "MyDataFactory09102017"