Document Details

Tutorial Transform Data Spark Powershell
Home / Scan #41 / Tutorial Transform Data Spark Powershell
Sad Tux - Windows bias detected
This page contains Windows bias

About This Page

This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
GitHub View on GitHub 📚 View on Microsoft Learn

Bias Analysis

Detected Bias Types
powershell_heavy
windows_tools
missing_linux_example
windows_first
Summary
The documentation demonstrates a strong bias toward Windows and PowerShell. All command-line examples are provided exclusively in PowerShell, with no mention of Bash, Azure CLI, or cross-platform alternatives. The tutorial assumes the use of Azure PowerShell throughout, and there are no instructions or guidance for users on Linux or macOS. The prerequisite section only references Azure PowerShell, and the workflow is tightly coupled to Windows-centric tooling and patterns.
Recommendations
  • Provide equivalent Azure CLI (az) commands alongside PowerShell for all deployment and management steps, as Azure CLI is cross-platform and widely used on Linux and macOS.
  • Explicitly mention that the tutorial can be followed on Linux/macOS, and provide guidance for setting up the environment on those platforms.
  • Add Bash shell script examples for key steps, such as uploading files to Azure Storage (using az storage blob upload or similar), and deploying resources.
  • In the prerequisites, include instructions for installing Azure CLI and/or using the Azure Portal as alternatives to PowerShell.
  • Where PowerShell is referenced, clarify that it is available on all platforms, but also offer alternative command-line approaches.
  • Consider restructuring the tutorial to present cross-platform or platform-neutral approaches first, or at least in parallel with Windows-specific instructions.
GitHub Create Pull Request

Scan History

Date Scan Status Result
2025-07-12 23:44 #41 cancelled Biased Biased
2025-07-12 00:58 #8 cancelled Clean Clean
2025-07-10 05:06 #7 processing Clean Clean

Flagged Code Snippets

    $pipelineName = "MySparkOnDemandPipeline" # Name of the pipeline
    
    Connect-AzAccount
    
    Get-AzSubscription
    
    Select-AzSubscription -SubscriptionId "<SubscriptionId>"    
    
     $df = Set-AzDataFactoryV2 -Location EastUS -Name $dataFactoryName -ResourceGroupName $resourceGroupName
    
    $df
    
    Set-AzDataFactoryV2LinkedService -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name "MyStorageLinkedService" -File "MyStorageLinkedService.json"
    
    Set-AzDataFactoryV2LinkedService -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name "MyOnDemandSparkLinkedService" -File "MyOnDemandSparkLinkedService.json"
    
    Set-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -Name $pipelineName -File "MySparkOnDemandPipeline.json"
    
    $runId = Invoke-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineName $pipelineName  
    
    $resourceGroupName = "ADFTutorialResourceGroup" 
    
    $dataFactoryName = "MyDataFactory09102017"
    
    New-AzResourceGroup -Name $resourceGroupName -Location "East Us" 
    
    while ($True) {
        $result = Get-AzDataFactoryV2ActivityRun -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineRunId $runId -RunStartedAfter (Get-Date).AddMinutes(-30) -RunStartedBefore (Get-Date).AddMinutes(30)
    
        if(!$result) {
            Write-Host "Waiting for pipeline to start..." -foregroundcolor "Yellow"
        }
        elseif (($result | Where-Object { $_.Status -eq "InProgress" } | Measure-Object).count -ne 0) {
            Write-Host "Pipeline run status: In Progress" -foregroundcolor "Yellow"
        }
        else {
            Write-Host "Pipeline '"$pipelineName"' run finished. Result:" -foregroundcolor "Yellow"
            $result
            break
        }
        ($result | Format-List | Out-String)
        Start-Sleep -Seconds 15
    }

    Write-Host "Activity `Output` section:" -foregroundcolor "Yellow"
    $result.Output -join "`r`n"

    Write-Host "Activity `Error` section:" -foregroundcolor "Yellow"
    $result.Error -join "`r`n"