Document Details

Transform Data Using Custom Activity
Home / Scan #2 / Transform Data Using Custom Activity
Sad Tux - Windows bias detected
This page contains Windows bias

About This Page

This page is part of the Azure documentation. It contains code examples and configuration instructions for working with Azure services.
GitHub View on GitHub 📚 View on Microsoft Learn

Bias Analysis

Detected Bias Types
powershell_heavy
windows_first
missing_linux_example
windows_tools
Summary
The documentation demonstrates a clear Windows bias. All command-line and scripting examples use PowerShell and Windows command patterns (e.g., 'cmd /c', '.exe' applications). There are no Linux/bash equivalents provided for creating or managing Azure Batch accounts, running pipelines, or specifying commands for custom activities. The documentation references Windows-native tools and patterns (such as .exe files and cmd commands) without mentioning or demonstrating Linux alternatives, and Windows approaches are always presented first or exclusively.
Recommendations
  • Provide equivalent bash/CLI examples alongside PowerShell commands for managing Azure Batch and running pipelines.
  • Include Linux-native command examples (e.g., using '/bin/bash -c' or shell scripts) in the 'command' property for custom activities.
  • Demonstrate usage of Linux executables (e.g., .sh scripts, ELF binaries) in addition to .exe files.
  • Clarify in the documentation that both Windows and Linux pool nodes are supported, and specify how to target each OS.
  • Reference Azure CLI and REST API methods for account and pool management, not just PowerShell.
  • Add a section or table comparing Windows and Linux usage patterns for custom activities.
GitHub Create Pull Request

Scan History

Date Scan Status Result
2026-01-14 00:00 #250 in_progress Biased Biased
2026-01-13 00:00 #246 completed Biased Biased
2026-01-11 00:00 #240 completed Biased Biased
2026-01-10 00:00 #237 completed Biased Biased
2026-01-09 00:34 #234 completed Biased Biased
2026-01-08 00:53 #231 completed Biased Biased
2026-01-06 18:15 #225 cancelled Clean Clean
2025-09-08 00:00 #105 completed Clean Clean
2025-08-17 00:01 #83 cancelled Clean Clean
2025-07-13 21:37 #48 completed Clean Clean
2025-07-09 13:09 #3 cancelled Clean Clean
2025-07-08 04:23 #2 cancelled Biased Biased

Flagged Code Snippets

$runId = Invoke-AzDataFactoryV2Pipeline -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineName $pipelineName

{
  "name": "MyCustomActivityPipeline",
  "properties": {
    "description": "Custom activity sample",
    "activities": [{
      "type": "Custom",
      "name": "MyCustomActivity",
      "linkedServiceName": {
        "referenceName": "AzureBatchLinkedService",
        "type": "LinkedServiceReference"
      },
      "typeProperties": {
        "command": "helloworld.exe",
        "folderPath": "customactv2/helloworld",
        "resourceLinkedService": {
          "referenceName": "StorageLinkedService",
          "type": "LinkedServiceReference"
        }
      }
    }]
  }
}

{
  "name": "MyCustomActivity",
  "properties": {
    "description": "Custom activity sample",
    "activities": [{
      "type": "Custom",
      "name": "MyCustomActivity",
      "linkedServiceName": {
        "referenceName": "AzureBatchLinkedService",
        "type": "LinkedServiceReference"
      },
      "typeProperties": {
        "command": "cmd /c echo hello world"
      }
    }]
  }
}

{
  "name": "MyCustomActivityPipeline",
  "properties": {
    "description": "Custom activity sample",
    "activities": [{
      "type": "Custom",
      "name": "MyCustomActivity",
      "linkedServiceName": {
        "referenceName": "AzureBatchLinkedService",
        "type": "LinkedServiceReference"
      },
      "typeProperties": {
        "command": "SampleApp.exe",
        "folderPath": "customactv2/SampleApp",
        "resourceLinkedService": {
          "referenceName": "StorageLinkedService",
          "type": "LinkedServiceReference"
        },
        "referenceObjects": {
          "linkedServices": [{
            "referenceName": "AzureBatchLinkedService",
            "type": "LinkedServiceReference"
          }]
        },
        "extendedProperties": {          
          "connectionString": {
            "type": "SecureString",
            "value": "aSampleSecureString"
          },
          "PropertyBagPropertyName1": "PropertyBagValue1",
          "propertyBagPropertyName2": "PropertyBagValue2",
          "dateTime1": "2015-04-12T12:13:14Z"
        }
      }
    }]
  }
}

while ($True) {
    $result = Get-AzDataFactoryV2ActivityRun -DataFactoryName $dataFactoryName -ResourceGroupName $resourceGroupName -PipelineRunId $runId -RunStartedAfter (Get-Date).AddMinutes(-30) -RunStartedBefore (Get-Date).AddMinutes(30)

    if(!$result) {
        Write-Host "Waiting for pipeline to start..." -foregroundcolor "Yellow"
    }
    elseif (($result | Where-Object { $_.Status -eq "InProgress" } | Measure-Object).count -ne 0) {
        Write-Host "Pipeline run status: In Progress" -foregroundcolor "Yellow"
    }
    else {
        Write-Host "Pipeline '"$pipelineName"' run finished. Result:" -foregroundcolor "Yellow"
        $result
        break
    }
    ($result | Format-List | Out-String)
    Start-Sleep -Seconds 15
}

Write-Host "Activity `Output` section:" -foregroundcolor "Yellow"
$result.Output -join "`r`n"

Write-Host "Activity `Error` section:" -foregroundcolor "Yellow"
$result.Error -join "`r`n"

startingNumberOfVMs = 1;
maxNumberofVMs = 25;
pendingTaskSamplePercent = $PendingTasks.GetSamplePercent(180 * TimeInterval_Second);
pendingTaskSamples = pendingTaskSamplePercent < 70 ? startingNumberOfVMs : avg($PendingTasks.GetSample(180 * TimeInterval_Second));
$TargetDedicated=min(maxNumberofVMs,pendingTaskSamples);