How to Retrieve and Use Secret Variables in Azure DevOps

Why Retrieve Secrets in Azure DevOps?

Why Retrieve Secrets in Azure DevOps?

When you’re building and deploying applications, you often need to handle sensitive values such as passwords, access tokens, and connection strings. Committing them into your repository is risky — anyone with repo access could see them. Instead, Azure DevOps lets you store secrets securely in a variable group (or Key Vault) and reference them in your pipeline at runtime.

The pipeline below demonstrates this approach:

1. Link a variable group (testbynhaila) that contains both secret (DatabasePassword, ApiToken) and non-secret (test) variables.
2. Generate a config file with placeholders.
3. Use sed to substitute the placeholders for real secrets from your variable group.
4. Debug by listing directory contents and showing how secrets appear as *** in logs.
5. Publish artifacts, so you can download or inspect the final config file (though it will contain the real secret values unmasked if you open the artifact after the pipeline).

Following this approach ensures that you can manage credentials in a secure, scalable, and auditable manner — without ever exposing them plainly in your repository or logs.

The Pipeline Example

trigger: none
pr: none

# 1) Reference a variable group named 'testbynhaila'.
#    This group contains three variables: 'test' (not secret), 'DatabasePassword' (secret), and 'ApiToken' (secret).
variables:
- group: testbynhaila

stages:
- stage: Deploy
  displayName: "Demonstrate Key Vault Secrets"
  jobs:
  - job: ShowSecrets
    displayName: "Generate and Replace"
    pool:
      vmImage: 'ubuntu-latest'

    steps:
    # Step A: Generate a placeholder config file
    - script: |
        echo "=== Step 1: Generate configfile.txt ==="
        # We'll simulate a config file with placeholders in uppercase for clarity:
        # __DATABASE_PASSWORD__ and __API_TOKEN__ will be replaced later by their real secret values.
        echo "DB_PASSWORD=__DATABASE_PASSWORD__" > $(Build.SourcesDirectory)/configfile.txt
        echo "API_TOKEN=__API_TOKEN__" >> $(Build.SourcesDirectory)/configfile.txt

        echo ""
        echo "Showing the newly generated file:"
        cat configfile.txt
      displayName: "Generate configfile.txt"
    
    # Step B: Debug the variable values, then replace placeholders with sed
    - script: |
        echo "=== Step 2: Debug and show variable values ==="
        # $(test) is a non-secret variable, so it will be displayed plainly.
        # $(DatabasePassword) and $(ApiToken) are secret; Azure DevOps masks them as *** in logs.
        echo "variable test is: $(test)"
        echo "DatabasePassword is: $(DatabasePassword)"
        echo "API Token is: $(ApiToken)"
        
        echo ""
        echo "=== Step 3: Replace placeholders with sed ==="
        # We use sed to replace '__DATABASE_PASSWORD__' and '__API_TOKEN__' in the config file
        # with the actual secret values from the variable group.
        sed -i "s|__DATABASE_PASSWORD__|$(DatabasePassword)|g" configfile.txt
        sed -i "s|__API_TOKEN__|$(ApiToken)|g" configfile.txt

        echo ""
        echo "=== Step 4: Show final file ==="
        # When we cat the final file, the secrets have been injected, but if you print them here,
        # Azure DevOps will still mask the actual values with *** in the logs.
        cat $(Build.SourcesDirectory)/configfile.txt
      displayName: "Use sed to replace placeholders"

    # Step C: Another debug step to confirm file location and contents
    - task: Bash@3
      displayName: "Debug | cat files "
      inputs:
        targetType: 'inline'
        script: |
          pwd
          echo "Build.SourcesDirectory is: $(Build.SourcesDirectory)"
          echo "Listing directory:"
          ls -l $(Build.SourcesDirectory)/
          echo "Showing file contents:"
          cat $(Build.SourcesDirectory)/configfile.txt

    # Step D: Publish the entire Build.SourcesDirectory as a pipeline artifact
    #         This artifact can be downloaded or viewed in the pipeline run summary.
    - task: PublishPipelineArtifact@1
      displayName: Publication du rapport
      inputs:
        path: $(Build.SourcesDirectory)/
        artifact: CLTest

1. Creating a Variable Group

First, I set up a variable group in Pipelines → Library. I named it “testbynhaila” and added three variables:

• ApiToken (secret)
• DatabasePassword (secret)
• test (plain text = “ok”)

Retrieve and Use Secret Variables in Azure DevOps

The idea is to store your sensitive or secret values in a variable group so you can reuse them in multiple pipelines — and keep them out of source control.

Why Mark Variables as Secret?

When you mark a variable as secret, Azure DevOps automatically masks its real value in the build logs. This means you’ll see *** instead of the actual password or token, which helps prevent accidentally leaking credentials.

2. Referencing the Variable Group

In my YAML file, I start by referencing testbynhaila:

variables:
- group: testbynhaila   # Contains 'test', 'DatabasePassword', 'ApiToken'

This tells the pipeline to load all the variables from that group. Now I can call them with $(VariableName) anywhere in my pipeline steps.

3. Generating the File (configfile.txt)

I created a stage called Deploy with a single job, ShowSecrets, on an Ubuntu agent. The first step generates a simple text file, configfile.txt, which contains placeholders for my secret values:

- script: |
    echo "=== Step 1: Generate configfile.txt ==="
    # We'll simulate a config file with placeholders in uppercase for clarity:
    echo "DB_PASSWORD=__DATABASE_PASSWORD__" > $(Build.SourcesDirectory)/configfile.txt
    echo "API_TOKEN=__API_TOKEN__" >> $(Build.SourcesDirectory)/configfile.txt

    echo ""
    echo "Showing the newly generated file:"
    cat configfile.txt
  displayName: "Generate configfile.txt"

1. DB_PASSWORD=__DATABASE_PASSWORD__
2. API_TOKEN=__API_TOKEN__

These placeholders will later be replaced by the real secrets.

4. Debug and Show Variables

Next, I have a step that prints out the variable values to confirm the pipeline is indeed picking them up:

- script: |
    echo "=== Step 2: Debug and show variable values ==="
    echo "variable test is: $(test)"
    echo "DatabasePassword is: $(DatabasePassword)"
    echo "API Token is: $(ApiToken)"
    
    ...
  displayName: "Use sed to replace placeholders"

• $(test) displays ok, since it’s not secret.
• $(DatabasePassword) and $(ApiToken) will be shown as *** because Azure DevOps masks secrets automatically.

Why the *** in Logs?

Azure DevOps recognizes these secret variables by name and value. Whenever you try to print them, it replaces the real text with *** to keep them hidden. That’s how it protects sensitive data.

5. Replacing Placeholders with sed

In the same script step, I run:

sed -i "s|__DATABASE_PASSWORD__|$(DatabasePassword)|g" configfile.txt
sed -i "s|__API_TOKEN__|$(ApiToken)|g" configfile.txt

• The -i flag edits the file in place.
• s|__DATABASE_PASSWORD__|$(DatabasePassword)|g swaps out the placeholder __DATABASE_PASSWORD__ for the secret value from $(DatabasePassword)—but the logs will still show *** if you print it.

Finally, I display the contents to confirm it’s replaced:

cat $(Build.SourcesDirectory)/configfile.txt

Because the secrets are replaced in the file itself, if you read the file’s contents in the logs, you’ll see DB_PASSWORD=*** and API_TOKEN=***.

6. Debug Bash Task

After that, I add a Bash@3 task just to show more debugging info:

- task: Bash@3
  displayName: "Debug | cat files "
  inputs:
    targetType: 'inline'
    script: |
      pwd
      echo "Build.SourcesDirectory is: $(Build.SourcesDirectory)"
      echo "Listing directory:"
      ls -l $(Build.SourcesDirectory)/
      echo "Showing file contents:"
      cat $(Build.SourcesDirectory)/configfile.txt

This is purely for debugging — it prints the current working directory, shows the files in it, and then prints out configfile.txt.

7. Publishing the Artifact

Finally, I publish everything in $(Build.SourcesDirectory)/ as a pipeline artifact:

- task: PublishPipelineArtifact@1
  displayName: Publication du rapport
  inputs:
    path: $(Build.SourcesDirectory)/
    artifact: CLTest

This means I can see (and download) the entire directory — including configfile.txt—as a pipeline artifact after the run completes. Note that the secrets in that file are replaced with their real values, so be mindful that you’re bundling them in an artifact.

Conclusion

• Library Variable Groups (like my testbynhaila) are the easiest way to store and reuse secrets across pipelines.
• Runtime variable syntax ($(MySecret)) is how you reference them in YAML.
• Sed is a quick way to replace placeholders in config files with your secrets at runtime.
• Azure DevOps automatically masks secret values in logs, showing *** instead of the real data.

This pipeline demonstrates the entire process: generating a file, replacing placeholders with secrets, debugging directory contents, and finally publishing everything as an artifact. It’s a great starting point for any scenario where you need to inject sensitive configuration into your build or deployment steps — without exposing those credentials in your source code!