Using Tokenization (Token Replacement) for Builds/Releases in vNext/TFS 2015

Microsoft has provided support for doing token replacements into build artifacts. Since this is provided as a “utility” task there’s no bias in how it’s used but you’ll likely find it most useful from your release workflow.

Inexplicably, Microsoft doesn’t actually provide this with TFS. Rather, you need to use their marketplace to download and install it. Still, as far as boneheaded designs go, it’s not so bad.

There are three things that it allows you to do:

  1. Do string-replacements using the environment.
  2. Do string-replacements using the environment-specific dictionary from the configuration file.
  3. Do environment-specific XPath replacements.

Note that:

  1. (1) and (2) will expect to find tokens that look like “[A-Za-z0-9._-]*” (e.g. “XYZ“, “ABC_DEF“, “GHI.JKL“, “MNO-PQR“), remove the underscores on the margins, and translate the periods to underscores.
  2. (2) and (3) are only possible if you provide a configuration file and if the source file is XML (obviously).

Installation

Download the Release Management Utility tasks extension (as a VSIX file). Install it using the instructions from my other post. You’ll now see a “Tokenize with XPath/Regular expressions” utility task.

For reference, this is the original project URL:

Extension-UtilitiesPack

Usage

The configuration file looks like:

{
   "Default Environment":{
      "CustomVariables":{
         "Token2":"value_from_custom2",
         "Token3":"value_from_custom3"
      },
      "ConfigChanges":[
         {
            "KeyName":"/configuration/appSettings/add[@key='TestKey1']",
            "Attribute":"value",
            "Value":"value_from_xpath"
         }
      ]
   }
}

If we’re going to do token-replacements over an App.settings file, the source file might look like:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>
      <add key="TestKey1" value="__Token1__" />
      <add key="TestKey2" value="__Token2__" />
      <add key="TestKey3" value="__Token3__" />
      <add key="TestKey4" value="__Token4__" />
    </appSettings>
    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.2" />
    </startup>
</configuration>

Since TestKey4 is not otherwise defined, it will either be replaced with itself (no net difference) or the value for an environment-variable named “Token4” if defined.

In my test situation, my build definition’s “Copy Files” task is copying the contents (“**\*”) of “$(Build.SourcesDirectory)\TestConsoleApplication\bin\Release” to “$(Build.ArtifactStagingDirectory)”. This is an example of the arguments to the “Tokenize with XPath/Regular expressions” task that I’ve added in my release-definition:

  • Source filename: “$(Build.ArtifactStagingDirectory)\drop\App.config.template”
  • Destination filename: “$(Build.ArtifactStagingDirectory)\drop\TestConsoleApplication.exe.config”
  • Configuration Json filename: “$(Build.ArtifactStagingDirectory)\drop\TokenEnvironmentConfig.json”

Personally, I’d prefer that parsing this be a little stricter: fail if can’t parse as JSON, fail if any tokens can not be resolved, and optionally allow it to fail if any XPath specifications don’t match anything.

Additional Notes

  • The RELEASE_ENVIRONMENTNAME variable is usually defined and used to determine the current environment. However, if it isn’t defined then the default environment searched in the configuration file is “default”.

  • The CustomVariables and ConfigChanges blocks don’t have to exist in the configuration file (won’t fail validation). What is there will be used.

  • For debugging, I’ve had issues enabling logging verbosity in PowerShell (with VerbosePreference). It would be easier adjusting the tasks\Tokenizer\x.y.z\tokenize.ps1 module in your build-agent’s directory to change all the instances of “Write-Verbose” to “Write-Host”. The module provides a lot of useful messaging if you can get it to show.

  • In the current version of the task, it provides a PowerShell 3 script in addition to the original PS script. I get the following message on my system: ##[error]Index operation failed; the array index evaluated to null. I downloaded the source-code to the VSIX that is available in the Marketplace, adjusted it to not use the PS3 version of the script, repackaged the VSIX, uploaded it to my on-premise TFS, and, bam, the token-replacement worked perfectly. The PowerShell 3 script is broken.

Advertisements