“Invalid Credentials” When Using TFX Tool to Login to TFS

The cross-platform TFX tool is, among other things, what you use to create new boiler-plate VSIX extensions (to populate with your own information) as well as to upload individual Release Management (vNext) tasks to TFS 2015. Previously, I’ve logged in using a fully-qualified UNC-formatted username (\domain\dustin.oprea) but this started, suddenly, not working for me. I have not applied any noteworthy updates. I did recently have to change my password (per group policy), but that may or may not have affected it.

As my web-based TFS UI indicated that I was logged-in as “DOMAIN\dustin.oprea”, I unsuccessfully tried logging in as that at the command-line. Eventually, I opened up a private session within the browser in order to log-in using that same username but the UI refused to log me out (even though it was a private session, I was always logged-in). I tried clearing web credential in Windows, but, alas, nothing was stored there (if that could’ve even helped).

I also restarted the system just in case there was credentials-related caching in the system, in or outside of TFS’ application pool.

It turns out that now logging-in with a fully-qualified email-account as my username works just fine.

For example:

C:\>tfx login --auth-type basic --service-url http://localhost:8181/tfs/DefaultCollection --username dustin@email.com
TFS Cross Platform Command Line Interface v0.3.20
Copyright Microsoft Corporation
> Password:
Logged in successfully

Thanks, TFS. Do you think you might unify how users are logging-in both at the command-line and in the UI? Maybe?

Thanks to Could Not Connect, Invalid Credentials, Bad Username/ID/Password for drawing my eye to this potential option.

Odd “find” Behavior Under Windows: “paths must precede expression”

I noticed a problem in the “find” version being delivered in the currently available GnuWin32. I have csproj files strewn throughout a tree. Most of the sln files are in the root. I can search for csproj‘s but I get an error with sln‘s.

D:\sc_root>c:\gnu\find . -name "*.csproj"
<...>
^C

D:\sc_root>c:\gnu\find . -name "*.sln"
c:\gnu\find: paths must precede expression
Usage: c:\gnu\find [-H] [-L] [-P] [path...] [expression]

Of course I’m already using quotes to prevent premature expansion but it’s not making a difference.

It looks like this might be a recent bug. If you sabotage Windows’ ability to expand before find can process the expression by using something Windows can’t understand, then it’ll work just fine:

c:\sc_root>c:\gnu\find . -name "*[.]sln"
<...>

Special thanks for this SO post.

Collections and $, @, % Variable References in MSBuild

A very useful and illuminating article:

MSBuild: PropertyGroup, ItemGroup, Item Metadata, and crazy $@% syntax

It also expertly articulates a common feeling:

It brings me great joy that the title of this blog post is both technically accurate and evokes the type of comic-book language I use when trying to figure out MSBuild variable syntax.

It expounds on how to express discrete items, lists, and dictionaries as well as how to refer to them; even implicitly looping over collections.

At the high-level:

  • $: Refers to scalar values. Renders an empty-string for collections.
  • @: Can be used to pass collections or flatten a collection into semicolon-delimited values if used in a scalar context (like embedding in a message task). Note that the individual values will first be normalized by expanding by semicolon and then be flattened by semicolon. So, extraneous semicolons will in fact be removed.
  • %: Expand the given task into N-tasks for each item in a collection.

It’s worth mentioning that %(VariableName) will not expand in such a way that it would result in doing the same thing twice.

For example, I have a list of two assemblies and two XML files of the same two names and would like to print them:

<Target Name="BeforeBuild">
  <ItemGroup>
    <File Include="$(MSBuildProjectDirectory)\AncillaryAssemblies\*.*" />
  </ItemGroup>
  <Message Text="Inner file: %(File.Filename)" />
</Target>

Output:

1>  Inner file: EntityFramework
1>  Inner file: EntityFramework.SqlServer

However, if I were to print the extensions, as well:

<Target Name="BeforeBuild">
  <ItemGroup>
    <File Include="$(MSBuildProjectDirectory)\AncillaryAssemblies\*.*" />
  </ItemGroup>
  <Message Text="Inner file: %(File.Filename) %(File.Extension)" />
</Target>

Output:

1>  Inner file: EntityFramework .dll
1>  Inner file: EntityFramework.SqlServer .dll
1>  Inner file: EntityFramework.SqlServer .xml
1>  Inner file: EntityFramework .xml

Something to keep in mind.

Publishing with ClickOnce via Visual Studio and TFS 2015 (vNext)

ClickOnce equips your published application to be able to check for updates and to, optionally, do the update automatically. See this post for more information.

To invoke this functionality from either the command-line or your build definition/process, you need to pass some additional project parameters to MSBuild.

ApplicationVersion (e.g. 1.0.1.6)
InstallUrl (e.g. http://clickonce.localhost/consoletestfrombuild/)
UpdateUrl (e.g. http://clickonce.localhost/consoletestfrombuild/)
PublishUrl (e.g. http://clickonce.localhost/consoletestfrombuild/)
UpdateEnabled (e.g. true)
UpdateMode (e.g. Foreground)
ProductName (e.g. TestConsoleApplication)

There is some disagreement as to whether the “ProductName” parameter is necessary.

The corresponding set of MSBuild arguments using the example values:

/target:publish /p:ApplicationVersion=1.0.1.6 /p:InstallUrl=http://clickonce.localhost/consoletestfrombuild/ /p:UpdateUrl=http://clickonce.localhost/consoletestfrombuild/ /p:PublishUrl=http://clickonce.localhost/consoletestfrombuild/ /p:UpdateEnabled=true /p:UpdateMode=Foreground /p:ProductName=TestConsoleApplication

If your polling locations are going to be UNCs (“\\path\\file”) rather than URLs then you’ll have to pass an additional argument:

/p:IsWebBootstrapper=false

To configure this under a TFS 2015 build-definition, simply add the parameters to the “MSBuild Arguments” field:

ClickOnce Settings in TFS 2015

References

Building ClickOnce Applications from the Command Line (Microsoft)

Deploying a click-once application with build vNext in Azure Blob Storage

Creating a ClickOnce Application in Visual Studio 2015

ClickOnce functionality allows you to create an application that checks for updates. It can do this before it starts (you’ll always run with the latest version, even if you will see the update-check when it starts) or after it starts (better for large updates). When you’re publishing, you can also specify the minimum acceptable version that is to be running, which will automatically do the update when in violation. It obviously follows that you can also specify whether the application is allowed to be run when offline (read: when it can’t check for updates).

Though most aspects of Microsoft development are unnecessarily complicated this is ridiculously easy.

Open project properties and go to the “Signing” panel:

ClickOnce - 1 - Certificate

Check the “Sign the ClickOnce manifests” checkbox. Click the “Create Test Certificate…” button and create a self-signed certificate (you can create an official one after you adopt and formalize the process):

ClickOnce - 1 - Certificate

Navigate to the “Publish” panel and fill in the path to push files and the UNC/URL that that directory can be accessed at for update polling and downloads:

ClickOnce - 2 - Paths and Updates

Click on the “Updates…” button and check “The application should check for updates”. Make sure to change where in the lifecycle the update should be performed if the default doesn’t work for you. Check “Specify a minimum required version for this application” and set that version if you want the update to be automatic. When you enable this for the first time, it will likely default to the version that is currently set to publish:

ClickOnce - 3 - Updates

Back in the top “Publish” panel, click on “Publish Now”. You may change the version factors of what you’re publishing, but, as the “Automatically increment revision with each publish” checkbox is checked by default, you shouldn’t have to:

ClickOnce - 4 - Publish

An Explorer window with the published files will probably be opened for you:

ClickOnce - 5 - Publish Files

Go ahead and click setup.exe to install. Forgive the unsigned-publisher security warning:

ClickOnce - 6 - Install - Prompt

You’ll also be presented with a security warning due to the self-signed certificate. Click on “More info” to expose the “Run anyway” button and then click it:

ClickOnce - 7 - Install - Self-Signed Certificate

Go ahead and do another publish (it’ll automatically push using a higher version than last time). Run the application. You’ll be prompted to update unless you changed the minimum-version in order to install automatically:

ClickOnce - 9 - Update Prompt

Using Tokenization (Token Replacement) for Builds/Releases in vNext/TFS 2015

Microsoft has provided support for doing token replacements into build artifacts. Since this is provided as a “utility” task there’s no bias in how it’s used but you’ll likely find it most useful from your release workflow.

Inexplicably, Microsoft doesn’t actually provide this with TFS. Rather, you need to use their marketplace to download and install it. Still, as far as boneheaded designs go, it’s not so bad.

There are three things that it allows you to do:

  1. Do string-replacements using the environment.
  2. Do string-replacements using the environment-specific dictionary from the configuration file.
  3. Do environment-specific XPath replacements.

Note that:

  1. (1) and (2) will expect to find tokens that look like “[A-Za-z0-9._-]*” (e.g. “XYZ“, “ABC_DEF“, “GHI.JKL“, “MNO-PQR“), remove the underscores on the margins, and translate the periods to underscores.
  2. (2) and (3) are only possible if you provide a configuration file and if the source file is XML (obviously).

Installation

Download the Release Management Utility tasks extension (as a VSIX file). Install it using the instructions from my other post. You’ll now see a “Tokenize with XPath/Regular expressions” utility task.

For reference, this is the original project URL:

Extension-UtilitiesPack

Usage

The configuration file looks like:

{
   "Default Environment":{
      "CustomVariables":{
         "Token2":"value_from_custom2",
         "Token3":"value_from_custom3"
      },
      "ConfigChanges":[
         {
            "KeyName":"/configuration/appSettings/add[@key='TestKey1']",
            "Attribute":"value",
            "Value":"value_from_xpath"
         }
      ]
   }
}

If we’re going to do token-replacements over an App.settings file, the source file might look like:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>
      <add key="TestKey1" value="__Token1__" />
      <add key="TestKey2" value="__Token2__" />
      <add key="TestKey3" value="__Token3__" />
      <add key="TestKey4" value="__Token4__" />
    </appSettings>
    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.2" />
    </startup>
</configuration>

Since TestKey4 is not otherwise defined, it will either be replaced with itself (no net difference) or the value for an environment-variable named “Token4” if defined.

In my test situation, my build definition’s “Copy Files” task is copying the contents (“**\*”) of “$(Build.SourcesDirectory)\TestConsoleApplication\bin\Release” to “$(Build.ArtifactStagingDirectory)”. This is an example of the arguments to the “Tokenize with XPath/Regular expressions” task that I’ve added in my release-definition:

  • Source filename: “$(Build.ArtifactStagingDirectory)\drop\App.config.template”
  • Destination filename: “$(Build.ArtifactStagingDirectory)\drop\TestConsoleApplication.exe.config”
  • Configuration Json filename: “$(Build.ArtifactStagingDirectory)\drop\TokenEnvironmentConfig.json”

Personally, I’d prefer that parsing this be a little stricter: fail if can’t parse as JSON, fail if any tokens can not be resolved, and optionally allow it to fail if any XPath specifications don’t match anything.

Additional Notes

  • The RELEASE_ENVIRONMENTNAME variable is usually defined and used to determine the current environment. However, if it isn’t defined then the default environment searched in the configuration file is “default”.

  • The CustomVariables and ConfigChanges blocks don’t have to exist in the configuration file (won’t fail validation). What is there will be used.

  • For debugging, I’ve had issues enabling logging verbosity in PowerShell (with VerbosePreference). It would be easier adjusting the tasks\Tokenizer\x.y.z\tokenize.ps1 module in your build-agent’s directory to change all the instances of “Write-Verbose” to “Write-Host”. The module provides a lot of useful messaging if you can get it to show.

  • In the current version of the task, it provides a PowerShell 3 script in addition to the original PS script. I get the following message on my system: ##[error]Index operation failed; the array index evaluated to null. I downloaded the source-code to the VSIX that is available in the Marketplace, adjusted it to not use the PS3 version of the script, repackaged the VSIX, uploaded it to my on-premise TFS, and, bam, the token-replacement worked perfectly. The PowerShell 3 script is broken.

TFS Release Management Build Agent Doesn’t Pickup Jobs

If your build tells you “Build Not started”, “Waiting for an available agent”, or “Waiting for an agent to be requested”, make sure that your agent is running (if as a service, then it’s called “VSO Agent”) as well as the “Visual Studio Team Foundation Background Job Agent” service. The latter feeds the former. In my case, I am using TFS 2015 and the latter appeared to be configured as “Automatic (Delayed Start)” but had to be configured as “Automatic” instead.

Uploading Release Management Tasks With a VSIX Extension Package

TFS 2015 lets you define build and release workflows comprised of discrete tasks. Although it provides a handful to you (build-specific, release-specific, general utility, etc..), you can also write your own “Extension Tasks” and upload them. I previously described how to use Microsoft’s tfx tool to upload a single task from the command-line.

You might also need to upload one or more tasks that have been packaged into a VSIX file (Microsoft’s Visual Studio Extension packaging format). These are essentially zip-files that have a manifest and one or more directories. In this case, those directories represent tasks (they each have a task.json file). Although you can probably do this from the command-line using the tfx tool, you can also use your TFS’s web UI to do it. Note that this process will need to be repeated for each collection that you’ll want the extension to be available in.

Click the “Shop” icon to open the corresponding menu and then click on “Browse of Extensions”:

tfsbrowseforextensions.png

Click on the “Manage extensions” button in the middle of the page. It’s not the same link/button as the previous one even if they have the same text:

release-management-extension-upload-3-manage-extensions-2.png

Click on “Upload New Extension”. Browse to the VSIX file and upload it.

release-management-extension-upload-4-extension-upload.png

After, you’re presented with a list of installed extensions:

release-management-extension-upload-5-extension-uploaded.png

Observe that the extension populated on to the previous local-management screen. Click on it:

release-management-extension-upload-6.png

Now, click on “Install”:

release-management-extension-upload-7.png

After installation, the new tasks will be available within the lists of available tasks.

Adding a Task to Your On-Premise TFS 2015 Server

TFS 2015 embeds previous vNext functionality to provide you workflow-driven, scriptable builds. This is based on establishing discrete tasks that can be reused between your build definitions and release definitions as relevant.

You can extend the basic set of tasks, which, though sufficient for very simple builds and releases, might quickly become insufficient. New tasks are called “extension tasks”. Discussing how to create one is out of scope for this post so we’re assuming that you already have a path that describes a task. At the very least, there will be a task.json file in it.

What comes next?

  1. Install Microsoft’s “tfx” tool.
  2. Added support for “Basic Authentication” to your on-premise TFS’ IIS server.
  3. Enable Basic Authentication on that IIS server.
  4. Login to your TFS using the tfx tool.
  5. Upload task.

Instructions

1. Install Microsoft’s “tfx” tool

This is a cross-platform project that requires NodeJS to be installed locally. The TFS-CLI project is hosted here: https://github.com/Microsoft/tfs-cli.

To install:

npm install -g tfx-cli

2. Add support for “Basic Authentication” to your on-premise TFS’ IIS server

On a workstation flavor of Windows (in my case, Windows 8.1):

  1. Go to “Programs and Features” -> “Turn Windows features on or off”.
  2. Navigate down through the tree of features: “Internet Information Services” -> “World Wide Web Services” -> “Security”.
  3. Enable “Basic Authentication” and click “OK”.

If you’re running a server flavor of Windows you’ll have to update the “Web Server” role instead:

(https://github.com/Microsoft/tfs-cli/blob/master/docs/configureBasicAuth.md)

windowsserverconfigurebasicauthfeature.png

Make sure to close and re-open IIS Manager afterward.

3. Enable Basic Authentication in IIS

Under the application, click “Authentication”, right-click on “Basic Authentication”, and click “Enable”. Make sure you enable “Basic Authentication” under the application, not the site. Setting it on the site will be ineffectual.

4. Login to your TFS using the tfx tool

C:\>tfx login --auth-type basic
TFS Cross Platform Command Line Interface v0.3.20
Copyright Microsoft Corporation
> Service URL: http://localhost:8181/tfs/DefaultCollection
> Username: yourdomain\dustin.oprea
> Password:
Logged in successfully

5. Upload task

I tested using the “XLD” task, here:

XL Deploy

Unzip and install by passing the directory path:

tfx build tasks upload --task-path xld

Now, if you go to your list of release tasks (where the XLD task ends-up), you’ll see it:

windowsserverconfigurebasicauthfeature.png