Sorting All Files in Tree by Modified-Time (in OS X)

This command had to be modified to run under OS X as it, and its heritage, don’t support the “-printf” parameter:

$ find . -print0 | xargs -0 stat -f '%m %N' | sort -k 1,1 -n

Tail of output in my situation:

1469286497 ./dev_import/data/160609_103423/.160609_103423.processing/style-color
1469286497 ./dev_import/data/160609_103423/.160609_103423.processing/upc
1469286541 ./bip/config/log.py
1469286566 ./dev/run_dev.sh
1469286597 ./dev_import/data/160609_103423/events.log.20160723-110946
1469286624 ./dev_import/data/160609_103423/.160609_103423.processing
1469286626 ./dev_import/data/160609_103423
1469286626 ./dev_import/data/160609_103423/events.log.20160723-111026

Using NuGet.Core to Get the Latest Version of a Package

Add the NuGet.Core package from NuGet and you’ll be in business. We use a NuGet config-file to get the one or more repositories that you might be using and hit them one at a time.

using NuGet;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Xml.XPath;

namespace LatestVersion
{
    class PackageNotFoundException : Exception
    {
        public string PackageName { get; private set; }

        public PackageNotFoundException(string packageName) : base(String.Format("Package [{0}] not found.", packageName))
        {
            PackageName = packageName;
        }
    }

    class NuGet
    {
        string nugetConfigFilepath;

        public NuGet(string nugetConfigFilepath)
        {
            this.nugetConfigFilepath = nugetConfigFilepath;
        }

        public IEnumerable<Tuple> GetSources()
        {
            XPathNavigator nav;
            XPathDocument docNav;
            string xPath;

            docNav = new XPathDocument(nugetConfigFilepath);
            nav = docNav.CreateNavigator();
            xPath = "/configuration/packageSources/add";

            foreach (XPathNavigator xpn in nav.Select(xPath))
            {
                string name = xpn.GetAttribute("key", "");
                string uri = xpn.GetAttribute("value", "");

                yield return new Tuple(name, uri);
            }
        }

        public SemanticVersion GetLatestVersion(string packageName)
        {
            foreach (Tuple source in GetSources())
            {
                string name = source.Item1;
                string uri = source.Item2;

                IPackageRepository repo = PackageRepositoryFactory.Default.CreateRepository(uri);

                // Passing NULL for `versionSpec` will return the latest version.
                IEnumerable packagesRaw = repo.FindPackages(packageName, null, false, false);
                IList packages = packagesRaw.ToList();

                if (packages.Count == 0)
                {
                    continue;
                }

                return packages[0].Version;
            }

            throw new PackageNotFoundException(packageName);
        }
    }
}

Best Argument-Processing for .NET/C#

After trying NDesk.Options and Fluent, I am nothing but impressed with CLAP (“Command-Line Auto-Parser”). It completely relies on reflection and parameter attributes (usually just one or two) to automatically marshal your values, assign defaults, enforce requiredness, and provide command-line documentation. It’s beautiful and, so far, flawless. Well done.

using CLAP;

namespace MyNamespace
{
    class Program
    {
        [Verb(IsDefault = true, Description = "Print the current version of the given package and, optionally, increment it.")]
        public void Version(
            [Description("Project path")]
            [Required]
            string projectPath,

            [Description("Package name")]
            [Required]
            string packageName,

            [Description("Base version to increment from (if lower than current, else use current)")]
            string baseVersion = null,

            [Description("Increment the version before returning")]
            bool increment = false
        )
        {
            // ...
        }
    }
}

If you don’t decorate with the “Required” attribute and don’t provide a default value the parameter will default to null. I explicitly set baseVersion to a default of null because I prefer being explicit.

Converting TFS 2015 Build Definition Names to Their Directory IDs

A TFS 2015 agent maintains a directory of build-space directories, where each corresponds to a single build definition. Each of these has a source-directory (“s”; usually automatically checked-out), binaries-directory (“b”; for intermediate binaries prior to publish-oriented cherry-picking), and artifact-staging directory (“a”; for artifacts to be published).

Not only is each of the build-space directories an integer with no clear mapping to the build-definition but this integer is different from one agent to the next. It turns out that there is a meta-information directory whose children are collection GUIDs. Furthermore, the children of those directories are additional directories with integer names. Each of these has a JSON file that contains build-definition information.

I quickly wrote a Python script to search for a given build definition and print the ID. The basic functionality is split into succinct, easily-callable methods for whatever other tasks you might have.

import logging
import os.path
import json

_LOGGER = logging.getLogger(__name__)


class Tfs(object):
    def __init__(self, agent_path):
        self.__agent_path = agent_path
        self.__srm_path = \
            os.path.join(
                self.__agent_path, 
                '_work', 
                'SourceRootMapping')

        _LOGGER.debug("Agent SRM path: [%s]", self.__srm_path)

    def collection_gen(self):
        for filename in os.listdir(self.__srm_path):
            filepath = os.path.join(self.__srm_path, filename)
            if os.path.isdir(filepath) is False:
                continue

            _LOGGER.debug("Collection GUID: [%s]", filename)
            yield filename, filepath

    def definition_gen(self, collection_path):
        for srm_id_raw in os.listdir(collection_path):
            _LOGGER.debug("SRM ID: [%s]", srm_id_raw)

            definition_info_filepath = \
                os.path.join(
                    collection_path, 
                    srm_id_raw, 
                    'SourceFolder.json')

            with open(definition_info_filepath) as f:
                yield json.load(f)

    def lookup_definition(self, definition_name):
        for collection_guid, collection_path in self.collection_gen():
            for definition in self.definition_gen(collection_path):
                current_definition_name = definition['definitionName']
                _LOGGER.debug("Definition: [%s]", current_definition_name)

                if current_definition_name != definition_name:
                    continue

                return definition

        raise ValueError("Could not find definition: {}".format(
                         definition_name))


def _configure_logging():
    sh = logging.StreamHandler()
    formatter = logging.Formatter('%(asctime)s [%(name)s %(levelname)s] %(message)s')
    sh.setFormatter(formatter)

    _LOGGER.addHandler(sh)
    _LOGGER.setLevel(logging.DEBUG)

if __name__ == '__main__':
    import argparse

    parser = argparse.ArgumentParser()

    parser.add_argument(
        'agent_path', 
        help="Agent path")

    parser.add_argument(
        'definition_name', 
        help="Build-definition name")

    args = parser.parse_args()

    _configure_logging()

    t = Tfs(args.agent_path)
    definition = t.lookup_definition(args.definition_name)

    print(definition['agent_builddirectory'])

Output:

D:\development\python>python tfs.py c:\tfs_build_agent ConsoleProject.Dev
2016-07-07 23:32:02,756 [__main__ DEBUG] Agent SRM path: [c:\tfs_build_agent\_work\SourceRootMapping]
2016-07-07 23:32:02,756 [__main__ DEBUG] Collection GUID: [2cf8d3cb-b8d4-49e1-bbdb-2aacf02f48c4]
2016-07-07 23:32:02,757 [__main__ DEBUG] SRM ID: [1]
2016-07-07 23:32:02,757 [__main__ DEBUG] Definition: [ConsoleProject.Dev]
1

Important Notes

  • The child directories of the SourceRootMapping directory will not exactly reflect the current build definitions. Some of them may represent build definitions that no longer exist.

  • One of the current work directories may be using an ID used by an old build-definition at some point in the past. So, if you are building a dictionary of work-directory IDs to build-definitions, you will have to use the build-definition’s ID or one of the timestamps described in the its JSON file to reconcile the latest build-definition to use that directory.

Searching for Specific Packages in NuGet

Package searches in NuGet hit the query API and the query API, by default, does a substring search. This obviously might get annoying when a simple string that you’re searching for might appear within any of the searchable characteristics of many different packages.

However, since this is the standard query interface, and you can pass more than just flat strings, you can also modify the query to do an exact search using the “PackageId” modifier:

C:\>nuget list zebus
Zebus 1.4.6
Zebus.Directory 1.2.10
Zebus.Directory.Cassandra 1.2.10
Zebus.Directory.Standalone 1.2.10
Zebus.Persistence 1.0.2
Zebus.Persistence.CQL 1.0.2
Zebus.Persistence.CQL.Testing 1.0.2
Zebus.Testing 1.4.6

C:\>nuget list PackageId:zebus
Zebus 1.4.6