Stopping the Go HTTP Server

It basically involves implementing our own net.Listener and monitoring a channel.

Stopping a listening HTTP Server in Go

Advertisements

Ridiculously Simple Google Hash Function

A Go implementation based on the paper “A Fast, Minimal Memory, Consistent Hash Algorithm”, describing an evenly distributed hashing function called the “Jump” algorithm:

func Hash(key uint64, numBuckets int) int32 {
    var b int64 = -1
    var j int64

    for j < int64(numBuckets) {
        b = j
        key = key*2862933555777941757 + 1
        j = int64(float64(b+1) * (float64(int64(1)<<31) / float64((key>>33)+1)))
    }

    return int32(b)
}

If you’re curious about that constant, it is “known to produce a good random number list” for 64-bit generators:

MSBuild/C#: How to Manage the Application Version Using a Text-File

Overview

C# applications have an “AssemblyInfo.cs” file that describe the assembly and executable versions of a project. Unfortunately, sometimes it is not possible to access this from the code. Other times, you need to drive this version from external sources (like a build system) and then use it for the build.

The approach this by keeping the version in a text-file:

  1. Manually set/update the version in a text-file.
  2. Install a package that helps us with string-replacements.
  3. Inject this to AssemblyInfo.cs during the build.
  4. Embed this file into the executing assembly.
  5. Extract this file file the executing assembly when you need to know it during execution.

The title of this post is a simplification for lack of an easy way to succinctly describe five steps in a couple of words.

Do It

Feel free to modify/customize these steps as suits your needs.

1. Create the Version File

Create a file called “executable.version” in the “Properties\” folder of your executable project. Make sure to include this in your project. In the “Properties” window, set “Build Action” to “Embedded Resource”.

2. Install the “MSBuild Community Tasks” NuGet Package

This is the “MSBuildTasks” package. This provides us a regular-expression string-replacement MSBuild task.

3. Create a template “AssemblyVersion.cs” File

Copy “Properties\AssemblyInfo.cs” to “Properties\AssemblyInfo.cs.use_this” and update the two version attributes as the bottom to be the following:

[assembly: AssemblyVersion("__EXECUTABLE_VERSION__")]
[assembly: AssemblyFileVersion("__EXECUTABLE_VERSION__")]

Make sure to include this new file in the project. Note that we name this so as to not have the “.cs” extension because, otherwise, Visual Studio will try to parse it and complain about the attributes being duplicated from the “AssemblyInfo.cs” file.

4. Add the Build Step

We are going to add a custom build target to inject the version. We personally chose to put this into a separate rules file in order to make it clear which of the build-logic was ours, but this is up to you. It would just as easily work if it were included at the bottom of your project-file. Create “Properties\build.targets” with the following:

(build.targets)

<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<!-- Inject a version from a text-file into AssemblyVersion.cs . We do this 
 so that it's easier for the application to know its own version [by 
 reading the text file].
 -->
 <Import Project="$(ProjectDir)..\packages\MSBuildTasks.1.5.0.196\tools\MSBuild.Community.Tasks.Targets" /> 
 <Target Name="InjectVersion" BeforeTargets="BeforeBuild">
 <!-- Read the version from our text file. This appears to automatically 
 trim (probably per line). This is located in the project root so 
 that we copy the file to the output-path rather than establishing 
 a whole Properties/ directory in the output path.
 -->
 <ReadLinesFromFile File="$(ProjectDir)Properties\executable.version">
 <Output TaskParameter="Lines" PropertyName="ExecutableVersion" />
 </ReadLinesFromFile>

<!-- Print it to the build output whether we're in debug-mode or not. -->
 <Message Importance="High" Text="Executable version is [$(ExecutableVersion)]"/>

<!-- Copy our template file to the output file. -->
 <Copy SourceFiles="$(ProjectDir)Properties/AssemblyInfo.cs.use_this" DestinationFiles="$(ProjectDir)Properties/AssemblyInfo.cs"/>

<!-- Do an RX replace of the version on to the token. -->

<ItemGroup>
 <WriteFiles Include='$(ProjectDir)Properties/AssemblyInfo.cs' />
 </ItemGroup>

<FileUpdate 
 Files="@(WriteFiles)"
 Regex="__EXECUTABLE_VERSION__"
 ReplacementText="$(ExecutableVersion)"
 />

<!-- Replace the cautionary note about how to use the file with one 
 saying that any changes will be lost (if made to the output file). 
 -->
 <FileUpdate 
 Files="@(WriteFiles)"
 Regex="// TEMPLATE:.+"
 ReplacementText="// THIS FILE IS GENERATED! Apply any changes to 'AssemblyInfo.cs.use_this', instead."
 />
 </Target>
</Project>

IMPORTANT: Notice that we have to import the build targets provided by the “MSBuildTasks” package:

$(ProjectDir)..\packages\MSBuildTasks.1.5.0.196\tools\MSBuild.Community.Tasks.Targets

For us, NuGet packages go into the “packages” directory that is in the parent directory of our project directory. Also notice that we have to embed the version for this NuGet package. If your package is a different version or is located in a different place, you will have to update the example to be accurate.

NOTE: One way to get around having to embed the version is to bypass putting this package in your “packages.config file” and, instead, do a manual NuGet install of this package from a build-task to your packages directory (whereever it is) while also passing the “-ExcludeVersion” argument so as to not put the version in the package’s directory name.

Now, import the “build.targets” file from your project file. Put it somewhere near the bottom. Since it will run before the “BeforeBuild” target, we put it before that (which will be commented-out unless you use it):

 <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
 <Import Project="Properties\build.targets" />

5. Reading the Version From the Application

At this point, you should be able to build your project. The only thing that might be considered a disadvantage to this method is that, every time you build your project from inside Visual Studio, you will be prompted to reload the “AssemblyInfo.cs” file because it has been updated from outside of VS even if it has not changed (which is no stupider than the amount of work that we are required to do in order to find our own version). It would be easiest to check the box in this popup that says to only tell you if you happen to have unsaved changes to a file that has been changed from outside VS.

In our case, we are using the CLAP command-line parser. So, we added a private “ExecutableVersion” getter on the class that we are using to handle our subcommands. Then, we added a “version” subcommand that reads and prints the new property. Code for the property:

(Program.cs)

private string executableVersion = null;

private string ExecutableVersion
{
    get
    {
        if (executableVersion == null)
        {
            Assembly assembly = Assembly.GetExecutingAssembly();
            string assemblyName = assembly.GetName().Name;

            // "Properties" is required since it is located in the 
            // Properties folder of the project and was thusly embedded 
            // as such.
            string filepath = assemblyName + @".Properties.executable.version";

            string[] names = assembly.GetManifestResourceNames();
            var stream = assembly.GetManifestResourceStream(filepath);
            if (stream == null)
            {
                throw new Exception(String.Format("Could not get resource-stream with name [{0}] for version content from assembly [{1}]. Available: {2}", filepath, assembly.FullName, String.Join(",", names)));
            }

            TextReader tr = new StreamReader(stream);
            executableVersion = tr.ReadToEnd().Trim();
        }

        return executableVersion;
    }
}

 

Creating TAR Archives in Go

A short program to show how to write TAR-GZ and TAR-XZ (LZMA) archives. Note that I have not included an example for TAR-BZ2 because there is no easily-findable public library for doing so.

package main

import (
    "archive/tar"
    "compress/gzip"

    "fmt"
    "os"
    "io"
    "time"

    "github.com/ulikunitz/xz"
)

func addFile(tw *tar.Writer, filepath string) {
    data := fmt.Sprintf("I am data: %s\n", filepath)

    h := new(tar.Header)
    h.Name = filepath
    h.Size = int64(len(data))
    h.Mode =  int64(0666)
    h.ModTime = time.Now()

    // write the header to the tarball archive
    if err := tw.WriteHeader(h); err != nil {
        panic(err)
    }

    // copy the file data to the tarball 
    if _, err := io.WriteString(tw, data); err != nil {
        panic(err)
    }
}

func createTarGz() {
    f, err := os.Create("output.tar.gz")
    if err != nil {
        panic(err)
    }

    defer f.Close()

    gw := gzip.NewWriter(f)
    defer gw.Close()

    tw := tar.NewWriter(gw)
    defer tw.Close()

    addFile(tw, "aa")
    addFile(tw, "bb/cc")
}

func createTarXz() {
    f, err := os.Create("output.tar.xz")
    if err != nil {
        panic(err)
    }

    defer f.Close()

    xw, err := xz.NewWriter(f)
    if err != nil {
        panic(err)
    }

    defer xw.Close()

    tw := tar.NewWriter(xw)
    defer tw.Close()

    addFile(tw, "dd")
    addFile(tw, "ee/ff")
}

func main() {
    createTarGz()
    createTarXz()
}

Examine the outputs:

$ tar tzf output.tar.gz 
aa
bb/cc
$ tar xz -O - -f output.tar.gz aa
I am data: aa
$ tar xz -O - -f output.tar.gz bb/cc
I am data: bb/cc

$ tar tJf output.tar.xz
dd
ee/ff
I am data: bb/cc
$ tar xJ -O - -f output.tar.xz dd
I am data: dd
$ tar xJ -O - -f output.tar.xz ee/ff
I am data: ee/ff

Using NuGet.Core to Get the Latest Version of a Package

Add the NuGet.Core package from NuGet and you’ll be in business. We use a NuGet config-file to get the one or more repositories that you might be using and hit them one at a time.

using NuGet;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Xml.XPath;

namespace LatestVersion
{
    class PackageNotFoundException : Exception
    {
        public string PackageName { get; private set; }

        public PackageNotFoundException(string packageName) : base(String.Format(&quot;Package [{0}] not found.&quot;, packageName))
        {
            PackageName = packageName;
        }
    }

    class NuGet
    {
        string nugetConfigFilepath;

        public NuGet(string nugetConfigFilepath)
        {
            this.nugetConfigFilepath = nugetConfigFilepath;
        }

        public IEnumerable&lt;Tuple&gt; GetSources()
        {
            XPathNavigator nav;
            XPathDocument docNav;
            string xPath;

            docNav = new XPathDocument(nugetConfigFilepath);
            nav = docNav.CreateNavigator();
            xPath = &quot;/configuration/packageSources/add&quot;;

            foreach (XPathNavigator xpn in nav.Select(xPath))
            {
                string name = xpn.GetAttribute(&quot;key&quot;, &quot;&quot;);
                string uri = xpn.GetAttribute(&quot;value&quot;, &quot;&quot;);

                yield return new Tuple(name, uri);
            }
        }

        public SemanticVersion GetLatestVersion(string packageName)
        {
            foreach (Tuple source in GetSources())
            {
                string name = source.Item1;
                string uri = source.Item2;

                IPackageRepository repo = PackageRepositoryFactory.Default.CreateRepository(uri);

                // Passing NULL for `versionSpec` will return the latest version.
                IEnumerable packagesRaw = repo.FindPackages(packageName, null, false, false);
                IList packages = packagesRaw.ToList();

                if (packages.Count == 0)
                {
                    continue;
                }

                return packages[0].Version;
            }

            throw new PackageNotFoundException(packageName);
        }
    }
}

Managing Visual Studio/TFS Workspaces at the Command-Line

List current workspaces for your user:

C:\temp>tf workspaces /collection:tfs.hostname/tfs/ /login:\, Collection: http://tfs.hostname/tfs
Workspace Owner Computer Comment
------------ ------------ --------- -------------------------------------------------------------
CODE1 Dustin Oprea DUSTIN-PC

Map a new one locally:

C:\temp>mkdir test-workspace
C:\temp>cd test-workspace
C:\temp\test-workspace>tf workspace /new /noprompt test-workspace /collection:<TFS URL PREFIX>/<COLLECTION NAME> /login:<DOMAIN>\<USERNAME>,<PASSWORD> 
C:\temp\test-workspace>tf workfold /map "$/<PROJECT NAME>" test-workspace

Populate it:

C:\temp\test-workspace>tf get "$/<PROJECT NAME>" /Recursive

If you want to dispose of it, then, first, unmap it:

C:\temp\test-workspace>cd ..
C:\temp>tf workfold /unmap test-workspace /workspace:test-workspace /login:<DOMAIN>\<USERNAME>,<PASSWORD>
C:\temp>tf workspace /delete /noprompt test-workspace

Now, recursively delete the workspace directory.

“Unable to create the workspace ” due to a mapping conflict.”

This is a natural extension of the above:

If you’d like to list workspaces pass the TFS user as the owner:

C:\>tf workspaces /computer:<COMPUTER NAME> /collection:<TFS URL PREFIX>/<COLLECTION NAME> /owner:<OWNER NAME>
Collection: <TFS URL PREFIX>/<COLLECTION NAME>
Workspace        Owner               Computer   Comment
---------------- ------------------- ---------- --------------------------------------------------------------------------------------
<WORKSPACE NAME> <OWNER NAME> <COMPUTER NAME> Workspace Created by Team Build

Delete the workspace mentioned in the error or found via the listing:

C:\>tf workspace /delete /collection:<TFS URL PREFIX>/<COLLECTION NAME> <WORKSPACE NAME>;<OWNER NAME>

Collections and $, @, % Variable References in MSBuild

A very useful and illuminating article:

MSBuild: PropertyGroup, ItemGroup, Item Metadata, and crazy $@% syntax

It also expertly articulates a common feeling:

It brings me great joy that the title of this blog post is both technically accurate and evokes the type of comic-book language I use when trying to figure out MSBuild variable syntax.

It expounds on how to express discrete items, lists, and dictionaries as well as how to refer to them; even implicitly looping over collections.

At the high-level:

  • $: Refers to scalar values. Renders an empty-string for collections.
  • @: Can be used to pass collections or flatten a collection into semicolon-delimited values if used in a scalar context (like embedding in a message task). Note that the individual values will first be normalized by expanding by semicolon and then be flattened by semicolon. So, extraneous semicolons will in fact be removed.
  • %: Expand the given task into N-tasks for each item in a collection.

It’s worth mentioning that %(VariableName) will not expand in such a way that it would result in doing the same thing twice.

For example, I have a list of two assemblies and two XML files of the same two names and would like to print them:

<Target Name="BeforeBuild">
  <ItemGroup>
    <File Include="$(MSBuildProjectDirectory)\AncillaryAssemblies\*.*" />
  </ItemGroup>
  <Message Text="Inner file: %(File.Filename)" />
</Target>

Output:

1>  Inner file: EntityFramework
1>  Inner file: EntityFramework.SqlServer

However, if I were to print the extensions, as well:

<Target Name="BeforeBuild">
  <ItemGroup>
    <File Include="$(MSBuildProjectDirectory)\AncillaryAssemblies\*.*" />
  </ItemGroup>
  <Message Text="Inner file: %(File.Filename) %(File.Extension)" />
</Target>

Output:

1>  Inner file: EntityFramework .dll
1>  Inner file: EntityFramework.SqlServer .dll
1>  Inner file: EntityFramework.SqlServer .xml
1>  Inner file: EntityFramework .xml

Something to keep in mind.