Receive Daily Digest Emails with Active Github Issues

go-github-reminders is a project that you can schedule in order to receive periodic emails of recently updated Github issues that you are involved in (technically, subscribed to) that you have not recently responded to.

For those of us with a lot to do, this keeps us on top of things.

Go: Parsing Time Expressions

go-time-parse will parse time expressions into time.Duration quantities. From the example:

actualDuration, phraseType, err := ParseDuration("24 days from now")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration/time.Hour/24, phraseType)

actualDuration, phraseType, err = ParseDuration("now")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration, phraseType)

actualDuration, phraseType, err = ParseDuration("12m")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration/time.Minute, phraseType)

actualDuration, phraseType, err = ParseDuration("every 6 hours")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration/time.Hour, phraseType)

Output:

24 [time]
0 [time]
12 [interval]
6 [interval]

Identifying Nearest Major Cities

go-geographic-attractor is a new project that indexes world city and population data and can match a given coordinate to either the nearest major city or the nearest city (if no major city is near) in near-instantaneous time.

From the example:

// Load countries.

countryDataFilepath := path.Join(appPath, "test", "asset", "countryInfo.txt")

f, err := os.Open(countryDataFilepath)
log.PanicIf(err)

defer f.Close()

countries, err := geoattractorparse.BuildGeonamesCountryMapping(f)
log.PanicIf(err)

// Load cities.

gp := geoattractorparse.NewGeonamesParser(countries)

cityDataFilepath := path.Join(appPath, "index", "test", "asset", "allCountries.txt.detroit_area_handpicked")
g, err := os.Open(cityDataFilepath)
log.PanicIf(err)

defer g.Close()

ci := NewCityIndex()

err = ci.Load(gp, g)
log.PanicIf(err)

// Do the query.

clawsonCoordinates := []float64{42.53667, -83.15041}

sourceName, visits, cr, err := ci.Nearest(clawsonCoordinates[0], clawsonCoordinates[1])
log.PanicIf(err)

// Print the results.

for _, vhi := range visits {
    fmt.Printf("%s: %s\n", vhi.Token, vhi.City)
}

fmt.Printf("\n")

fmt.Printf("Source: %s\n", sourceName)
fmt.Printf("ID: %s\n", cr.Id)
fmt.Printf("Country: %s\n", cr.Country)
fmt.Printf("City: %s\n", cr.City)
fmt.Printf("Population: %d\n", cr.Population)
fmt.Printf("Latitude: %.10f\n", cr.Latitude)
fmt.Printf("Longitude: %.10f\n", cr.Longitude)

Go: Read and Browse Ext4 Filesystems in User-Space

go-ext4 is a pure Go implementation of an Ext4 reader with support for reading the journal. An example of how to walk the file-structure:

inodeNumber := InodeRootDirectory

filepath := path.Join(assetsPath, "hierarchy_32.ext4")

f, err := os.Open(filepath)
log.PanicIf(err)

defer f.Close()

_, err = f.Seek(Superblock0Offset, io.SeekStart)
log.PanicIf(err)

sb, err := NewSuperblockWithReader(f)
log.PanicIf(err)

bgdl, err := NewBlockGroupDescriptorListWithReadSeeker(f, sb)
log.PanicIf(err)

bgd, err := bgdl.GetWithAbsoluteInode(inodeNumber)
log.PanicIf(err)

dw, err := NewDirectoryWalk(f, bgd, inodeNumber)
log.PanicIf(err)

allEntries := make([]string, 0)

for {
    fullPath, de, err := dw.Next()
    if err == io.EOF {
        break
    } else if err != nil {
        log.Panic(err)
    }

    description := fmt.Sprintf("%s: %s", fullPath, de.String())
    allEntries = append(allEntries, description)
}

sort.Strings(allEntries)

for _, entryDescription := range allEntries {
    fmt.Println(entryDescription)
}

// Output:
//
// directory1/fortune1: DirectoryEntry
// directory1/fortune2: DirectoryEntry
// directory1/fortune5: DirectoryEntry
// directory1/fortune6: DirectoryEntry
// directory1/subdirectory1/fortune3: DirectoryEntry
// directory1/subdirectory1/fortune4: DirectoryEntry
// directory1/subdirectory1: DirectoryEntry
// directory1/subdirectory2/fortune7: DirectoryEntry
// directory1/subdirectory2/fortune8: DirectoryEntry
// directory1/subdirectory2: DirectoryEntry
// directory1: DirectoryEntry
// directory2/fortune10: DirectoryEntry
// directory2/fortune9: DirectoryEntry
// directory2: DirectoryEntry
// lost+found: DirectoryEntry
// thejungle.txt: DirectoryEntry

This project is used to directly read the filesystem, file, and journal data without the support of kernel or the FUSE interface. Therefore, no elevated privileges are required.

Go: Image-Processing Microservice

Imaginary is a fun little package that can be Dockerized and deployed next to the rest of your services to offload image-processing:

Fast HTTP microservice written in Go for high-level image processing backed by bimg and libvips. imaginary can be used as private or public HTTP service for massive image processing with first-class support for Docker & Heroku. It’s almost dependency-free and only uses net/http native package without additional abstractions for better performance.

Creating TAR Archives in Go

A short program to show how to write TAR-GZ and TAR-XZ (LZMA) archives. Note that I have not included an example for TAR-BZ2 because there is no easily-findable public library for doing so.

package main

import (
    "archive/tar"
    "compress/gzip"

    "fmt"
    "os"
    "io"
    "time"

    "github.com/ulikunitz/xz"
)

func addFile(tw *tar.Writer, filepath string) {
    data := fmt.Sprintf("I am data: %s\n", filepath)

    h := new(tar.Header)
    h.Name = filepath
    h.Size = int64(len(data))
    h.Mode =  int64(0666)
    h.ModTime = time.Now()

    // write the header to the tarball archive
    if err := tw.WriteHeader(h); err != nil {
        panic(err)
    }

    // copy the file data to the tarball 
    if _, err := io.WriteString(tw, data); err != nil {
        panic(err)
    }
}

func createTarGz() {
    f, err := os.Create("output.tar.gz")
    if err != nil {
        panic(err)
    }

    defer f.Close()

    gw := gzip.NewWriter(f)
    defer gw.Close()

    tw := tar.NewWriter(gw)
    defer tw.Close()

    addFile(tw, "aa")
    addFile(tw, "bb/cc")
}

func createTarXz() {
    f, err := os.Create("output.tar.xz")
    if err != nil {
        panic(err)
    }

    defer f.Close()

    xw, err := xz.NewWriter(f)
    if err != nil {
        panic(err)
    }

    defer xw.Close()

    tw := tar.NewWriter(xw)
    defer tw.Close()

    addFile(tw, "dd")
    addFile(tw, "ee/ff")
}

func main() {
    createTarGz()
    createTarXz()
}

Examine the outputs:

$ tar tzf output.tar.gz 
aa
bb/cc
$ tar xz -O - -f output.tar.gz aa
I am data: aa
$ tar xz -O - -f output.tar.gz bb/cc
I am data: bb/cc

$ tar tJf output.tar.xz
dd
ee/ff
I am data: bb/cc
$ tar xJ -O - -f output.tar.xz dd
I am data: dd
$ tar xJ -O - -f output.tar.xz ee/ff
I am data: ee/ff

Using the Google Maps Client Library for Go in AppEngine

The default HTTP transport implementation for Go isn’t supported when running in AppEngine. Trying to use it will result in the following error:

http.DefaultTransport and http.DefaultClient are not available in App Engine. See https://cloud.google.com/appengine/docs/go/urlfetch/

To fix this, you need to use the http.Client implementation from AppEngine’s urlfetch package (imported from google.golang.org/appengine/urlfetch).

uc := urlfetch.Client(ctx)

options := []maps.ClientOption {
    maps.WithHTTPClient(uc),
    maps.WithAPIKey(GoogleApiKey),
}

c, err := maps.NewClient(options...)
if err != nil {
    panic(err)
}

nsr := &maps.NearbySearchRequest{
    Location: &maps.LatLng {
        Lat: latitude,
        Lng: longitude,
    },
    Radius: radius,
    OpenNow: true,
    RankBy: maps.RankByProminence,
    Type: maps.PlaceTypeRestaurant,
}

psr, err := c.NearbySearch(ctx, nsr)
if err != nil {
    panic(err)
}