Image Processing: Identifying Groups of Bracketed Images

A new project to recursively scan through a tree of images and determine which were captured as part of an auto-bracketing image capture operation.

bracketed_image_finder

Example:

$ bif_find tests/assets/images --json
[
    {
        "entries": [
            {
                "exposure_value": 0.0,
                "rel_filepath": "DSC08196.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": -0.7,
                "rel_filepath": "DSC08197.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": 0.7,
                "rel_filepath": "DSC08198.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": -1.3,
                "rel_filepath": "DSC08199.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": 1.3,
                "rel_filepath": "DSC08200.JPG",
                "timestamp": "2019-02-13T00:31:51"
            }
        ],
        "type": "periodic"
    },
    {
        "entries": [
            {
                "exposure_value": 0.0,
                "rel_filepath": "DSC08201.JPG",
                "timestamp": "2019-02-13T00:32:09"
            },
            {
                "exposure_value": -0.7,
                "rel_filepath": "DSC08202.JPG",
                "timestamp": "2019-02-13T00:32:09"
            },
            {
                "exposure_value": 0.7,
                "rel_filepath": "DSC08203.JPG",
                "timestamp": "2019-02-13T00:32:10"
            },
            {
                "exposure_value": -1.3,
                "rel_filepath": "DSC08204.JPG",
                "timestamp": "2019-02-13T00:32:10"
            },
            {
                "exposure_value": 1.3,
                "rel_filepath": "DSC08205.JPG",
                "timestamp": "2019-02-13T00:32:10"
            }
        ],
        "type": "periodic"
    }
]

Receive Daily Digest Emails with Active Github Issues

go-github-reminders is a project that you can schedule in order to receive periodic emails of recently updated Github issues that you are involved in (technically, subscribed to) that you have not recently responded to.

For those of us with a lot to do, this keeps us on top of things.

Go: Parsing Time Expressions

go-time-parse will parse time expressions into time.Duration quantities. From the example:

actualDuration, phraseType, err := ParseDuration("24 days from now")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration/time.Hour/24, phraseType)

actualDuration, phraseType, err = ParseDuration("now")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration, phraseType)

actualDuration, phraseType, err = ParseDuration("12m")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration/time.Minute, phraseType)

actualDuration, phraseType, err = ParseDuration("every 6 hours")
log.PanicIf(err)

fmt.Printf("%d [%s]\n", actualDuration/time.Hour, phraseType)

Output:

24 [time]
0 [time]
12 [interval]
6 [interval]

Identifying Nearest Major Cities

go-geographic-attractor is a new project that indexes world city and population data and can match a given coordinate to either the nearest major city or the nearest city (if no major city is near) in near-instantaneous time.

From the example:

// Load countries.

countryDataFilepath := path.Join(appPath, "test", "asset", "countryInfo.txt")

f, err := os.Open(countryDataFilepath)
log.PanicIf(err)

defer f.Close()

countries, err := geoattractorparse.BuildGeonamesCountryMapping(f)
log.PanicIf(err)

// Load cities.

gp := geoattractorparse.NewGeonamesParser(countries)

cityDataFilepath := path.Join(appPath, "index", "test", "asset", "allCountries.txt.detroit_area_handpicked")
g, err := os.Open(cityDataFilepath)
log.PanicIf(err)

defer g.Close()

ci := NewCityIndex()

err = ci.Load(gp, g)
log.PanicIf(err)

// Do the query.

clawsonCoordinates := []float64{42.53667, -83.15041}

sourceName, visits, cr, err := ci.Nearest(clawsonCoordinates[0], clawsonCoordinates[1])
log.PanicIf(err)

// Print the results.

for _, vhi := range visits {
    fmt.Printf("%s: %s\n", vhi.Token, vhi.City)
}

fmt.Printf("\n")

fmt.Printf("Source: %s\n", sourceName)
fmt.Printf("ID: %s\n", cr.Id)
fmt.Printf("Country: %s\n", cr.Country)
fmt.Printf("City: %s\n", cr.City)
fmt.Printf("Population: %d\n", cr.Population)
fmt.Printf("Latitude: %.10f\n", cr.Latitude)
fmt.Printf("Longitude: %.10f\n", cr.Longitude)

Go: Read and Browse Ext4 Filesystems in User-Space

go-ext4 is a pure Go implementation of an Ext4 reader with support for reading the journal. An example of how to walk the file-structure:

inodeNumber := InodeRootDirectory

filepath := path.Join(assetsPath, "hierarchy_32.ext4")

f, err := os.Open(filepath)
log.PanicIf(err)

defer f.Close()

_, err = f.Seek(Superblock0Offset, io.SeekStart)
log.PanicIf(err)

sb, err := NewSuperblockWithReader(f)
log.PanicIf(err)

bgdl, err := NewBlockGroupDescriptorListWithReadSeeker(f, sb)
log.PanicIf(err)

bgd, err := bgdl.GetWithAbsoluteInode(inodeNumber)
log.PanicIf(err)

dw, err := NewDirectoryWalk(f, bgd, inodeNumber)
log.PanicIf(err)

allEntries := make([]string, 0)

for {
    fullPath, de, err := dw.Next()
    if err == io.EOF {
        break
    } else if err != nil {
        log.Panic(err)
    }

    description := fmt.Sprintf("%s: %s", fullPath, de.String())
    allEntries = append(allEntries, description)
}

sort.Strings(allEntries)

for _, entryDescription := range allEntries {
    fmt.Println(entryDescription)
}

// Output:
//
// directory1/fortune1: DirectoryEntry
// directory1/fortune2: DirectoryEntry
// directory1/fortune5: DirectoryEntry
// directory1/fortune6: DirectoryEntry
// directory1/subdirectory1/fortune3: DirectoryEntry
// directory1/subdirectory1/fortune4: DirectoryEntry
// directory1/subdirectory1: DirectoryEntry
// directory1/subdirectory2/fortune7: DirectoryEntry
// directory1/subdirectory2/fortune8: DirectoryEntry
// directory1/subdirectory2: DirectoryEntry
// directory1: DirectoryEntry
// directory2/fortune10: DirectoryEntry
// directory2/fortune9: DirectoryEntry
// directory2: DirectoryEntry
// lost+found: DirectoryEntry
// thejungle.txt: DirectoryEntry

This project is used to directly read the filesystem, file, and journal data without the support of kernel or the FUSE interface. Therefore, no elevated privileges are required.

go-exif-knife: One Exif Command-Line Tool to [Nearly] Rule Them All

go-exif-knife is a tool that will allow you to parse Exif from JPEG and PNG images and to do a brute-force parse of Exif embedded in any other format. You can cherry-pick specific IFDs or tags to print, and print them both as normal and JSON-formatted text. You can can also print parsed GPS data and timestamps and even produce a Google S2 geohash from the GPS data, and dump the thumbnail. If using JPEG or PNG, you can also update or add new Exif data.

This project is built on top of go-jpeg-image-structure, go-png-image-structure, and go-exif. PNG added support for Exif only in the last year, and this project was in service of providing useful Exif support for PNG.

Binary downloads are available here.

 

 

Go: Exif Reader/Writer Library

The go-exif project is now available. It allows you to parse and enumerate/visit/search/dump the existing IFDs/tags in an Exif blob, instantiate a builder to create and construct a new Exif blob, and create a builder from existing IFDs/tags (so you can add/remove starting from what you have). There are also utility functions to make the GPS data manageable.

There are currently 140 unit-tests in the CI process and tested examples covering enumeration, building, thumbnails, GPS, etc…

I have also published go-jpeg-image-structure and go-png-image-structure to actually implement reading/writing Exif in those corresponding formats. PNG adopted Exif support in 2017 and this project was primarily meant to provide PNG with fully-featured Exif-writer support both via library and via command-line tool.

go-exif includes a command-line utility to generally find and parse Exif data in any blob of data. This works for TIFF right off the bat (TIFF is the underlying format of Exif), which I did not specifically write a wrapper implementation for.