Author: dustin

Measure Internet Speed from CLI

Use speedtest-cli:

$ curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python -
Retrieving speedtest.net configuration...
Testing from Comcast Cable (73.1.128.16)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Broadwave (Fort Lauderdale, FL) [43.78 km]: 22.155 ms
Testing download speed................................................................................
Download: 232.72 Mbit/s
Testing upload speed......................................................................................................
Upload: 10.07 Mbit/s

Jenkins: Accessing REST API for a Specific Plugin

Some plugins may extent the Jenkins API with additional functionality. To [blindly] determine if this is the case for a particular plugin of interest, first enumerate the list of installed plugins:

https://<jenkins server>/pluginManager/api/json?depth=2

Then, find the plugin in the list and get the value from its “shortName” field:

Selection_20193009-23:57:31_01.png

That value is the name used in the API URL. In this case, that URL would be https://<jenkins server>/gerrit-trigger​. This might be enough to get you what you need, though, depending on the plugin, you might have to reverse-engineer the plugins’ sourcecode in order to find which further nouns/entity subpaths are available from here. You might do a Google search and then grep for the paths that you find from that in order to discover siblings.

This is not meant to be a complicated post, though the information is made complicated by the effort required to find the information.

go-exfat

I’m a Linux guy, but the lure of a new filesystem spec can not be denied.

This is a read-only exFAT impementation in pure Go. This provides a direct API, but several command-line tools were also made available with which to inspect the filesystem’s structure. There’s 90% test-coverage and an A+ on the score-card.

https://github.com/dsoprea/go-exfat

Go: An In-Memory ReadWriteSeeker

The SeekableBuffer type is similar to a bytes.Buffer in that you can perform both reads and writes against it, but has the following two main differences:

  1. You can seek on it.
  2. After writing to it, the current position will be on the byte following whatever you wrote. This is the typical behavior for a file resource on almost any platform but not for a bytes.Buffer.

Eccentric usage of seek, read, and write behavior, such as the following, will work as expected:

  • seeking but not writing
  • seeking past the end of the file and writing
  • seeking past the end of the file and reading
  • writing N+M bytes when positioned only N bytes from the end of the file

Usage (from the unit-tests):

sb := NewSeekableBuffer()

// Write first string.

data := []byte("word1-word2")

_, err := sb.Write(data)
log.PanicIf(err)

// Seek and replace partway through, and replace more data than we
// currently have.

_, err = sb.Seek(6, os.SEEK_SET)
log.PanicIf(err)

data2 := []byte("word3-word4")

_, err = sb.Write(data2)
log.PanicIf(err)

// Read contents.

_, err = sb.Seek(0, os.SEEK_SET)
log.PanicIf(err)

buffer := make([]byte, 20)

_, err = sb.Read(buffer)
log.PanicIf(err)

// `buffer` currently has "word1-word3-word4".

Image Processing: Identifying Groups of Bracketed Images

A new project to recursively scan through a tree of images and determine which were captured as part of an auto-bracketing image capture operation.

bracketed_image_finder

Example:

$ bif_find tests/assets/images --json
[
    {
        "entries": [
            {
                "exposure_value": 0.0,
                "rel_filepath": "DSC08196.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": -0.7,
                "rel_filepath": "DSC08197.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": 0.7,
                "rel_filepath": "DSC08198.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": -1.3,
                "rel_filepath": "DSC08199.JPG",
                "timestamp": "2019-02-13T00:31:50"
            },
            {
                "exposure_value": 1.3,
                "rel_filepath": "DSC08200.JPG",
                "timestamp": "2019-02-13T00:31:51"
            }
        ],
        "type": "periodic"
    },
    {
        "entries": [
            {
                "exposure_value": 0.0,
                "rel_filepath": "DSC08201.JPG",
                "timestamp": "2019-02-13T00:32:09"
            },
            {
                "exposure_value": -0.7,
                "rel_filepath": "DSC08202.JPG",
                "timestamp": "2019-02-13T00:32:09"
            },
            {
                "exposure_value": 0.7,
                "rel_filepath": "DSC08203.JPG",
                "timestamp": "2019-02-13T00:32:10"
            },
            {
                "exposure_value": -1.3,
                "rel_filepath": "DSC08204.JPG",
                "timestamp": "2019-02-13T00:32:10"
            },
            {
                "exposure_value": 1.3,
                "rel_filepath": "DSC08205.JPG",
                "timestamp": "2019-02-13T00:32:10"
            }
        ],
        "type": "periodic"
    }
]

Python: Parsing XML and Retaining the Comments

By default, Python’s built-in ElementTree module strips comments as it reads them. The solution is just obscure enough to be hard to find.

import xml.etree.ElementTree as ET

class _CommentedTreeBuilder(ET.TreeBuilder):
    def comment(self, data):
        self.start('!comment', {})
        self.data(data)
        self.end('!comment')

def parse(filepath):
    ctb = _CommentedTreeBuilder()
    xp = ET.XMLParser(target=ctb)
    tree = ET.parse(filepath, parser=xp)

    root = tree.getroot()
    # ...

When enumerating the parsed nodes, the comments will have a tag-name of “!comment”.

ssl: Promoting Existing Client Socket to SSL in C/C++

You may be in a situation where something else produces the sockets for you (such as an event-loop) or you otherwise need to manage the socket rather then allowing something else to.

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>

int main(int argc, char *argv[])
{
    int sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (!sockfd) {
        printf("Error creating socket.\n");
        return -1;
    }

    struct sockaddr_in sa;
    memset (&sa, 0, sizeof(sa));

    sa.sin_family = AF_INET;
    sa.sin_addr.s_addr = inet_addr("172.217.2.196");
    sa.sin_port = htons (443); 

    socklen_t socklen = sizeof(sa);
    if (connect(sockfd, (struct sockaddr *)&sa, socklen)) {
        printf("Error connecting to server.\n");
        return -1;
    }

    SSL_library_init();
    SSLeay_add_ssl_algorithms();
    SSL_load_error_strings();

    const SSL_METHOD *meth = TLSv1_2_client_method();
    SSL_CTX *ctx = SSL_CTX_new (meth);

    SSL *ssl = SSL_new (ctx);
    if (ssl == NULL) {
        printf("Could not create SSL context.\n");
        return -1;
    }

    SSL_set_fd(ssl, sockfd);

    int err = SSL_connect(ssl);
    if (err <= 0) {
        printf("Could not connect.\n");
        return -1;
    }

    printf ("SSL connection using %s\n", SSL_get_cipher (ssl));

    // Do send/receive here.

    return 0;
}

Adapted from openssl-in-c-socket-connection-https-client, and works with both OpenSSL and BoringSSL.