Go: Build-Time Variables

Go allows you to override global variables at build time during the linker stage:

$ go build -o /tmp/prog-custom -ldflags "-X main.overrideableValuePhrase=123456" ./cmd/prog

Requirements:

  • It must be a global variable and not a constant.
  • It might be in the package that you are telling it to build. This definitely works for executables and, with the removal of support for binary-only packages (BOPs), it probably doesn’t apply whatsoever to intermediate packages.
  • It must be a string (so, process it from the init() function).

It doesn’t matter whether it is an exported or unexported symbol.

 

Go: Write RPC-Connected Plugins

Plugins are Go’s system for developing shared-libraries. However, this is backed by a general system that can also have alternative implementations. In this case, you can write a plugin in Go that runs from one system and load that plugin in Go, on the fly, from a million other systems. Courtesy of Hashicorp.

https://github.com/hashicorp/go-plugin

 

 

Python: Command-Line Completion for argparse

argcomplete provides very useful functionality that you will basically get for free with just a couple of steps.

Implementation

Put some markup below the shebang of your frontend script in the form of a comment:

#!/usr/bin/env python
# PYTHON_ARGCOMPLETE_OK

The BASH-completion script argcomplete will basically identify and scan any script with a Python shebang that is used with BASH-completion. This entails actually running the script. In order to minimize how much time is spent loading scripts that don’t actually use argcomplete, the completion script will ignore anything that does not have this comment directly following the shebang.

Next, add and import for the argcomplete package and run argcomplete.autocomplete(parser) after you have configured your command-line parameters but before your call to parser.parse_args() (where parser is an instance of argparse.ArgumentParser). This function will produce command-line configuration metadata and then terminate.

That is it. Note that it is not practical to assume that everyone who uses your script will have argcomplete installed. They may not be using BASH (BASH is the only well-supported shell at this time), they may not be using a supported OS, and/or any commercial environments that adopt your tools may be server environments that have no use for command-line completion and refuse to support it. Therefore, you should wrap the import with a try-except for ImportError and then only call argcomplete.autocomplete if you were able to import the package.

Installation

To install autocomplete, the simplest route is to merely do a “sudo pip install argcomplete” and then call “activate-global-python-argcomplete” (this is a physical script likely installed to /usr/local/bin. This only has to be done once and will install a non-project-specific script that will work for any script that is equipped to use argcomplete. For other configuration and semantics, see the project page.

Custom String Template Format in Python

This might be necessary if you, for example, want to apply your own set of replacements to a string argument that will be passed to you by another mechanism that applies its own set of replacements.

This example supposes that you might want to use square-brackets instead of the standard curly-brackets.

import string
import re

_FIELD_RE = re.compile(r'\[([a-zA-Z0-9_]+)\]')

class CustomReplacer(string.Formatter):
    def parse(self, s):
        last_stop_index = None
        for m in _FIELD_RE.finditer(s):
            token_name = m.group(1)

            start_index, stop_index = m.span()

            if start_index == 0:
                prefix_fragment = ''
            elif last_stop_index is None:
                prefix_fragment = s[:start_index]
            else:
                prefix_fragment = s[last_stop_index:start_index]

            last_stop_index = stop_index

            yield prefix_fragment, token_name, '', None

cr = CustomReplacer()
template = 'aa [name]      bb [name2] cc dd [name3]'

replacements = {
    'name': 'howard',
    'name2': 'mark',
    'name3': 'james',
}

output = cr.format(template, **replacements)
print(output)

Output:

aa howard      bb mark cc dd james

Note that no formatting is supported with our custom replacer (though it could be added, with more work). If any formatting specifiers are provided, they will fail the regular-expression match and be ignored.

Measure Internet Speed from CLI

Use speedtest-cli:

$ curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python -
Retrieving speedtest.net configuration...
Testing from Comcast Cable (73.1.128.16)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Broadwave (Fort Lauderdale, FL) [43.78 km]: 22.155 ms
Testing download speed................................................................................
Download: 232.72 Mbit/s
Testing upload speed......................................................................................................
Upload: 10.07 Mbit/s

Jenkins: Accessing REST API for a Specific Plugin

Some plugins may extent the Jenkins API with additional functionality. To [blindly] determine if this is the case for a particular plugin of interest, first enumerate the list of installed plugins:

https://<jenkins server>/pluginManager/api/json?depth=2

Then, find the plugin in the list and get the value from its “shortName” field:

Selection_20193009-23:57:31_01.png

That value is the name used in the API URL. In this case, that URL would be https://<jenkins server>/gerrit-trigger‚Äč. This might be enough to get you what you need, though, depending on the plugin, you might have to reverse-engineer the plugins’ sourcecode in order to find which further nouns/entity subpaths are available from here. You might do a Google search and then grep for the paths that you find from that in order to discover siblings.

This is not meant to be a complicated post, though the information is made complicated by the effort required to find the information.

Python: Parsing XML and Retaining the Comments

By default, Python’s built-in ElementTree module strips comments as it reads them. The solution is just obscure enough to be hard to find.

import xml.etree.ElementTree as ET

class _CommentedTreeBuilder(ET.TreeBuilder):
    def comment(self, data):
        self.start('!comment', {})
        self.data(data)
        self.end('!comment')

def parse(filepath):
    ctb = _CommentedTreeBuilder()
    xp = ET.XMLParser(target=ctb)
    tree = ET.parse(filepath, parser=xp)

    root = tree.getroot()
    # ...

When enumerating the parsed nodes, the comments will have a tag-name of “!comment”.

ssl: Promoting Existing Client Socket to SSL in C/C++

You may be in a situation where something else produces the sockets for you (such as an event-loop) or you otherwise need to manage the socket rather then allowing something else to.

#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>

int main(int argc, char *argv[])
{
    int sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (!sockfd) {
        printf("Error creating socket.\n");
        return -1;
    }

    struct sockaddr_in sa;
    memset (&sa, 0, sizeof(sa));

    sa.sin_family = AF_INET;
    sa.sin_addr.s_addr = inet_addr("172.217.2.196");
    sa.sin_port = htons (443); 

    socklen_t socklen = sizeof(sa);
    if (connect(sockfd, (struct sockaddr *)&sa, socklen)) {
        printf("Error connecting to server.\n");
        return -1;
    }

    SSL_library_init();
    SSLeay_add_ssl_algorithms();
    SSL_load_error_strings();

    const SSL_METHOD *meth = TLSv1_2_client_method();
    SSL_CTX *ctx = SSL_CTX_new (meth);

    SSL *ssl = SSL_new (ctx);
    if (ssl == NULL) {
        printf("Could not create SSL context.\n");
        return -1;
    }

    SSL_set_fd(ssl, sockfd);

    int err = SSL_connect(ssl);
    if (err <= 0) {
        printf("Could not connect.\n");
        return -1;
    }

    printf ("SSL connection using %s\n", SSL_get_cipher (ssl));

    // Do send/receive here.

    return 0;
}

Adapted from openssl-in-c-socket-connection-https-client, and works with both OpenSSL and BoringSSL.