Go: Build-Time Variables

Go allows you to override global variables at build time during the linker stage:

$ go build -o /tmp/prog-custom -ldflags "-X main.overrideableValuePhrase=123456" ./cmd/prog

Requirements:

  • It must be a global variable and not a constant.
  • It might be in the package that you are telling it to build. This definitely works for executables and, with the removal of support for binary-only packages (BOPs), it probably doesn’t apply whatsoever to intermediate packages.
  • It must be a string (so, process it from the init() function).

It doesn’t matter whether it is an exported or unexported symbol.

 

Git: Use git-sizer to Identify Large Space Usage in Your History

https://github.com/github/git-sizer

Notice that it cites the revisions at the bottom:

$ git-sizer -v
Processing blobs: 2049                        
Processing trees: 1056                        
Processing commits: 432                        
Matching commits to trees: 432                        
Processing annotated tags: 0                        
Processing references: 12                        
| Name                         | Value     | Level of concern               |
| ---------------------------- | --------- | ------------------------------ |
| Overall repository size      |           |                                |
| * Commits                    |           |                                |
|   * Count                    |   432     |                                |
|   * Total size               |   143 KiB |                                |
| * Trees                      |           |                                |
|   * Count                    |  1.06 k   |                                |
|   * Total size               |   885 KiB |                                |
|   * Total tree entries       |  21.0 k   |                                |
| * Blobs                      |           |                                |
|   * Count                    |  2.05 k   |                                |
|   * Total size               |  33.8 MiB |                                |
| * Annotated tags             |           |                                |
|   * Count                    |     0     |                                |
| * References                 |           |                                |
|   * Count                    |    12     |                                |
|                              |           |                                |
| Biggest objects              |           |                                |
| * Commits                    |           |                                |
|   * Maximum size         [1] |  1.32 KiB |                                |
|   * Maximum parents      [2] |     2     |                                |
| * Trees                      |           |                                |
|   * Maximum entries      [3] |    66     |                                |
| * Blobs                      |           |                                |
|   * Maximum size         [4] |  4.05 MiB |                                |
|                              |           |                                |
| History structure            |           |                                |
| * Maximum history depth      |   413     |                                |
| * Maximum tag depth          |     0     |                                |
|                              |           |                                |
| Biggest checkouts            |           |                                |
| * Number of directories  [5] |    27     |                                |
| * Maximum path depth     [6] |     4     |                                |
| * Maximum path length    [7] |   105 B   | *                              |
| * Number of files        [6] |   137     |                                |
| * Total size of files    [8] |  4.64 MiB |                                |
| * Number of symlinks         |     0     |                                |
| * Number of submodules       |     0     |                                |

[1]  a52f2b2c86baa9617ed66bc0b3301d57bf763ed3
[2]  fa562d025143096dc8ae0c2294114cd0b4443945 (refs/stash)
[3]  4ca9fdb84fec68c38d6061441996fd15b9494e9d (d2a75bc4b9743a3decf9e6cd5cb4a8670d57f30d^{tree})
[4]  62918aac7359d32b4b342db8d65a5a2a5172215d (86c3344be1fc590ee3ebe9eb7117e91c4ad21450:command/gozipinfo/gozipinfo)
[5]  2b5b648f5e6488420c22ba9c318fc6bfc4ce2a47 (461bbd7555360a202d1ab16f02c992581791a7d2^{tree})
[6]  853678df5f5fe5ec58b169d0fe3878e376c53e3e (refs/heads/dustin/profiling/temp_path^{tree})
[7]  8b03f6bab9d3369fec1fe322f5d3f159a220d95a (1b385e1714b7904fca85943d0f27c7f772af6d0f^{tree})
[8]  a557ab9aff02167856530cf536163d349e44fdfe (86c3344be1fc590ee3ebe9eb7117e91c4ad21450^{tree})

Go: Write RPC-Connected Plugins

Plugins are Go’s system for developing shared-libraries. However, this is backed by a general system that can also have alternative implementations. In this case, you can write a plugin in Go that runs from one system and load that plugin in Go, on the fly, from a million other systems. Courtesy of Hashicorp.

https://github.com/hashicorp/go-plugin

 

 

Python: Command-Line Completion for argparse

argcomplete provides very useful functionality that you will basically get for free with just a couple of steps.

Implementation

Put some markup below the shebang of your frontend script in the form of a comment:

#!/usr/bin/env python
# PYTHON_ARGCOMPLETE_OK

The BASH-completion script argcomplete will basically identify and scan any script with a Python shebang that is used with BASH-completion. This entails actually running the script. In order to minimize how much time is spent loading scripts that don’t actually use argcomplete, the completion script will ignore anything that does not have this comment directly following the shebang.

Next, add and import for the argcomplete package and run argcomplete.autocomplete(parser) after you have configured your command-line parameters but before your call to parser.parse_args() (where parser is an instance of argparse.ArgumentParser). This function will produce command-line configuration metadata and then terminate.

That is it. Note that it is not practical to assume that everyone who uses your script will have argcomplete installed. They may not be using BASH (BASH is the only well-supported shell at this time), they may not be using a supported OS, and/or any commercial environments that adopt your tools may be server environments that have no use for command-line completion and refuse to support it. Therefore, you should wrap the import with a try-except for ImportError and then only call argcomplete.autocomplete if you were able to import the package.

Installation

To install autocomplete, the simplest route is to merely do a “sudo pip install argcomplete” and then call “activate-global-python-argcomplete” (this is a physical script likely installed to /usr/local/bin. This only has to be done once and will install a non-project-specific script that will work for any script that is equipped to use argcomplete. For other configuration and semantics, see the project page.

Custom String Template Format in Python

This might be necessary if you, for example, want to apply your own set of replacements to a string argument that will be passed to you by another mechanism that applies its own set of replacements.

This example supposes that you might want to use square-brackets instead of the standard curly-brackets.

import string
import re

_FIELD_RE = re.compile(r'\[([a-zA-Z0-9_]+)\]')

class CustomReplacer(string.Formatter):
    def parse(self, s):
        last_stop_index = None
        for m in _FIELD_RE.finditer(s):
            token_name = m.group(1)

            start_index, stop_index = m.span()

            if start_index == 0:
                prefix_fragment = ''
            elif last_stop_index is None:
                prefix_fragment = s[:start_index]
            else:
                prefix_fragment = s[last_stop_index:start_index]

            last_stop_index = stop_index

            yield prefix_fragment, token_name, '', None

cr = CustomReplacer()
template = 'aa [name]      bb [name2] cc dd [name3]'

replacements = {
    'name': 'howard',
    'name2': 'mark',
    'name3': 'james',
}

output = cr.format(template, **replacements)
print(output)

Output:

aa howard      bb mark cc dd james

Note that no formatting is supported with our custom replacer (though it could be added, with more work). If any formatting specifiers are provided, they will fail the regular-expression match and be ignored.

Measure Internet Speed from CLI

Use speedtest-cli:

$ curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python -
Retrieving speedtest.net configuration...
Testing from Comcast Cable (73.1.128.16)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Broadwave (Fort Lauderdale, FL) [43.78 km]: 22.155 ms
Testing download speed................................................................................
Download: 232.72 Mbit/s
Testing upload speed......................................................................................................
Upload: 10.07 Mbit/s