Hint/Trick

Register New Executable Formats In The Linux Kernel

If the kernel identifies that a file you’re trying to execute has a known sequence of magic bytes, it will happily execute it. However, you might want to run certain formats without needing to pass them to another tool. In as much as these are usually binary tools, you can prepend a magic-bytes preamble (or use an existing sequence, if your binary format already declares one) and then associate that with a certain loader in the running kernel using binfmt_misc. Access is usually via procfs (currently /proc/sys/fs/binfmt_misc/register). Cross-execution with a virtualized process is a common use-case for this.

Note that this doesn’t really apply to text files (it’s all data) because you’d need to inject gibberish at the top of your file to use this functionality when using a shebang (which is meant for this case) should work just fine.

This is a walkthrough of how to do this for compiled Python or LUA source-code: https://twdev.blog/2024/01/docker_multi_platform

Since both of the example compilation formats include magic bytes, you can just write a tool to acquire those from libraries (at least with Python) and call binfmt_misc in one step.

You can also write configs to be automatically loaded at boot. There appears to be no way to list registered formats, with the only recourse being to sift the config paths (probably empty by default) or to check your kernel config to see which supported formats were included in that build.

Embedding Python in PostgreSQL Functions

Example:

CREATE OR REPLACE FUNCTION url_quote (url text)
RETURNS TEXT
AS $$
    from urllib.parse import quote
    return quote(url)

$$
LANGUAGE 'plpython3u';

SELECT url_quote('https://www.postgresql.org/docs/12/plpython-data.html#id-1.8.11.11.3');

Getting Started with Postgres Functions in PL/Python

How to Render Django Templates Without Loading Django (or Its Configuration System)

#!/usr/bin/env python3

import django.template
import django.template.engine

def _main():
    e = django.template.engine.Engine()

    body = """\
aa {{ test_token }} cc
"""

    t = django.template.Template(body, engine=e)

    context = {
        'test_token': 'bb',
    }

    c = django.template.Context(context)
    r = t.render(c)

    print(r)


_main()

Output:

$ ./test_render.py 
aa bb cc

Python: Substitute Values Into YAML During Load

There’s definitely a case to be made for automatically and efficiently applying environment variables or another set of replacements into a YAML-based config or set of instructions on load. This example uses PyYAML. We’ll use Python’s built-in string templating to replace tokens like “$NAME” with values from a dictionary. It will fail, as it should, if the name is not in the given dictionary.

import string
import yaml

def load_yaml(f, context):

    def string_constructor(loader, node):

        t = string.Template(node.value)
        value = t.substitute(context)

        return value


    l = yaml.SafeLoader
    l.add_constructor('tag:yaml.org,2002:str', string_constructor)

    token_re = string.Template.pattern
    l.add_implicit_resolver('tag:yaml.org,2002:str', token_re, None)

    x = yaml.load(f, Loader=l)
    return x

y = """\
aa: bb
cc: dd $EE ff
"""

context = {
    'EE': '123',
}

d = waw.utility.load_yaml(y, context)
print(d)

Output:

{'aa': 'bb', 'cc': 'dd 123 ff'}

Installing VMWare Remote Console (VMRC) On Arch/Manjaro

When you download the Remote Console bundle installer from VMWare and run it, the UI will start and simply fail with “Installation was unsuccessful” error. There is no console output, seems to be no log, and no apparent option to enable verbosity. Since I’m using Manjaro, I’d rather not use the package, which is only available in AUR.

A successful install is not terribly difficult. You can start the install using a command-line installer rather than the GUI:

sudo ./VMware-Remote-Console-11.2.0-16492666.x86_64.bundle --console

There will be some prompting that isn’t present in the GUI. When it asks you:

System service scripts directory (commonly /etc/init.d).:

…enter an alternative directory. That’s it. In my case, it didn’t even have to exist.

The install will take a minute expanding things, and then deposit the “vmrc” executable within the executable search-path. The browser should now be able to find it when the remote console wants to open the viewer.

Raspberry Pi: Expand Root Filesystem

The “expand_rootfs” option is no longer available in raspi-config, at least in my situation. This is a Raspberry Pi 4 and I used the Imager to provision my SD card.

It turns out that the functionality is still accessible, though:

raspi-config --expand-rootfs

Go: Build-Time Variables

Go allows you to override global variables at build time during the linker stage:

$ go build -o /tmp/prog-custom -ldflags "-X main.overrideableValuePhrase=123456" ./cmd/prog

Requirements:

  • It must be a global variable and not a constant.
  • It might be in the package that you are telling it to build. This definitely works for executables and, with the removal of support for binary-only packages (BOPs), it probably doesn’t apply whatsoever to intermediate packages.
  • It must be a string (so, process it from the init() function).

It doesn’t matter whether it is an exported or unexported symbol.

 

Go: Write RPC-Connected Plugins

Plugins are Go’s system for developing shared-libraries. However, this is backed by a general system that can also have alternative implementations. In this case, you can write a plugin in Go that runs from one system and load that plugin in Go, on the fly, from a million other systems. Courtesy of Hashicorp.

https://github.com/hashicorp/go-plugin