cidrex; ferment; libxo

To try to combat yet-another flare-up of the long covid “hey, you’re awake!” annoyance, I set out to reproduce the functionality of a recent Golang utility I discovered using C and Zig. Along the way, I learned about a super neat C-based library for emitting structured program output. So, today’s Bonus Drop showcases all three items.

TL;DR

(This is an AI-generated summary of today’s Drop using Ollama + llama 3.2 and a custom prompt.)

• cidrex: This tool expands IP addresses and CIDR ranges into individual IP addresses, providing a complete list of all addresses within a given range. (https://github.com/d3mondev/cidrex)
• ferment: cidrex functionality recreated in C using Zig’s build system to create cross-platform binaries for arm64 and amd64 linux systems. (https://github.com/hrbrmstr/ferment)
• libxo: A library called that enables applications to generate output in multiple formats, including text, XML, JSON, and HTML, allowing developers to create a single codebase that can produce different output styles based on runtime decisions. (https://libxo.readthedocs.io/en/latest/index.html#)

cidrex

As part of some reporting automation, I needed to expand a dynamic list of individual IPv4 + IPv6 addresses potentially interspersed with the same types of addresses in CIDR blocks.

A CIDR range refers to a block of IP addresses represented using Classless Inter-Domain Routing (CIDR) notation. In this notation, an IP address is followed by a slash and a number—for example, 192.168.1.0/24. The number after the slash indicates the number of bits in the network prefix, which defines the size of the network. This allows for flexible allocation of IP addresses by specifying networks of varying sizes, rather than adhering to the fixed sizes dictated by the older classful addressing system.

There are plenty of ways to do said expansion, but I’m constantly looking for new, shiny objects. That’s how I came across cidrex, a useful, super-focused command-line utility 100% designed for expanding IP addresses and CIDR ranges in my specific use case.

cidrex can read IP addresses and CIDR ranges from either a specified file or standard input (stdin). It expands CIDR ranges into individual IP addresses, providing a complete list of all addresses within a given range. It further provides options for us to filter results to show only IPv4 addresses, only IPv6 addresses, or both types.

The basic syntax for using cidrex is cidrex [OPTIONS] [filename]. If no filename is provided, cidrex reads from stdin. Command-line options include -4 or --ipv4 to print only IPv4 addresses, -6 or --ipv6 to print only IPv6 addresses, and -h or --help to display the help message.

The author of cidrex built it for bug bounty hunters, who often need to perform network scanning. For them, the tool can quickly generate a list of all IP addresses in a subnet for vulnerability scanning. In firewall configuration, it expands CIDR ranges to individual IPs for precise firewall rule creation. For log analysis, it converts CIDR notations in logs to individual IPs for more detailed analysis. In IP management, it helps visualize all addresses within a given IP range for network planning.

As a command-line tool, cidrex can be easily integrated into shell scripts or used in combination with other Unix tools using pipes. For example, the command cat input.txt | cidrex -6 | grep "::" > ipv6_results.txt would expand the IP ranges in input.txt, filter for IPv6 addresses, then further filter for addresses containing “::”, and finally save the results to a file.

cidrex is written in Go, making it easy to install with no runtime dependencies. You can install it using Go’s package manager with the command go install github.com/d3mondev/cidrex@latest. This will download, compile, and install the cidrex binary in your Go bin directory.

ferment

The aforementioned cidrex does what I need, but also found myself needing something a bit mentally engaging to try to get back to sleep. While I’m a big fan of Golang CLIs, they end up being yuge, and this one is around 2.5 MB on disk. That’s…alot…for such a simple tool. So, I decided to recreate it in C, and then figure out how to use Zig’s build system to create cross-platform binaries for arm64 and amd64 linux systems. Thus begat ferment.

It does the same thing cidrex does, though it places an inherent limit on the number of IPv6 addresses will be expanded from a v6 CIDR, since they can get yuge, and it would most likely be an error that such a CIDR showed up in a real-world use case (you can comment that logic out of the C code to let yourself go wild).

As I was poking at how to test the results of my C hack, I happened to do a man wc on macOS 15, as I didn’t remember if there was a way to just get the number of lines sans all annotations, and noticed this at the top of the entry:

--libxo
  Generate output via libxo(3) in a selection of different human
  and machine readable formats. See xo_parse_args(3) for details
  on command line arguments.

Head to the next section to see what that’s for.

The ferment C code is nothing special, and I spent most of the time on the Zig build script, since I wanted to make the aforementioned release binaries. While it’s possible to use either clang or gcc to perform cross-platform builds, you need to do some legwork to ensure you have all the support files necessary to perform said builds. Zig abstracts all that away, and the build script is small enough to include here, and I think it’s likely readable enough to trust Drop readers to see what it’s doing:

const std = @import("std");

pub fn build(b: *std.Build) void {
  const target = b.standardTargetOptions(.{});
  const optimize = b.standardOptimizeOption(.{});

  _ = b.addExecutable(.{
    .name = "ferment",
    .target = target,
    .root_source_file = b.path("main.c"),
    .optimize = optimize,
  });

  // Linux AMD64 Build
  const linux_amd64_build = b.addSystemCommand(&.{
    "zig",                     "cc",
    "-target",                 "x86_64-linux-gnu",
    "-O3",                     "-o",
    "bin/ferment-linux-amd64", "main.c",
  });

  // Linux AArch64 Build
  const linux_aarch64_build = b.addSystemCommand(&.{
    "zig",                       "cc",
    "-target",                   "aarch64-linux-gnu",
    "-O3",                       "-o",
    "bin/ferment-linux-aarch64", "main.c",
  });

  // Create a custom step to build all targets
  const build_all = b.step("build-all", "Build for all targets");
  build_all.dependOn(&linux_amd64_build.step);
  build_all.dependOn(&linux_aarch64_build.step);

  // Make the default build step build all targets
  b.getInstallStep().dependOn(build_all);
}

It works super-fast and builds a local build cache, so future builds are even faster.

I won’t go back to being a C-first coder, but it’s nice to exercise those muscles every now and again, and I’ll take a 25K binary over a 2.5 MB one any day.

libxo

libxo (GH) (wiki) is a useful library developed by Juniper Networks that enables applications to generate output in multiple formats, including text, XML, JSON, and HTML. It allows developers to create a single codebase that can produce different output styles based on runtime decisions. The library supports features like multiple output streams, pluralization, color output, syslog integration, humanized output, internationalization, and UTF-8 encoding.

Folks at Juniper Networks recognized the need for a flexible output generation system that could cater to various use cases, from command-line interfaces to web applications. That led to the development of this library, which has since become an integral part of FreeBSD’s core system.

It provides a simple API centered around the xo_emit function, which uses format strings with field descriptors to specify output content and structure. This approach allows developers to easily generate different output formats without significant code changes.

Apple incorporated libxo support in some of the command-line utilities shipped with macOS. Only a few have the support:

$ rg -c "libxo "
man1/wc.1:2
man1/w.1:2
man1/df.1:2
man1/last.1:3

But it is so much nicer having structured output:

$ w --libxo json | jq
{
  "uptime-information": {
    "time-of-day": "11:12",
    "uptime": 962173,
    "days": 11,
    "hours": 3,
    "minutes": 16,
    "seconds": 13,
    "uptime-human": " 11 days,  3:16,",
    "users": 2,
    "load-average-1": 2.84,
    "load-average-5": 5.02,
    "load-average-15": 6.00,
    "user-table": {
      "user-entry": [
        {
          "user": "hrbrmstr",
          "tty": "console",
          "from": "-",
          "login-time": "24Sep24",
          "idle": "11days",
          "command": "-"
        },
        {
          "user": "hrbrmstr",
          "tty": "s003",
          "from": "-",
          "login-time": " 3:17",
          "idle": true,
          "command": "-zsh"
        }
      ]
    }
  }
}

$ ps -ef | wc --libxo json | jq
{
  "wc": {
    "file": [
      {
        "lines": 1044,
        "words": 10060,
        "characters": 192537
      }
    ]
  }
}

If I ever have to make another C-based CLI utility, I’m absolutely going to use this to add structured output support.

FIN

Drop a reply or a private note if you’d like other examples.

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev ☮️