Deepfield IP Checker; HTML-XML-utils; MapTheNet

What’s this?…A weekday Drop?!

Yep!

Three cool finds in a row managed to (at least temporarily) push past both the ennui that has set in for various reasons I am not at liberty to publicly disclose and the long covid cardio & brain fog relapse.

(Hopefully this is a catalyst for getting back to the regular cadence, and restoring Tuesday Drops to their typography roots.)

Also, no TL;DRs for a bit as I’m growing pretty weary of “AI”.

Deepfield IP Checker

Back in 2025 I briefly mentioned the IP checker I made at my previous employer. It tells you if your IP was seen in the sensor network of said employer. Thanks to some operational incompetence (not by me), that service was temporarily busted, but is back online. However, you don’t need to rely on just it to test if your external IP address has been seen breaking bad.

The always spiffy Jérôme Meyer of the Nokia Deepfield team used their vast DDoS telemetry network data to build their IP checker.

In his own tooted words:

We sit on a lot of #DDoS telemetry, so we built a way to give a bit of it back. Load the page and it answers two questions about the address you are connecting from:

1. Has it been observed taking part in DDoS attacks recently? (In other words, is there a bot in your house?)
2. Has it been on the receiving end of a botnet attack command recently?

It checks and reports both your v4 and your v6 addresses, though the IPv6 half is still mostly a courtesy, since most DDoS botnets have not discovered it.

A clean report looks like what you see in the section header.

And you can get the same info at the terminal with:

≫ curl --silent https://check.deepfield.net/api/check | jq
{
  "ip": "70.#.#.#",
  "family": 4,
  "seen_in_ddos_samples": false,
  "matched_prefix_length": 32,
  "meaning": "DDoS attack traffic left this address during attacks Deepfield captured; this describes the address, not the person using it",
  "observation_set": {
    "addresses": 4378631,
    "generated_at": 1787638056,
    "age_seconds": 11469
  },
  "targeted_by_c2_command": false,
  "targeted_meaning": "named as a destination in a botnet command-and-control attack command; this address was pointed at, not the source"
}

Whenever you’re on the network of a fam/friend, hit up their checker and show them the results, then encourage them to use it regularly. Attackers truly value the “clean” IP space of residences and small businesses and are working overtime to compromise as many of them as possible.

Between Nokia’s tool and the thing I wrote, you can get some solid visibility into whether a given network has been naughty or nice.

HTML-XML-utils

After seeing the Deepfield checker, I hacked together a curl + htmlq one-liner to get the info on the CLI. After posting that to Mastodon, Jérôme kindly directed me to the API call seen in the previous section.

However, right after using htmlq, I decided to do some Kagi-ing for any latest and greatest alternative to htmlq. What I found is great, but is also 28 years old (O_O)!

Bert Bos has been the sole maintainer (verifiable with curl -s https://www.w3.org/Tools/HTML-XML-utils/ChangeLog | rg "^[[:digit:]]" | sed -e 's/^..........//' |sort -u) of HTML-XML-utils which provides the following set of tools:

cexport (1)          - create headerfile of exported declarations from a C file
hxaddid (1)          - add ID's to selected elements
hxcite (1)           - replace bibliographic references by hyperlinks
hxcite-mkbib (1)     - expand references and create bibliography
hxcopy (1)           - copy an HTML file while preserving relative links
hxcount (1)          - count elements and attributes in HTML or XML files
hxextract (1)        - extract selected elements
hxclean (1)          - apply heuristics to correct an HTML file
hxprune (1)          - remove marked elements from an HTML file
hxincl (1)           - expand included HTML or XML files
hxindex (1)          - create an alphabetically sorted index
hxmkbib (1)          - create bibliography from a template
hxmultitoc (1)       - create a table of contents for a set of HTML files
hxname2id            - move some ID= or NAME= from A elements to their parents
hxnormalize (1)      - pretty-print an HTML file
hxnum (1)            - number section headings in an HTML file
hxpipe (1)           - convert XML to a format easier to parse with Perl or AWK
hxprintlinks (1)     - number links & add table of URLs at end of an HTML file
hxremove (1)         - remove selected elements from an XML file
hxtabletrans (1)     - transpose an HTML or XHTML table
hxtoc (1)            - insert a table of contents in an HTML file
hxuncdata (1)        - replace CDATA sections by character entities
hxunent (1)          - replace HTML predefined character entities to UTF-8
hxunpipe (1)         - convert output of pipe back to XML format
hxunxmlns (1)        - replace "global names" by XML Namespace prefixes
hxwls (1)            - list links in an HTML file
hxxmlns (1)          - replace XML Namespace prefixes by "global names"
asc2xml, xml2asc (1) - convert between UTF8 and &#nnn; entities
hxref (1)            - generate cross-references
hxselect (1)         - extract elements that match a (CSS) selector

It’s in every package manager I use, so you should be able to quickly install it.

While a tad more verbose than the htmlq idiom I used, this dance gets the Deepfield IP check status from the non-API endpoint:

curl --silent https://v4.check.deepfield.net | \
  hxclean | \
  hxselect -s '\n' -c '.headline'
Not seen in the DDoS attacks we captured
Not named as a victim in any botnet command we saw

The web is still an awful place, and HTML is busted on pretty much every site (to some degree), so the hxclean step is always a good idea, and the hxnormalize step is necessary to clean it up a bit, and the incantation for hxselect says to put a newline after each found selector, with -c saying to print the contents vs. the tag + text.

hxwls also seems pretty useful:

≫ (curl -s "https://dailydrop.hrbrmstr.dev/2026/02/12/drop-771-2026-02-12-system-of-a-markdown/" | hxclean | hxwls) 2>/dev/null | egrep -v "(dailydrop|wp|wordpress|grav)" | grep "^ht"
https://github.com/needle-tools/md-browse
https://github.com/blackboardsh/electrobun
https://github.com/mixmark-io/turndown
https://github.com/needle-tools/md-browse
https://github.com/blackboardsh/electrobun
https://github.com/mixmark-io/turndown
https://mixmark-io.github.io/turndown/

Most of the rest have manual pages, but are kind of niche, and many of them are intended to weave together a publication.

Always nice to have more tools in the toolbox!

MapTheNet

While I may not have been in a regular Drop cadence, links/resources I might have covered in one have been blathered about on Mastodon over the past couple months, including this one.

I discovered MapTheNet from the daily Site Observatory update experiment I’ve been running. It looks at web traffic from a different lens than the boring ol’ “stats” perspective. One idiom it performs is traditional bot vs AI bot vs Fediverse vs visitor classifications, and when there are named misses it notifies and adds new classification rules.

MapTheNet’s bot hit the site yesterday and I poked around at their project.

It appears to be a non-profit, volunteer-run project crawling the public web at the domain level to build a link graph. It claims to not store page content or personal data and also to be a robots.txt-respecting crawler (the user agent is MapTheNetBot/1.0). They do have an opt-out form and have stamped CC BY 4.0 on all exports. There’s a broken public API at /api/ and some interactive page that seemed broken when I visited it.

They offer three gzipped CSVs under /exports/:

The domain was registered on 2026-01-01, and the datasets are a bit stale despite me seeing their crawler this week.

I’ll check back on it in a month or two to see if anything changes.

FIN

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on:

• Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev
• Bluesky via <https://bsky.app/profile/dailydrop.hrbrmstr.dev.web.brid.gy>

☮️