Glyph-ByT5; opentype.js; SpotiFont

Today’s TT focuses mostly on some technical underpinnings of fonts, and I’m kind of stoked about the promise of the first section.\

TL;DR

(This is an AI-generated summary of today’s Drop)

• Glyph-ByT5: Glyph-ByT5 is a customized text encoder designed to improve text rendering in image generation models by focusing on character-level understanding and glyph alignment. This approach significantly enhances text accuracy in generated images, achieving nearly 90% accuracy compared to less than 20% with previous models. https://glyph-byt5.github.io/
• opentype.js: opentype.js is a JavaScript library for parsing and writing OpenType and TrueType fonts. It allows developers to manipulate fonts directly in the browser or Node.js environments, supporting advanced text rendering features like ligatures, kerning, and Arabic text rendering. https://opentype.js.org/
• SpotiFont: Spotify introduced a new custom font called “Spotify Mix,” developed with Dinamo Typefaces. This variable font is designed to reflect the dynamic nature of audio culture and will be used across Spotify’s branding and content, enhancing visual consistency and user experience. https://newsroom.spotify.com/2024-05-22/introducing-spotify-mix-our-new-and-exclusive-font/

Glyph-ByT5

Current text-to-image generation models like DALL-E and Stable Diffusion often struggle to render text accurately within generated images. The text frequently contains spelling mistakes, misshapen letters, or incorrect character spacing.

The core issue lies in how these models process text input. They break down text into tokens (e.g., words or sub-word chunks) rather than processing individual characters. This makes it difficult for the model to truly understand how words are spelled at the character level.

The researchers behind Glyph-ByT5 (GH) (arXiv) identified two key requirements for accurate text rendering:

1. character awareness: the text encoder needs to understand text at the individual character level, not just tokens.
2. glyph alignment: the encoder needs to align the text with the rendered glyph images (the visual shapes of characters).

They created this customized text encoder by fine-tuning an existing character-aware encoder (ByT5) on a carefully curated dataset of text prompts paired with rendered glyph images.

They then integrated this Glyph-ByT5 encoder into an image generation model called SDXL. This new model, dubbed “Glyph-SDXL”, showed a massive improvement in text rendering accuracy from less than 20% to nearly 90% on their design image benchmark.

Glyph-SDXL could now render paragraphs of text with high spelling accuracy and proper multi-line layout planning within specified text boxes. After further fine-tuning on real scene images, it also substantially improved text rendering in open-domain photographs.

In essence, by customizing the text encoder to truly understand characters and align with rendered glyphs, the researchers overcame a fundamental limitation in current text-to-image models regarding accurate text generation.

There’s tons more to explore in the paper and repo.

I don’t get excited about many aspects of this AI devolution, but this project has all sorts of potential, and I’m looking forward to seeing where it turns up.

opentype.js

opentype.js (GH) is a JavaScript library designed for parsing and writing OpenType and TrueType fonts. It gives us the ability to access and manipulate the letterforms of text directly from the browser or Node.js environments. This is quite useful for apps that require custom font rendering, text manipulation, or font creation.

They pack a ton of things into the tin, including (cribbing from the GH README):

1. Font Parsing and Writing:
   • opentype.js can read and write OpenType (OTF) and TrueType (TTF) fonts, supporting both TrueType glyf and PostScript cff outlines.
   • It also supports Web Open Font Format (WOFF) and WOFF2, with the latter offering better compression but at the cost of increased library size.
2. Glyph Access and Manipulation:
   • The library provides raw access to glyphs, allowing developers to modify them as needed. This includes creating Bézier paths from text, which can be used for custom rendering or animations.
   • It supports composite glyphs, such as accented letters, and can handle kerning (adjusting the spacing between characters) using GPOS or the kern table.
3. Advanced Text Rendering:
   • opentype.js supports ligatures (combined characters), hinting (instructions to improve font rendering at small sizes), and Arabic text rendering.
   • It also supports emojis and other SVG or COLR/CPAL color glyphs, making it versatile for modern web applications.
4. Cross-Platform Compatibility:
   • The library runs in both browser and Node.js environments, making it suitable for a wide range of applications from web development to server-side font processing.

Folks can install it via npm with npm install opentype.js or use it directly from a CDN.

Example usage in a browser with ES6 modules:

 import { parse } from "https://unpkg.com/opentype.js/dist/opentype.mjs";
 parse(...);

(Yes, it’s that straightforward.)

Whether you are building a web application that needs precise control over text rendering or a Node.js service that processes fonts, opentype.js has everything you need.

There are tons of examples in the README, and the interactive explorer is also kind of fun to use!

SpotiFont

(I’m including this because Spotify is pretty sincere about the reasons for developing this new font. I know not everyone or every org can, today, commission their own font. But, I suspect “AI-customized fonts” to be a reality very soon, and it may help to make such a customized font a reality for some subset of those folks/orgs. I do hope that AI is trained on fonts billionaires & VCs paid for.)

Spotify has introduced a new custom font called “Spotify Mix,” developed in collaboration with Berlin-based foundry Dinamo Typefaces. This new typeface replaces the previous Circular font and is designed to reflect the dynamic and evolving nature of audio culture. Spotify Mix blends classic and contemporary styles, incorporating shapes reminiscent of sound waves to evoke a rhythmic feel. The font features a combination of sharp angles and smooth curves, giving it a distinctive character that feels quintessentially Spotify.

Spotify Mix is a variable font — a Drop fave techincal detail — allowing for changes in weight, width, slant, and other variables without requiring separate files. This flexibility enables Spotify to customize the font for various applications, from playlist titles and artist names to album covers and marketing campaigns. The new typeface will be used across all content written in Latin-based scripts and Vietnamese, gradually rolling out over the coming weeks.

The introduction of Spotify Mix is part of Spotify’s effort to create a cohesive and recognizable brand identity. The new font will also update the company’s logo, ensuring a consistent visual experience across the app, website, and marketing materials. By developing its own bespoke typeface, Spotify aims to enhance its visual branding and provide a more engaging user experience.

FIN

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev ☮️

6 min read

Drop #475 (2024-05-28): Typography Tuesday