TextDiffuser; Typography.js; Featured Foundry: Open Foundry

Looks like this is going to be one of “those weeks”. We’ll still get all the editions out, but I’m a day behind thanks to an unexpected surge in fam needs and work-work.

TL;DR

This is an AI-generated summary of today’s edition.

Aside: Perplexity goes back-and-forth between not having links, putting links up front, and doing this “here” link business. It also randomly alternates between numeric bullets and dashes (I try to always sub digits for dashes). Using it daily for this section absolutely demonstrates the stochastic nature of these beasts. I’m going to be playing with the prompt a bit over the coming weeks to see if it’s possible to get it to be more deterministic.

• The blog post begins with a discussion on TextDiffuser, a novel approach that addresses the challenge of generating accurate and coherent text in images using diffusion models. The author explains the concept of diffusion models and how they work. TextDiffuser uses a two-stage method: layout generation and image generation. The authors of TextDiffuser have also contributed the first large-scale text images dataset, MARIO-10M, and collected a benchmark to evaluate text rendering quality. The original work has already been updated, which readers can explore here.
• The second section introduces Typography.js, a JavaScript library that provides a toolkit for building websites with an emphasis on typographic design. The library is based on design concepts such as typographic scales and vertical rhythm, and allows users to set font weights, typefaces, and other typographic properties. It comes with a number of pre-existing typographic themes, but users can also create their own custom font theme here.
• The final section features Open Foundry, a free platform for curated open-source typefaces. Although the platform seems to have stopped adding to the collection, there are still many striking and fun typefaces for readers to explore and potentially use in their next project here.

---

TexDiffuser

I was running afoul of the art community the other day by trying to create a graphic with text on it for use in a presentation with one of the diffusion models, which got me poking at where we are in terms of the current stat of getting typography/text into image generation, which led me to the resource/topic of this section.

TexDiffuser (GH) (live) is a novel approach that addresses the challenge of generating accurate and coherent text in images using diffusion models.

As a refresher or introduction (since we just used a technical term that is likely familiar to most, but not all, reader) a “diffusion model” is a type of generative model that creates new data, such as images or sounds, by simulating a natural process called diffusion. This process involves two main steps: adding noise to the data and then learning to reverse this noise addition. You’ve seen the results of this quite a bit since November of 2022.

Start with an original image, then — in steps — add random speckles of noise to it, until it’s completely hidden in the noise. This is similar to how molecules spread out from an area of high concentration to an area of low concentration until they are evenly distributed, which is the natural process of diffusion.

After the data (our image) is fully noised, the diffusion model then learns to reverse this process. It effectively learns to remove the noise step by step, going backwards until it recovers the original clear image. This reverse process is not trivial; the model has to learn from many examples of noised and clear images to be able to do this effectively.

In the case of TextDiffuser, the method consists of two stages: layout generation and image generation. In the first stage, a Transformer-based encoder-decoder model generates character-level segmentation masks that indicate the layout of keywords in images from text prompts. In the second stage, a diffusion model generates images conditioned on the text prompt and the generated layout.

The paper’s authors contributed the first large-scale text images dataset, MARIO-10M, containing 10 million image-text pairs with text recognition, detection, and character-level segmentation annotations. They have also collected a benchmark to evaluate text rendering quality.

And, of course, there’s already a big update from the original work that you can also play with.

Despite how neat this work is, we (thankfully) still seem to have a long way to go before we also usurp humans in this space with AI, as can be seen, below.

Typography.js

Typography.js (GH) is a handy JavaScript library built by Kyle Mathews (the Gatsby dude). It provides a toolkit for building websites with an emphasis on typographic design that comes with a bonkers number of pre-existing typographic themes.

Kyle based this project on a few design concepts, including typographic scales and vertical rhythm. One can think of a typographic sizing scale, sort of like a musical scale. Being deliberate about the progression helps to maintain consistency and visual harmony across a design (just like the audio counterpart). Vertical rhythm, a concept we’ve covered before, involves the vertical spacing and arrangement of type on a page. Properly implemented, it helps to improve readability and the overall visual aesthetic of a design. This library also lets us set font weights, typefaces, and other typographic properties.

If none of the 30 built-in themes is suits you, you can also create a custom font theme. Doing so is as straightforward as crafting a config file, running a bit of code, and having it generate the CSS.

Featured Foundry: Open Foundry

Open Foundry is a “free platform for curated open-source typefaces; to highlight their beauty, activate ideas and encourage exploration.”

They seem to have stopped adding to the collection, but there are some striking and fun typefaces to explore and, perhaps, use in your next project.

FIN

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev ☮️