Long and Short Viewing Relationships

When we design a variable typeface, we mostly concentrate to represent optimal visual performance of our typeface for a specific viewing angle – mainly the front view. How the typeface looked on other viewing angles was out of the designer’s control and as such disregarded. These flaws became noticeable in the work of the Swiss Graphic Designer Wolfgang Weingart and his photo-optical experiments in 1965 called “Examination of the Letter M”, in which he photographed the letter “M“ from different viewing angles to produce astounding type design variations. Although Weingart primarily did not intend his experiments for legibility and readability, he provided plenty of questions to think about when designing a typeface that can be read on multiple viewing angles. The effect of small and large visual angles on typography has already been explored and documented across in the medium of print in subtle ways. For example the research from Sofie Beier shows that letter boldness benefits on smaller visual angles than larger visual angles.

Exploring Angles

The aim of this project is to design a similar experiment, but in a digital context and to combine these approaches and findings with type through machine learning (ML) and variable fonts. While one could argue that machine learning (ML) may have its disadvantages in terms of privacy, it gives a glimpse into the possibilities of variable typography in new context aware spaces. Things like world location, acceleration, reading perspective/distance and ambient light all were not possible before, and now with the introduction of sensors and machine learning they become accessible to the designer. By understanding these advantages, we can communicate variable typography in new ways. As an example, the displayed typeface above (figure 1) showcases a variable typeface smoothly interpolating between various axes. Yet, the typeface solely operates as an animation and lacks responsiveness to the reader’s behavior in the given context. So the question stands: how can we instruct the typeface to adapt its axes seamlessly according to the reader’s viewing angle in real-time on the device using machine learning?

Machine Learning

One potential strategy to enable the typeface designed (figure 1) to adapt to the reader’s viewing angle is to utilize a face detection ML library that assesses the user’s proximity to the screen by detecting their face’s position in relation to the frame. There are many face recognition libraries available out there, but for this experiment, I opted to install the PoseNet library from ml5.js* (figure 2) to analyze the user’s camera feed and identify facial landmarks in the browser. Ml5.js is a user-friendly machine learning library that simplifies Machine Learning for designers by providing an accessible means of rendering graphical output to the browser.

Having loading the PoseNet library into the browser (which, by the way, is accompanied in a series of online tutorial by Daniel Shiffman on his YouTube channel, offering beginner-friendly machine learning tutorials and challenges), I was able to process facial landmarks detection like the eyes, nose, and ears, directly in the browser. This approach enabled me to manipulate the data according to my preferences. After having these data I utilized p5.js* to visualize facial landmarks like the eyes, nose, and ears, using the processed data generated by ml5.js (figure 3). To ensure a balanced calculation of the user’s position, I chose the nose as the central point of both the face and the entire screen. With the information furnished by PoseNet, I was able to pinpoint the exact position of the nose on the screen (as shown in figure 4). However, the data provided by the browser was too detailed and very complex to be edited with p5.js. Therefore I created a 12-column grid with p5.js to serve as a reference point for facial landmarks (figure 5). This approach allowed me to calculate the nose’s position based on its location on the grid (e.g., number 1, 2, or 3) and provide more specific information about the location of facial landmarks, in this case the nose.

The Typeface

The variable typeface developed for this project has been designed with a focus on adjustability and responsiveness to the reader’s viewing angle. The typeface includes three axes: the right master, left master, and weight master, which are all meant to be controlled by the machine learning library, PoseNet. The right and left master axes enable the typeface to tilt in the corresponding direction, providing a dynamic appearance that changes based on the reader’s perspective. In contrast, the weight master axis adjusts the thickness of the font, creating further visual interest as the typeface adapts to different angles.

Exploring Angles

After successfully setting up facial landmark detection using PoseNet and visualizing the data with p5.js, the focus of the experiment shifted to integrating a variable typeface onto the canvas. In the example below (as depicted in figure 7) you can see how the typeface is reacting to the user’s position on the screen. This is happening due to the fact that I am changing font-variation-settings as the nose is moving left and right to the screen. For example, if the user’s nose was detected in grid column 2, the font weight could be set to 500, and vice versa (figure 8). This approach provided greater control over the context of the typeface, as the user’s position could be determined and the typeface adjusted accordingly (figure 9).

Final Prototype

In this prototype, the interpolations of the variable typeface occur within the constraints of the built-in FaceTime HD camera located near the top edge of the retina MacBook Pro display. While this camera setup was sufficient for the purposes of this experiment, there is certainly room for improvement and expansion in future iterations. For example, a wide-angle camera (as one seen in the latest Ipads) could be used to capture a broader field of view and provide more flexibility in terms of viewing angles. Additionally, non-linear interpolation could be explored as a means of easing and animating the text as it is approached from different angles. These considerations could further enhance the user experience and make the variable typeface more adaptable to a wider range of digital contexts.