Introduction

Handling "level of detail" (LOD) is a common problem in video games. Typically, you have high-quality art assets (models, textures, normal maps, videos, etc) and you simplify them, in various ways, depending on how big they appear on screen. After all, it's a waste to draw a million-polygon object if it is far away and only occupies a handful of pixels.

So, the LOD process generally starts with a high-quality object, which is reduced as needed. Eventually, it might be too far away or offscreen, so it is not drawn at all. These LODs can be created automatically, be hand-crafted, or both.

From the first commercial game I worked on. These are not in-game LODs, but various stages of being made game-ready, by hand.

However, in my game, the player can zoom in forever, which kind of turns the LOD problem on its head. Now, instead of shedding detail as we move away from some maximum-fidelity version, there *is no* maximum-fidelity version. We generate more detail on demand. It is effectively infinite detail — the closer you look, the more there is.

This presents some unique challenges, relative to traditional LODs. At the same time, many of the core concerns are the same. I'll discuss some of these below.

Traditional LODs

There has been a long span of time (still ongoing) in the history of video games characterized by a drive towards realism. The next generation of hardware is more powerful, so games can have more polygons, bigger textures, better lighting, etc. etc. These become bullet points on the latest marketing materials, and consumers keep buying new GPUs and consoles, feeding this cycle. But the real world is still far more complex than any computer can simulate, so it remains a struggle to use the latest technology to get ever-closer to that goal.

I enjoy technology, and have followed various developments in video games. I didn't fully realize until I had to think about this topic in depth that there's some part of me that gets annoyed when (1) some cool demo shows new possibilities and then (2) upon seeing the demo, it's clear that there is still a fundamental limit that can't be passed. This is usually made worse by the fact that the demo-giver invariably glosses over the downsides of their approach, focusing only on hyping up the advantages.

Some examples:

• Oldie but goodie: "Unlimited Detail Technology!" (2010) [note 1]
• Sparse Virtual Textures, akin to id's "Mega Textures"
• Atomontage (2018), or an older video (2010), or a newer one (2022)
• The venerable Speedtree
• Epic's "Nanite" geometry — high-level introduction or a nice, detailed SIGGRAPH talk

In each of those demos, there are times when they zoom in to greater and greater detail, demonstrating the ability of their system to handle the LOD problem smoothly and impressively. Then, inevitably, they stop zooming in, and I feel somehow cheated. It was just getting good — keep zooming in! I want to see microbes! Then atoms!

Of course, they can't do that, because there is generally some "ground truth" maximum-resolution data they're working with. It might be scanned from the real world, or was procedurally-generated offline, but it's typically sitting on disk somewhere, and it is what it is. Now, there's no reason why they *couldn't* keep generating more detail — fractal zoom programs have been around for a long time, after all. Those generate new detail as needed. But that doesn't fit into the typical AAA art pipeline, and normal games just don't have a need for it. I wished there were a game that did.

Pinhole Universe has allowed me to scratch this itch [note 2]. I knew it could be done, but I couldn't find a great example of someone having done it already [note 3].

Challenges With Infinite Detail

Art in my game differs from typical video game art in a few important ways:

• It's all generated by algorithms, at runtime.
  • This means: no textures on disk, no meshes, etc.
  • The artist and programmer must be one person.
• When you zoom in on an object, it creates sub-objects, to expose the next layer of detail.
  • These can be a pretty direct representation of the original, or a transition to "new" objects, such as a solid-colored fish becoming individual scales, or a tuft of hair becoming individual strands (see below).
  • This provides the key thing I want: you never "hit the limit" when zooming in; there is always more to see. But we still have to generate it, somehow.

Some example LOD transitions.
Enlarged to show detail — like on a cereal box.

Thus, at no point is there a "maximum resolution" reference object available — there is no maximum resolution at all. That means we cannot downsample to generate the lower-detail versions, because there is no starting point to downsample from. Instead, we can only up-sample, starting with a given simple object. We need to create more detail that somehow "matches" the parent object, but while still creating something new, out of thin air.

In a very real sense, this game only has one object — "the universe" — and as you zoom in on it, more detail is exposed, from galaxies to planets down to oceans, cities, cats, poop, rocks, fingernails, atoms, etc.

The essential challenge, then, is: given a simple object, create a higher LOD version that we can plausibly transition into, as we get closer. In some ways, this is harder than the traditional LOD problem (we have to create something out of nothing, rather than working from a reference point), and in other ways, it is easier (we have more freedom).

This raises the question: "what makes a good transition?"

From Something to Something More

When transitions go poorly, it feels wrong, but it can be hard to enunciate *why*, or know how to make it better. Some aspects of an image might transition poorly, yet go unnoticed, while for other aspects, even a slight deviation from expectations can be jarring. A chunk of this comes down to how the human visual system works.

Our eyes are good at noticing some information, such as brightness of light, silhouettes of objects, faces, and sudden movements. They are less good at noticing other things, such as color/hue, slower movements, oblique angles, etc. Some information is discerned "pre-attentively" — it happens without us being aware of it (examples). These concepts underlie the hierarchy of data visualization choices, painterly techniques, optical illusions, and more. And LODs too, of course.

I haven't solved all of these problems, but they're good concerns to keep in mind. For Pinhole Universe, there are a few main techniques that I've found work pretty well, pretty often.

• Avoid a realistic art style. More of a meta-technique, really. This is one reason why games like Tempest or Pac-Man seem timeless, but graphics in a game from 10+ years later look pretty dated. People have a lot of experience with reality, so they know when it looks wrong. They don't have as much experience with your made-up art style.
  • Here, a 2D style is a further advantage. People live in 3D, and have more experience and expectations surrounding it.
• Preserve the silhouette. The edges of objects are not only something we notice well, but often have sudden, high contrast changes — such as a dark object on light background, or a jump in hue. Even small changes to the silhouette are jarring.
• Use subtle color variations. Saturation changes are less noticeable than hue changes. Hue changes are less noticeable than brightness changes. I'm using the Oklab perceptual color space, which makes these things easier to express than in standard RGB. The hair strands above and the snowflakes below exhibit this.
• Fade, rather than popping. See the flower video, above.
• Use fused/composite shapes. This is fairly specific to Pinhole Universe's peculiar graphics system, but is an easy and cheap way to display more detail earlier.
  
• Keep transitions small onscreen. This has a direct tension with performance issues, so is not always possible — if objects are small, you need more of them to cover the screen. But obviously, smaller transitions will be less noticeable, so less distracting.
• For a rough transition, at least make it interesting. It's easier to forgive a jarring transition if it's entertaining and/or if the destination is somehow satisfying. See the flower video above (with the added orange dot), and the mite zoom video, where various hairs, fronds, etc. appear somewhat suddenly. If your eye is upset but another part of your mind is tantalized, it can work out.

So far we've been discussing LODs as a purely visual effect. With traditional LODs, this is usually the end of the discussion, since the ideal goal is known (i.e. reality, or at least the highest-definition version of it that we can display), and we just try to get as close to that as possible. But when we generate new detail on demand and are not necessarily following reality, there's not only the question of how similar the new LOD *looks*, but also how well the new objects *conceptually* match.

Ultimately, the LOD/transition problem comes down to managing expectations. The visual system you carry around with you has certain built-in expectations, and when they are not met, something feels wrong. At the same time, your mental model of the world also has certain built-in expectations, and when they are not met, it can also feel wrong.

Consider these:

• Expectation not met: Disappointment. The transition feels bad.
• Expectation met: It's okay. Perhaps not even noticeable. This is the goal for normal LOD systems — to be unnoticed.
• Expectations exceeded or flipped: This can be fun! In fact, it is a big source of the fun in Pinhole Universe (and life in general?). What you see next is not always what you expect (that would be boring), but we still want it to "feel right" and be convincing, somehow [note 4].

This is a big topic, and one of the main artistic and design struggles for me, making this game.

Here are some examples of zooming in on a snowflake:

We have an expectation of what a snowflake is, and what it's made of, even if we don't have a detailed mental image. We also have familiarity with bricks. And we know that snowflakes are not made of bricks. So, when we see the transitions in the top row, there is some mental dissonance. Also, flat bricks are a little bit boring; in the top-left, the transition is harder to forgive since it's so plain. Adding just a little color variation (top-right) makes it slightly more engaging.

Moving away from bricks entirely seems to work even better, as in the bottom row. Perhaps because this looks a little more like some "ice crystals" concept we have in our mind, the transition feels less harsh. Making the shapes jut out from the silhouette, as in the bottom-right case, also seems somehow more "crystal-y" and satisfying, to me — it's a rougher visual transition, but more fun to see.

Weighing all of these factors is ultimately an artistic decision — what effect do you want to achieve? It's hard to know without trying and evaluating with your own eyes (another point in favor of rapid iteration), but keeping these guidelines in mind can help guide your efforts.

Sometimes, even real-world transitions and imagery are not what we expect. In this way, even actual photographs of the real world can seem "wrong" or "fake" to us. Making a believable world is not just about being realistic, it's about matching people's inner expectations. And their expectations may not match reality very well at all! This can be especially true when getting outside the range of normal human perception — like when things are very big or very small.

Some pictures of real life, perhaps not fitting expectations.

So, there is a constant tension between (1) the risk of jarring the player too much, making an unpleasant experience, (2) challenging the player's expectations somewhat, to create a new fun experience, and (3) making artistic choices about what I want to portray, and how to do it. There is no silver bullet solution for this, alas.

Know What You're Giving Up

I mentioned earlier being annoyed when people hype up their technology without mentioning the corresponding downsides. I don't mean to hype, but lest I be remiss, here are some counterpoints to the choices I've made:

• Avoiding realism: Go too far in this direction, and you have a "fractal explorer" sort of game, detached from reality. It can be pretty, and interesting, but may feel somewhat alien. It can get boring since it's not relatable on a human level. In one sense, creating your own non-realistic art style is more difficult, since you need to come up with something new but still somehow convincing. In another sense it is easier, since people bring fewer expectations (which of the two is harder is an age-old art debate).
• Performance: It takes a lot of CPU to constantly generate things. Luckily, this game doesn't have much else going on — no physics, animations, networking, etc.
• Creating Content: Since the art is generated by code, you can't benefit from the traditional division of labor between artists and programmers.
• Fidelity: One nice thing about textures is that they are jam-packed with high-frequency data. We don't have textures; instead, we just use more objects. So, each object is relatively flat, simple, and low-frequency. This is not a general limitation, just one for my game — we *could* generate just as much detail as is found in traditional textures, but I decided against it, for both practical and artistic reasons.

Wrangling with these issues has been fun and educational. Also difficult. Much of the programming, engineering, and mathematical side of this project has been about solving specific problems in precise ways. But the visual side is an intersection of science, art, design choices, and brute-force experimentation. Even in this very minimal sort of game (no multiplayer, no 3D or animation, etc), it's a forest of possibilities to explore. It would take more than a lifetime, really. But I hope to do it justice!

Thanks for reading.
– John

Discussion:

• Reddit
• Hacker News

Footnotes:

1. • Key quote: "Ideally, we would prefer a lettuce not to have any flat sides at all"
   • Joking aside, the start of the video is actually a decent layman's overview of LOD issues.
   • Take a drink when they say "unlimited" (it's not, of course).
2. Truly, this project scratches several itches for me. I had been working on it for some years already before I realized how central to my personality some of its concepts are. It has been an illuminating experience, in that way (:
3. Incidentally, if you know of a game that does, I'd love to add it to my big list of prior art and inspiration — please drop me a line.
4. Undermining expectations while still being satisfying is what makes for enjoyable art of all kinds — whether it's video games, painting, cooking, or comedy. It's hard to pull off consistently.

If you like these articles, feel free to sign up for the mailing list:

Subscribe

(we never share your address)

We also have some social media links, if you want to follow or contact us.

And if you feel like buying me a cup of tea, I won't say no (: You can use Ko-fi or Patreon .