Interactive Illustrations

This is an idea I’ve been toying with for quite some time. Letting programming and painting meet somewhere in the neighborhood of illustration / animation / motion graphics / creative coding / games. There’s not much of a concept yet, it will take some trial and error, but a few central ideas are:

  • each illustration should work as a stand-alone still image
  • I want to “add magic” by animating things (the moving pictures in Harry Potter would be an ideal)
  • some interactivity should be there, otherwise it would just be a video

So here’s a first demo, where I explored how to do something like this in a web browser. This is from a painting I recently developed from an old inktober sketch, titled “Sugar Raid” for obvious reasons 🙂

Click on the image to open the interactive version in a new window:

Sugar Raid!

A few technical details

The illustration itself was done in Procreate (the iPad painting app), where each moving object resides on its own layer. The implementation of the demo required:

  • exporting the layers as image files
  • cutting and resizing the layer images, also generating a Javascript snippet with size and offset information for each layer
  • writing Javascript code (using the quite awesome three.js library) to render all the layers – essentially another implementation of how a painting app composes the layers. I use custom shaders to support blend modes such as screen, overlay, etc.
  • animating layers – pretty straightforward in three.js
  • as a boon, I added a particle system simulation (the exploding sugar cube)
  • the painful part: testing in various browsers on the desktop and smart phones