This is an idea I’ve been toying with for quite some time. Letting programming and painting meet somewhere in the neighborhood of illustration / animation / motion graphics / creative coding / games. There’s not much of a concept yet, it will take some trial and error, but a few central ideas are:

• each illustration should work as a stand-alone still image
• I want to “add magic” by animating things (the moving pictures in Harry Potter would be an ideal)
• some interactivity should be there, otherwise it would just be a video

So here’s a first demo, where I explored how to do something like this in a web browser. This is from a painting I recently developed from an old inktober sketch, titled “Sugar Raid” for obvious reasons 🙂

Click on the image to open the interactive version in a new window:

A few technical details

The illustration itself was done in Procreate (the iPad painting app), where each moving object resides on its own layer. The implementation of the demo required:

• exporting the layers as image files
• cutting and resizing the layer images, also generating a Javascript snippet with size and offset information for each layer
• writing Javascript code (using the quite awesome three.js library) to render all the layers – essentially another implementation of how a painting app composes the layers. I use custom shaders to support blend modes such as screen, overlay, etc.
• animating layers – pretty straightforward in three.js
• as a boon, I added a particle system simulation (the exploding sugar cube)
• the painful part: testing in various browsers on the desktop and smart phones