Creating videos with Processing

Other than interactive live applications, Processing can can also be used to generate cool animations from frame sequences! It‘s not so hard, but you need to consider a few things.

Randomness

If your animation uses random values, be aware that random() creates different results each time the program is run. To make sure you always get the same good-looking frame sequence of your choice, use randomSeed().

Recording the frame sequence

Set resolution and frame rate in setup():

void setup()
{
size(640, 640);
frameRate(30);
}

The frame rate doesn’t have to be the same as in the final video, you can speed up or slow down the sequence when creating the video file.

To save the first 100 animation frames, include this snippet:

if (frameCount <= 100)
{
saveFrame(“frames-###.jpg”);
print(“.”);
}

Video file creation

On a Mac, this is easy – you can use the built-in Quicktime player tool to create a video from your image sequence.

On a PC, a good option is ffmpeg. One way to install is as part of the ImageMagick suite, which is a very handy collection of image processing tools.

It’s a command line tool with a billion options. A minimal command to generate an MP4 file from an image sequence looks like this:

ffmpeg -framerate 30 -i frames-%03d.jpg -c h264 output.mp4

Video format considerations

Depending on how you want to use your video, you’ll want to make sure format, frame rate, and image size are appropriate.

For instance, to post a video on instagram, the recommendations are:

  • Aspect 1.9 to 1:1
  • a minimum of 600×315, resp 600×600 px
  • maximum width 1080 px (anything bigger is downscaled)
  • 3 sec minimum duration, 60 sec max.
  • 30 fps maximum frame rate
  • H.264 or VP8 codecs

Happy recording!

Resonance

Ever since I started drawing, trying to understand how ‘real artists’ work and think, I was often surprised to learn about something I knew from my own profession! Even between two activities so apparently far apart as pure logic and abstract thinking on one side and making colorful marks with emotional impact on the other, one can find resonance.

Here’s the shortlist of some of my obvious and not-so-obvious findings:

Problem solving

Programming is obviously a problem solving activity. It was really news to me how artists would view much of their work as problem solving, though. It may be a concept artist, who is given the task of designing a vehicle within given constraints until tomorrow, or a painter figuring out the perfect composition, shape language, lighting, color palette…

I think the analogy carries pretty far. To solve your problems, you can work methodically top/down (resp. rough to detailed), or employ trial and error. If you didn’t solve your fundamental problems properly early on, they will come and bite you later (I haven’t heard anyone talking about a ‘bug-ridden picture’ so far, but they certainly exist 🙂 ).

Attention to detail

This seems quite obvious. What is a single misplaced character breaking a program, can be one wrong line ruining your sketch. And you should absolutely know what every element in your program / picture is doing to help achieve its goal. Useless clutter, anyone?

Teams of introverts

It seems both programmers and artists generally prefer to be left alone to work on their piece for endless hours. In professional reality, it is important, though, to communicate a lot with others, as projects get bigger and bigger all the time. Feedback is also a topic here (code reviews, art critique).

Life-long learning

To me, at least, doing something that you’ve never done before is the essence of software development, always expanding your mental toolbox, so you can tackle bigger tasks the next time. There is no finish line, where you have learned everything there is and will be. The same seems to be true for passionate artists, as far as I can tell.

Creative bursts and discipline

In coding, painting, and I guess all creative endeauvours, there are spurts of energy and inspiration, followed by long stretches of filling in the gaps, where great discipline is required to get through the project. Understanding how much making art also requires discipline and quite simply showing up and getting to work helped me a great deal. You’re not a failed artist, if you aren’t inspired every day. You’re not a bad developer, if you can’t produce brilliant code like a machine.

Composition and architecture

As much as a good initial composition can decide over success or failure of a picture, the software architecture of your system can determine the fate of your project. There are functional as well as pleasing designs, concepts such as symmetry and readability in both worlds.

Looks too easy

From personal experience, I know the complete inability of, say, upper management in some companies, to understand why all of this software business is taking so long, when certainly it can’t be that complicated and the computer is doing the work, after all. And if things seem easy and fast, nobody can see it’s the result of years of training and countless hours of practice. Then again, you’ll hear ‘oh, I could never do that’ very, very often 🙂

I’ve heard the same, anecdotally, from working artists.

Style

Obvious in the arts, but there’s personal style in programming as well! Even if everyone in your team is following the same coding guidelines (where to put the braces, etc.), chances are you’ll be able to recognize who in your team wrote a piece of code after a while. Also, you’ll find people able to adapt their style to fit into whatever project they’re working on, while others value their freedom of expression higher than anything else – they might be brilliant, but substantially harder to work with.

I could elaborate on some of these topics, but this is essentially what I found so far, Anything missing? Probably 🙂

It might be worth writing about fundamental differences between making a drawing and making software, too. Have to think a bit about that.

Clara & Pong chapter 4 is done

page 72!

A major milestone: chapter four of my comic “Clara & Pong” is done – 72 pages into the storyline. If you can read German, please head over to clarapong.com to check it out!

Before moving on to drawing the final chapter, I thought it might be interesting to share some details about my process. Since this is – especially for a relative beginner like me – a massive project, with around 100 pages, running already for more than two years – it’s pretty important to me to have a smooth workflow.

Story pipeline

It all starts with the script in text form. The major lesson I learned: Know where you’re going! I did have the major story points in mind from the beginning, and that helped me tremendously in keeping the story together. Mind you, I didn’t have all details fleshed out before starting to draw, but I could always be certain I wouldn’t draw myself into a corner.

I guess the “Right Way” to do this would have been:

  • write the script
  • draw the storyboard
  • layout the pages
  • render the pages

In my case, it turned out to work best incrementally, in overlapping “pipeline stages” with three tracks:

  • detailed script, chapter by chapter
  • storyboard, scene by scene, feeding from finished script parts
  • layout, page by page, feeding from finished storyboard panels

So, I finished the script first, the storyboard later (just a few weeks ago), and by now am conveniently pulling off panels from the storyboard, assembling pages (obviously, I’ll make corrections and put in new ideas all the time, but so far I never had to go back and change major parts of the story).

The storyboard

might look messy to you, but really provides great overview

The only non-digital part of the pipeline 🙂 I draw my storyboard on big sheets of paper, in a 6×6 grid. The drawings are just barely enough to show what’s happening in each panel, so I get the sequence down. (The picture only shows chapters 3 to 5 – before, I didn’t have a consistent approach.)

Layout & scribbles

Picking panels from the storyboard sheet, I assemble the layout for the next page. Usually I try to end each page with a transition into the next page – a smaller or larger cliffhanger, if you will. For layout and scribbling, I use my iPad Pro, moving stuff around and drawing in layers until I’m happy enough to move on to inking. Important: Make sure there’s enough space for the text!

Inking & text

layer setup

Next, I transfer the scribble to the PC (a Mac, in my case), where I import it into ClipStudio Paint, as a background image layer. My template already has two vector layers for foreground and background lines. Dividing the page up into panels gives me separate folders for each panel. The image on the left shows the layer setup of the finished page.

I love drawing in vectors, because it allows for easy corrections of shapes. When I’m done, I usually go over all the lines and adjust line weights to keep them consistent across the page.

After adding text and drawing the balloons around it, I move on to coloring the page.

Coloring

Comic palette

Here, I usually work on just one foreground and one background color layer for the whole page. I select all the frame folders, and choose “combine”, which results in a new folder masking out all the frame border areas of the page. I place my two color layers in that folder.

My coloring style is very simple, so I typically can just fill areas with flat colors. I restrict myself to a fixed color palette – which I did carefully extend over the first few chapters, but has now become very stable. Most importantly, it contains bright and dark values in warm and cool grays, plus the essential colors for my characters. A palette like this is really helpful for keeping things consistent and efficient.

… and done

Upload the page to the my website, announce it on FB, and that’s it, really. It’s all working pretty smooth and painless by now, so I can keep my pace of doing one page a week. With the completed storyboard, I now know how many pages are lying ahead, so projections show it’s quite possible to finish by the end of the year. Yay!

Clara & Pong

In 2016, I started my own web comic revolving around a girl named Clara and her best friend Pong, a red panda. These were just short weekly strips, which I initially inked by hand, scanned in, and colored digitally. Here’s one of the first episodes, where Pong receives his name:

After about a year, it was time to start a bigger story about my two heroes! I didn’t realize how big it would become, though: The story has now reached 70 pages, with one published page per week. The final book will be around 100 pages, hopefully finished by the end of 2018. It’s only German so far, but I hope to do a translation once I’m done with the first pass. Here are a few sample pages:

This continues to be a huge and rewarding personal adventure. Just as in software development: When you don’t even know what you don’t know about what it takes to finish a project, that’s where the fun is and where the best learning happens.

Lightbox Trace 2.0 with In-App feature

I recently published version 2.0 of my iPad App Lightbox Trace, introducing a filter panel as an optional In-App feature. This really comes in handy, when your sketches don’t have enough contrast or you want to desaturate a colored image before tracing. Check it out!

Unfortunately, I ran into a problem with enabling the In-App purchase, so it just didn’t work for about a week. If you tried and got an error message about not being able to connect to the App Store, please try again. It really should be working now. If there are still any problems, please send me an e-mail. I appreciate your support!

New iOS app: Lightbox Trace

I’m currently spending a lot of time drawing on my iPad Pro, and needed a way to transfer my digital sketches to drawing paper. Essentially, a lightbox with the ability to display an image. Since I still had my old first-generation iPad lying around, I developed a simple little app to put it to use again: Lightbox trace.

  • Load an image from photos or the clipboard
  • Scale, position, rotate as desired
  • lock the screen – the app now ignores all touch events, so you can put a piece of paper on the display and trace the image
  • display brightness is automatically increased to the maximum
  • you can also just show white, for tracing from one paper to another

I’ve found it to be quite useful – please try it (it’s free!) out and let me know if there is anything you might want to be added.