Real-time Mosaic Video Studio

Index

Overview

Ben Fry created AsciiVideo, a Processing sketch which transforms live video into ASCII text art, and shared his approach with the Processing language community. Instead of using brightness-sorted letters for pixels, this studio will take his work a step further and use brightness-sorted images instead.

The methods that you'll learn in this studio were used to create the video installation "Barbie Mirror", which translates live video into a moving mosaic of tiny Barbie doll faces. Create the same mosaic effect with your own image collections.

Get inspired! Ben Fry, Daniel Rozin, Golan Levin, Camille Utterback, Brian Knep, Philip Worthington

Ingredients:

Steps in the process:

  1. Collect thumbnail images and tour Processing.app
    1. Collect thumnail images. /try it
    2. Tour Processing application. /try it
      Download completed processingIntro0b sketch
  2. Walk through AsciiVideo sketch and modify video source:
      Download AsciiVideo sketch
    1. Add comments, subtract unnecessaries. /try it
      Download completed mosaicPresenterStep1a sketch
    2. Edit to use an existing movie file instead of live video. /try it
      Download completed mosaicPresenterStep1b sketch
  3. Switch from ASCII characters to thumbnail images:
    1. Pair character-related variables with new thumbnail-related variables.
      Comment "* Preview *" where related code changes will occur. /try it
      Download completed mosaicPresenterStep2a sketch
    2. Get thumbnail images (skipping file reading code) incorporated in place of characters. /try it
      Download completed mosaicPresenterStep2b sketch
    3. Get thumbnail image files read and working, with help from Daniel Shiffman. /try it
      Download completed mosaicPresenterStep2c sketch
    4. Repair vertical start position for placing thumbnails. /try it
      Download completed mosaicPresenter sketch
  4. At this point, we've got a basic working mosaic video presenter application. Next, let's create an application to get those thumbnail images analyzed and ready for sorting so that the presenter app can use them.

  5. Create a sort-ready directory of brightness-prefixed thumbnail images:
    1. Borrow useful code from mosaicPresenter/AsciiVideo code.
      Comment "* Preview *" what major code chunks need to do. /try it
      Download completed imageSorter3a sketch
    2. Calculate brightness of each image file. /try it
      Download completed imageSorter3b sketch
    3. Prefix brightness to image filename and save copy in "sort" subdirectory. /try it
      Download completed imageSorter sketch
  6. Fine-tuning options
    1. Quick experiments to improve your real-time mosaic. /try it
      Download completed mosaicPresenterStep4a sketch
      Download completed imageSorterStep4a sketch
    2. Generate .csv file showing how thumbnails map to 256 levels of brightness (a.k.a. histogram). /try it
      Download completed imageAnalysisCSV sketch
Skip to the end and download all the finished sketches instead

Let's begin!

Next: Step 0a: Collect thumbnail images and tour Processing.app - Collect thumbnails