Processing

Processing is an open source programming language and environment for people who want to program images, animation, and interactions.

The pogg aka Processing Theora/Ogg Java Library allows to read TheArtCollider streams (or any Ogg/Theora video for that matter) into processing.

Example code reading video from the Collider using Pogg can be found here.

Pogg currently only supports decoding. For encoding & transmitting videos data back to TheArtCollider some tricks have to be used.

Basically we’re saving an image-sequence to disk and encode this sequence in real-time using command-line tools (ffmpeg2theora and oggfwd; see shell-scripts in General Usage Information).

The processing part therefore becomes very simple; at the end of the draw() function the current video-frame is dumped to disk. For synchronization (fixed framerate) the draw routine should be wrapped with a ‘millis()' timer:

// global variables
int FrameCounter = 0;
String FrameCounterOUT;
int Folder = 0;
int lastTime=0;

void doDraw(){
  // YOUR DRAW CODE

  FrameCounterOUT = nf(FrameCounter, 3);
  saveFrame(Folder+"/frame_"+FrameCounterOUT+".jpg");
  FrameCounter ++;
  if(FrameCounter > 950) {
    // OSX "open" requires '-g' to not bring the application to the foreground.
    // on Gnu/Linux or Windows simply using the command will suffice.
    String[] params = { "-g", "/usr/local/bin/blob_announce.command" };
    open(params);
    FrameCounter = 0;
    if(Folder == 0)  Folder = 1;
    else             Folder = 0;
  }
} // end draw()

void draw(){
  // TODO: check for wrap-around/overflow
  int now=millis();
  if (now-lastTime >= 65){ // 1sec/15fps = 65ms
    doDraw();
    lastTime=now;
}

The shell script - here invoked as usr/local/bin/blob_announce.command - which calls ffmpeg2thera and oggfwd is available for download. Note that it’ll require some minor modifications (Path to Processing folder, Project Name, etc.). More information below.

ffmpeg2theroa and oggfwd binary packages for OSX (10.5 & 10.6 PPC and i386) are available from http://theartcollider.org/software/. On Gnu/Linux they are available in most distributions, simply install them with a package manager.

The Source-code and Windows versions are available from Jan’s website; many thanks for providing these tools:

• ffmpeg2theora
• oggfwd

The aforementioned shell script is a rather pragmatic hack that came to be during the Linz Workshop. The locking mechanism and two folder-solution came to be due to the fact that the open command on OSX did not support passing commandline arguments to the shell script.

With a bit of java experience it should be straight forward to improve or rewrite the tool. It should even be possible to directly pipe single frames into ffmpeg rather than saving them to disk first.