Overfloater

I’ve been quite busy these days.

I’ve been answering questions at the JavaScript InfoVis Toolkit Google Group, fixing a couple of bugs for the JIT and putting a lot of effort in V8-GL.

Oh right, I was almost forgetting that I have a day job also :S

Anyway, V8-GL kind of took another direction. I’ve been working on OpenGL ES 2.0 and OpenGL 2.1 bindings.

OpenGL ES 2.0 bindings are almost complete. Other bindings (like OpenGL 2.1, GLUT and GLU) need some extra work, but they are functional.

Dean helped a lot also. He just translated Vlad’s metatunnel example to work with the new OpenGL ES 2.0 bindings. The result’s pretty impressive:

You can find some videos of “Metatunnel” implemented in Processing or Canvas3D at Youtube.

The good thing about the V8-GL example is that’s a Desktop app coded in JavaScript using V8.

This post is about performing advanced camera movement in OpenGL.
We’ll use the same Radiohead’s HoC dataset we used in the previous post.

Once again, the quality of the youtube video is pretty lame. You can right click here and save link as… to download a high quality version of the video (~100MB).
I strongly recommend you to see the high quality video

Camera Instructions

Camera movement is made of Translations and/or Rotations.
We want to provide our camera model with instructions of the type:

• [ Translate from to ]
• [ Rotate from to rotation_axis ]
• [ Translate ...; Rotate ... ]

As the last example shows, multiple transformations can be done at the same time (translations and rotations).

The definition for a transformation type is:

type camera_op =
  | Translate of Gl.point3 * Gl.point3
  | Rotate of float * float * Gl.vect3

A camera instruction is a list of these operations (camera_op) and a number specifying the number of frames this transformation should take (i.e the duration of the transformation).

So, for example, this instruction:
( [ Translate( (100., 100., 100.), (0., 0., 0.) ) ], 300. )
translates the camera from (100, 100, 100) to (0, 0, 0) in 300 frames, that is in 10 seconds (at 30 frames per second).

Translation is done by simple interpolation. The interpolation formula for translating from A to B is something like this:
A + (B – A) * delta with delta in (0, 1).

Transitions

It would be nice if camera movement, besides being linear, could also perform other advanced transitions, like the ones used in Fx.Transitions by Mootools.
Some of these transitions are: Quadratic, EaseIn, EaseOut, EaseInOut, Back, Sine, etc…

These effects are achieved by applying functions to the delta value, changing the way it increases or descreases its value.
A possible interface for a Transition module is:

type trans = Linear | Quart
type ease = None | EaseOut | EaseIn | EaseInOut

val linear : 'a -> 'a
val quart : float -> float

val ease_in : ('a -> 'b) -> 'a -> 'b
val ease_out : (float -> float) -> float -> float
val ease_in_out : (float -> float) -> float -> float

val get_transition : trans -> float -> float
val get_ease : ease -> (float -> float) -> float -> float

val get_animation : trans -> ease -> float -> float

By using Transition.get_animation Quad EaseInOut delta we can change the timing of our animation from this:

into this:

Our camera instructions are then defined as:

type camera_op_list = (camera_op list) * float * (trans * ease)

For example:
( [ Translate( (100., 100., 100.), (0., 0., 0.) ) ], 300. (Quad, EaseOut))

The Camera Model

A possible interface for the camera model is:

class camera_model :
  object
    val mutable animations : camera_op list
    val mutable time : float
    val mutable total_frames : float
    val mutable transition : Transition.trans * Transition.ease

    method get_time : float
    method step : unit
    method draw : unit
    method translate : Gl.point3 -> Gl.point3 -> float -> unit
    method rotate : float -> float -> Gl.vect3 -> float -> unit
    method set_animations :
      camera_op list * float * (trans * ease) -> unit
  end

The camera_model instance variables contain the destructured camera_op_list type elements: animations, total_frames and transition.
We also provide individual methods for handling translations and rotations. These methods simply compute a delta value, apply the interpolation and then call GlMat.translate3 or GlMat.rotate3.

The 40 line implementation looks like this:

class camera_model =
 object (self)
  val mutable total_frames = 0.
  val mutable time = 0.
  val mutable transition = (Linear, None)
  val mutable animations = []

  method get_time = time

  method set_animations ans =
    let (x, y, z) = ans in
      animations <- x;
      total_frames <- y;
      transition <- z;
      time <- 0.

  method step =
    if time < total_frames then
      time <- time +. 1.

  method translate start last delta =
    let (trans, ease) = transition in
    let delta_val = Transition.get_animation trans ease delta in
    let (x, y, z) = start in
    let (x', y', z') = last in
    let DVertex(a, b, c, d) = Interpolate.cartesian
                                (DVertex(x, y, z, 0.))
                                (DVertex(x', y', z', 0.))
                                delta_val
    in
      GlMat.translate3 (a, b, c)

  method rotate start last vec delta =
    let (trans, ease) = transition in
    let delta_val = Transition.get_animation trans ease delta in
    let ang = Interpolate.cartesian_float start last delta_val in
    GlMat.rotate3 ang vec 

    method draw =
      let delta = time /. total_frames in
        List.iter (fun anim ->
          match anim with
            | Translate(start, last) ->
               self#translate start last delta
            | Rotate(start, last, vec) ->
               self#rotate start last vec delta ) animations
end

Timeline

Now that we have our camera model, we need a “timeline” object that can pass intructions to the camera at different stages of the animation.
We define a class-less object timeline that holds a list of camera transformations to be executed at a specific frame of the animation:

let timeline =
  object (self)
    val mutable frame = 0.
    (* Starting frame number, camera_instructions *)
    val camera_timeline = [
      (1.,   (* camera_instructions *));
      (310., (* camera_instructions *));
      (631., (* camera_instructions *) ]

    method get_frame = frame

    method tick =
      frame <- frame +. 1.;
      self#update_camera;

    method update_camera =
      try
        let camera_anim = List.assoc frame camera_timeline in
          cam#set_animations camera_anim;
        with
          | Not_found -> ()
end

Download and Use

This is all I’ve done to handle camera movement.
I’m not an advanced OpenGL/OCaml developer, so any comment/suggestion about my understanding of OCaml/OpenGL is very welcome.
You can download the source here.
You can compile the source with:

ocamlc -g str.cma -I +camlimages ci_core.cma ci_jpeg.cma ci_bmp.cma
-I +lablGL lablglut.cma lablgl.cma interpolate.ml transition.ml
camera.ml loader.ml main.ml -o main

Last part of this “trilogy” will be about particle transformations in OpenGL.
Hope you enjoyed it!

So I was looking for some excuse to learn OCaml + OpenGL, and I run into Radiohead’s House of Cards video dataset hosted at google code.
The dataset is made of many CSV files, one for each frame of the HoC video.
The data is also shipped with an application that uses Processing to create an image for each frame of the video.

I decided to do the same program in OCaml + OpenGL: for each CSV file, the program loads it, renders it in OpenGL, and then saves that rendered data into a jpg (or bmp) image.

You can merge the generated image frames with the sample mp3 provided at google code, by using ffmpeg:

ffmpeg -f image2 -r 30 -i ./img%d.jpg -sameq -i 1.mp3 ./out.mpeg -pass 2

Anyway, the result is quite interesting, and it gives us a good ground to build better visualizations:

(This youtube video quality is pretty lame, I’d recommend you to right click here and save link as…).

This post is going to be about the making of this simple application.
Further posts on this “project” will cover advanced camera movement and particle transformations.

The Code

This app was made in one single file, but it contains two important parts:

A data object containing information about the location of the CSV and generated image files, along with some methods to load CSV files and save OpenGL rendered pictures into image files (bmp and jpeg formats).
This object uses camlimages for saving images in different formats, and the OpenGL/GLUT bindings provided by lablGL.

(* Loads csv frames and saves the rendered OpenGL image *)
let data =
  object (self)
    val path_to_file = "path_to_folder_containing_csv_files"
    val path_to_image_file = "path_to_folder_that_will_contain_imgs"
    val mutable current_frame = 1
    val total_frames = 2101
    val time_interval = 33

    method get_time_interval =   time_interval

    method load_file filename =
      let channel = open_in (path_to_file ^ filename) in
      let ans = ref [] in
        try
          while true do
            let line = input_line channel in
            let sp = split (regexp ",")
                        (sub line 0 (pred (length line))) in
            match List.map float_of_string sp with
              | [ x; y; z; d ] ->
                ans := DVertex (x, y, z, d) :: !ans
              | _ -> raise (Invalid_argument "not a depth vertex")
          done;
          !ans
        with End_of_file | Invalid_argument _ ->
                 close_in_noerr channel; !ans

    method save_image =
        let img_rgb = new OImages.rgb24 600 400 in
        let pixels = GlPix.read
          ~x:0 ~y:0
          ~width:600 ~height:400
          ~format:`rgb ~kind:`ubyte
        in
        let praw = GlPix.to_raw pixels in
        let raw = Raw.gets ~pos:0 ~len:(Raw.byte_size praw) praw in
        let w = GlPix.width pixels in
        let h = GlPix.height pixels in
        for i = 0 to pred (w * h) do
          let color_rgb = { r = raw.(i * 3 + 2);
                                  g = raw.(i * 3 + 1);
                                  b = raw.(i * 3 + 0) }
          in
            img_rgb#set (i mod w) (i / w) color_rgb;
        done;
        img_rgb#save (path_to_image_file ^ "img" ^ (string_of_int current_frame) ^ ".jpg")
                              None []

    method next_frame =
      current_frame <- (current_frame + 1) mod total_frames;
      if current_frame = 0 then
        current_frame <- total_frames;
      self#load_file ((string_of_int current_frame) ^ ".csv")
end

This object is included in the main.ml file, which is the main entry point for the OpenGL application.
This file defines functions for initializing and binding events to the main openGL app. You'll find this code familiar if you know some OpenGL.

open Str
open String
open Color
open VertexType

(* Initializes openGL scene components*)
let init width height =
    GlDraw.shade_model `smooth;
    GlClear.color (0.0, 0.0, 0.0);
    GlClear.depth 1.0;
    GlClear.clear [`color; `depth];
    Gl.enable `depth_test;
    GlFunc.depth_func `lequal;
    GlMisc.hint `perspective_correction `nicest

(*  Draws the image*)
let draw () =
  GlClear.clear [`color; `depth];
  GlMat.load_identity ();
  GlMat.translate3 (-150.0, -150.0, -400.0);
  GlDraw.begins `points;
  List.iter (fun (DVertex (x, y, z, d)) ->
    let color = d /. 255. in
      GlDraw.color (color, color, color);
      GlDraw.vertex ~x:x ~y:y ~z:z ()) data#next_frame;
  GlDraw.ends ();
  Glut.swapBuffers ();
  data#save_image

(* Handle window resize *)
let reshape_cb ~w ~h =
  let
    ratio = (float_of_int w) /. (float_of_int h)
  in
    GlDraw.viewport 0 0 w h;
    GlMat.mode `projection;
    GlMat.load_identity ();
    GluMat.perspective 45.0 ratio (0.1, 1300.0);
    GlMat.mode `modelview;
    GlMat.load_identity ()

(* Handle keyboard events *)
let keyboard_cb ~key ~x ~y =
  match key with
    | 27 (* ESC *) -> exit 0
    | _ -> ()

(* A timer function setted to draw a new frame each time_interval ms *)
let rec timer value =
  Glut.postRedisplay ();
  Glut.timerFunc ~ms:data#get_time_interval
                 ~cb:(fun ~value:x -> (timer x))
                 ~value:value

(*  Main program function*)
let main () =
  let
    width = 640 and
    height = 480
  in
    ignore (Glut.init Sys.argv);
    Glut.initDisplayMode ~alpha:true ~depth:true ~double_buffer:true ();
    Glut.initWindowSize width height;
    ignore (Glut.createWindow "Radiohead HoC");
    Glut.displayFunc draw;
    Glut.keyboardFunc keyboard_cb;
    Glut.reshapeFunc reshape_cb;
    Glut.timerFunc ~ms:data#get_time_interval
                   ~cb:(fun ~value:x -> (timer x))
                   ~value:1;
    init width height;
    Glut.mainLoop ()

let _ = main ()

You can download the source here.
You can compile the files with the bytecode compiler:

ocamlc -g str.cma -I +camlimages ci_core.cma ci_bmp.cma ci_jpeg.cma
-I +lablGL lablglut.cma lablgl.cma main.ml -o main

.
Just remember that you have to install OCaml + lablGL + camlimages to be able to use this.

Any comment about the code will be well appreciated, since I'm an OCaml beginner .