4. Calibration Programming Exercises (40 points): Implement the direct parameter calibration method in order to (1) learn how to use SVD to solve systems of linear equations; (2) understand the physic

4. Calibration Programming Exercises (40 points): Implement the direct parameter calibration method in order to (1) learn how to use SVD to solve systems of linear equations; (2) understand the physical constraints of the camera parameters; and (3) understand important issues related to calibration, such as calibration pattern design, point localization accuracy and robustness of the algorithms. Since calibrating a real camera involves lots of work in calibration pattern design, image processing and error controls as well as solving the equations, we will mainly use simulated data to understand the algorithms.  As a by-product we will also learn how to generate 2D images from 3D models using a “virtual” pinhole camera. The calibration procedure has the following three steps:

1. Calibration pattern “design”. Generate data of a “virtual” 3D cube similar to the one shown in here of the lecture notes in camera calibration. For example, you can hypothesize a 1x1x1 m3 cube and pick up the coordinates of 3-D points on corners of each black square in your world coordinate system. Make sure that your data is sufficient for the following calibration procedures. In order to check the correctness of your data, draw your cube (with the control points marked) using Matlab (or whatever tools you are selecting). I have provided a piece of starting code in Matlab for you to use.
2. A “virtual” camera and its images. Design a “virtual” camera with known intrinsic parameters including focal length f, image center (ox, oy) and pixel size (sx, sy).  As an example, you can assume that the focal length is f = 16 mm, the image frame size is 512*512 (pixels) with (ox,oy) = (256, 256), and the size of the image sensor  inside your camera is 8.8 mm *6.6 mm (so the pixel size is (sx,sy) = (8.8/512, 6.6/512) ). Capture an image of your “virtual” calibration cube with your virtual camera in a given pose (i.e., R and T).  For example, you can take the picture of the cube 4 meters away and with a tilt angle of 30 degree. Use three rotation angles alpha, beta, gamma to generate the rotation matrix R (refer to the lecture notes in camera model – please check the correctness of the R equation especially for signs).  You may need to try different pose in order to have a suitable image of your calibration target. In your report, please clearly list the parameters you used for generating your calibration image. They are called the “ground truth” of the parameters.
3. Direction calibration method: Estimate the intrinsic (fx, fy, aspect ratio a, image center (ox,oy) ) and extrinsic (R, T and further alpha, beta, gamma) parameters. Use SVD to solve the homogeneous linear system and the least square problem, and to enforce the orthogonality constraint on the estimate of R.  You are asked to do the following:

        i.      Use the accurately simulated data (both 3D world coordinates and 2D image coordinates) to the algorithms, and compare the results with the “ground truth” data (which are given in step (a) and step (b)).  Remember you are practicing a camera calibration, so you should pretend you know nothing about the camera parameters (i.e. you cannot use the ground truth data in your calibration process). However, in the direct calibration method, you could use the knowledge of the image center (in the homogeneous system to find extrinsic parameters) and the aspect ratio (in the Orthocenter theorem method to find image center). 

      ii.      Show experimentally whether the unknown aspect ratio matters in estimating the image center, and how the initial estimation of image center affects the estimating of the remaining parameters.  Give a solution to solve the problems if any and implement it.

    iii.      Accuracy Issues. Add in some random noises to the simulated data and run the calibration algorithms again. See how the “design tolerance” of the calibration target and the localization errors of 2D image points affect the calibration accuracy. For example, you can add 0.5 mm random error to 3D points and 1.0 pixel random error to 2D points. Also analyze how sensitive of the Orthocenter method is to the extrinsic parameters in imaging the three sets of the orthogonal parallel lines. (* extra points:10)

In all of the steps, you should give you results using either tables or graphs, or both of them.