PSNR Quality Analysis Methodology

The quality analysis comparison performed between CineForm's 10-bit Visually Perfect codec and the DVCPRO HD codec was performed at CineForm's facilities by CineForm employees.  The analysis uses visual inspection, image differencing, and PSNR (peak signal-to-noise ratio) calculations as measures of image quality.  PSNR analysis uses a standard mathematical model to measure an objective difference between two images.  It is commonly used in the development and analysis of compression algorithms, and for comparing visual quality between different compression systems.  PSNR is calculated by the following formula:

 PSNR = 20*log10 (255 / RMSE)  // For 8bit inputs.

Where RMSE is the square root of the mean squared error for the entire image.  

Pseudo code (for those who wish to perform their own PSNR analysis):

For (each pixel)

{

            Difference  = Pixel from Image A – Pixel from Image B
            SummedError = SummedError + Difference * Difference

 } 

MeanSquaredError = SummedError / Number of Pixels
RMSE = sqrtt (MeanSquaredError)
PSNR = 20*log10 (255 / RMSE)

For the Quality results shown in the table, PSNR was calculated on the uncompressed luma (Y) channel of the YUV signal.  Luma is the best channel for showing errors the human eye is most likely to perceive.  Also, by analyzing the luma channel directly we avoid any PSNR error that may occur in YUV to RGB conversion; therefore we get a more accurate measure of the codec’s quality performance.

NOTE: The Luma channel for both sources (CFHD 10-bit and DVCPRO HD 8-bit) was scaled up to a 16-bit value for analysis (i.e. the data was normalized – requiring 255 in the formulas to be 65535.)  If you perform your own analysis, be weary that most off-the-shelf PSNR tools will only consider 8-bit sources.

PSNR was calculated using the first-captured image as the reference image, not the original 1920x1080 source image.  The reason for this is the DVCPRO HD codec always resamples 1920x1080 frames to 1280x1080, substantially reducing (by 1/3) its spatial resolution.  Consequently, using the original image as the reference for PSNR analysis would have further reduced the measured PSNR of the DVCPRO HD codec.  Consequently the PSNR analysis uses the first-captured image as the reference for both codecs. 

To perform each step in the multi-generation testing, a 1920x1080 uncompressed RGB frame was converted to 10-bit YUV 4:2:2 and presented to each codec.  For testing the DVCPRO HD codec the data was sent across HD-SDI for compression in the AJ-HD130DCP DVCPRO HD deck.  For testing CineForm's CFHD codec the uncompressed YUV source was presented to the production version of CineForm's software codec that ships with Prospect HD.  This effectively simulates a DVCPRO HD camera acquiring its first image, or the CineForm codec acquiring its first image from an uncompressed camera head.  By performing the PSNR analysis in this way we are testing the performance of each codec for multi-generational characteristics – i.e. we are testing how well it preserves the capture quality through a series of rendering stages.  No additional compression was used in the signal processing paths of either codec.

The PSNR graph on the Quality Analysis page shows the boundary generally considered the dividing point between "visually lossless" and "visible".  This boundary is subjective, as some people will see errors that others will not.  In practice, it becomes very difficult to see differences for PSNR greater than 50dB between the source and the compressed image.

The difference images were calculated by comparing the source RGB image to the resulting compressed images.  (Remember, the PSNR analysis used the first-captured image, not the original image).  This is why both codecs show some loss in their first generation.  Some small differences are due to the RGB-to-YUV conversion (common to most compressed video formats), however most of the error shown with the DVCPRO HD codec indicates the lower resolution nature of this compressed format.  The difference images where created by extracting uncompressed frames from the timeline (in RGB.) These frames were loading into an image-processing tool (GIMP) where the source and resulting compressed images were "differenced" (not subtracted which can hide errors.)  These difference images were lifted to a middle grey by setting the brightness to 127, a luminance region where the eye is more sensitive.  In order to enhance the visibility of codec-induced errors, the contrast for all difference images was lifted about 400%.

Have more questions? Submit a request

0 Comments

Article is closed for comments.
Powered by Zendesk