017 Camera Image Quality Assessment

Digital imaging technology has today become ubiquitous. Digital cameras are finding uses in an ever increasing number of products, in everything from endoscopy equipment to consumer cameras and cars. Thus, a growing number of engineers are becoming exposed to camera systems in one way or the other. Even though many new applications of cameras are aimed at machine vision applications, there is still a wide and growing range of uses for cameras delivering images for human vision. Therefore, the need for image quality metrics delivering perceptually correlated results is continuously expanding. Furthermore, a systematic and reproducible as well as accurate methodology must be developed in order to guarantee usable results. An important part of this is the standardization of image quality assessment methods.

One can often hear that it is not possible to make an assessment of image quality because of its subjective nature. The aim of this course is to show that the opposite is true, and to provide methods to perform this assessment. To obtain the most accurate determination of image quality, a statistical treatment of results from subjective experiments is in principle necessary. Such methodologies are well-established and in use at many different locations. The drawback of this methodology is that it is time and resource consuming and therefore expensive. In contrast, objective measurements of image quality can be fast, automated and highly reproducible. The drawback is that many times they give results that do not reflect the actual perception of image quality. In this course, objective methods that do provide good correlation with subjective experiments will be presented and ways to design custom methods that correlate with subjective results will be shown. Additionally, most of the objective methods discussed may be used in applications where humans are not intended to observe the images, i.e., machine vision

The course starts with defining image quality and presenting a set of image quality attributes, combinations of which will determine the overall perceived quality of a certain image. Each individual attribute will be explained in detail. Subjective image quality assessment will then be described with focus on the most commonly encountered methods. This will end up in presenting an efficient way to perform subjective experiments in a way that is reproducible and comparably quick; the image quality ruler method. An application of this method to video will also be discussed. The course then continues with a detailed discussion of the camera and its components: the lens, image sensor, image processing, but also illumination systems. Next, the most extensive part of the course follows, with explanations of objective measurement methods for assessing the degradations of the image quality attributes previously defined. Among these attributes are sharpness, noise, color, etc. Existing standards are discussed and put into context. Then, methods to make the link between the results of the objective measurements and results from subjective experiments are presented. Having knowledge of the measurement methods is only half the story. In order to perform accurate measurements, the so-called protocols must be established. These provide guidance for how to perform the actual measurements, and include instructions for how to set up lab equipment, how many images to capture, etc. In the next part of the course, a discussion of how to set up an image quality testing lab, including design of the lab space, measurement equipment, test charts, etc, is made. This will provide a solid foundation for understanding the testing protocols. The course ends with putting all parts taught together by providing an example of a benchmarking experiment in which a set of cameras are compared according to the methods described in the course.

Image scientists, photographers, engineers, or managers who want to learn more about image quality and how to evaluate still as well as video cameras for different applications.