Visual attention Models for saliency detection in Spectral images

visualattention We have tested the hypothesis that Visual Attention Models can be more effective for saliency detection when using multispectral images compared to conventional RGB images. We have performed adaptations of six well-known Visual Attention Models to allow them to be used with multispectral images as input. The models are: Itti, GBVS, RARE, BMS, LDS and ResNet (see Preliminary results using standard metrics for saliency prediction assessment (Area Under Curve (AUC), Normalized Scan-path Saliency (NSS) and Information Gain (IG)) indicate that using spectral information in the Visible and Near Infrared range can produce more accurate saliency maps than using standard RGB images.

Spectral imaging system based on Transverse Field Detectors

spectral_imagingOne of the main research interests in the Color Imaging Lab is spectral imaging. We work on designing spectral imaging systems that overcome current devices' limitations (size, cost, time of capture, uncontrolled conditions, etc). We work with spectral capturing devices such as systems based on Liquid Crystal Tunable Filters (LCTF), Bragg grating hyperspectral imagers, filter wheel high speed cameras, multispectral line-scan sensors, etc. We design and optimize the systems for general or specific spectral imaging applications. We develop tools to use them and improve their limitations. One of the cut-edge technologies we work with is that of Transverse Field Detectors (TFD). They are CMOS-based filter-less multispectral silicon image sensors. They are still in a prototype stage of development, and we proposed an imaging system combined with Multispectral Filter Arrays, to obtain up to 36 spectral bands images in a quick capture. A journal article as well as some congress communications have been published about this system. We also collaborate in international research projects with European companies to develop a system for high resolution hyperspectral imaging of gonio-chromatic effect coatings.

Optimization of a multispectral line scan camera for spectral reflectance estimation

Timo1 Timo2 Spectral imaging systems have been used for spectral measurements for several decades, mainly for scienti c purposes, and were usually linked to costly applications. Thanks to research in imaging science and machine learning and the technological advancement in recent years, spectral imaging became feasible for various industrial and even consumer applications. This research deals with line-scan multi-spectral imaging systems for spectral reflectance and color measurements. The most important aspects under consideration are optimization of spectral properties of the optical components, image registration and estimation of surface spectral reflectances. We focus on a particular system design, in which multiple color filtered RGB images with distinct spectral content are acquired at the same time. These images correspond to different viewpoints of the scanning scene due to the mechanical arrangement of camera sensor and optics. By optimizing the system's optical component spectral properties, the amount of spectral information acquired can be increased and the spectral reflectance estimation can be improved. We propose a filter selection framework and demonstrate that optimization for various line-scan system configurations results in an improvement of spectral and color measurement performance. Multi-channel image registration is required to account for viewpoint differences and other sources of image channel misalignment. We develop a calibration scheme for planar scanning objects and propose scene-adaptive registration for non-planar scanning objects. For our 12-channel laboratory imaging system, subpixel accuracy is achieved. Based on the registered multi-channel image data, spectral reflectance estimation can be performed. Physical and empirical estimation methods are considered, and we propose a logarithmic kernel function for kernel ridge regression. We experimentally compare performance of various estimation methods for simulated and measured camera response data and consider different noise levels and number of spectral channels. Empirical estimation performance is influenced by model training. We compare various training sample selection approaches and propose an application dependent selection scheme. Further, adaptive training methods from related literature are unified conceptually and evaluated systematically. We show that the aforementioned aspects of line-scan multi-spectral imaging system design are critical for spectral and color measurement, and that application specific design is often beneficial to improve system performance.

High Dynamic Range imaging

HDR granadaOur goal is the study of High Dynamic Range Imaging techniques in conjunction with Spectral Imaging. Among other results we have developed an algorithm to estimate the bracketing sets needed to capture the full dynamic range of an HDR scene, without the need of any prior knowledge of information about image content. Therefore we call the method blind. Besides, it is applicable to any camera, whatever the shape of its Camera Response Function (CRF) is. Linear or not. For this reason we call it universal. The method works on-line, as the exposure times are estimated during the capturing process and not after some previous image acquisition. It is also minimal, as every single shot taken is used to compose the final HDR radiance map of the scene, and the number of shots to cover the full dynamic range of the scene is also minimal. However, the method can be easily tuned so that the user can decide if it is more crucial to have a minimum bracketing set (shorter capturing time) or a higher signal to noise ration (SNR).

Design of a capture system for measurement of Vegetation indices

vegetationIn collaboration with the Ecology Research group of the CEAMA Institute ( we are developing a capture system with two Raspberry Pi cameras coupled to optical filters, embedded in a 3D-printed support structure. The aim of the research project is to demonstrate the correlation between the NDVI (Normalized Differencial Vegetation Index) obtained from spectral data on-site and the camera response ratios obtained by the designed capture system. The NDVI indices are defined as ratios of integrated reflectance signals in the NIR and the VIS ranges (usually obtained wirh Remote Sensing devices). Once the testing phase is over, the system will be placed within a natural environment to perform regular checks of the vegetation vital activity in the surrounding area. Preliminary experiments have already proven that the two cameras coupled with filters offer a better correlation to the NDVI data than conventional systems designed for measuring these indices, consistent in a single camera coupled to a blue filter. In the figure, we present a false-color vegetation index image captured in these preliminary experiments, in which we demonstrate that the system is able to distinguish between recently harvested and dry leaves.

Hyperspectral Images of effect coatings for car paints

car paintThis project is linked to a collaboration between our group and Prof. R. Huertas of the Optics Department of the University of Granada, along with the company Akzonobel. The aim of the project is to design a capture system for obtaining spectral images of surfaces covered by effect coatings, commonly used in the car industry. These samples present a spatial structure due to the sparkles of small flakes introduced in the base paint substrate. The fine texture cannot be resolved by conventional spectral spot measurement devices, and as a result there is a mismatch between the predicted and the experimentally measured color difference distributions for these samples. In the high-resolution hyperspectral images, the individual flakes will be resolved and so independent measurements of color for the sparkle and the basic paint surface will be made possible. The project represents the first stage towards a future development of an adequate instrument for the spectral measurement of the effect coated samples. As capture device, we are currently using an LCTF filter coupled with a monochrome camera. akzonobel logo

Hyperspectral Imaging of outdoor scenes with a Bragg-grating based spectral imaging device

jia_res1 The acquisition of spectral reflectance factor image data in an outdoor environment is a challenging task, mostly due to nonstatic scene content and illumination. In this project, we proposed a work-flow for this task using a commercial Bragg-grating-based hyperspectral imager that can capture the visible and near-infrared part of the light spectrum. The work-flow involves focus position and exposure time estimation, illumination scaling, and image registration, among other procedures. Most of them generally apply to hyperspectral imaging, while some are specific to a Bragg-grating-based hyperspectral imaging device when dealing with specific challenges in outdoor environments. We have conducted some experiments to evaluate the quality of the acquired image data and discussed some limitations of the technology for spectral imaging of outdoor scenes. Fourteen urban scene spectral images acquired using the proposed approach are already publicly available to the scientific community under a Creative Commons license. They can be downloaded via the following link: jia_res2

Analysis of natural scenes under daylight

Under construction. Sorry!

De-weathering algortihms based on camera responses ratio

Under fog or haze images are degraded due to scattering and attenuation of atmospheric particles, reducing the contrast and visibility, changing the color and making the object features difficult to identify by a humans and by computer vision systems. This degradation depends on the distance, the density of atmospheric particles and on the wavelength. Dehazing is a multidisciplinary challenge, since it requires the knowledge of several fields: meteorology to model fog or haze, physical models to predict how light is affected by the atmosphere, and computer vision & image processing to recover the parameters of the scene. Recently technological advances in image sensors and spectral filtering have allowed the proliferation of multispectral and hyperspectral systems for image capture in a wide range of applications. However, they have not yet been used in dehazing methods despite their potential. We have developed a method in which the original image is recovered from the constancy of the response of the ratio of the RGB channels of a camera under different illuminants. This method only requires a previous segmentation of the image to group the pixels of the scene located at the same distance. Another advantage is that it is not necessary to know any atmospheric parameter.