We have tested the hypothesis that Visual Attention Models can be more effective for saliency detection when using multispectral images compared to conventional RGB images. We have performed adaptations of six well-known Visual Attention Models to allow them to be used with multispectral images as input. The models are: Itti, GBVS, RARE, BMS, LDS and ResNet (see http://saliency.mit.edu/results_mit300.html). Preliminary results using standard metrics for saliency prediction assessment (Area Under Curve (AUC), Normalized Scan-path Saliency (NSS) and Information Gain (IG)) indicate that using spectral information in the Visible and Near Infrared range can produce more accurate saliency maps than using standard RGB images.

Digital imaging techniques are widely used in the examination of cultural heritage objects such as paintings, documents, plasterworks, posters, maps, etc. All these items are subject to the action of degradation agents as well as the intervention of conservation and restoration proffesionals. Studying in deep these pieces we can detect degraded or restored areas, detect materials such as pigments, study, model and predict the appearance of conservation products such as varnishes, binders or consolidants, simulate the color appearance under different illuminants, etc.

Summing up, spectral imaging opens a huge amount of possibilities for the accurate study of cultural heritage. Currently we are working together with various departments of the University of Granada like the department of Analitical Chemistry in the Faculty of Science or the department of Painting in the Faculty of Art. We are studying art paintings, paper documents, plasterworks, varnishes, pigments, binders, consolidants, inks, etc. We are designing and developping new capturing and processing techniques to overcome the limitations of current spectral imaging frameworks, as well as new techniques and metrics to evaluate the effect of different agents on the appearance of the heritage items.

In foggy or hazy conditions, images are degraded due to the scattering and attenuation by the atmospheric particles, reducing the contrast and visibility, distorting the color and decreasing the visibility of the content in the scene and making the object features difficult to identify by humans and by computer vision systems. This degradation depends on the distance between the object and the acquisition device, the type and the density of atmospheric particles and also on the wavelength.

Dehazing is a multidisciplinary challenge, since it requires the knowledge of several fields: meteorology to model fog or haze, physical models to predict how light is affected by the atmosphere, and computer vision & image processing to recover the parameters of the scene.

Recently technological advances in image sensors and spectral filtering have allowed the proliferation of multispectral and hyperspectral systems for image capture in a wide range of applications. However, they have not yet been used in dehazing methods despite their potential.

In 2015, at the Color Imaging Lab, we proposed a method in which the original image is recovered from the constancy of the response of the ratio of the RGB channels of a camera under different illuminants. This method only requires a previous segmentation of the image to group the pixels of the scene located at the same distance. Another advantage is that it is not necessary to know any atmospheric parameter.

Currently we are studying how well known dehazing algorithms perform with hyperspectral images in the visible and NIR ranges.

Congenital colour vision deficiency (CVD) affects 8% of Caucasian males and 0.5% of Caucasian females, who face difficulties with daily tasks and are also disqualified from particular professions. The study of color vision deficiencies has recently gained attention from social networks due to the appearance of companies that have developed what they call “aids” for subjects with CVD. This viral phenomenon does not go unnoticed by many researchers who study the effect of this type of passive aid from different paradigms.

From different strategies and paradigms we study, at the Color Imaging Lab, the efficacy of these “aids”, concluding, with the scientific support of several research papers (i.e. Optics Express, Optics Letters, Sensors, Coloration Technology) that filters can assist in certain specific colour-related tasks, enabling discrimination between some colours, and consequently between objects, but not allowing the subject with CVD to have comparable colour vision with those individuals, who have normal colour vision, because filters are unable to expand the colour gamut of subjects. Coloured filters do not make/help CVD observers to get a color vision more similar to normal observers, and even less so either “correct” or “cure” CVD.

Within this topic, we have introduced an innovative parameter that allows us to evaluate the so-called “relevant colors” in a painting; in other words, the number of colors that would stand out for an observer when just glancing at a painting. These relevant colors allow us to characterize the color palette of a scene and, on this basis, can be used as instances to segment any color image. Using a set of more than 4,000 paintings our approach is obtaining an average number of 18 relevant colors. This number partially agrees with the total 11–15 basic color names usually found in other categorical (i.e. color naming) color studies.

One of the main research interests in the Color Imaging Lab is spectral imaging. We work on designing spectral imaging systems that overcome current devices' limitations (size, cost, time of capture, uncontrolled conditions, etc). We work with spectral capturing devices such as systems based on Liquid Crystal Tunable Filters (LCTF), Bragg grating hyperspectral imagers, filter wheel high speed cameras, multispectral line-scan sensors, etc. We design and optimize the systems for general or specific spectral imaging applications. We develop tools to use them and improve their limitations. One of the cut-edge technologies we work with is that of Transverse Field Detectors (TFD). They are CMOS-based filter-less multispectral silicon image sensors. They are still in a prototype stage of development, and we proposed an imaging system combined with Multispectral Filter Arrays, to obtain up to 36 spectral bands images in a quick capture. A journal article as well as some congress communications have been published about this system. We also collaborate in international research projects with European companies to develop a system for high resolution hyperspectral imaging of gonio-chromatic effect coatings.

Spectral imaging systems have been used for spectral measurements for several decades, mainly for scientic purposes, and were usually linked to costly applications. Thanks to research in imaging science and machine learning and the technological advancement in recent years, spectral imaging became feasible for various industrial and even consumer applications. This research deals with line-scan multi-spectral imaging systems for spectral reflectance and color measurements. The most important aspects under consideration are optimization of spectral properties of the optical components, image registration and estimation of surface spectral reflectances. We focus on a particular system design, in which multiple color filtered RGB images with distinct spectral content are acquired at the same time. These images correspond to different viewpoints of the scanning scene due to the mechanical arrangement of camera sensor and optics. By optimizing the system's optical component spectral properties, the amount of spectral information acquired can be increased and the spectral reflectance estimation can be improved. We propose a filter selection framework and demonstrate that optimization for various line-scan system configurations results in an improvement of spectral and color measurement performance. Multi-channel image registration is required to account for viewpoint differences and other sources of image channel misalignment. We develop a calibration scheme for planar scanning objects and propose scene-adaptive registration for non-planar scanning objects. For our 12-channel laboratory imaging system, subpixel accuracy is achieved. Based on the registered multi-channel image data, spectral reflectance estimation can be performed. Physical and empirical estimation methods are considered, and we propose a logarithmic kernel function for kernel ridge regression. We experimentally compare performance of various estimation methods for simulated and measured camera response data and consider different noise levels and number of spectral channels. Empirical estimation performance is influenced by model training. We compare various training sample selection approaches and propose an application dependent selection scheme. Further, adaptive training methods from related literature are unified conceptually and evaluated systematically. We show that the aforementioned aspects of line-scan multi-spectral imaging system design are critical for spectral and color measurement, and that application specific design is often beneficial to improve system performance.

Our goal is the study of High Dynamic Range Imaging techniques in conjunction with Spectral Imaging. Among other results we have developed an algorithm to estimate the bracketing sets needed to capture the full dynamic range of an HDR scene, without the need of any prior knowledge of information about image content. Therefore we call the method blind. Besides, it is applicable to any camera, whatever the shape of its Camera Response Function (CRF) is. Linear or not. For this reason we call it universal. The method works on-line, as the exposure times are estimated during the capturing process and not after some previous image acquisition. It is also minimal, as every single shot taken is used to compose the final HDR radiance map of the scene, and the number of shots to cover the full dynamic range of the scene is also minimal. However, the method can be easily tuned so that the user can decide if it is more crucial to have a minimum bracketing set (shorter capturing time) or a higher signal to noise ration (SNR).

In collaboration with the Ecology Research group of the CEAMA Institute (http://www.ceama.es/en) we are developing a capture system with two Raspberry Pi cameras coupled to optical filters, embedded in a 3D-printed support structure. The aim of the research project is to demonstrate the correlation between the NDVI (Normalized Differencial Vegetation Index) obtained from spectral data on-site and the camera response ratios obtained by the designed capture system. The NDVI indices are defined as ratios of integrated reflectance signals in the NIR and the VIS ranges (usually obtained wirh Remote Sensing devices). Once the testing phase is over, the system will be placed within a natural environment to perform regular checks of the vegetation vital activity in the surrounding area. Preliminary experiments have already proven that the two cameras coupled with filters offer a better correlation to the NDVI data than conventional systems designed for measuring these indices, consistent in a single camera coupled to a blue filter. In the figure, we present a false-color vegetation index image captured in these preliminary experiments, in which we demonstrate that the system is able to distinguish between recently harvested and dry leaves.

This project is linked to a collaboration between our group and Prof. R. Huertas of the Optics Department of the University of Granada, along with the company Akzonobel. The aim of the project is to design a capture system for obtaining spectral images of surfaces covered by effect coatings, commonly used in the car industry. These samples present a spatial structure due to the sparkles of small flakes introduced in the base paint substrate. The fine texture cannot be resolved by conventional spectral spot measurement devices, and as a result there is a mismatch between the predicted and the experimentally measured color difference distributions for these samples. In the high-resolution hyperspectral images, the individual flakes will be resolved and so independent measurements of color for the sparkle and the basic paint surface will be made possible. The project represents the first stage towards a future development of an adequate instrument for the spectral measurement of the effect coated samples. As capture device, we are currently using an LCTF filter coupled with a monochrome camera.

The acquisition of spectral reflectance factor image data in an outdoor environment is a challenging task, mostly due to nonstatic scene content and illumination. In this project, we proposed a work-flow for this task using a commercial Bragg-grating-based hyperspectral imager that can capture the visible and near-infrared part of the light spectrum. The work-flow involves focus position and exposure time estimation, illumination scaling, and image registration, among other procedures. Most of them generally apply to hyperspectral imaging, while some are specific to a Bragg-grating-based hyperspectral imaging device when dealing with specific challenges in outdoor environments. We have conducted some experiments to evaluate the quality of the acquired image data and discussed some limitations of the technology for spectral imaging of outdoor scenes. Fourteen urban scene spectral images acquired using the proposed approach are already publicly available to the scientific community under a Creative Commons license. They can be downloaded via the following link: http://colorimaginglab.ugr.es/pages/Data#__doku_ugr_hyperspectral_image_database