Saturation.vs.Intensity Distributions of Quality Color Images
Transcripción
Saturation.vs.Intensity Distributions of Quality Color Images
SATURATION VS. INTENSITY DISTRIBUTIONS OF QUALITY COLOR IMAGES Alfredo Restrepo (Palacios) Laboratorio de Senales, Dpt. Ing. Eléctrica y Eectrónica, Universidad de los Andes A.A. 4976, Bogota, Colombia [email protected] ABSTRACT No-reference image quality assessment is an image characterization task of sorts. We explore the chromatic characterization of color images and relate it to image quality. Poor image quality is usually related to a loss of contrast; contrast is measured along the color dimensions of luminance, saturation and hue (which is a circular variable). The distribution of saturation as a function of luminance, both at the local and at the global level, is an informative type of measurement. Likewise, location-distribution plots give interesting image characterization tools for each of the dimensions of hue, saturation and luminance. 1. INTRODUCTION Other than fidelity (as in full-reference quality assessment), two main aspects of image quality are readability and aesthetics. Readability refers to the easiness with which a human observer interprets an image while aesthetics is an even more subjective, although important aspect. Image characteristics that determine readability are the luminance, hue and saturation contrasts. Image cahracteristics that modify the aesthetics of the image are hue balance, hue shifts, variety along the dimensions of hue, saturation and luminance, and also the distribution of dispersion as a function of location for each of the these components of color. For natural images, the illumination of the scene is a determinant of the aesthetics of the resulting image as well, and it is conceivable to estimate the quality of the illumination in the scene with the image data (e.g. with local and global white balance measurements). High dynamic-range images result from scenes with strong shadows or with several, different sources of light. Colorfulness is higher under direct light conditions than under diffuse illumination (e.g. overcast skies.) Indeed, hue is a more invariant color component than saturation; nevertheless, changes in the spectral contents of the illuminant (e.g. from daylight to incandescent tungsten light, which is less blue and therefore yellower,) change the metameric equivalences of the colors of objects in a scene, photo or painting. Likewise, the resulting color of human skin is an important determinant of the quality of a color image. Full-reference, image-quality measurement, a fidelity issue, is usually not a choice in the case of images of antique documents, where a well-preserved original is not likely to exist any more [3]. In fact, we focus on qality measures that result from the image itself. In this context, the idea of image quality assessment is related to that of image characterization. Color, the Eye and the Camera. For natural images, fidelity is a difficult issue if the ”original” is ”the scene as a human oberver saw it.” The human eye has three main modes of vision, regarding the level of the intensity of the illumination of the scene being watched. For very low illumination intensities, only the rods respond and we have scotopic vision (below 0.034 cd/m2 , with at maximal absortion at 507 nm and pretty much insensitive to large wavelengths, near 700 nm (spectral oranges)), which is achromatic and rather peripheric, making the issue of fidelity a tricky one in the case of scotopic vision. For higher levels of illumination, also the cones respond and we have mesotopic vision, with a strong perceived component of bluish colors. At still more intense illuminations, (above 3.4 cd/m2 ,) the rods saturate and do not respond any more, only the cones do (cone pigments having peak absortions at 445 nm (S), 535 nm (M) and 575 nm (L); an overall maximun sensitivity at 555 nm (a spectral yellow)) results and we have photopic vision, trichromatic and foveated. By using sensors with different sensitivities and different exposure times, cameras may overcome the limitations of the eye at low-level illumination. For the visualization of the image we have color prints (glossy or matte) and luminous screens. As Hunt hass remarked ”Many color pictures depict brightly-lit outdoor scenes and are viewed at lower levels of illumination” [5]. Poor image quality is usually related to a loss of luminance, of contrast and of saturation. Many pigments fade away in time and the optics of cameras and projectors may defocus image regions. As Hunt remarks, there is the ”... unavoidable tendency of all processes to produce losses in colourfulness. Along the dimensions hue, saturation and luminance, the image can be characterized at the local (intermediate betwee pixel and global) level in terms of the corresponding distributions of dispersion versus location; for this, we use midrange-range plots [7]. The distrubution of saturation as a function of luminance is considered at a local level, using four color spaces (three well-known spaces, plus a new one) of the type hue-saturation-luminance. Also, the distribution of saturation as a function of luminance is considered at the pixel level, using the range and midrange of the triple (R, G, B), for four types of chromaticity: red-yellow (ry=oranges), yellow-green (yg=cetrines), green-blue (gb=cyans) and bluered (br=purples). Likewise, we compute (crude) circular hue histograms on the basis of these four chromaticity bins, that characterize the image as having either a limited chromaticity distribution or a balanced one, at the global level. We concentrate on quality estimators that involve chromatic aspects; luminance aspects have been considered for example in [7], in the context of virtual antique document restoration. 3. SPACES OF THE TYPE HUE-SATURATION-LUMINANCE In the RGB cube, call the origin (0, 0, 0) pure black, the point (1, 1, 1) pure white, the line segment between them the achromatic segment Φ and the plane through the origin orthogonal to Φ, the chromatic plane Π. Call the faces of the cube with points for which min(R, G, B) = 0, the dark corner of the cube, and those for which max(R, G, B) = 1, the light corner. Call the polygon formed by the edges of the cube of points (R, G, B) with min(R, G, B)=0 and max(R, G, B)=1, the chromatic hexagon, see Figure 2; finally, call each of the triangles that have Φ as one its sides and a point on the chromatic hexagon as one of its vertices, a (constant-) hue triangle or a chromatic triangle; see Figure 2. Geometrically, the luminance component is a measure of the distance of the RGB color point from pure black, the saturation is a measure of the distance from the point to Φ and the hue is a measure of the angle that the projection of the point on Π makes with the projection of pure red. Color points on Φ (called achromatic colors) have an undefined hue and the colors on each chromatic triangle have the same hue. As in the local case, we denote 2. ON CONTRAST IN HUE-SATURATION-LUMINANCE SPACES The contrast contents of an image is an indicator of image quality, mainly in the readability aspect. We are sensitive to contrast in the three dimensions of hue, saturation and luminance, and the phenomenom of simultaneous contrast induction can be observed for each of the three cases. In the hue variable, a colored region induces its complementary color (the one that under additive color combination produces an achromatic gray) on the immediate surrounds. Likewise, we have Mach bands for the component of saturation; see Figure 1. Fig. 1. Mach bands of saturation. Half left parts of squares have hsv hsi and hsl coordinates at 0.3, 0.8, 0.7, and (right) runge’s r, θ, φ = 0.2, 0.3(2π), 0.7; half right parts have 0.3, 0.3, 0.7 and runge’s 0.4, 0.3(2π), 0.7. We measure image contrast at the local level, with a moving window, by computing the range (i.e. max−min is denoted ρ) for the cases of luminance and saturation. (The midrange (max + min)/2 is denoted µ).) Fig. 2. A labeling {0, 1, 2, 3, 4, 5} of the regions of the cube, depending on the possible orderings of the triple R, G, B amd the corresponding partition of the RGB cube. the range max(R, G, B) − min(R, G, B) as ρ, which is an unnormalized measure of saturation [4]. The midrange max(R,G,B)+min(R,G,B) is the luminance component of the 2 HSV color space; see Figure 3. For the color space HSI, the projection of the color point C = [R, G, B] on the line Φ is given by [I, I, I], where I = R+G+B . Writting the projec3 √ 3a tion of C on Π as [a, b, −(a+b)], we have α = 2√a2 +b 2 +ab and H = arcos(α), if G ≥ B, and H = −arcos(α) if G ≤ B. The saturation comonent S is given by S := 1 − min(R,G,B) = µ2 −min . For a given hue H, the possible I 6I values of the pair (S, I) are bouded above by a segment of I0 hyperbola {(S, I) : I = I0 +(1−I , S ∈ [0, 1]}, where 0 )S 0 I0 = 1+med , where med0 = median(R0 , G0 , B 0 ) and 3 0 0 0 (R , G , B ) is the point on the chromatic hexagon that is the vertex of the chromatic triangle that contains the point. HSI Fig. 3. At left, schematic min, median, max, range and cuasirange. At right, the caped cylinder of image HSI space. image space is thus not a ”complete cylinder”; see Figure 3 and [6]. For the space HSV, we denote its components as h, ρ min = max . s, max. The saturation is given by s = 1 − max The luminance component is given by the max. As in the case of the HSL system, the hue component h is of the form ±ρ1 n 1 h = ±ρ 6ρ + 3 , n ∈ {0, 1, 2} so that 6h = ρ + 2n and in fact, we can make the correspondence 2n + sign(±ρ1 ) and the codes {−1, 0, 1, 2, 3, 4} for the permutations, as mentioned above; see Figure 2. In [6], it is proposed a spherical space θ, κ, λ, named after Otto Runge; the space is obtained by shifting the cube in ambient space so that the central point (intermediate gray) ends up at the origin, and contracting the cube to a ball of radius 1/2. The color attributes that are made explicit by using spherical coordinates on the resulting Runge ball are those of hue θ, grayness γ (and colorfulness κ = 1 − γ) and lightness λ; they are defined on the basis of spherical coordinates (r, θ, φ) on the resulting sphere, as γ = 1 − 2r ∈ [0, 1] which measures distance from intermediate gray (colorfulness measures that from the surface boundary of the ball,) hue θ ∈ [0, 2π), λ = π−φ π ∈ [0, 1] is a measure of the distance from pure black. In the image space, where the achromatic axis points vertically upwards. 4. CHARACTERIZATION LUMINANCE-SATURATION The distribution of the saturation component along the luminance dimension is an important way of characterizing a color image. In addition to one such local characterization, we consider a pointwise one as well. 4.1. Pixelwise Saturation vs Luminance Distribution For each pixel out of 3 pixels by column and by row, for the images in Figure 5 we compute the range and midrange of the triple (R, G, B), which are measures of pixel color saturation and pixel luminance, respectively, and plot the range as a function of the midrange. TABLE 1. Pointwise Sat.vs.Lum, by region and by chromaticity type (Gabriela) C D c L (Gabriela) total 0 0.5365 0.1845 0.2790 oranges 0 0.4530 0.1564 0.3906 0 0.8941 0 0.1059 cetrines cyans 0 0.9554 0.0213 0.0233 purples 0 0.4415 0.3459 0.2126 (Grotta) total 0 0.4067 0.4991 0.0939 0 0.1365 0.7410 0.1221 oranges purples 0 0.8967 0.1192 0.0111 (Ababoles) total 0.0474 0.6827 0.1903 0.0796 0.2259 0.0826 oranges 0.1088 0.5827 cetrines 0.0180 0.7348 0.1748 0.0723 0 0.6074 0.0828 0.3098 cyans purples 0 0.2295 0.0656 0.7049 (Venezia2) total 0.1272 0.5550 0.2067 0.1110 oranges 0.1283 0.5515 0.2084 0.1118 cetrines 0 0.8571 0.0476 0.0952 cyans 0 1 0 0 purples 0 1 0 0 (Mercado1) total 0.0208 0.6269 0.1610 0.1913 oranges 0.0438 0.6308 0.2099 0.1154 cetrines 0 0.7321 0.1570 0.1109 cyans 0 0.4906 0.0458 0.4636 purples 0 0.3442 0.0658 0.5885 (Mercado2) total 0.0119 0.8211 0.0721 0.0950 oranges 0.0340 0.7449 0.1069 0.1142 cetrines 0 0.9311 0.0434 0.0255 cyans 0 0.5167 0.0597 0.4236 purples 0 0.8161 0.0865 0.0964 (Venezia1) total 0 0.4008 0.0865 0.5126 oranges 0 0.3842 0.0741 0.5416 cetrines 0 0.6532 0.0270 0.3198 cyans 0 0.3859 0.1136 0.5005 purples 0 0.3487 0.0120 0.6393 (Window) total 0.0017 0.8679 0.0186 0.1117 oranges 0.0022 0.9565 0.0213 0.0201 cetrines 0 0.8534 0.0207 0.1259 cyans 0 0.7698 0.0226 0.2075 purples 0 0.4575 0.0036 0.5389 code DLcC’ DLcC’ DLc’C’ DLcC’ DcLC’ cDLC’ cDLC’ DcLC’ DcLC DcLD DcLC DcLC’ LDcC’ DcCL DcCL DLcC’ DL’c’C’ Dc’L’C’ DcLC DcLC DcLC’ DLcC’ LDcC DcLC DcLC DcLC’ DLcC’ LDcC’ DcLC DcLC DcLC’ LDcC’ LDcC’ DcLC DcLC DLcC’ DLcC’ LDcC’ We plot the results globally and also discriminating according to chromaticity, observe the µ-ρ triangles for oranges 2π (added over 0 ≤ θ ≤ 2π 6 ), cetrines (added over 6 ≤ θ ≤ 2π 2π 4π 3 ), cyans (added over 3 ≤ θ ≤ 3 ), purples(added over 4π 3 ≤ θ ≤ π). See Figure 5, where a set of 8 test images is presented,7 where the global distributions are plotted and 7 where the distributions are discriminated by chromaticity. Each chromatic triangle of the RGB cube maps in a bijective fashion to the µ − ρ triangle. Points on the chromatic hexagon get mapped to (µ, ρ) = (0.5, 1). The achromatic line is mapped to the base of the triangle. Points on the faces of the dark (resp. light) corner get mapped to the left (resp. right) edge of the triangle. The µ − ρ triangle is subdivided as shown in Fig 4, into four subtriangles, called Highly Chromatic (labeled C), Moderately Chromatic (labeled c), Light (labeled L) and Dark (Labeled D). The points on each subtriangle are added up and the results are shown in Table 1. The labels of the regions are ordered according to their resulting relative weights and a code (a ”Word Descriptor”) results; see the last column of Table 1. Fig. 5. Set of images considered: Mercado1, Venezia1, Mercado2, Ababoles, Grotta, Gabriela (photo by Nary Kim), Venezia2 and Window 5. HUE BALANCE Fig. 4. With reference to the RGB cube, the µ − ρ triangle can be read as indicated 4.2. Local Saturation.vs.Luminance Distributions Above the pixel level and below the global level, we have the local level. For each of the 4 color spaces HSL, HSI, HSV and RUNGE, and considering partially overlapping windows of size 5 × 5, we compute scatter plots of local saturation versus local luminance; see Figure 8. For sun-lit images, it is typical to have an overall decrement of saturation with luminance. The plots for the HSI space should be read with care since the the pair (luminance, saturation) lives in a subset of the square [0, 1]2 , depending on the hue value. Likewise, the grayness component of RUNGE space is, broadly speaking, the opposite of saturation. Even though there are very pleasing images that are nearly monochromatic, we consider that variety in hue contents is a useful indicator of color image quality. Thus, from the circular histogram of hue with 4 bins, we compute an index of uniformity. For a uniform histogram, each bin has a value of 0.25; on this basis, we define measures of nonuniformity ν (which has a maximum of 1.5) and uniformity of the histogram. See Tables 2 and 3. The nonuniformity is given by ν = |ry − 0.25| + |yg − 0.25| + |gb − 0.25| + |br − 0.25| while the uniformity is given by 1.5−ν 1.5 TABLE 2. Chromaticity Percentages chr. sector → ry yg gb mercado1 0.4733 0.3230 0.1298 mercado2 0.3449 0.4370 0.0717 venezia1 0.3998 0.0487 0.4824 venezia2 0.9918 0.0013 0.0000 grotta 0.3622 0 0 ababoles 0.3261 0.6619 0.0095 gabriela 0.5269 0.0179 0.0902 window 0.8018 0.0485 0.0178 br 0.0626 0.1360 0.0518 0.0000 0.2154 0.0018 0.3205 0.0371 Fig. 6. Pixelwise distribution of saturation (measured as the range of (R,G,B)) as a function of luminance (measured as the midrange of (R,G,B)) for the images Mercado1, Venezia1, Mercado2, Ababoles, Grotta, Gabriela, Venezia2 and Window. Fig. 7. Pixelwise luminance-saturation distributions, discrimiTABLE 3. Uniformity histogram uniformity mercado1 0.5973 mercado2 0.6172 venezia1 0.4789 venezia2 0.0064 0.3538 grotta ababoles 0.3489 gabriela 0.5072 window 0.2011 ν 0.6040 0.5742 0.7817 1.4903 0.9643 0.9766 0.7393 1.1984 6. LOCATION-DISPERSION DISTRIBUTION OF HUE Here we consider the local variation (circular dispersion) of hue as a function of lcal hue. The local hue is measured with the circular mean while the local dispersion is measured with a circular range by measuring first the circle gap and then the circular range is given by 2π − gap; for more details, see [1]. See Figure 9. nated by chromaticity, as: oranges (ry, at left), cetrines (yg), cyans (gb), purples (br, at right); images Mercado1 (top), Venezia1, Mercado2, Ababoles, Grotta, Gabriela, Venezia2 and Window (bottom). 7. FINAL REMARKS The color reproduction of natural scenes is a complex theme and, in practice, the aesthetics of the image usually takes precedence over the fidelity. Different types of vision, depending on the intensity of the illumination make the issue more difficult. Conceivably, scotopic scenes can be reproduced with black and white projections in darkened rooms. Image quality as the result of image characterization, is a rich subject. Different requirements determine the quality of an image, depending on the subject of the image and on the field of application. From the data presented, calling the region of oranges, warm; that of cetrines, sour, that of cyans, cool and that of purples, sweet, we characterize image Mercado 1 as warm and chromatically balanced, Mercado 2 as quite balanced, chromatically. Venezia1 is warm and cool, moderately saturated and rather balanced, chro- Fig. 8. Local Distributions of Saturation versus Luminance: HSL (left), HSI, HSV and, Grayness versus Lightness of RUNGE space (right).Images: Mercado1 (top), Venezia1, Mercado2, Ababoles, Grotta, Gabriela and Venezia2 (bottom) matically. Venezia2 is warm, very monochromatic and has a low hue contrast. Grotta is warm, has an atypical distribution of saturation versus luminance, is unsaturated, it is very monochromatic and has a low hue contrast. Ababoles is sour, very monochromatic and has a low hue contrast. Gabriela is warm and sweet, moderately saturated and rather chromatically balanced. Window is warm, very dark, unsaturated but has a large hue contrast. Due to lack of space we do not explore the µ − ρ distribution of the saturation component; that is the subject of a forthcoming paper. That of luminance has been explored in [7]. 8. REFERENCES [1] Alfredo Restrepo, Carlos Rodriguez and Camilo Vejarano, “Circular processing of the hue variable,” VISAPP 2007, Barcelona, V.1, pp. 69-76, 2007. [2] Alfredo Restrepo, Stefano Marsi and Giovanni Ramponi, “HSV-Domain enhancement of high contrast images,” VISAPP09, Lisbon, Portugal, 2009. [3] Filippo Stanco, Alfredo Restrepo and Giovanni Ramponi, “Virtual Restoration of Antique Books and Photographs,” CRC, 20010. Fig. 9. Location-dispersion plots corresponding to the hue component; window = 5×5. Images: Mercado1 (top-left), Venezia1, Mercado2, Ababoles, Grotta, Gabriela, Venezia2 and Window (bottom-right). The horizontal hue axis should be interpreted as circular. (The plot exists on a cylinder.) [4] A. Hanbury and J. Serra, “A 3D-polar coordinate colour representation suitable for image analysis,” Technical report PRIP-TR-77, TU, Wien, 2002. [5] R.W.G. Hunt, “The Reproduction of Color,” Wiley (fifth ed), 2004. [6] Alfredo Restrepo and Luisa Junco, “On color modification in hue-saturation-luminance spaces (submitted),” VISAPP 2010, Angers, France, 20010. [7] Alfredo Restrepo and Giovanni Ramponi, “Word descriptors of image quality based on local dispersion.vs.location distributions,” EUSIPCO 08, Lausanne, Switzerland, 2008.