RESEARCH ARTICLE
Lightness perception and lightness illusions
6 September 2023 – Vol 1, Issue 1.
“Every light is a shade, compared to the higher lights, till you come to the sun; and every shade is a light, compared to the deeper shades, till you come to the night.”
—John Ruskin, 1879.
Abstract
A grey surface in sunlight may have a much higher luminance than it has in the shade, but it still looks grey. To achieve the task of ‘lightness constancy,’ the visual system must discount the illumination and other viewing conditions and estimate the reflectance. Many different physical situations, such as shadows, filters, or haze, can be combined to form a single, simple mapping from luminance to reflectance. Human lightness computation is imperfect, but performs well in most natural situations. Lightness illusions can reveal the inner workings of the estimation process, which may involve low-level, mid-level, and high-level mechanisms. Mid-level mechanisms, involving contours, junctions, and grouping, appear to be critical in explaining many lightness phenomena.
Introduction
The amount of light coming to the eye from an object depends on the amount of light striking the surface and on the proportion of light that is reflected. If a visual system made just a single measurement of luminance, acting as a photometer, there would be no way to distinguish a white surface in dim light from a black surface in bright light. Yet humans can usually do so, and this skill is called lightness constancy.
The constancies are central to perception. An organism needs to know about meaningful world-properties: colour, size, shape, etc. These properties are not explicitly available in the retinal image, but are extracted by visual processing. The grey shade of a surface is one such property. To extract it, luminance information must be combined across space.
Figure 1: The simultaneous contrast effect.
In figure 1, for example, we see the well-known simultaneous contrast effect, which demonstrates a spatial interaction in lightness perception. The two smaller squares are the same shade of grey, but the square in the dark surround (left) appears lighter than the square in the light surround (right).
This is, of course, an illusion, which may be viewed as a quirky failure of perception. But such illusions are also revealing, allowing us to examine the inner workings of a system that functions remarkably well. Here we consider how lightness illusions can inform us about lightness perception.
Levels of processing
The visual system processes information at many levels of sophistication. At the retina, there is low-level vision, including light adaptation and the centre-surround receptive fields of ganglion cells. At the other extreme is high-level vision, which includes cognitive processes that incorporate knowledge about objects, materials, and scenes. In between there is mid-level vision. Mid-level vision is simply an ill-defined region between low and high. The representations and the processing in the middle stages are commonly thought to involve surfaces, contours, grouping, and so on. Lightness perception seems to involve all three levels of processing.
The Gestalt approach
The Gestalt psychologists emphasized the importance of perceptual organization,
much of it is based on mechanisms that might be characterized as mid-level. The key concepts include grouping, belongingness, good continuation, proximity, and so on.
Figure 2: Variants on the Koffka ring. (a) The ring appears almost uniform. (b) When split, the two half-rings appear distinctly different. (c) When shifted, the two half-rings appear quite different.
Koffka offered an example of how simultaneous contrast can be manipulated by changing spatial configuration. The ring in figure 2(a) appears almost uniform in colour. When the stimulus is split in two, as shown in figure 2(b), the two half-rings appear to be different shades of grey. The two halves now have separate identities, and each is perceived within its own context.
The left and right half-blocks are slid vertically, shown in figure 2(c), and the new configuration leads to a very different perceptual organization and a strong lightness illusion. Grouping cues such as good continuation indicate that the left semicircle is connected to the light region on the right, and the right semicircle is connected to the dark region on the left.
Atmospheres and X-junctions
Figure 3 shows an illusion using X-junctions. The centres of the two diamond shaped regions are physically the same shade of light grey.
However, the one on the right seems to lie in haze, while the one on the left seems to lie in clear air.
Figure 3: An illusion of haze. The two marked regions are identical shades of grey. One appears clear and the other appears hazy.
T-junctions and White’s illusion
White’s illusion is shown in figure 4. The grey rectangles are the same. This is surprising: By local contrast, the left ones should look darker than the right ones. The left rectangles have a long border with white and a short border with black. The illusion is reversed from the usual direction. This effect has been interpreted in terms of the T-junctions (Todorovic, 1997). Patches straddling the stem of a T are grouped together for the lightness computation, and the cross-bar of the T serves as an atmospheric boundary. (cf. Anderson, 1997, for an alternate approach).
Figure 4: White’s illusion. The grey strips are all the same shade of grey
Zaidi, Spehar, and Shy (1997) have shown that the action of T-junctions can be so strong that it overpowers traditional grouping cues such as coplanarity. Therefore the grouping rules for the lightness computation evidently differ from those underlying subjective belongingness.
Constructing a new illusion
In the ‘criss-cross’ illusion of figure 5, the small tilted rectangles in the middle are all the same shade of grey. Many people find this hard to believe.
Figure 5: The criss-cross illusion. The small tilted rectangles are all the same shade of grey.
The figure was constructed by the following principles: The multiple psi-junctions along the vertical edges establish strong atmospheric boundaries. Within each vertical strip there are three luminances and multiple edges to establish articulation. The combination of tricks leads to a strong illusion.
Some observers see the image in terms of transparent strips; others see it merely as a flat painting. However, all subjects see a strong illusion instantly.
The snake illusion
Similar principles can be used to construct a figure with X-junctions. Figure 6a shows an illusion we call the snake illusion (Somers and Adelson, 1997). The figure is a modification of the simultaneous contrast display shown at the right. The diamonds are the same shade of grey and they are seen against light or dark backgrounds.
Figure 6: The snake illusion. All diamonds are the same shade of grey. (a) The regular snake: the diamonds appear quite different. (b) The ‘anti-snake’: the diamonds appear nearly the same. The local contrast relations between diamonds and surrounds are the same in both (a) and (b).
A set of half-ellipses have been added along the horizontal contours. The X-junctions aligned with the contour are consistent with transparency, and they establish atmospheric boundaries between strips. The statistics within a strip are chosen so that the diamonds are at extrema within the strip distributions. Note that the ellipses do not touch the diamonds, so the edge contrast between each diamond and its surround is unchanged.
Figure 6b shows a different modification in which the half-ellipses create a sinuous pattern with no junctions and no sense of transparency. The contrast illusion is weak; for most subjects it is almost gone. Thus, while figures 6a and 6b have the same diamonds against the same surrounds, the manipulations of the contour greatly change the lightness percept. In effect, we can turn the contrast effect up or down by remote control.
Why should the illusion of figure 6b be weaker than in the standard simultaneous contrast? We have various observations suggesting that the best atmospheric boundaries are straight, and that curved contours tend to be interpreted as reflectance. The sinuous contours of figure 6b are not seen as atmospheric boundaries, and therefore the adaptive window is free to grow and to mix statistics from both light and dark strips.
Summary
Illusions of lightness and brightness can help reveal the nature of lightness computation in the human visual system. It appears that low-level, mid-level, and high-level factors can all be involved. In this chapter we have emphasized the phenomena related to mid-level processing.
Our evidence, along with the evidence of other researchers, supports the notion that statistical and configural information are combined to estimate the lightness mapping at a given image location.
__
This is an abridged version of the article published as Chapter 24 in M. Gazzaniga, ed., The New Cognitive Neurosciences, 2nd ed. Cambridge, MA: MIT Press, 339-351, 2000. With thanks to Dr. Gil Dekel for his editing contribution to this abridged version, in 2023.
References and further reading
Adelson, E. H. (1993). Perceptual organization and the judgment of brightness. Science, 262, 2042-2044.
Adelson, E. H., and Anandan, P. (1990). Ordinal characteristics of transparency. Proceedings of the AAAI-90 Workshop on Qualitative Vision, 77- 81.
Adelson, E. H., and Pentland, A. P. (1996). The Perception of Shading and Reflectance. In D. Knill and W. Richards (eds.), Perception as Bayesian Inference (pp. 409-423). New York: Cambridge University Press
Anderson, B. L. (1997). A theory of illusory lightness and transparency in monocular and binocular images: the role of contour junctions. Perception, 26, 419-453.
Arend, L. (1994). Surface Colors, Illumination, and Surface Geometry: Intrinsic-Image Models of HumanColor Perception. In A. Gilchrist (Ed.), Lightness, Brightness, and Transparency (pp. 159-213). Hillsdale:Erlbaum.
Barrow, H. G., and Tenenbaum, J. (1978). Recovering intrinsic scene characteristics from images. In A. R. Hanson and E. M. Riseman (Eds.), Computer Vision Systems (pp. 3-26). Orlando: Academic Press.
Beck, J., Prazdny, K., and Ivry, R. (1984). The perception of transparency with achromatic colors. Perception and Psychophysics, 35(5), 407-422.
Cornsweet, T. (1970). Visual Perception. New York: Academic Press.
Gilchrist, A. L., and Cataliotti, J. (1994). Anchoring of surface lightness and multiple illumination levels. Investigative Ophthalmology and Vision Science (supplement), 35(4), 2165.
Gilchrist, A. L. (1977). Perceived lightness depends on perceived spatial arrangement. Science, 195(4274), 185- 187.
Gilchrist, A. L., and Jacobsen, A. (1983). Lightness constancy through a veiling luminance. Journal of Experimental Psychology: Human Perception and Performance, 9(6), 936- 944.
Gilchrist, A. L., Kossyfidis, C., Bonato, F., Agostini, T., Cataliotti, J., X., L., Spehar, B., and Szura, J. (in press). A new theory of lightness perception. Psychological Review.
Hochberg, J. E., and Beck, J. (1954), Apparent spatial arrangement and perceived brightness. Journal of Experimental Psychology, 47, 263-266.
Hupfeld, Herman, lyrics to “As Time Goes By,” Harms, Inc., ASCAP 1931. Also appears in Casablanca, Warner Brothers, 1942.
Katz, D. (1935). The World of Colour. London: Kegan Paul.
Knill, D., and Kersten, D. (1991). Apparent surface curvature affects lightness perception. Nature, 351, 228- 230.
Land, E. H., and McCann, J. J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61, 1-11.
Li, X., and Gilchrist, A. (Submitted). Relative area and relative luminance combine to anchor surfacelightness values.
Metelli, F. (1974). Achromatic color conditions in the perception of transparency. In R. B. McLeod and H. L. Pick (Eds.), Perception, Essays in Honor of J.J. Gibson . Ithaca, NY: Cornell University Press.
Reid, R. C., and Shapley, R. (1988). Brightness induction by local contrast and the spatial dependence of assimilation. Vision Research, 28, 115-132.
Ross, W., and Pessoa, L. (in press). Contrast/filling-in model of 3-D lightness perception. Perception and Psychophysics.
Sinha, P., and Adelson, E. H. (1993). Recovering Reflectance in a World of Painted Polyhedra. Proceedings of Fourth International Conference on Computer Vision (pp.156-163) Berlin; May 11-14, 1993.
Somers, D. C., and Adelson, E. H. (1997). Junctions, Transparency, and Brightness. Investigative Ophthalmology and Vision Science (supplement), 38, S453.
Spehar, B., DeBonet, I., and Zaidi, Q. (1996). Brightness Induction from Uniform and Complex Surrounds: A General Model. Vision Research, 36, 1893-1906.
Todorovic, D. (1997). Lightness and junctions. Perception, 26(4), 379-394. https://doi.org/10.1068/p260379
Lightness and retinex theory. Zaidi, Q., Spehar, B., and Shy, M. (1997). Induced effects of backgrounds and foregrounds in three-dimensional configurations: the role of T-junctions. Perception, 26, 395-408.
Edward Adelson is Professor of Vision Science at MIT, in the Department of Brain and Cognitive Sciences, and the Computer Science and Artificial Intelligence Laboratory (CSAIL). Prof. Adelson has published widely in the areas of human vision, computer vision, and computer graphics. His current research focuses on artificial touch sensing for robotics.
