Creativity of light
5 July 2024 – Vol 2, Issue 3.
The play of light and matter is a daily spectacle that significantly influences our world, affecting everything from the colours we perceive to the complex dance of shadows. Renowned artists like Caravaggio, Rembrandt, and Turner have harnessed light as a medium to create immersive and dynamic works. Light’s ability to transform our perception of spaces has been utilized for architectural effects throughout history. In contemporary times, artists continue to innovate with light, crafting installations that meld light, sound, and movement. The understanding of how light interacts with matter is pivotal in visual arts, and indeed artists employ the principles of light and shadow to craft illusions of three-dimensional forms in their drawings. They can even leverage the principle of interference to fabricate 3D holograms, resulting in diverse visual effects and perceptions. In recent times, the use of drones and lasers at music festivals and celebratory events has introduced an additional dimension of visual stimulation.
Light is an electromagnetic wave that interacts with matter in two primary ways: absorption and scattering. Absorption occurs when light is incident on an object and transfers its energy to the object. This energy can excite electrons to higher energy states, and when these electrons return to their ground state, they can re-emit the light. Depending on the material, different wavelengths are absorbed by a different amount, and the light emitted can be of a different colour to that incident on the material.
Materials that appear blue under white light are generally blue because they absorb all other visible wavelengths apart from blue, and hence the blue is scattered from the material. Scattering is the process by which light deviates from its original path. This can happen through reflection, refraction, or diffraction, and the size and shape of an object significantly affect how the light is scattered. For instance, smaller particles tend to scatter shorter wavelengths of light (towards the blue end of the spectrum), while larger particles scatter longer wavelengths (towards the red end of the spectrum), hence the sky appears blue. When particles are larger than the wavelength of light, a phenomenon known as Mie scattering occurs, and this can be observed on mountain mist due to the sun, or around streetlights. The absorption and scattering of a material determine how that material interacts with light. For instance, a material that absorbs most of the light that hits it will appear dark, because little light is reflected back to our eyes. Conversely, a material that reflects most of the light will appear bright.
Figure 1: Mie scattering – a mountain halo. Photo by Miha Rekar, taken at Kališče (at 1540 metre), Preddvor, Slovenia. Photo in public domain.
Since materials scatter light in different ways, artists can exploit a variety of mediums to create visually stunning works. In painting materials, light interacts with pigment particles through absorption and scattering, where white pigments dominate in light scattering and coloured pigments dominate in light absorption. This interaction of light and matter allows artists to create opaque paint films, obscuring anything behind the film, while also allowing a combination of colours to create desired lighting effects. In the case of film strips, light scattering is crucial for image formation. When projected light is scattered off the film strip, a visible image is created, where the degree of scattering depends on the film’s composition and structure, with different film strips producing distinct colours depending on their chemical composition. Sculptures, on the other hand, offer a three-dimensional canvas for light scattering, where the interplay of light and shadow on the sculpture’s surface enhances its depth and texture, bringing the artwork to life. This can be seen particularly in the hollow-face illusion, where a concave mask of a face appears as a normal convex face when viewed from a distance.
Artificial Intelligence (AI) has made significant strides in a wide range of endeavours, including image generation, which is the focus of this article. Models like DALL-E, Stable Diffusion, and Midjourney have led the way in text-to-image innovation. These models take text prompts and produce images that match the description using machine learning and deep neural networks. They have potential applications in numerous fields, such as graphic design, video game development, advertising, and marketing. However, an important question, which we discuss here, is whether these generated images are physically correct. In other words, do these generated images obey the rules that describe the physics of light.
AI has been a game-changer in many activities, including content creation, where it has revolutionized how we produce videos, images, music, and writing. Using AI in generating content has resulted in increased efficiency, reduced costs, and improved quality, and has been applied to create realistic images of photographs, oil paintings, and 3D structures. AI can also help artists visualise more complex art and sculptures. For instance, LightX’s AI sculpture generator uses artificial intelligence to craft breathtaking images of 3D sculptures using just a single photograph. This advanced technology can empower artists to experiment with sculpting and design, bringing their artistic concepts to life. However, to generate realistic images accurately, deep learning networks need to understand the physics of light. We can evaluate the degree to which AI has understood or correctly applied the rules of light physics by asking it to generate images that offer the opportunity for careful analysis.
As shown in figure 2, DALL-E 3 can be used to generate an image of a straw in a glass of water. The rules of physics denote that the angle of the straw should be different below and above the water line, owing to the effect of refraction. This is a direct consequence of the difference in the refractive index of the water and air. This generated image can then be compared to a real photograph of a straw in a glass of water. Whilst the generated and the real image have slightly different viewing angles and the two glass containers are not identical, there is a strong similarity between the angle of the straws below and above the water line. In this case, it appears that the AI has indeed generated an image that appears to follow the physics of refraction.
Figure 2: (Left) Image of a straw in a glass of water generated using DALL-E 3 using the prompt “red straw in a glass of still water on a white surface”. (Right) Image of a photograph of a straw on a white piece of paper taken using an iPhone SE 2020 edition. The angles of the straw above and below the water are highlighted by blue and yellow dotted lines. Images © the authors.
Figure 3 shows a generated image, using DALL-E 3, of a miniature sculpture of Stone Henge placed on a mirrored surface, which provides the opportunity to evaluate the capability of the AI in following the rules of reflection. A second image shows the result after flipping the generated image on the horizontal axis, to allow for a more detailed comparison of the reflected component of the image. Whilst the reflection is mostly correct, there is a notable error in the position of one of the Sarsen stones, and this is also highlighted in the figure. In addition, whilst the shadows and texture are similar in both the original and the reflection, there are subtle differences in the general angles of the Sarsen stones. The conclusion here is that, in this case, the AI has failed to perfectly follow the physics of reflection.
Figure 2: (Left) Image generated using DALL-E 3 using the prompt “A miniature sculpture of Stone Henge on a reflective surface”. (Right) The vertical mirror image, with the yellow circle highlighting a significant error in the reflection. Images © the authors.
As artificial intelligence technologies ‘learn’ to understand the physics of light, the journey towards accurate physics-based artwork continues to progress. While more training data and novel network architectures are necessary, it is likely that we are steadily progressing towards a future where AI can assist artists in creating realistic imagery on demand. This could bring forth a world where artists can effortlessly translate their visions into digital or physical canvases with unprecedented accuracy and creativity. AI also holds the potential to revolutionise the materials used in art by uncovering new ways to manipulate light. By understanding the absorption and scattering properties of currently unused compounds and materials, AI may help artists discover innovative paints and materials that can unlock new and exciting visual capabilities.
At a glance:
© Journal of Creativity and Inspiration.
Images copyrights as specified.
Dr. James A Grant-Jacob, MInstP, is a Senior Research Fellow at the Optoelectronics Research Centre, University of Southampton. He has worked in a variety of areas, ranging from high harmonic generation and phase retrieval, to DNA sequencing, laser-induced forward transfer (LIFT), pulsed laser deposition, laser fabrication, laser machining, sensing, medical imaging and artificial intelligence.
Dr. Ben Mills is a Principal Research Fellow and world-leader in the application of deep learning for laser machining. With over 150 papers, 20 invited talks, and £ 11 million grant income, he was recently awarded “The Top 100 in Photonics” and the AILU “Innovation Award”.
