Art: computer and human interpreting skills

Thursday 24th December 2009
The Golden Mean and Rectangle. mceinc/goldsqre.htm

While some researchers find computers lack the the high level pictorial information to place art in a period, the human eye fares better at facial expressions recognition over a minimum 100 millisecond moving image.

Researchers from the University of Girona and the Max Planck Institute in Germany have shown that some mathematical algorithms provide clues about the artistic style of a painting. The composition of colours or certain aesthetic measurements can be quantified by a computer, but machines are still far from being able to interpret art as people do.

(Right)Seated woman with bent knee_Egon Schiele 1917.jpg.

But how does one place an artwork in a particular artistic period? The question is raised by scientists from the Laboratory of Graphics and Image in the University of Girona and the Max Planck Institute for Biological Cybernetics, in Germany.

Researchers have shown that certain artificial vision algorithms mean a computer can be programmed to "understand" an image and differentiate between artistic styles based on low-level pictorial information. But human classification strategies, include medium and high-level concepts.

Low-level pictorial information encompass brush thickness, type of material and composition of the palette of colours. But medium-level information differentiates between certain objects and scenes, as well as the type of painting i.e.landscape, portrait, still life. High-level information takes account of historical context and knowledge of artists and artistic trends.

While "it will never be possible to precisely determine mathematically an artistic period nor to measure the human response to a work of art, .. we can look for trends", Miquel Feixas, one of the study authors published in the journal Computers and Graphics, tells SINC.

Researchers analysed various artificial vision algorithms used to classify art, finding that certain aesthetic measurements (calculating "the order" of the image, based on analysing pixels and colour distribution), as well as the composition and diversity of the palette of colours, can be useful. 

The team worked with people with little knowledge of art, showing them more than 500 paintings done by artists from 11 artistic periods. Participants were "surprisingly good" at linking the artworks with the corresponding artistic period, showing the high capacity of human perception.

Beyond implications for philosophy and art, scientists want to apply research in developing image viewing and analysis tools, classifying and searching for collections in museums, creating public informative and entertainment equipment, and to better understand the interactions between people, computers and works of art.

Beauty, order and complexity
The mathematician George D. Birkhoff undertook the earliest work of this kind in 1933, when he tried to formalise the notion of beauty with an aesthetic measurement defined as the relationship between order and complexity.

Philosopher Max Bense converted this into a measurement of information based on entropy (disorder or diversity).

According to Bense, "to create is to select", within a range of elements (a palette of colours, sounds, phonemes, etc).

The creative process can be seen as channel transmitting information between the palette and artist and objects or features of an image. This concept provides a powerful tool for analysing composition and the visual attention ("saliency") of a painting. "Categorizing  art: comparing humans and computers." Christian Wallraven, Roland Fleming, Douglas Cunningham, Jaume Rigau, Miquel Feixas, Mateu Sbert.  Computers & Graphics 33 (4): 484-495, 2009.

Human vision needs movement to interpret

Communication is a central aspect of everyday life, a fact reflected in the variety of ways eople exchange information, not just with words, but using their face and body. Scientists from the Max Planck Institute for Biological Cybernetics in Tübingen, Germany, found out that we are able to recognise facial expressions in motion - for example in a movie - far better than in a static photograph. The video sequence needs to be at least as long as one tenth of a second to gain this dynamic advantage. (Journal of Vision, December 7th, 2009) Image below courtesy: Christian Wallraven / Max Planck Institute for Biological Cybernetics

A facial expression can state a lot but the thoughtful, baffled, ignorant, happy faces require their companion faces for best comprehension A nod indicates understanding, a frown may say: “Please explain that again!”

Scientists from the Max Planck Institute for Biological Cybernetics discovered that we are able to classify an expression much better when it moves naturally rather than when it is “frozen” in a photograph. In order to gain the advantage of dynamic information, we need to see the expression moving for at least 100m/secs. In a shorter video sequence, our brain is less capable of interpreting the facial motion.

Some expressions rely on changed head orientation, for example, a nod or a shake of the head, others on  complex deformation of facial parts, such as wrinkling our nose to signal disgust or a frown.

In order to examine to what extent we are able to recognise – based on facial expressions – the mood of a person with whom we are interacting, scientists showed participants pictures of humans with various different expressions.

Among them were simple, emotional expressions, as “happy” and “sad”, and more complex ones as agreement, confusion, or surprise, usually used to emphasise or modify statements in a conversation. 

To investigate whether these expressions are recognised more easily in motion or in static pictures, a short video sequence was shown to the participants. The video recordings began at a neutral expression, showed an emotion, and ended at the last frame before the face started to head back to a neutral expression.

The frame used in the static conditions was the last, so-called ‘peak’ frame of each dynamic sequence. The participants were then asked to identify the expressions based on the shown sequence or single frame.

In further experiments, video sequences were converted to photographic  series shown to the participants. The expressions were still recognised more accurately in the video sequence. This showed the dynamic advantage is not due to the presence of multiple images, but that some form of dynamic information is being used.

To figure the degree to which facial expression recognition relies on natural movement, the frames were presented as a movie, in a random order. Comparisons of the performance in this scrambled condition to the original video sequence shows that the recognition rates were still higher in the original than in the scrambled version.

The chronological direction is of importance as well. If  video sequences are temporally reversed, they are identified less accurately. Finally, the more temporal information we receive, the better we are able to recognise expressions - at least up to 100 milliseconds.

Results show that neither pictures, nor motion alone are of importance, but that we need a combination of the correct temporal sequence and the correct facial motion to reliably interpret facial expressions.

“Facial expressions, like gestures and body motion, are a dynamic phenomenon and need to be investigated with the help of video sequences in order to get a better understanding of the dynamic information that is being processed”, says Dr. Christian Wallraven, (right) co-author both the studies. “Our results also have implications for the area of computer animation, since its goal is to create artificial avatars and facial animations that are able to communicate realistically and believably”, says the physicist and perception scientist.

Custom Search

Scotland, Computer News in Scotland, Technology News in Scotland, Computing in Scotland, Web news in Scotland computers, Internet, Communications, advances in communications, communications in Scotland, Energy, Scottish energy, Materials, Biomedicine, Biomedicine in Scotland, articles in Biomedicine, Scottish business, business news in Scotland.

Website : beachshore