Researchers train AI to attribute paintings based on detailed brushstroke analysis

A figure from the research paper "Discerning the painter’s hand: machine
learning on surface topography" showing four paintings analysed in row A, topographic data in row B and machine learning attributions of different areas of each canvas in row C Courtesy Case Western Reserve University
Art historians may have a new tool for settling the attribution of disputed paintings using artificial intelligence (AI) thanks to research by a cross-disciplinary team led by physicists at Case Western Reserve University in Cleveland, Ohio. The research, published in November in the journal Heritage Science, shows how machine learning analysis of small sections of topographical scans of paintings—some as tiny as half a millimeter—was able to attribute the works to the correct artist with up to 96% accuracy. The technology could eventually also help identify which artists were responsible for different areas of a painting made by multiple artists or produced by an artist’s workshop, and help tell authentic works from forgeries.
The project differs from others that have sought to harness AI to settle questions of attribution and authenticity in that most previous research in this area has been based on machine analysis of high-resolution images of paintings, not the painted surfaces of the canvases themselves.
“The idea was that analysing the brushstroke was going to create a fingerprint,” says Kenneth Singer, a physics professor at Case Western Reserve who led the research. “We found that even at the brush bristle level, there was a fair level of success in sorting the attribution. Frankly we don’t really understand that, it’s kind of mind boggling actually when you think about it, how the paint coming off a single bristle is indicative of what we’re calling the artist’s unintentional style.”
The project focused on analysis of sets of paintings made specially by students at the Cleveland Institute of Arts, who were tasked with painting copies of a photograph of a water lily. The research involved training convolutional neural networks (CNNs) with three-dimensional scans of the paintings’ surfaces made with a profilometer. By dividing the canvases into tiny square patches for analysis, the CNNs identified each artist’s “unintentional style” or “fingerprint”. The AI was then able to correctly attribute other paintings by matching the artists’ unintentional styles in the textures of brushstrokes.
The team behind the research is now looking for additional tests of its AI’s capabilities. It collaborated with conservation firm Factum Arte to analyze a topographic scan of El Greco’s Portrait of Juan Pardo de Tavera (1609), which was severely damaged during the Spanish Civil War and extensively restored.
“This is a painting we have an answer key to, because we have photos of the destroyed painting and the current painting, so we’re able to make a map of the areas that were conserved, and [the AI] was able to identify those areas,” Singers says. “But there was another section of the painting that it identified as conserved that wasn’t obvious, so we’re going to have a painting conservator in Spain look at the painting to see what’s going on.”
Now the research team is turning its attention to paintings produced by multiple artists trying to replicate the style of one painter in their studio or workshop. Discerning between the hand of a Renaissance master, that of his star pupil and those of his lesser-known assistants has long been a topic of heated debate among art historians and Old Masters specialists, often with large sums of money hanging in the balance when works go to auction. The researchers hope to develop “unbiased and quantitative methods to lend insight into disputed attributions of workshop paintings”, they write. To that end, they are working with artists from the Cleveland Institute of Art again to create brand new paintings in a workshop process, with multiple artists working on each canvas in a unified style.
In addition to the student painters and other members of Case Western Reserve’s physics department such as Michael Hinczewski, key collaborators on the research included the university’s chair of art history Elizabeth Bolman and Dean Yoder, the conservator of paintings at the Cleveland Museum of Art. The endeavour was a true marriage of art and science.
“The project came about from an idea of a student of mine, who at the time had just started dating an art history student,” Singer says. “They went to a conference on art and science and had the idea of using this profilometer we have in one of our labs for doing surface topography. I agreed to do it and then, after a while, all my students got involved and the collaboration grew. Those two students are married now, by the way.”
The next application for the AI could be to test it on media that have less surface texture than paintings, Singer says, like watercolours or drawings. “Those would seem to be more challenging,” he says, “but what I’ve learned in this project is that I shouldn’t be as skeptical as I usually am, because this artificial intelligence is surprisingly good.”

source