Visitors to the Mauritshuis museum can use their smartphone to experience Rembrandt van Rijn’s The Anatomy Lesson of Dr. Nicolaes Tulp in augmented reality, thanks to Dutch design agency CapitolaVR.
To mark 350 years since the passing of the 17th-century Dutch painter, CapitolaVRcreated the Rembrandt Reality app that uses augmented reality to let people “step inside” one of the artist’s most famous paintings.
The app offers visitors a virtual reality experience of the anatomical dissection that took place in 1632, as depicted by the artist, in a bid to explore new ways of interacting with historic artworks using technology.
Users can interact with virtual holographic 3D objects that have been scanned from a real life reconstruction of the painting using devices compatible with Apple‘s ARKit or Google‘s ARCore augmented reality platforms.
“We wanted to give people the opportunity to actually walk into the artwork and explore the painting and the story behind it with innovative technology,” said CapitolaVR’s head of digital David Robustelli.
“The AR app offers people a different way of exploring art both visually and on an educational level,” Robustelli added. “As you ‘physically’ step into the painting you are fully immersed and learn its history while exploring the way it was created by one of the world’s most famous painters.”
To recreate the artwork in virtual form, CapitolaVR selected “lookalikes” to pose as each of the characters in the scene, using makeup and 17th-century costumes to make them appear as similar to the painted figures as possible.
The agency then used a 3D scanner made up of 600 reflex cameras to scan each actor and the setting – which was reconstructed from the original room at the Waag in Amsterdam – before combining these scans and fine-tuning the textural details via 3D modelling software.
“This is a new way of looking at art, which catapults the experience of looking at art into the future,” said Emilie Gordenker, director of the Mauritshuis. “Rembrandt was a pioneering painter. He used new techniques to arrive at a new visual language.”
“350 years later, he continues to encourage us to innovate,” she continued. “Using augmented reality technology, you can enter the anatomical theatre through a portal, whether you are at home or outside. You become a witness to a seventeenth-century anatomy lesson and look over Rembrandt’s shoulder.”
The creators then used ARKit and ARCore to place the scanned, holographic 3D objects into a virtual environment that can be accessed by users via an app on their smartphone, where they can interact with the 3D objects to trigger various animation effects.
“With ARKit and ARCore, a huge audience now has access to augmented reality with their current devices,” Robustelli told Dezeen. “As devices are becoming more powerful with each release, this offers a lot of opportunities for creatives and developers.”
“We wanted to realise an experience where art meets tech not only to immerse users within a painting in a totally different way, but also to educate them and attract a younger audience,” he continued. “Tech is becoming part of our lives and can enrich experiences and show them on a whole new and interactive level.”
The creators applied various lighting effects to achieve Rembrandt’s characteristic chiaroscuro painting technique – strong contrast between light and dark – and added particle effects to imitate the look of dust illuminated by beams of light.
“We were not just trying to remake the painting in 3D, but to create a portal to the past, to the moment the painting was created,” said the designer.
“For instance, to create the [flayed] arm – one of the most significant elements of the painting – we had to do a complete study on the anatomy of the arm to create this part in 3D as authentically as possible,” he added.
There are several “hotspots” marked within the AR experience, which delve deeper into the history of the painting by zooming in on specific details that can’t be seen at first glance, as a voiceover guides users through the experience and explains what the user is seeing.
For example, viewers can see through 3D scans that Rembrandt originally depicted the cadaver with an amputated hand, which the subject lost as a punishment for theft before his execution, before painting over this with a fully formed arm.
“Both artificial intelligence (AI) and AR are techniques that specifically help the end user by providing them with information that they couldn’t have easily accessed a couple of years ago,” said Robustelli. “Art is a perfect example of how AI and AR can be used in this perspective.”
“It basically is the perfect combination of the physical world being enriched with the digital and virtual world to enhance the users’ experience with information and visuals and allow them to interact with this.”
“It shifts from one-way communication to two-way communication,” he continued. “The user is in control and that makes it very interesting for a large audience.”
Visitors to the Mauritshuis museum can download the Rembrandt Reality app free of charge from the Apple app and Google Play stores, to be fully immersed in The Anatomy Lesson of Dr Nicolaes Tulp.
Augmented reality also made an appearance at London’s Serpentine Gallery, where a 3D digital rendering of performance artist Marina Abramovi? wandered around the space in a mixed-reality art installation.
Augmented reality takes museum-goers inside Rembrandt painting [Dezeen]