This MosAIc algorithm is developed by MIT students along with Microsoft team. This is new AI based MosAIc algorithm which can identify subtle visual similarities between artworks created in vastly different media, cultures, and time periods. In short we can say that this algorithm helps to identify the similarities between two art-pieces and these art pieces is made by some artists and the artists never met before in their life, but still some similarities in their work. So the aim of this project is to find those similarities.
Let's see some more details of this algorithm:-
Now, if we think of how the idea of this project came from?
Mosaic project sponsor Mark Hamilton, Research Engineer pursuing a PhD at MIT was inspired by a visit to an exhibit at the Rijksmuseum.
This exhibit compared works from the studios of Rembrandt and Velazquez, two prolific artists that never met before, still told similar stories in their works.
He began thinking of possible ways the same idea could be achieved by AI, and developed mosaic algorithm which searches for similar works of art based on a starting image and a feature of that image such as the medium, a color, or the culture (e.g. “French”).
Below is one of the example or output of this algorithm:-
From the above figure, we can see giving input image blue and white dutch banyan dress image, we can see multiple outputs from this image from this 2 museums by culture as well as media.
Based on a starting image of a Banyan, the algorithm finds similar works of art from different artists in the Rijksmuseum and The Met collections. This algorithm matches the dutch banyan image to a resulting matching blue & white music instrument.
The lead author of the paper mark has told that:-
The algorithm picked up on commonalities of form between the banyan and a string instrument, but also the color of the blue and white motif which originated in Ming Dynasty and spread through Europe. You can see evidence of cultural exchanges happening through the outputs of this algorithm
- - Lead author of the paper( Mark hemilton)
Below is the results on open access artwork from the Metropolitan Museum of Art and Rijksmuseum using culture and media,top row text, as conditioners, taken from the research paper.
Here is another output of this algorithm:-
Above example explains the similarity between the query image and any requested match. Unimportant pixels have been masked. The subject of the work is selected as an important contribution to the similarity.
Till now we have seen what this mosaic algorithm is and what it doing . Let's check a very brief of the logic behind this algorithm.
They apply conditional KNN data-structures on the combined open-access collections of the two museums defined earlier to create an algo which is capable of identifying artistic connections across time, space, culture and media. It puts similar images in a tree-like structure and traverses it till they find the closest result
From the above figure,in left hand side tree the colored blocks are the image within leaf node, while on right hand side is the result of applying conditioning on the particular class can prune the nodes. For more details about it you can find full research paper here
For building this algorithm mark hemilton said that
Building this algorithm was challenging as the aim was to match images that weren't just similar in color & style, but also in meaning and theme.
- Mark hemilton
One more note from the author regarding this project to take this project as enhancement or added features for curators,but not an replacement. Here are the words from the author:-
Our method is good at building a particular type of exhibition: unlikely pairs of art that span barriers and share common structure,The world’s art exhibits are infinitely varied, each with their own thesis, and this is what makes art wonderful. We hope the community can see this method as adding a distinct voice to the chorus, not replacing it.
- mark hemilton
You can Explore more:- https://microsoft.github.io/art/
Full research paper available here
REFERENCES
