Learning how to draw has never been easier, and as interesting. The innovative mobile app SketchAR does this using augmented reality, machine learning and neural networks. It is the first assistant powered by AI for drawing using augmented reality.
The best part about the Lithuanian start-up is that you can draw on paper, but you can also draw on walls, which allows you to make your own awesome murals. So, the app put virtual images on a paper or a wall in order to let you trace drawings from your phone.
But, how does it work?
Just hold up your phone and watch as virtual images appear on top of your surroundings. The app will work better on A4/A5 sizes of paper. However, you can also put any image on a wall and subsequently trace it. Note that to paint on a wall you need to use smartphones with ARKit and ARCore.
Of course, augmented reality glasses or a mix with these, are the best way to draw without limits and it’s then that true synergy is able to take place combining various things such as art and technology.
Innovation Origins had a short interview with founder, CEO, and mural artist Andrey Drobitko.
How did you come up with the idea for SketchAR?
The initial idea was as a tool for artists like myself; a convenient and straightforward method for tracing sketches onto a large surface. Later we discovered that for those who are looking to acquire new life skills – ‘learning to draw’ is one of the most frequently searched items on the internet.
Do you think that there are many start-ups tackling the same issue?
We don’t think so. We know that we are the only app in this domain, because of the technology we developed which works on flat, blank surfaces.
What makes SketchAR different from other similar start-ups?
We understand two spheres in depth: art and tech. The second thing is our Markerless CV-tracking of flat surfaces powered by machine learning and neural networks. We analyzed hundreds of thousand of videos from our users (with their permission) when we taught our CV core. The ideas was to ‘see’ different types of surfaces, hands, paper, and their structure.
And as I said, there is no other app which teaches drawing using AR, and whch does not use markers (like QR codes).
How would you describe your technology?
Augmented reality is the most straightforward way to take a first step. The first line you draw will lead to endless growth for you.
Did you create the technology used in SketchAR?
Yes. Since 2016 we focused only on how to make the camera understand such complex surfaces like a blank piece of paper or a wall.
What has been the biggest obstacle that you have had to overcome during the whole SketchAR process?
‘ The fear of blank paper’ is not only the fear our users have, but one of the fundamental problems for computer optics. There was no solution when we started. We had to create that ourselves. Even software like ARKit or ARCore cannot work with this. We spent more than three years just focusing on this issue.
Was there a moment when you thought of giving up?
I’ve never think about giving up. It’s just rock’n’roll.
What has been the most gratifying moment?
Every time I see users who created something using SketchAR, I feel proud of the product and our team. The whole product is about a new era of art, education, creative, etc. Every day is an adventure.
And the second moment was when we won the most prestigious award for digital products – Webby Awards. SketchAR won the Best Use of Augmented Reality Product of the Year Award 2018.
What can we expect from the coming year/years?
We expect billions of dollars, Lamborghini, and a yacht 100m long [- joking]. Actually, we are busy with developing something really amazing. You will get to see it.
Within the next 5 years we will be the leading virtual assistant for creativity, highly personalized; and SketchAR will be the most powerful CV-tech when it comes to the environment.
Who is the SketchAR customer?
Novices in art, mainly those between 20 and 35 years old. They are people who have always wanted to but never did get into drawing.
Can you tell me a bit about the feedback you’ve gotten?
Most of the feedback, of course, is that people want to use SketchAR with AR-glasses. But not everyone has a smartphone yet. So when AR glasses are on the mass market, we will have the most personalized assistant, because of data collection that is already used in smartphones.
Why focusing on drawing?
Because our passion is art and tech. Also, I am a pretty famous mural artist.
Innovation Origins is an independent news platform, which has an unconventional revenue model. We are sponsored by companies that support our mission: spreading the story of innovation. Read more here.
On Innovation Origins you can always read articles for free. We want to keep it that way. Have you enjoyed this article so much that you want to contribute to independent journalism? Click here: