[smart_track_player url=”<<enclosure url=”https://traffic.libsyn.com/secure/allanmckay/Ep275_-_KeenTools_PUB_V3.mp3″ length=”101545496″ type=”audio/mpeg” /></enclosure>” title=”275 – KeenTools Founder Roman Belov” social=”true” social_twitter=”true” social_facebook=”true” social_gplus=”true” ]
Episode 275 — KeenTools Founder Roman Belov
Roman Belov is the Founder of KeenTools, a Blender add-on for 3D modelling of human faces and heads based on photos. The models can be used for sculpting, animation, tracking or anything else in Blender or facial tracking with FaceTracker in Nuke; or to be exported for further modifications in any other 3D software.
In this Podcast, Allan interviews Roman about the origins and development of this software, its current usage and its potential, as well as the future of cloud based innovations.
KeenTools’s Website: https://keentools.io
KeenTools on LinkedIn: https://www.linkedin.com/company/keentools
KeenTools on Twitter: @keen_tools
KeenTools on Instagram: @keentools
HIGHLIGHTS:
[04:05] Roman Belov Introduces Himself and Talks About His Background
[10:50] Roman Talks About the VFX Industry in Russia
[13:07] KeenTools Development and Its Release
[18:33] The Nuke Origins of KeenTools
[27:44] Is Blender the Platform of the Future?
[41:02] Allan and Roman Discuss the Need for Cloud Based Innovations
[47:40] The Potential Uses for KeenTools
EPISODE 275 — KEENTOOLS FOUNDER ROMAN BELOV
Hello, everyone! This is Allan McKay.
Welcome to Episode 275! I’m interviewing Roman Belov, the Founder of KeenTools which is a really powerful piece of software that a lot of major studios use for big feature films. It has so much potential and it’s about to really take off! I wanted to talk to Roman about the technology of facial matchmoving.
In December, I may be doing a big challenge similar to the one I’d done before: Your Best Year Yet. I will be announcing it soon! Next week, I’m doing a Podcast with Mark Toia, the Director of feature film Monsters of Man.
Please take a few seconds and share this Podcast with others.
Let’s dive in!
FIRST THINGS FIRST:
[1:07:23] One of the biggest problems we face as artists is figuring out how much we’re worth. I’ve put together a website. Check it out: www.VFXRates.com! This is a chance for you to put in your level of experience, your discipline, your location — and it will give you an accurate idea what you and everyone else in your discipline should be charging. Check it out: www.VFXRates.com!
KEENTOOLS FOUNDER ROMAN BELOV
[04:05] Allan: Thanks so much for taking the time to chat! Do you want to quickly introduce yourself?
Roman: I don’t know how to start. I’m just the Founder and Lead Researcher at KeenTools. KeenTools does tools for VFX artists and 3D modelers.
[04:37] Allan: For you, what’s your background? Were you always involved in 3D or in programming?
Roman: Well, it’s a long story. It all began when I was 4 years old. And my parents wanted a bright future for me. They brought me to two different schools: one was an art school, the other — a chess school.
[05:11] Allan: If you added music in there, you would’ve ticked all three.
Roman: Art school was more about painting. My entire life has been about two different tracks. One was about math and physics and logic, and the other has been about visual stuff like painting. When I finished that art school, I had 9 years of drawing and I did that in high school. Then, I found a new area which was cinema. At the same time, I was studying math at the St. Petersburg State University, [as well as] computer science. It was always for me KeenTools became a tradeoff between two different parts of myself. I’ve been in the VFX industry since 2008. I started compositing in Shake and it was fun.
[07:15] Allan: In the past week, Shake has come up a lot for some reason. I love Shake so much! But then Apple got a hold of it.
Roman: I switched to Nuke only in version 6. For me, it was a combination of different skills. I understand mathematics and I can feel the picture and understand how it should look. The idea was pretty natural for me. I expected for different tools to do this or that, but at the same time when I would use them, they didn’t do that. That’s why we decided to develop something and that’s how KeenTools started.
[08:49] Allan: At that time when you started working on KeenTools, were you working in the industry? Or did you do it more as a hobby?
Roman: I had my own studio and we created mostly commercials. For some of them, we needed VFX, of course. We also created some short films which were more for fun. But speaking about the industry, in Russia there is no cinema industry. Every new film is something outstanding. There is a tv industry, but no cinema industry. It’s probably like a small European film industry. In Russia, there is something like 5 VFX companies, no more!
[10:51] Allan: I’ve had Armen Yahin, the CEO of Main Road Post, on the Podcast (www.allanmckay.com/87).
Roman: That’s one of the best studios in Russia, probably the best one! At least, it’s the studio I like the most. They work on a really low budget with top deadlines. It’s really interesting how they organize everything.
[11:35] Allan: That’s what was cool about chatting with him. It’s so weird how he started his company and how they manage their team (and how not everyone wanted to adapt). In general, I thought that was really fascinating.
Roman: Something I’ve never seen at other companies! The cool thing about being the Founder at KeenTools is that I travel a lot and I visit a lot of studios. I’ve met with VFX supervisors and they show me their studios and how they work, what pains they have. I’ve never seen anything similar to Main Road Post anywhere.
[13:07] Allan: I’m just curious, was KeenTools the first commercial plugin you’ve ever made?
Roman: First of all, KeenTools is not a plugin. It’s a set of plugins. They aren’t commercial, they are free. We started with several free plugins. But they’re very handy. I know people use them a lot. The last plugin, we showed to everyone. There is a huge difference between something that looks like a product and can be used by everyone; and something that can be used for your needs. It’s a really long road. It’s an interesting story because we started with a prototype and the prototype is really easy and fast to do. But then it takes several months to create an actual product. It’s a long and tedious process of software development. It’s about testing your approaches and showing them to users, listening to their feedback. The thing I like the most is when you’re trying to combine some state of the art mathematics and make it available to users and make it something they can touch, that’s the most fascinating thing for me.
[15:24] Allan: What was it like when you put it in front of artists? Obviously, you’re working internally. How early were you getting feedback from artists?
Roman: We did something called continued delivery. We released everything as early as possible. Yesterday, we had a release, for instance. Usually, companies try to create big releases once a year, or twice a year. They create some features and a marketing plan, and we have a completely different approach. We release the nightly build every day and everyone can use this stuff if they’re okay with not having a properly tested solution. But from time to time, we test everything and do a public release. Currently, we try to do one release per 3 weeks. As the new feature is ready, we release it. The same was with the first version as well. As soon as we created something, we released it on Nukepedia. We got great feedback that we were doing the right things. To be honest, it’s probably not that good for marketing. It’s great to create a big buzz with all these great features but at the same time, I believe that marketing is not something we could be famous for. We focus on the tools and the product.
[18:33] Allan: Just to talk about Nukepedia for a second: What was your decision to support Nuke based on?
Roman: First of all, we love Nuke. It’s really hard to say that because I know many people who don’t like it. At the same time, for me Nuke is something that’s really good as a platform. And Nukepedia is the ecosystem. There are a ton of different plugins and we believe that Nuke could be a huge platform in the future. It has great potential! That’s why we created a plugin for Nuke. KeenTools came from an actual job. It was a feature film with 500 shots of objects tracking. We each had a moving object. We looked around and there were different standalone solutions, like 3DEqualizer. We tried them heavily and we understood that we wanted to simplify and shorten the pipeline as much as possible. Because when you have some standalone thing, you have to track something first then import that track inside Nuke; and if there is something wrong, you have to re-track and reexport. Also the problem is the only person that sees the final picture is the actual compositor. It’s really handy when a compositor can fix something on the fly. That’s why we decided to create it in Nuke, but it was for our needs originally. We had 500 shots, 4 people in the studio and 3 months for our deadline. And we understood that we needed to create something to expedite the process. We created a rough prototype first. We failed the deadline. We did it in 4-5 months.
[22:32] Allan: That amount of work and so few people!
Roman: And it was really fast. We did several shots per day. We created a pipeline and everything was inside Nuke. That was the reason for creating KeenTools.
[23:05] Allan: I love when software is built by necessity. You had to look for a better solution which is the best way solutions are made.
Roman: Actually, it wasn’t a cool idea. It was a huge pain!
[23:30] Allan: But that’s what I mean that it was born out of necessity. You needed to do this and there was a big disconnect in tools. That’s where Maya was supposed to create solutions with PowerAnimator, PowerModeler. It was very segmented. Having an all in one solution made sense.
Roman: We are working on the plugin version in Maya as well.
[24:22] Allan: And that makes sense. But I like that you started out with Nuke being the destination. Do you think that in the future people will start working in a compositing environment, like Eddy and Vray, and Red Shift?
Roman: I think that most likely people will try to minimize the number of software they use. I believe that’s because of minimizing the number of scenes between software — because all this import / export stuff could be a nightmare. At the same time, I can’t believe it would be in Nuke. Most likely, I bet on Blender. It evolves rapidly. It already has a compositing part. When I look at what people do in Blender, it’s just amazing! What is also very important is that they don’t have a lot of legacy. In Maya, there is a huge legacy. Nuke is better but they’re trying to reimplement an interval engine. They have to write a new engine from scratch. But they have a good compromise. But it’s closed source stuff. If I were to start now, I would start with Blender.
[27:45] Allan: Just to touch on Blender, I’ve been following it since 2003. I’ve been fascinated by it but I never opened the package. More recently, I’ve asked people about their fascination with it. You’ve given the best answer I’ve heard so far. The answer I’ve gotten before has been, “Because it’s free.” If you’re developing a plugin, that’s not a good enough reason. In terms of its trajectory, what are your thoughts why so many people tend to use Blender and why would it be the future?
Roman: Well, it’s probably because it’s free. That’s pretty important. It’s not about money, to be honest. It’s more about the ecosystem. If you want to have something that’s a platform for everything else, this platform would be great. With KeenTools, we’re limited by the number of users that use Nuke. Nuke is more than 10 times more expensive than our tool. And there have been times when people would buy Nuke just so that they could use KeenTools. For them, it was okay from a business angle. At the same time, when we released a plugin for Blender, our audience increased by 10 times. Of course, that’s a different audience because some of them won’t pay you. But it’s a big audience and it’s different. There aren’t so many VFX companies in our industry, less than 100. We know all of them and they all use our tools. This market isn’t really big. As a platform, I believe that Blender could be more interesting not only for users, but for developers. That could make it explode in terms of abilities. At the moment, I believe there is only one piece of software that could do that — and it’s Blender. Honestly, I don’t really like lots of things in Blender. There are a ton of things that could be improved. It’s open source. If I don’t like something, I can improve it.
[33:22] Allan: Did you get the email yesterday that Autodesk acquired Blender?
Roman: Sorry? What?!
[33:38] Allan: I’m kidding! But going back to [what you were saying]: By being an open source, you’re creating a community. That’s a great way to look at it! Your user base grew substantially which I think is powerful. This week, I talked to someone about building software. I was talking about TyFlow which is free. The product that they would develop wouldn’t be as great if they didn’t have people chiming in. I’ve seen that before. Having that community give you feedback in real time is huge! Nuke will be for compositors a lot of the time and they aren’t FX people and they have little interest in doing FX. Whereas in Blender, you’ll attract that audience automatically. Plus, they aren’t paying a huge overhead to be a part of it.
Roman: Speaking of platforms, I also think that Blender isn’t modern enough to be central. Especially now, people understand how important it is to have something in the cloud and to have a way for collaborative manipulation on different scripts. Blender is not ready for it yet. There are some solutions but most of them aren’t handy. I believe that the solution for the future would have to host in the cloud. It’s much more efficient in terms of money. And you would use the hardware you need and you don’t buy something more powerful than you need. That’s why there is a possibility that the VFX industry will be more distributed than it was. I believe that no solution is ready for such challenges yet because they were all created when everything was done locally. Everyone was trying to buy the best hardware to work locally. Now, everything has changed. Data science was local as well, but now it’s mostly in the cloud. Basically, the same should happen with VFX as well. There were several tries, like from the Foundry, but I think they were too early. But I think in the next 5 years, we’re going to see these solutions. There are so many common features between softwares. For example with manipulating 3D space, we have 5-10 different ways of doing it. That’s why we had to create a standalone. That’s why I believe it should be standardized.
[41:02] Allan: It’s interesting how a lot of innovation that’s trending right now isn’t new. This stuff has been around for 30 years. Bill Gates’s dream was to have every household have a computer. Now, we want to get the computers out of the house and do everything virtually. I remember back 1991 reading a magazine where Nintendo would give you the download for every game. Now, we’re moving in that direction. With COVID-19, everyone is looking to work via VPN. Now, having these virtual computer has a much bigger need. I think the next 2-3 years will be fascinating to watch.
Roman: I believe some solutions are waiting for other solutions. Speaking of data science and AI based on deep learning, so much of it was developed over 50 years ago. But mathematics were waiting for graphics cards. When they [became] available, everything exploded. Now we have Deep Fakes and lots of different deep learning networks. Nobody could expect that. There were several waves of popularity. The last wave of popularity for 3D Cinema was because of Avatar. That’s when everyone decided they needed 3D tvs, solutions, films. Now, we don’t see such demand. The same goes for cloud solutions. There was a demand for them 15 years ago but now the internet is much faster and we’re more distributed.
[45:37] Allan: I was talking with John Brennick at Digital Domain. He wanted me to ask whether you’re planning to incorporate machine learning into KeenTools. Do you see that?
Roman: We’ve actually already implemented it. We’re going to release it fairly soon, by the end of the year, perhaps. It’s a pretty natural way to snap the face. But at the same time, the main issue with the approach is that you can’t control them. If something goes wrong, you have no way to fix it. That’s why we’re looking at different approaches for controllable Deep Fakes, ones that you can manipulate. It’s not that easy to create.
[47:40] Allan: Do you think that’s something that could blow up? Digital Domain is pretty secretive with their Deep Fake stuff. But I think there is so much potential there! Most people look at it as a 2D solution. But do you think there is potential to team with production in the future, to get their information you really need to take things further?
Roman: First of all, it’s not the first time we’ve teamed up with a production company to work on our tools. Because it’s important to have some connection with the real world and the issues they’re having. Digital Domain has an amazing software development department. I believe that at some point, we’ll develop such features like Deep Fakes bundled with some 3D formations, based on real examples. One of the main issues for me is that I don’t have lots of different friends in the VFX industry. Of course, I know everyone here, in Russia. But at the same time, the main problem in VFX is that everything is covered by an NDA. Companies should be interested in teaming up with other companies. We usually speak with guys who are artists but often they cannot share their material with us, to explain what issues they’re having (because of the NDA’s). That’s the issue why we can’t show all the examples of how KeenTools has been used in films. Sometimes we ask VFX companies to create a tracking reel on how they used our tools but those cases are rare. Basically, [the same goes for] research. Usually, it’s hard to start the research but we’re really open to it. When John writes to us about it, we’d be open to doing mutual research. Of course, we want to use the result of that research later. It’s all up for a negotiation but we’re open to it!
[53:32] Allan: I do think that case studies in marketing can be overlooked but they’re so powerful. There are these amazing tools out there known in the industry, but other people don’t know about them. Maya was the first to market themselves well. Everyone had to get that! And it became the selling point for many studios. When you’re doing something so innovating and new, KeenTools has the potential to change how we’re doing things. It’s about making it that next buzzword. I think people who are using it realize how you’ve done that successfully, with object tracking and facial capture. It’s grown so much beyond that! I think there is so much potential!
Roman: I believe in the same idea. We have a pretty niche product. It’s just a small part of the process. But our users love KeenTools and that’s one of the strongest feedbacks we can have from our users. On the other hand, it’s about our own marketing. The word of mouth is a powerful and most effective way to market.
[57:36] Allan: I’m curious what feature films have used KeenTools (that you can talk about)?
Roman: I can’t say that there is something specific or a particular film of which we’re proud. At some point, almost every other film uses KeenTools. I’m not sure about which cases. I have no way of getting this knowledge. I know it’s been used in Marvel films.
[58:30] Allan: I was going to say you’re doing a terrible job of selling your tool! I don’t want to name names, but I’m pretty sure it’s been used in Captain Marvel and so many other shows! You might not hear about it, but a lot of studios are using it!
Roman: I know a lot of tv series use it too! Game of Thrones, Gotham, Lucifer, The Walking Dead. Everytime you see some weird stuff on top of the face, most likely if it was edited in comp — it was done with KeenTools.
[59:55] Allan: I do think it’s pretty revolutionary especially since you can have it in comp, in Nuke. It’s like Smart Vector. There are some tools that come along and it means you can get things pretty accurate within minutes. Before you’d have to hack a solution. That’s why Smart Vector in Nuke is so powerful.
Roman: KeenTools could be really effective combined with Smart Vector. You can track the object first, then unwrap it in 2D space. Then apply Smart Vector to log all movement in UV space. Then reapply everything back. In fast movements and rotations, Smart Vector fails. In UV space, everything is stabilized. That’s one of the solutions people have used.
[1:01:51] Allan: In terms of the future — and whatever you can talk about — are there specific directions you want to take KeenTools?
Roman: We have three different vectors, always. One of them we’re working on Maya and AfterEffects version. The other one for current features. For example, it’s for AI stuff or faces, or better deformation models, facial mashes. It’s about improving the current KeenTools. The last one is new things. We’re working on full body tracking and different parts. We believe that’s important research.
[1:03:29] Allan: I have people do my tracking for me and nothing goes as planned. I’m really excited to see where things go. It’s one of those tools that has the potential on how it can change things. I think you’re being pretty modest, but every production can make use of KeenTools. If there is an actor in a project, they can use your tool!
Roman: Hopefully! I believe that it could be widely used. It’s still a niche but at the same time, all capture of physical actions of actors will be tracked and I strongly believe that in 10 years we’ll have completely different pipelines because of new technologies. It has already been changing in the last 20 years. The cool thing is that I don’t want to switch to other areas: There is a lot of professional tracking in the entertainment industry, like Instagram and Snapchat. But they have completely different approaches. The main value for us is to track things as precisely as possible. That’s the main aspect for us! We heavily invest in the quality of our research. Facebook wants to track as well as possible but it’s not the main point for them. These are different values. I like our values more because they’re more about science.
[1:07:19] Allan: That’s cool! Where can people go to find out more about KeenTools?
Roman: Basically, our website is the main source of information (https://keentools.io) and our social networks.
[1:07:41] Allan: This has been so much fun! I’m so excited to see the future of your product!
Roman: Thank you as well for your time, Allan! It was great to chat with you!
I hope you enjoyed this Episode. I want to thank Roman for taking the time to do this Podcast. This was a lot of fun! I highly recommend checking out KeenTools!
I will be back next week with Mark Toia, the Director of feature film Monsters of Man. Mark has a background in visual effects. He is also the face of RED Cameras. One of the interesting things about his film is that Mark has financed it himself, which is highly unusual for an action packed sci fi feature!
I will be back next week!
Rock on!
Click here to listen on iTunes!
Upload The Productive Artist e-book.
Allan McKay’s Facebook Fanpage.
Allan McKay’s YouTube Channel.