How Motion Control Powers Virtual Production¶

▶ How Motion Control Powers Virtual Production — Watch on YouTube

Hey guys, we're here at BMP Studio in Hey guys, we're here at BMP Studio in Tampa and today we're going to make the Tampa and today we're going to make the Tampa and today we're going to make the robot work with their LED wall. Let's go robot work with their LED wall. Let's go robot work with their LED wall. Let's go plug it in. Hey guys, welcome back. Today we're Hey guys, welcome back. Today we're talking about integrating motion control talking about integrating motion control talking about integrating motion control with virtual production and more with virtual production and more with virtual production and more specifically ICVX, which stands for in specifically ICVX, which stands for in specifically ICVX, which stands for in camera visual effects. Now, I'm not camera visual effects. Now, I'm not camera visual effects. Now, I'm not going to go too deep into what virtual going to go too deep into what virtual going to go too deep into what virtual production or ICVF effects are, as production or ICVF effects are, as production or ICVF effects are, as there's already so many incredible there's already so many incredible there's already so many incredible YouTube videos out there, and I'm pretty YouTube videos out there, and I'm pretty YouTube videos out there, and I'm pretty sure you guys are more interested in sure you guys are more interested in sure you guys are more interested in learning about how Moco fits into the learning about how Moco fits into the learning about how Moco fits into the equation. So, we're going to stay equation. So, we're going to stay equation. So, we're going to stay focused on that. With ICVFX, the goal is focused on that. With ICVFX, the goal is focused on that. With ICVFX, the goal is to film real practical elements and to film real practical elements and to film real practical elements and computerenerated elements in real time computerenerated elements in real time computerenerated elements in real time with a real camera. The trick is to with a real camera. The trick is to with a real camera. The trick is to render and project the virtual render and project the virtual render and project the virtual environment onto a background or an LED environment onto a background or an LED environment onto a background or an LED wall. And in order for it to look and wall. And in order for it to look and wall. And in order for it to look and feel convincing, the perspectives must feel convincing, the perspectives must feel convincing, the perspectives must line up as close as possible. This means line up as close as possible. This means line up as close as possible. This means that we need to know both the camera's that we need to know both the camera's that we need to know both the camera's position and orientation as it relates position and orientation as it relates position and orientation as it relates to the surface that we're projecting the to the surface that we're projecting the to the surface that we're projecting the backgrounds onto. ICV effect studios use backgrounds onto. ICV effect studios use backgrounds onto. ICV effect studios use tracking systems to pinpoint the real tracking systems to pinpoint the real tracking systems to pinpoint the real camera's location and orientation. It camera's location and orientation. It camera's location and orientation. It sends that data to the virtual world so sends that data to the virtual world so sends that data to the virtual world so that the background can be rendered and that the background can be rendered and that the background can be rendered and projected from the correct perspective. projected from the correct perspective. projected from the correct perspective. The only problem with this is that there The only problem with this is that there The only problem with this is that there will always be inherent latency as the will always be inherent latency as the will always be inherent latency as the tracking system can't predict the future tracking system can't predict the future tracking system can't predict the future and it doesn't know where you're and it doesn't know where you're and it doesn't know where you're planning to move the camera. It can only planning to move the camera. It can only planning to move the camera. It can only tell the computer where the camera is tell the computer where the camera is tell the computer where the camera is after it locates it. Moco solves this after it locates it. Moco solves this after it locates it. Moco solves this problem because it already knows where problem because it already knows where problem because it already knows where the camera is and where it will be since the camera is and where it will be since the camera is and where it will be since it's literally just running a program it's literally just running a program it's literally just running a program that you get to design. Uh since I'm that you get to design. Uh since I'm that you get to design. Uh since I'm using Push Moco and the software doesn't using Push Moco and the software doesn't using Push Moco and the software doesn't know where I'm going to be moving the know where I'm going to be moving the know where I'm going to be moving the camera because it can't predict the camera because it can't predict the camera because it can't predict the future and it doesn't know what I'm future and it doesn't know what I'm future and it doesn't know what I'm going to do. you're going to see there's going to do. you're going to see there's going to do. you're going to see there's a little bit of a little bit of a little bit of latency. So, I move the camera, the latency. So, I move the camera, the latency. So, I move the camera, the virtual camera moves behind this one. virtual camera moves behind this one. virtual camera moves behind this one. Uh, this is what happens with the Uh, this is what happens with the Uh, this is what happens with the majority of tracking solutions. But once majority of tracking solutions. But once majority of tracking solutions. But once you've program the move and you're you've program the move and you're you've program the move and you're running the move, you can send a delay running the move, you can send a delay running the move, you can send a delay to the physical robot uh to account for to the physical robot uh to account for to the physical robot uh to account for any latency and make sure that they're any latency and make sure that they're any latency and make sure that they're nice and dialed. A fun way to dial in nice and dialed. A fun way to dial in nice and dialed. A fun way to dial in your latency is to program a fast pan your latency is to program a fast pan your latency is to program a fast pan and observe which camera moves first. If and observe which camera moves first. If and observe which camera moves first. If the real camera moves first, add a delay the real camera moves first, add a delay the real camera moves first, add a delay to the robot. If the virtual camera to the robot. If the virtual camera to the robot. If the virtual camera moves first, reduce the delay to the moves first, reduce the delay to the moves first, reduce the delay to the robot and repeat this process until you robot and repeat this process until you robot and repeat this process until you have everything dialed. So, I'll just have everything dialed. So, I'll just have everything dialed. So, I'll just open up my miscellaneous setups. We'll open up my miscellaneous setups. We'll open up my miscellaneous setups. We'll start with 15 ticks just to show you start with 15 ticks just to show you start with 15 ticks just to show you what it looks like with an extreme what it looks like with an extreme what it looks like with an extreme delay. We'll run the delay. We'll run the delay. We'll run the move. And as you can see, the virtual move. And as you can see, the virtual move. And as you can see, the virtual camera moved first, completed the full camera moved first, completed the full camera moved first, completed the full move, and then the real camera caught up move, and then the real camera caught up move, and then the real camera caught up to it. So, we'll just reduce the delay. to it. So, we'll just reduce the delay. to it. So, we'll just reduce the delay. We'll go back to two ticks, which seems We'll go back to two ticks, which seems We'll go back to two ticks, which seems to be pretty good for this system. to be pretty good for this system. to be pretty good for this system. We'll back on the move. And as you can We'll back on the move. And as you can We'll back on the move. And as you can see, it's keeping up nicely and they're see, it's keeping up nicely and they're see, it's keeping up nicely and they're pretty dialed. So that's it. At this pretty dialed. So that's it. At this pretty dialed. So that's it. At this point, you might be wondering how to point, you might be wondering how to point, you might be wondering how to line up the perspective of the real line up the perspective of the real line up the perspective of the real camera and the virtual camera. Flare camera and the virtual camera. Flare camera and the virtual camera. Flare knows where the camera is in relation to knows where the camera is in relation to knows where the camera is in relation to its own zero point, but it has no idea its own zero point, but it has no idea its own zero point, but it has no idea where the camera is in relation to the where the camera is in relation to the where the camera is in relation to the stage zero point. We call this step stage zero point. We call this step stage zero point. We call this step making them relative. The easiest way making them relative. The easiest way making them relative. The easiest way that I found to make the stage relative that I found to make the stage relative that I found to make the stage relative with the robot is to find something that with the robot is to find something that with the robot is to find something that has nice perspective lines like this and has nice perspective lines like this and has nice perspective lines like this and line it up to where it's touching the line it up to where it's touching the line it up to where it's touching the LED wall in the real world and then LED wall in the real world and then LED wall in the real world and then extend it with a similar shape behind extend it with a similar shape behind extend it with a similar shape behind the mesh inside of Unreal Engine into the mesh inside of Unreal Engine into the mesh inside of Unreal Engine into the virtual world. Pilot the camera the virtual world. Pilot the camera the virtual world. Pilot the camera around to observe the object from many around to observe the object from many around to observe the object from many different angles and move the virtual different angles and move the virtual different angles and move the virtual camera around until the perspectives camera around until the perspectives camera around until the perspectives line up properly. It gets even better. line up properly. It gets even better. line up properly. It gets even better. Since the camera path is already Since the camera path is already Since the camera path is already programmed, it can be sent to post teams programmed, it can be sent to post teams programmed, it can be sent to post teams as well if there's any cleanup work that as well if there's any cleanup work that as well if there's any cleanup work that needs to be done. If you have an needs to be done. If you have an needs to be done. If you have an opportunity to record the camera data on opportunity to record the camera data on opportunity to record the camera data on set, it's definitely a good idea to do set, it's definitely a good idea to do set, it's definitely a good idea to do that as the camera will already be that as the camera will already be that as the camera will already be relative to the stage in the scene. So, relative to the stage in the scene. So, relative to the stage in the scene. So, I would highly recommend taking the I would highly recommend taking the I would highly recommend taking the extra time to record and make any notes extra time to record and make any notes extra time to record and make any notes if you're able to. Don't stress though if you're able to. Don't stress though if you're able to. Don't stress though because any project file can be opened because any project file can be opened because any project file can be opened in Flare at any point in time and all in Flare at any point in time and all in Flare at any point in time and all the camera data can be exported in a the camera data can be exported in a the camera data can be exported in a multitude of different ways for various multitude of different ways for various multitude of different ways for various 3D programs. The main takeaway here is 3D programs. The main takeaway here is 3D programs. The main takeaway here is that motion control robotics and ICVF that motion control robotics and ICVF that motion control robotics and ICVF effect studios were basically made for effect studios were basically made for effect studios were basically made for each other. This combination of each other. This combination of each other. This combination of technology and innovation really takes technology and innovation really takes technology and innovation really takes things to the next level. And honestly, things to the next level. And honestly, things to the next level. And honestly, it's just super fun. Well, that's it for it's just super fun. Well, that's it for it's just super fun. Well, that's it for this episode. Thanks again for tuning this episode. Thanks again for tuning this episode. Thanks again for tuning into MRMC Academy. Don't forget to like into MRMC Academy. Don't forget to like into MRMC Academy. Don't forget to like and subscribe and feel free to leave any and subscribe and feel free to leave any and subscribe and feel free to leave any questions or comments down below and hit questions or comments down below and hit questions or comments down below and hit me up on Instagram if you'd like to me up on Instagram if you'd like to me up on Instagram if you'd like to connect. Until next time, peace. connect. Until next time, peace. connect. Until next time, peace. [Music] Heat. Heat. [Music]