Machin-X: Digital Puppetry

This is a new demo reel for Argentinian digital puppeteer Mario Mey that shows off his digital characters performing en Español at various live events (his character Pinokio 3D was mentioned here back in 2010) . He creates and performs his "Marionetas Digitales" (digital puppet) characters using Blender 3D and PureData, a real-time graphical dataflow programming environment for audio, video, and graphics.

You can see Mario at work and get a look at his production process in this video, however it was recorded in Spanish.

This is an early test of Kinect and Blender working together using OSCeleton. This is a custom coded solution that's different than the NI Mate software that I mentioned in my previous post. As you can see, it's not quite production ready yet, but it's still very, very exciting to see Blender and Kinect working together. Full details can be found here.

Great work Machinima Studios!

NI Mate is software that imports Kinect motion capture data in real-time into 3D apps and they've announced that Blender is the first application that they're supporting. When I was working on the aborted Panda Puppet project several years ago I realized that one of the things Blender really needed in order to become a practical and effective tool for digital puppetry was a straight forward method of accepting motion capture data in real-time, so I think this is very exciting news and long overdue!

It should be really interesting to see what people start doing with this new capability in Blender. 

NI Mate is now available in open beta for both Windows and Mac OS X (Linux support is coming soon). You can register to participate in the beta and download all of the necessary software at www.ni-mate.com.

Via BlenderNation.

The latest version of Blender 3D was released recently and has a bunch of great new features. One of the most impressive is motion tracking, which enables you to reconstruct camera animation from video and film footage, and composite 3d rendered object into movie clips. The Blender community has been busy exploring the possibilities of this new feature and several people have been attempting to use it for rudimentary motion capture.

As you can see above and below, the results so far look pretty good!





If you're feeling ambitious and want to try this for yourself you can find a tutorial here.

I've always believed that Blender had the potential to become an ideal (and totally free) professional tool for creating digital puppetry. Although I don't believe that this works in real-time - I haven't had a chance to try it myself yet - this is a huge step in that direction and suggests that we can look forward to a lot more amazing things in the future.

Via BlenderNation.

It's been far too long since I've updated Machin-X, so to try to catch up here's a look at bunch of interesting things that have been happening in the past few months:

Carnegie Mellon University and Disney Research are working on a system for "motion capture on the go" that utilizes body mounted cameras, which opens up some intriguing technological possibilities (via Tech Crunch).

In Australia, Jason Saragih is working on an interesting system for doing facial puppetry. His work on facial expression recognition looks particularly impressive so far.

Check out the visual insanity that is the Unreal 3 graphics engine.

Using the Kinect for digital puppetry keeps getting easier.

Digital puppetry with Wiimotes, using the Unity 3D engine.

Blender continues to be expanded and refined. The latest stable version (and last of the 2.5 series) Blender 2.59 was recently released and a number of exciting new features are being planned for future releases, including better support for motion capture and an improved real-time animation system.

Finally, old puppetry meets new technology with the "Naked Puppet" digital sock puppet.

There seems to be a lot of interesting things brewing on the digital puppetry front these days. In fact, I'm going to go out on a limb and suggest that 2011 might be a breakthrough year for real-time animation.

I hope to get back to sharing some of the exciting things that are happening out there in the new year, but until then enjoy "Pinokio 3D" a digital puppet created using Blender that debuted at the 14th Annual Circus, Clowns and Street Shows Argentine Conference in Argentina. The video is in Spanish, but more info is available (in English) in this thread on the BlenderArtists Forum.

As you can see in this demo, Blender's Game Engine has come a long way in a short time (the video above was rendered in real-time, not frame-by-frame). Actually, the GE is capable of even more than what this video demonstrates; the latest version (2.49) was recently released and includes a bunch of cool features including video textures and real-time dome rendering as well as improved modifier support, speed, logic and physics.

As the GE gets better and better, more and more people seem to be interested in using Blender to create Machinima and digital puppetry. You can download the latest version of Blender from Blender.org.

The next release of Blender, version 2.5 is a major overhaul of the program's underlying code, some of which dates back to 1994(!). I'm especially excited about the proposed new animation system “Animato” and the new event system, which you can see demonstrated below:



Although I don't think the GE is the focus on the 2.5 release, you can bet that at least a few improvements are on the way. Experimental builds of Blender 2.5 can be downloaded from Graphicall, but they may not be stable and aren't necessarily suitable for production use yet.

Work continues with Panda Puppet slowly, but surely. I haven't been able to share much of what I have been doing lately because my work is focused on two client projects that I can't really blog about at this point, but I am starting up my Bear Town web series project again and plan to be using Panda Puppet heavily on that in the coming months.

What has me most excited right now though is that "Ashsid" - who wrote a lot of the original Blender scripts that much of Panda Puppet is based on - has been working on a cool new video texture plugin for Blender's game engine that allows you to combine real-time graphics with various other sources like video files, live video, rendered 3d scenes, etc. inside Blender.

This has all sorts of interesting applications, the coolest of which is tracking the movement of a camera and applying that movement to a Blender object. Here's a demo of it in action with Suzanne (the famous Blender monkey head) combined with live webcam video:



What excites me most about this is that it enables puppeteers working with monitors to have physical puppets and digital ones interact in real-time. That isn't a new idea of course, but what's great about this is that now anyone can download the software and try it themselves.

Nice work Ashsid!

This was just a quick experiment that I did yesterday...I took a head that was modeled and rigged by Calvin over at BlenderArtists.org and set it up so that it could be puppeteered right inside a standard Blender window (i.e. not using the game engine) with a joystick. It was quick to set-up and fairly easy to do, I may experiment a bit more with this.

A physics engine is a type of computer program used in video games and real-time animation to simulate different aspects of real-world physics like gravity, mass, velocity, and friction. A relatively new and fairly powerful physics engine called Bullet is being used in the latest version of Blender and to help promote its capabilities a Rube Goldberg Device contest was recently held with Blender users modeling elaborate machines using Bullet's Rigid Body physics.

You can see all of the entries here, but my favourite by far is the one above by Chris Plush, who's better known in Blender circles as "Blengine". When you watch this remember that the animation is generated in real-time. Don't believe me? If you have Blender on your computer you can download Chris' entry (link goes directly to a .blend file) and see it in real-time for yourself.

Now while this is real-time animation it's really more of a physics simulation than an example of digital puppetry. Although you can do some really basic things using Bullet's Ragdoll Physics, decent character control isn't yet possible with Bullet. It is on the developer's to-do list though and once it's implemented I think all sorts of new and interesting possibilities will open up.

Plans are afoot to add motion tracking capability to Blender. This is still in the earliest stages of development, but it looks like the idea is to add the ability to both match-move camera shots and track motion on a object that can be assigned to Blender controls. These were two huge features I really wanted for Panda Puppet (and dreaded the prospect of having to code on my own) and now it looks like they'll end up built right in to Blender.

How cool is that?