Unity3D And Microsoft Kinect? Hell Yeah!

January 28, 2011 By Gabriele Maidecchi

It all started with an email that I – naturally – ignored from my CEO, saying he was fiddling with unofficial MacOS X drivers for the Kinect ’cause he saw some post around and he was all hyped up.

Of course after a few minutes there was some photo on his Facebook page about him going to buy a Kinect (before even having the drivers up and running, I add).

I didn’t give much importance to it until I came to work the next day to find our CTO dancing and jumping like he was on drugs.

Seeing a Kinect pointed directly to him made me realize he hadn’t just lost his mind.

So where’s Unity3D in all this?

Take a look at the video and tell me what do you think:

Of course the drivers-fiddling doesn’t make it a mainstream solution, but there are many uses for something like this, from motion capture to museums/installations/public exhibitions and whatnot. And drivers are getting better and better, and let’s not forget, it’s all completely legal (hey I wouldn’t be posting this otherwise).

Filed Under: Research, Tech Tagged With: 3d, animation, dancing, gaming, kinect, microsoft, mocap, modeling, motion capture, party game, social games, technology, unity, videogames, vidoegaming

If you liked this post, why don't you Subscribe to this blog?

Comments

Will Goldstone says:

January 29, 2011 at 00:18

Guys this is totally awesome, would love to know more about what you’ve done to achieve this! thanks for all this inspirational work
Will Goldstone recently posted..Unity3DWorkcom Job Portal Launched
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  January 31, 2011 at 10:38
  
  Francesco wrote something about it in a comment below, but we’ll probably write a more technical post later on, for now we’re gonna study it a bit more
Maria Pavel from CNA Training says:

January 29, 2011 at 21:02

Hi Gabriele,

I just can’t live without modern technology, I love the kinect. Can’t wait to play something through kinect. It’s way too awesome, and the video totally rocks. Thanks for sharing this great article and awesome video!

Best regards,

Maria
Maria Pavel@CNA Training recently posted..CNA Online Training And Certification
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  January 31, 2011 at 10:39
  
  This “hack” surely cannot be mainstream, but it can have its uses, we’re sure of that
Amir says:

January 30, 2011 at 19:09

Very Fun The post he was all hyped about was probably mine. The Unity wrapper code for OpenNI / Kinect, some videos and some example projects are available on my blog and git:

http://blog.tinkerheavy.com
https://github.com/tinkerer/UnityWrapper
Amir recently posted..Kinect Poi
- Francesco Marcantoni says:
  
  January 31, 2011 at 10:33
  
  Hi Amir!..
  
  Yes! you’re right! We found your post inside Unity3D Forum and we started learning it..
  
  We took your Unitywrapper sourcecode for the DLL and we started to write the code from the ground to get the orientation for the bones to attach it to the avatar!
  
  Now we’ll start to understand how to use it.
  - Amir says:
    
    January 31, 2011 at 10:51
    
    i frigin lovit! I’m releasing a new video right now where i use a torch and first-person view to light other stuff on fire:
    http://www.youtube.com/watch?v=FdI8Lcxb1pU
    Amir recently posted..Kinect Poi
    - Gabriele Maidecchi
      Twitter: maidoesimple
      says:
      
      February 1, 2011 at 15:25
      
      Oh that’s pretty nice, we are already at work to release something else, probably Friday if time allows, stay tuned
Ali Mujtaba says:

January 31, 2011 at 01:45

I think a kinect MMORPG based game will be pretty interesting and probably a big hit.
Ali Mujtaba recently posted..How To Treat And Manage Tinnitus
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  January 31, 2011 at 10:50
  
  I am not really sure about that, MMO players like to play’n'relax, opposed to run, move etc
  I saw the video of Kinect used to control a Warcraft character, it was cool as a proof of concept but nothing I could see myself using in the long run
John from Beginner Guitar Lessons says:

January 31, 2011 at 21:53

Hey Gabriele,

That is some pretty cool stuff in the video. I think there are many different things you can do with the Kinect that most people just don’t know yet. I haven’t played any games using the Kinect yet, but after watching your boss in this video I think it will be fun to see all the different things I can do with it. Thanks for sharing as always Gabriele.
John@Beginner Guitar Lessons recently posted..How to Play Electric Guitar Songs Easily – The Simple Tips You Don’t Know About
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 1, 2011 at 15:26
  
  Yes we are already thinking about some practical uses for it, with Unity environments backing it up of course, I am glad you liked it.
  By the way, that one in the video wasn’t my CEO but my CTO, Francesco Gallorini, it wasn’t much clear from my post, sorry about that
felipe says:

February 1, 2011 at 18:43

hi, i want to know what price is the application that http://player.vimeo.com/video/19299107 samples and if you can do something for us and seemed to opensim and second life
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 2, 2011 at 10:09
  
  The system runs on custom-made graphics, made in Unity3D, I am afraid I see it kinda hard to make it interact with a Second Life based solution.
  If what you are interested in is something custom-made, we can talk about it, if it’s just Second Life, I am afraid there’s not much at the moment we can do.
  - felipe says:
    
    February 2, 2011 at 16:20
    
    can make this application, I have connected the Kinect in win 7 of 64 bit with OPENNI
    
    tnnks
  - felipe says:
    
    February 5, 2011 at 03:23
    
    hello, I can download it somewhere
    - Gabriele Maidecchi
      Twitter: maidoesimple
      says:
      
      February 7, 2011 at 15:37
      
      I am sorry but it’s not up for download
      - deewahyu says:
        
        February 9, 2011 at 05:11
        
        if i want copy your code, how much the license prize? thanks.
Adrienne says:

February 2, 2011 at 19:39

Hey Gabriele,

Wow, that’s so cool. I’m totally ignorant when it comes to that type of technology but I can still admire those who know how to use it. I’m really impressed and found your video just down right neat.

Thanks for sharing.

Adrienne
Adrienne recently posted..100 Blog Comment Challenge
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 4, 2011 at 11:46
  
  Thank you Adrienne, believe me it was also fun to experiment with it, I can say we’ve been doing progresses – even though this is for now just a side-project – and we’ll be releasing a new video soon, thanks for your support!
nazimwarriach from YouLove Coupons
Twitter: PakistanHotline
says:

February 3, 2011 at 13:07

Its fun for me to watch a video in which your CTO is dancing. The video stuff is pretty cool. I think there are many different things can be done with the Microsoft Kinect.
Nice to read about this latest technology? I will definitely try it with my computer.
nazimwarriach@YouLove Coupons recently posted..iContact Coupon Code
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 4, 2011 at 11:47
  
  During the week we’ve been testing out new things, even if we didn’t have time to actually shoot a video as well. But we’ll do it soon, just stay tuned, I’m planning to release something next week.
Charles Baratta
Twitter: huntingtonlive
says:

February 4, 2011 at 22:57

This is simply Cool. The synchronization is also perfect.
I want to know more about this stuff.. or more like how to work with it.
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 7, 2011 at 15:38
  
  I am sure that once we have it more under control we’ll be showing examples of practical uses, for now I think we’re gonna release a new video this week, if we manage to find the time, sadly it’s not experiments’n'fun all the time hehe
marc
Twitter: kinect_hacks
says:

February 12, 2011 at 17:27

Great hack i can see a bright future for this kind of usage, you should consider visiting kinect hacks to see more awesome projects, some are great.
btw nice explosion
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 14, 2011 at 16:45
  
  Thanks for the link, I’ll forward it to our tech people, glad you liked our explosion
marc
Twitter: kinect_hacks
says:

February 12, 2011 at 17:28

Is it just me or … lol i cant see any of my posts…….
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  February 14, 2011 at 16:46
  
  I see one post was blocked by Akismet ’cause of the malformed link (I presume), but the others went through no problem
  - marc
    Twitter: kinect_hacks
    says:
    
    February 15, 2011 at 23:05
    
    yes I see it, my mistake thanks.
Nick from Blackberry application developer says:

March 8, 2011 at 10:07

Modern technology makes 3d modelling so real and actually on-line one. Love it! And your CTO is a truly artistic guy!
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  March 8, 2011 at 18:50
  
  I will be sure to pass along the compliment, glad you liked it.
Saif
Twitter: stereografix
says:

May 22, 2011 at 05:34

Awesome work with the Unity Integration – Is there a chance to collaborate? My company specialises in stereoscopic 3D and I am working on building some demos using Unity with a lenticular 9-view auto-stereoscopic (glasses-free) 3D TV. If you are interested drop me an email!
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  May 22, 2011 at 15:33
  
  I will show your comment and website to my tech team Saif, and will be in touch with you in any case in the coming week. Thanks for your comment and glad you liked out experiment.
hanen says:

July 21, 2011 at 00:50

hello,
i am starter for using kinect , my problem is that the kinect show only the highest aprt of body the movement of waist is fixed can you help me
thanks
- Gabriele Maidecchi
  Twitter: maidoesimple
  says:
  
  July 22, 2011 at 18:15
  
  I believe you just stand too close to it. Take some steps behind and it’ll recognize legs as well. It’s a problem we had too ’cause our desks were too close to each other.

Trackbacks

Tweets that mention Unity3D And Microsoft Kinect? Hell Yeah! | Esimple Studios Blog -- Topsy.com says:

January 29, 2011 at 00:35

[...] This post was mentioned on Twitter by Will Goldstone, Wes McDermott, Maurizio Furno, Thomas Alex. Semple, Esimple Studios and others. Esimple Studios said: Unity3D And Microsoft Kinect? Hell Yeah! http://bit.ly/edWSbH [...]
Unity3D And Microsoft Kinect | ARCH Virtual says:

January 29, 2011 at 00:55

[...] and Realtime Motion Capture 28 January 2011 No CommentThis is a pretty impressive demo from esimple showing Microsoft Kinect being used to control an avatar in Unity3D. This hack, and the eventual [...]
Learn Unity 3D Game Engine » Blog Archive » Kinect and Unity esimplestudios demo says:

January 31, 2011 at 12:48

[...] View blog post [...]
LEGO Hero William Furno 7167 - American doll collectors Barbie dolls clothes & baby gift ideas says:

February 2, 2011 at 15:55

[...] Unity3D And Microsoft Kinect? Hell Yeah! | Esimple Studios Blog [...]
6 Points To Make People Totally Interested In What You Do | Esimple Studios Blog says:

February 4, 2011 at 17:45

[...] last week’s post about our Kinect experiment with Unity3D it’s been a constant buzz about that video, and how we did that, and what are our plans for [...]
A Guy, a Zombie And a Dog Using Unity3D And Microsoft Kinect | Esimple Studios Blog says:

February 8, 2011 at 18:38

[...] another Kinect video this week, we just needed a lil more time to work on stuff. If you remember my previous post on the matter, I mentioned this is just a side-project whose potential we are actually still exploring. It does [...]
Augmented Reality Experiments With Unity3D | Esimple Studios Blog says:

February 18, 2011 at 17:13

[...] among the ones we are currently developing – lining up next to our anaglyph package and the Kinect experiments – is the Augmented Reality tests we have in progress and which allow us to put [...]
Unity3D and Microsoft Kinect at GDC says:

February 19, 2011 at 01:31

[...] technologies and connecting them. That is what Esimple Studios did a few weeks back and they posted the video on their blog of a Unity3D scene controlled with a Kinect device. A really nice example! Unity3D and Kinect [...]
What Can You Learn From Your Blog's Niche? | Esimple Studios Blog says:

March 1, 2011 at 17:45

[...] user-base (duh) with content that is kinda unique (I haven’t seen many posts out there about Kinect or augmented reality used with Unity3D models and [...]
The New Microsoft Kinect | Game Skill says:

October 26, 2011 at 02:46

[...] Nintendo started it and now Sony and Microsoft are trying to bank off from their success. Read on fo…ty to make your real life actions control your player on the screen; and now Microsoft has launched its attempt to expand on the success of the Nintendo Wii with the Kinect and controller free gaming. The Kinect, which was launched on Nov. 20, is anticipated to sell close to five million copies in the first quarter of the year 2011 – but is it really better than the Wii? I could make this really easy, rarely ever is anything better than the original. But when you break it down, the Kinect is the first of its kind. No other console is capable of controller free gaming, and though the use of the Xbox Avatars may make it seem like you are playing a game on the Wii with your friendly Mii’s, it is quickly realized that you are not. The Kinect is not like anything you have played before. While playing your game your character mimics every movement you make. Waiting for your opposition to serve the ball in volleyball while playing Kinect Sports, or celebrating a strike while playing bowling, your player still moves around and replicates your every move. Not only is game play with the Kinect one of a kind, but so are the unique features it comes with. The camera and built in microphone allows you to video chat with other friends on XBOX Live; if your friend doesn’t have a Kinect you can use the microphone and chat with your friend without wearing the embarrassing headset that they are still forced to wear. Lastly, and perhaps one of the coolest features, you can talk to your Xbox and it will listen to your commands. Just simply say “Xbox” and then whatever you want it to do – and it will do it. There is no explaining how playing the Kinect truly feels, you’ll just have to try it yourself to truly understand. Just remember not to do anything embarrassing because your character will certainly do the same embarrassing movement – and oh yeah, the Kinect takes pictures as you play and will remind you of your embarrassing gestures after your game is complete. [...]