I actually got everything working just in time for the big performance. Yesterday they used my project on stage, live. This is a small piece of evidence.



And here are some pictures.

Bringing fire into WPF isn't that simple. You need to make code witch colours every pixel separately. But for every problem there is a solution: Microsoft's XNA. This is a set of tool created to make video games and I came across this video.




His code is found here. Ill modify his code to let my project work.
Copyright John O Brien (his twitter)

The webcamtexture in Unity isn't going to work.
The frame rate of the webcamtexture is to low (crappy when moving to fast)
Cant get the skeleton to track properly on the real video feed, because of the static hip joint. We found something different: WPF (graphic subsystem of .Net framework) and the Kinect Toolkit from Coding4fun. This uses the 1.0 SDK witch gives better detection.

Got the detection working, we will work on the fire later.

(sorry for the crappy lighting, and yes my cat is overweight :D)

Found out how to get video feed into unity using webcamtexture.

Had some bad news...
The band would like to see the fire on the real girl, something like this.

Just gave the presentation, overall feedback: your project looks awesome!
Here is the powerpoint (in dutch).

I have to give a second presentation the 2e march, to see my progress. I'll show them the little clips I posted here before. They asked me to review the feedback from before:

Less informal language use: did this on the second document I had to give them.

How to test? Well you can drag points while running the unity program.

I just drag the joints were I want them. No need to actually stand up :D.

The project was shown to the band, they liked it! The activation was perfect, no mixed activations or anything. They liked the fire animation. BUT they thought the stick figure had to be better. So I experimented with this, a free 3D (female) character pack. This was the 3D model work of Mixamo, much better then mine. This is the start, still a bit buggy (watch the accordion centre).

Got all the detections working.



I am doing this with "pose" recognition. For example, the first movement is this

What we do for test is the following:

(OtherHand = left hand, ActivationHand = right hand)

(YPositionOtherHand < YPositionHip)
        &&
        (YPositionActivationHand > YPositionHead)

We get the Y coordinates from the required joints en do this test. If true, a timer starts running, if you hold the pose for X seconds, fire animation will start.

Exams are done, I restarted the project. And the detection of moves is going pretty good. Having one problem: this move is hard to test :D.

The 1.0 SDK is out, but this doesn't work with my Unity project with uses the beta. No problem, we will keep going at it with the beta version.

The kinect SDK v1.0 is out.

Going to let this project rest for a while, going to start learning for my exams in January.

I just received this video, these are the moves that have to be detectable. I'll know what to do after the exams.

Just did my presentation here is the powerpoint (in dutch). They had the following remarks:

- Don’t use informal language (This refers to a document I had to give, not very important here).
- How are you going to test this? (actual tests)

Did a little 3D modeling this is the result :p.

The head looks fine by me, the cloth = crap. Will try to find a solution for this.
Problems I faced with this cloth:

Somtimes the cloth just stops moving.

Solution: the cloth has something called: random acceleration. This makes sure the cloth is always moving (a little).

The cloth moves slow

There are too much calculation being done.
Solution: make the cloth out of less polygons.

I know for which band I'm going to do the project:
Mental Nomad. Here is a picture of there mascot, they would like to see him instead of "Rainbowman" (the stick figure with the rainbow color in previous post). This is going to be hard I don't know nothing about 3D modeling.

I have to give a presentation on 22 nov. Made this screen capturing for it.

What I did was this:

Found unity electricity.
Made a script, so that I can make game object follow one of my joints.
Voila!

(For screen cap I used Hypercam, it's free. There probably are better program's out there, post them here in the comments)

PS: code will come online soon!

Just found this site. They give a step by step explanation on how to get the windows SDK working with Unity.

(UPDATE (03/03/2012): This site has been updated to a "new" version of the kinect wrapper, here is a link to the old project.)

You can still do the steps they talk about, in the old project. You can also find more info about this on the site of Jeremy, specifically here.

Downloaded everything, this is me in my room, testing out the Kinect SDK.

Just found out who my "assistant" is going to be. (Every bachelor project is given by an assistant from the VUB) Lode Hoste, he is the "Kinect guy" at the VUB and has given me some info:

- Download the Kinect SDK (beta)
- Download the 3D engine Unity
- Combine the two

Project assignment

Just got the good news, I got the "augmented reality during live music performance" project.

I could post the project proposal, but it would only be visible to VUB students.

In short: I need to create some sort of "animation" live on a stage, using the motion sensor Kinect. They gave me a link to this video, as an example.