KINECTlogy (XBOX KINECT Hack)

[Deadstring67]

---

Controlling weapons with Kinect



It was only a matter of time before someone would figure out how to weaponize their Kinect. Hacker [Jonas Wagner] was fiddling with his Kinect one day and thought that it would be cool to launch missiles simply by gesturing. Not having any real missiles on hand, he settled for controlling a USB-powered foam missile launcher instead.

He mounted a webcam to the top of his rocket launcher to record video of his victims, and with a bit of Python along with the libfreenect ilbrary he was well on his way to world cubicle dominance. The Kinect waits for him to pull his hand out of its holster in dramatic fashion, monitoring his movements for tracking purposes. Once the launcher has been armed, the Kinect watches for [Jonas] to pull his hands out of frame before firing the rocket.

[Deadstring67]

Kinect hack makes April Fools’ prank a reality



Unless you have been hiding out in a cave for the last week or so, you have heard about this year’s April Fools’ joke from Google. Gmail Motion was purported to be an action-driven interface for Gmail, complete with goofy poses and gestures for completing everyday email tasks. Unfortunately it was all an elaborate joke and no gesture-based Gmail interface is forthcoming…at least not from Google.

The team over at the USC Institute for Creative Technologies have stepped up and made Google’s hoax a reality. You might remember these guys from their Kinect-based World of Warcraft interface which used body motions to emulate in-game keyboard actions. Using their Flexible Action and Articulated Skeleton Toolkit (FAAST), they developed a Kinect interface for Gmail which they have dubbed the Software Library Optimizing Obligatory Waving (SLOOW).

Their skeleton tracking software allows them to use all of the faux gestures Google dreamed up for controlling your inbox, however impractical they might be. We love a good April Fools’ joke, but we really enjoy when they become reality via some clever thinking.

[Deadstring67]

R/C car controlled by an iPad or Kinect



R/C cars can be tons of fun, but sometimes the fun runs out after awhile. [Gaurav] got bored of steering around his R/C car with its remote, so he built an interface that lets him control the car using two different motion-detecting devices.

He built an HTML5 application for his iPad, which allows him to steer the car around. As you can see in the video below, the application utilizes the iPad’s tilt sensor to activate the car’s motors and steering depending on where on the screen he has moved the guide marker.

The second steering method he devised uses his Kinect sensor to track his movements. His hand gestures are mapped to a set of virtual spaces similar to those which the iPad uses. By moving his hands through these areas, the Arduino triggers the car’s remote just as it does with the iPad.

The actual remote control interface is achieved by wiring the car’s remote to an Arduino via a handful of opto-isolators. The Arduino is also connected to his computer via the serial port, where it waits for commands to be sent. In the case of the iPad, a Python server waits for commands to be issued from the HTML5 application. The Kinect’s interface is slightly different, with a C# application monitoring his movements and sending the commands directly to the serial port.

[Deadstring67]

Real-time wireframe video effect overlay with Kinect



[Francois] over at 1024 Architecture has been working on a project we think you’ll be likely to see in a professional music video before too long. Using his Kinect sensor, ha has been tracking skeletal movements, adding special effects to the resulting wire frame with Quartz Composer. While this idea isn’t new, the next part is. He takes the QC tweaked video stream and then projects it back over the performer using MadMapper to match the video to the body movements, recording the resultant display.

The project started out with a few hiccups, including a noticeable delay between the body tracking and the display. It caused the performer to have to move more slowly than he would like, so things had to be tweaked. [Francois] first tested the latency between his computer and the projector by displaying a timecode representation on the screen as well as via the projector. He found the projector to have a latency of 1 frame at 60 fps, which wasn’t too bad. This led him to believe the culprit was his Kinect, and he was right. There was a 6 frame delay, so he locked the video output to 30 fps in hopes of cutting that delay in half.

Video:

[Deadstring67]

Clever hack tethers a Kinect sensor to the PS3



[Shantanu] has been hard at work writing code and experimenting with some preexisting Kinect software to get the sensor to talk to his PS3. The Kinect is hooked up to a PC, which captures all of his movements with OpenNI. Those movements are mapped to PS3 controls via NITE, a piece of middleware used for interpreting gestures into commands. All of the captured button presses are then relayed to the PS3 over a Bluetooth connection using DIYPS3Controller.

As you can see in the video below, the solution works pretty well for what should be considered pre-alpha code. He has been able to map several custom gestures to button presses, and the Kinect does an overall decent job tracking his limbs and translating their movements to on-screen actions. The actual in-game use is a bit rough at the moment, but aside from the infancy of the code, you have to remember that these games were never meant to be played with the Kinect.

[Deadstring67]

Giving “sight” to the visually impaired with Kinect



A helmet-mounted Kinect sensor is placed on the subject’s head and connected to a laptop, which is stored in the user’s backpack. The Kinect is interfaced using custom software that utilizes depth information to generate a virtual map of the environment. The computer sends information to an Arduino board, which then relays those signals to one of three waist-belt mounted LilyPad Arduinos. The LilyPads control three motors, which vibrate in order to alert the user to obstacles. The group even added voice notifications via specialized markers, allowing them to prompt the user to the presence of doors and other specific items of note.

[Deadstring67]

Kinect Controlled Tesla Coil



The London Hackspace crew was having a tough time getting their Kinect demos running at Makefair 2011. While at the pub they had the idea of combining forces with Brightarcs Tesla coils and produced The Evil Genius Simulator!

After getting the go ahead from Brightarcs and the input specs of the coils they came up with an application in Openframeworks which uses skeletal tracking data to determine hand position. The hand position is scaled between two manually set calibration bars . The scaled positions then speeds or slows down a 50Hz WAV file to produce the 50-200Hz sin wave required by each coil. It only took an hour but the results are brilliant, video after the jump.

[Deadstring67]

3D modeling out of thin air



[sonsofsol] has combined several open source software packages and a little electronics know-how to create one of the more useful Kinect hacks we have seen lately. His project enables him to manipulate 3D models in GEM simply by moving his hands about in front of his Kinect sensor. Using OpenNI and Ubuntu, all of his actions are tracked by the computer and translated into actions within the GEM 3D engine.

To make things easier on himself, he also constructed a pair of electronic gloves that interface with the system. Using an Arduino, the gloves send different complex commands to the 3D modeling software, just by touching different pairs of fingers together.

[Deadstring67]

Chilling drinks with your friends’ faces



3D printing of Kinect-mapped models seems to be all the rage lately. [Nirav] caught the bug and has developed software which allows him to join in the fun. Frustrated by the lack of documentation and source code for the Fabricate Yourself project, he set out to create his own open-source process for scanning people and objects to share with the hacking community.

His software allows you to aim the Kinect and capture a 3D scan of any object, after which you need to use MeshLab or similar software to turn the scan into a STL file for printing. He says that the process is a bit tedious at the moment, but he is working hard to condense it down into a single step.

While he can scan and print pretty much anything he wants, his ultimate goal is to create ice cube trays for his friends featuring molds of their faces. The project has a lot of promise, though we’re not sure about our friends crunching on our faces after finishing their drink.

[Deadstring67]

Kinect home theater control



[Harishankar] has posted a video on his blog demonstrating the ability to control devices using the Microsoft Kinect sensor via IR. While controlling devices with Kinect is nothing new, he is doing something a little different than you have seen before. The Kinect directly interfaces with his Mac Mini and tracks his movements via OpenNI. These movements are then compared to a list of predefined gestures, which have been mapped to specific IR functions for controlling his home theater.

Once the gestures have been acknowledged, they are then relayed from the Mac via a USB-UIRT to various home theater components. While there are not a lot of details fleshed out in the blog post, [Harishankar] says he will gladly forward his code to you if you request it via email.

Source:
code42tiger: Controlling TV and Set-top box with Kinect