Copied from:
http://beatsportable.com/forum/viewtopic.php?f=4&t=1023&p=1346#p1346
(Keripo = Philip Peng)
=============================================================
Finished up writing the full report with instructions, etc. Includes all relevant documents, source code modifications, photos, and of course a video of the actual demo ; )
Report: 2011-05-09 ESE350 Final Project - Report.pdf
Video: Beats, Advanced Rhythm Game - 1.5.5b Kinect PoC Demo [Android]
Files: http://beatsportable.com/static/kinect/
Blog: Dance With Your Hands
For installation instructions, download all the files in the "install/" folder and skip down to "5. b)" in the report.
Happy hacking!
~Keripo
=============================================================
Tuesday, May 10, 2011
Monday, May 9, 2011
Demo!
Following the successful implementation of our project base goal, we proceeded to add in the multi-input feature, allowing the player to input more than one directional input at once.
On top of that, the slowdown issue was fixed and the game is essentially 100% playable now! Check out our demo video above!
 |
| Demo day setup |
 |
| Demo day setup |
 |
| Demo day setup |
Integrating Modded ofsample Functionalities with Beats
Fri 4/13/11
Integrating Beats with Kinect was much trickier than we originally thought. The main obstacle we faced was that the Kinect refused to become active when we booted Beats up. After many hours of trial and error, we finally managed to keep the Kinect active (evident from its green LED) continuously while running Beats.
The next step is to modify the Beats source code to respond appropriately when the Kinect detects the corresponding inputs. This was accomplished fairly painlessly with Phil modifying his own codes. Following another lengthy episode of debuggings and adjustments, we finally arrive at SUCCESS!
The Beats+Kinect ran at about 15-20 frames per second. We tried to narrow down what was causing the major slowdown but could not find the source as we were thoroughly exhausted by then. Due to lag, the player would have to raise their hand into the Kinect's detection zone about half a second prior to the arrows reaching to the bottom. However, the game was ultimately playable, and we were able to achieve 10+ combos with acceptable accuracy.
Integrating Beats with Kinect was much trickier than we originally thought. The main obstacle we faced was that the Kinect refused to become active when we booted Beats up. After many hours of trial and error, we finally managed to keep the Kinect active (evident from its green LED) continuously while running Beats.
The next step is to modify the Beats source code to respond appropriately when the Kinect detects the corresponding inputs. This was accomplished fairly painlessly with Phil modifying his own codes. Following another lengthy episode of debuggings and adjustments, we finally arrive at SUCCESS!
 |
| Success! |
The Beats+Kinect ran at about 15-20 frames per second. We tried to narrow down what was causing the major slowdown but could not find the source as we were thoroughly exhausted by then. Due to lag, the player would have to raise their hand into the Kinect's detection zone about half a second prior to the arrows reaching to the bottom. However, the game was ultimately playable, and we were able to achieve 10+ combos with acceptable accuracy.
Modifying ofsample
Thur 4/28/11
As mentioned previously, the package we downloaded included the ofsample source code. After, many trials of figuring out what does what in the sample code, we made the appropriate changes and compiled the code in Eclipse. The code was altered to have similar functionalities as the code we wrote for Kinect + Ubuntu.
We made a minor modification, changing the 4 corner + 1 middle input boxes into 4 input boxes laid out horizontally. This is because of the switch we made from the StepMania simulator on Ubuntu to Beats on Android.
Downloads:
Modified ofsample source codes: ofsample.zip
Modified ofsample compiled apk: ofsamplemod.apk
As mentioned previously, the package we downloaded included the ofsample source code. After, many trials of figuring out what does what in the sample code, we made the appropriate changes and compiled the code in Eclipse. The code was altered to have similar functionalities as the code we wrote for Kinect + Ubuntu.
We made a minor modification, changing the 4 corner + 1 middle input boxes into 4 input boxes laid out horizontally. This is because of the switch we made from the StepMania simulator on Ubuntu to Beats on Android.
 |
| Modifying ofsample in Eclipse |
 |
| Running openFrameworks Sample |
 |
| Running openFrameworks Sample |
Modified ofsample source codes: ofsample.zip
Modified ofsample compiled apk: ofsamplemod.apk
Saturday, April 30, 2011
Kinect + Android
Tue 4/26/11
After many researches, we found a guide from Japan (conveniently in English) that allowed us to work with Kinect on our Beagleboard running Android.
This guide used these open-source software:
OpenFrameworks
ofxKinect(Kinect for OpenFrameworks)
ofxAndroid(Android for OpenFrameworks)
Steps 3 and 4 in the guide were the necessary steps to achieve our goal.
Step 3. Change Android's configuration
we added the following line
Step 4. Upgrade Android's kernel
libusb MUST neet "USB FS". And Kinect MUST neet "High Speed
USB(USB2.0)".
Check the kernel option in .config
Upgrading the kernel involved recompiling the it and took quite a while for us to finish (upwards of 20-30 minutes).
Now that Kinect can be recognized through the USB on Beagleboard, we downloaded the Full "OpenFrameworks x kinect x Android" Development Environment for beagleboard provided by the guide as well. The extracted files included an openFrameworks sample that we compiled into an .apk file. Using the same procedures to install Beats on Android+Beagleboard in the previous post, the openFrameworks sample installed and ran smoothly.
After many researches, we found a guide from Japan (conveniently in English) that allowed us to work with Kinect on our Beagleboard running Android.
This guide used these open-source software:
OpenFrameworks
ofxKinect(Kinect for OpenFrameworks)
ofxAndroid(Android for OpenFrameworks)
Steps 3 and 4 in the guide were the necessary steps to achieve our goal.
Step 3. Change Android's configuration
we added the following line
mount usbfs none /proc/bus/usb -o devmode=0666into the init.rc file (after all file permissions and before services)
Step 4. Upgrade Android's kernel
libusb MUST neet "USB FS". And Kinect MUST neet "High Speed
USB(USB2.0)".
Check the kernel option in .config
CONFIG_USB_DEVICEFS=yCopy new kernel onto sd card
Upgrading the kernel involved recompiling the it and took quite a while for us to finish (upwards of 20-30 minutes).
Now that Kinect can be recognized through the USB on Beagleboard, we downloaded the Full "OpenFrameworks x kinect x Android" Development Environment for beagleboard provided by the guide as well. The extracted files included an openFrameworks sample that we compiled into an .apk file. Using the same procedures to install Beats on Android+Beagleboard in the previous post, the openFrameworks sample installed and ran smoothly.
At first glance, the sample looks promising with fairly good and stable frame rates. However, the new OpenFrameworks with ofxKinect addon would require us to learn how to code with the new library and reprogram the functionalities we achieved with Kinect+Ubuntu.
New SD card + Android Running Beats on Beagleboard
Sun 4/25/11
We procured a new micro SD card from RadioShack and repeated the same steps in our previous post to boot the Beableboard into Android. As a precaution, a fan kept the board cool at all times to hopefully avoid the same overheat problem that occurred before.
Recall the several network and installation issues that prevented us from getting Phil's Beats to work. This time, we managed to discover a hackish way to open and run the application's .apk file without the need for internet access on the Beagleboard (credits to Phil for finding this out). It involved three simple steps:
1. Placing the .apk package onto the SD card's 3rd partition. (This is done on a pc)
2. Insert SD card into Beagleboard and boot into Android.
Select web browser and enter
file:///sdcard//<appname>.apk
in the address bar.
3. Follow the on screen steps to complete installation and open the application.
We were delighted to see that the Beats game had a great performance and ran at the maximum 60 fps almost constantly!
Our next milestone will be getting Kinect to work with Android like it did with Ubuntu. However, this is likely to be more challenging given the relatively poor documentation on this topic.
We procured a new micro SD card from RadioShack and repeated the same steps in our previous post to boot the Beableboard into Android. As a precaution, a fan kept the board cool at all times to hopefully avoid the same overheat problem that occurred before.
Recall the several network and installation issues that prevented us from getting Phil's Beats to work. This time, we managed to discover a hackish way to open and run the application's .apk file without the need for internet access on the Beagleboard (credits to Phil for finding this out). It involved three simple steps:
1. Placing the .apk package onto the SD card's 3rd partition. (This is done on a pc)
2. Insert SD card into Beagleboard and boot into Android.
Select web browser and enter
file:///sdcard//<appname>.apk
in the address bar.
3. Follow the on screen steps to complete installation and open the application.
We were delighted to see that the Beats game had a great performance and ran at the maximum 60 fps almost constantly!
Our next milestone will be getting Kinect to work with Android like it did with Ubuntu. However, this is likely to be more challenging given the relatively poor documentation on this topic.
Beagleboard Not Fast Enough for Ubuntu
Sat 4/23/11
With Kinect working on Beagleboard running Ubuntu, we proceeded to download and installed Stepmania Simulator (Pump It Up game-mode). A screenshot can be seen below.
To our disappointment, the game runs at very low frame rate (less than 5 fps), so it is virtually unplayable.
This drove us to reconsider running Android again, especially after seeing the blazing speed of 2D and 3D graphics demos that came with the Texas Instruments' pre-built Android image.
With Kinect working on Beagleboard running Ubuntu, we proceeded to download and installed Stepmania Simulator (Pump It Up game-mode). A screenshot can be seen below.
 |
| Stepmania (SM-SSC) Running on Beagleboard with Ubuntu |
To our disappointment, the game runs at very low frame rate (less than 5 fps), so it is virtually unplayable.
This drove us to reconsider running Android again, especially after seeing the blazing speed of 2D and 3D graphics demos that came with the Texas Instruments' pre-built Android image.
Subscribe to:
Posts (Atom)