Madlib Karaoke

Microphone Recognittion

2009-02-28T14:52:00.001-08:00

The key components to user interaction in the M.K. system is to know where the microphone is being held.

By knowing X/Y position of the microphone, we can assume a very general position of the user. Essentially user body center is < 1 meter from microphone. There are better ways to get user position, but this one is the easiest, since we are leveraging the the technology that the user is ALREADY holding.

The original concept was to build a band of IR LEDs that circle the microphone. A problem with this is knowing how close the user is to the camera. The hack solution is to provide a narrow stage that limits where the user can position themselves. A better solution is for the microphone system to let the computer know its exact position.

For this reason, ARToolkit was considered. ARToolkit faducial can be rigged as a backlight. This would give X,Y,Z and tilt position. Very powerful for user interaction. Unfortunately, ARToolkit is built for Augmented Reality,and seems to assume the user and camera are in same position - heads up or see through displays. For a 7 inch faducial we get 50 feet distance from faducial to camera. Assuming a 1 foot covering is largest we can give user, then we can expect no more than a 100 foot distance from camera, or 8 feet.

In looking at positioning under this system, the ideal is to place camera and projector as close to one another as possible. This model was suggested by Zach Lieberman, whose work with Open Framework makes him an expert - for sure. It seems to make sense, since you are minimizing skew between projection and vision systems. Problem is that projectors typically require 15 feet for proper projection throw. This ofcourse depends on size of the projection. Were camera and projector next to each other, then faducial would be ... about 2 feet big! This would be cumbersome and a huge obstruction.

So for now, back to drawing board.

motion tracking in flash

2009-01-16T13:29:00.000-08:00

This video shows a pretty cool implementation of motion tracking using flash.

Tracking Multiple Objects Using a Webcam from chris teso on Vimeo.

Full blog post here.

DEERTANK

2009-01-16T12:56:00.001-08:00

one last one... testing some other body types... Let me know if im getting to crazy.

i give you lobster crotch

2009-01-16T12:24:00.000-08:00

ok heres one more before the weekend. IM experimenting with a color system now also.. like the body parts the coloration should be generated at random like the parts.

MR TEEEEEEEEEEEETH

2009-01-14T12:20:00.001-08:00

Im trying a new body type here and seeing if you guys can code in like an IF THEN statement. LIke IF the body is this THEN stop adding parts... I present MR TEEEEEEEEETH.

TITFISH

2009-01-08T08:34:00.001-08:00

Here is the my latest effort TITFISH. I am finding more and more that the way to have these things work (multiple character generation) is to remove most of the joints. That way placement is easier to asses.

Playing with AS3

2009-01-06T16:32:00.000-08:00

One of the first things I ran into was the lack on an onEnterFrame. It seems that event listeners are the way to go.

Here's what I did today. There is no analysis yet, but it tracks the x and y locations of the mouse for a period of thirty seconds (when running at 30 frames/second).

I just wanted to check that I am using the best approximation of an onEnterFrame function.

----------
var xLocations:Array=new Array();
var yLocations:Array=new Array();
var testFreq=3;
var counter=0;

//300 at 10 sample per second is 30 seconds
for (var i=0; i<300; i++) {
xLocations.push(0);
yLocations.push(0);
}

addEventListener(Event.ENTER_FRAME,TrackMouse);

function TrackMouse(event:Event) {
counter++;
if (counter>=testFreq) {
counter=0;
xLocations.push(mouseX);
xLocations.splice(0,1);
yLocations.push(mouseY);
yLocations.splice(0,1);
}
}
-------------

-Andy

Capturing CD+G

2009-01-06T14:29:00.001-08:00

I have access to over 3000 karaoke songs on the typical format, CD+G, G standing for graphics. There is an excellent explanation of this data system here.

It appears that the words are ACTUALLY a video that is compressed and hidden in the available stream on regular CDs. This means that for me to translate the files into the madlib karaoke format, I will need to take the ripped video file, place it in a video editor, and get the time points for each new line.

It would be great to find some video analysis software that does video-to-text, but I don't know of any that exists :(

motion library

2009-01-06T14:22:00.000-08:00

Andy is working on creating a motion library that translates mouse positions over time into specific motions.

This library will be activated by data from the mouse for now, but will soon be received from IR coming from the microphone. We will be using XML Sockets in flash to communicate between the IR recognition and Flash

Initial Madlib Karacters

2008-12-19T09:17:00.000-08:00

Johnny's working on creating characters for Karaoke that can be randomly generated from different parts.

This is a link to the character file:
http://www.davidjimison.info/workinprogress/madlib/charcters.zip

Madlib Karaoke

2008-12-17T12:22:00.000-08:00

The current version of madlib karaoke uses a series of different files to enable the madlibs to occur.

http://www.davidjimison.info/workinprogress/madlilbKaraoke_01.zip

karaTimer.fla - this file is the master file, provides the menu, and then loads the appropriate lyrics and such
songs.as - this contains all necessary lyrics and timepoints
*.swf - these are the flash files which play in the background during the song currently they are just launched to play
*.mp3 these files are the songs which play in conjunction with the visuals and lyrics

ANDY - Open up the various files and take a look around. See if you have any questions regarding how everything is currently running. Things can definitely be made more efficient in terms of how the arrays and data is being handled. However for now, once you have finished getting a handle of what is going on in the code, try to substitute one of the animations with the one that you created with the mousetracker.

Spinning bird

2008-12-14T18:52:00.000-08:00

Here's the new version. The bird now flies around the mouse, which I replaced with a simple graphic so that the bird could go in front of it.

All of the instances of _ymouse or _xmouse have been replaced by variables that are checked every frame to be equal to the mouse location.

Also, while playing with xScale, I made this, which is fairy simple, but I liked what it did.

I've been using my own hosting, but is there anywhere else that you would prefer me to be putting these?

And I can email you the .fla if you would like it.

Mouse Follower

2008-12-10T20:06:00.000-08:00

http://www.andymasteroffish.com/files/Eyebeam3.swf

A flash animation with three characters that follow the mouse
-The walking lady slowly follows the mouse along the ground. Once she is in the same place, she looks up, however, if the mouse moves, it takes her a little while to figure out what happened. The likelihood of her realizing that the mouse has moved increases with each frame that she spends confused.

-The dog is excited about getting to the mouse. The further away he is, the faster he'll move. Once he gets there he sits down, but jumps up occasionally.

-The bird flies toward the mouse in the air and rotates to face the mouse. He's the only one that work on both the x and y plane.

Creating Beta Version

2008-12-10T19:26:00.000-08:00

Today we decided upon a strategy for creating the next version of Madlib Karaoke. This version will feature the current ability to replace words in Karaoke songs, but also enable interaction through movement of the microphone.

New to the team, Andy Wallace, will be developing a symbol library of movements, allowing particular movements from the mic to trigger specific interactions.

The plan is as follows:

STAGE 1 - December 20th

Create a 2d sprite in flash have it move on x and y axis to follow mouse
Add parameter to tweak speed that the sprite moves to the mouse
Replace mouse data with x/y data from XML sockets from text file
Use java app to send randomized x/y data to flash via XML sockets

STAGE 2 - January 20th

Develop a system to monitor x/y input over time, allowing detection of particular shapes. This will require some abstraction of the data.
Successfully test this with mouse as the input
Modify Java App so that it sends both randomized data and particular shapes

STAGE 3 - Feburary 1st

Use Open Frameworks to receive x/y position of mic with IR band
Send via XML sockets to Flash