Final Project- Flappy Shadow

Final Project

by Tong W,  Kai H

Concept & Reference

    The idea of this  project comes from our childhood memory- shadow puppet. Before we grown up and become more sophisticated to this world , we usually play with and connection meanings and objects with a more creative way. A stick can be a sword of a warrior. A shadow can be a monster hiding in the room. But most of us lose this ability of imagination. In this project, we want to waken people’s early  habit of imagination and revivify the shadow into our living environment.

    Our reference is “The Treachery of Sanctuary” from Chris Milk’s. The installation art takes viewers through three stages of flight through the use of Kinect controllers and infrared sensors. Viewers could come closer to the screen and start waving their hands. The “shadow” of their arms are transformed into two giant wings and move as viewers waving their arms and walk closer toward or further from the screen.


Design & Prototype



    In the original design, we want users can first create a real shadow with their bird gesture, which is the easiest shadow animal gesture. Then, the shadow can fly into the screen become a graphic bird which represent the imagination being vivified. At last, bird goes to the projection and live in the real environment.   

    We choose Kinect sensor, but leap motion or the combination flex sensor and accelerometer as our input sensor is owing to that it have a great range of sensing zone and intuitional experience. For the output, we use overlook viewpoint to mimic the shape of shadow. The bird image is draw by illustrator and animated by P5.js.

    The first problem we faced is the position arrangement of the flashlight, projection and the Kinect. They influence each other and the movement of users.The second problem is the control of Kinect. We spent lots of times to erase the noise when Kinect detect more than one person. At last, we originally want to use three.js to show a 3D bird in the browser but fail.


Technique & Source Code



ICM -Data -Week 7

Create a sketch that uses an external data source. Plan to present in class using the classroom computer or your own laptop if you prefer. Here are some ideas:

1.Track personal data over the course of a few days. Enter the data manually into a JSON file and visualize the results.
2.Count word frequencies in two different text sources and visualize the results.
3.Use weather data to affect elements in a sketch.
4.Gather data from a google spreadsheet and use in a sketch.
5.If you are interested in maps, check out ITP 2nd year Cris’s Mappa p5.js library.

Please download file here:

ICM -Objects and Arrays -Week 5

Design a sketch in an object-oriented fashion. Follow these steps and see how far you get.
1. Write a class with a constructor function and generate one single object.
2. Put one or more functions in the class function and call them on the object.
3. Try making two objects without an array. Can you make them different?
4. Duplicate the object using an array and make as many as you like!

ICM -Functions: the basics -Week 4

Explore re-organizing your code. Keep setup() and draw() as clean as possible, and do everything in functions that you create yourself:
1.Break code out of setup() and draw() into functions.
2.Use a function to draw a complex design (like this) multiple times with different arguments.
3.Write a function to that returns the result of a mathematical operation that you need to do several times in your code.

ICM -Animation and Interaction -Week 2 + 3

Create a sketch that includes:
* One element controlled by the mouse.
* One element that changes over time, independently of the mouse.
* One element that is different every time you run the sketch.
* (You can choose to build off of your week 1 design, but I might suggest starting over and working with one or two simple shapes in order to emphasize practicing with variables. See if you can eliminate all (or as much as you can) hard-coded numbers from the sketch.)

Work with rule-based animation, motion, and interaction.
1.Try making a rollover, button, or slider from scratch.
2.Create an algorithmic design with simple parameters.
3.Tie the above two together and have an interface element control the visual design or behavior of other elements in your sketch.

ICM -Introduction and Drawing -Week 1

How Computation Applies To My Interests?

What I really care most about is how we can merge computation technology into more “place and events” in our life. Academically speaking, I want to take part in the carnival of ubiquitous computing. I want to experiment with new user interface combining much more than touch screen.

Source: Harvard Innovation Lab
It’s hard not to be excited by the computation technology living at this age. We witness how computer after effect live people’s imagination into real life, we experience how iPhone change almost everything in our life and not to mention how Facebook reform our society structure and the way people connect to each other. They are tons of chances that attract us to become not only a technology consumer, but a creator. For myself, it an advertising campaign called GUMPLAY in Japan.
GUMPLAY is mean to help kids forming the habit to brush teeth. The advertising agency gamify the dull routine action of brushing teeth into a battle toward germs in the mouth with a little gadget made up of sensors attached on the toothbrush. This project demonstrate the capability of computation technology to endues virtual meaning toward real life and provokes my initial interest toward this field.
But computation means a lots more than that. Computation combined with internet provides possibilities of automation, virtualization and decentralization. On these basis, we build countless parallel spaces and even teach robots being a human. I hope I won’t miss this journey of future history and I hope I can hands on and get dirty to realize those envision. That’s me interest toward computation and the reason why I’m here at ITP.


Source Code:


Setting up the development environment.
Why program?  What is program?
High-level programming language / Low-level
Client-side / Server-side
Java / Processing
C++ / OpenFramework
Javascript / P5.js