ICM Midterm: Drag & Drop Utterances

At some point early this semester, I convinced myself that programming was beyond me. Until now, I’ve been tripping up and getting discouraged by little obstacles as I go along, basically sweating the small stuff. However, this week’s ICM assignment – documenting an idea and first steps for a Processing project – enabled me to develop bigger-picture objectives that are helping me to overcome the mental obstacles. Having some goals in mind, I’ll have a far better idea of what questions to ask!

In last week’s class, I proposed my initial project idea. Somewhat inspired by my recent ‘Word of The Day’ text parsing assignment, I was thinking of creating a project entitled “Today.” The idea was to form a new Processing sketch to record each day of my life, in a journal format.

My first idea for the ICM midterm.

"Today" was my first idea for the Processing project.

Some of the data, such as the date, weather, Twitter feed, Flickr feed and perhaps the top news headlines would be filled in automatically from APIs, XML and/or manually parsed data using the loadStrings() command. In a bid to do a little bit of Processing every day, I also planned to include a simple yet adorable ‘Sketch of the Day’ (by the hand of yours truly) in there, too. I thought about this idea for quite a while, and drew up an example of what the journal would look like – and as I visualized it, I realized that using a template to record every day of my life wasn’t quite as engaging, or interactive, as the project that I really wanted to create.

Returning to the drawing board, I came up with a few bigger-picture projects that I’d like to create in Processing. I’ve documented my ideas below, along with my thoughts on how I might tackle these projects. Figuring out the steps to complete one or more of these is the motivation I need to really delve further into Processing.

Drag & Drop Utterances
Magnetic Poetry
is a wonderful thing – I love playing around with the words on the fridge while waiting for the kettle to boil.  Since beginning at ITP, however, I rarely have the chance to loiter in front of the fridge (and that kettle never boils anyway, no matter how much I try not to watch it).  So, this project will harness the power of Processing, the ITP vocabulary and computer screens as makeshift “fridges” to create interactive ITP sentences/poetry. A working title for this project could be “Drag & Drop Utterances.”

Drag & Drop Utterances

Drag & Drop Utterances

Step 1: Create one word as an object on a plain background: http://itp.nyu.edu/~kjk311/ICM2009/poetry/
Step 2: Figure out how to drag the word from one location and drop it at another intended location – possibly using some combination of mouseClicked(), mouseDragged(), mousePressed() and mouseReleased() functions.
Step 3: Create a list of ITP/poetry words to use.
Step 4: Create array of word objects from list. Each word object must be independently movable like the one in Step 1.
Step 5: Place the array of words in random order in the “word bin.” Figure out how to select, drag and drop them without screwing up anything.
Step 6: Create cool new background (cooler than a fridge), or series of backgrounds.
Step 7: Segue into PComp/Arduino territory to figure out a physical interface (project the image on Smart Board? touchscreen?)

Idea #2: Kaleidoscope
Processing lets us make cool patterns, as do kaleidoscopes. And, a kaleidoscope project could be a good opportunity to brush up on my math skills, because patterns need to be mirrored and reproduced at certain angles to create the kaleidoscope effect.

Step 1: Research kaleidoscope patterns, and figure out exactly how they work. What’s the general structure? To what extent do shapes move randomly? What axes do they need to move on?
Step 2: Create pattern, based on findings from Step 1.
Step 3: Create reflections of patterns.
Step 4: Create physical interface for kaleidoscope – using dials?

Phantom limbs, cognitive/sensory function and virtual tactility (and Kat's nightmares)

Amputation is one of my greatest fears. I can’t count the number of times I’ve awoken with a jerk and immediately checked to see that my limbs are still there. The existence of phantom limbs is new to me – after the initial traumatic experience of amputation, I can’t fathom what it must be like to sense things as if the missing limb remained in place. Cognitive function is a crazy thing.

In chapter seven of “We Create What We See,” Donald Hoffman argues that we construct everything that we sense and perceive, including all that we taste, hear, smell and feel. We also construct the location and point in time in which we sense and perceive things. Of all the senses, touch poses “the biggest impediment.”

If we construct all that we feel, we “should be able to construct feeling in any body part, even if that part is amputated.” Conversely, if an individual’s constructive processes or cognitive functions are impaired, that individual will have difficulty with touch, even if the body part remains. Therefore, it’s possible to create virtual realities -  or, more accurately, “virtual tactilities” – based on touch sensation.

I’m relieved that Donald Hoffman concludes this chapter with a discussion of the medical possibilities afforded by these findings, because I kind of expected him to go down the gaming/virtual worlds route. Once upon a time, when Second Life was the Next Big Thing, I played around with it for a while. Cool concept and all, but I felt it was a waste of real-life time to hang out in a virtual world that felt so, um, virtual. I’m all for using my imagination and whatnot, but I have enough to do in this joint without pottering around another world learning to fly, walk and talk.

Returning to Hoffman, this was a pretty intriguing read. I’d be interested in checking out any PComp projects (or those from more advanced classes) that explore or employ virtual tactility.

Pcomp Lab: Servo Motor Control with an Arduino

This lab introduced us to the Servo, a small motor with an output shaft that can be easily used to create and control motion with a microcontroller.  Most servos allow for up to 180 degrees of movement, and can be used in applications involving levers, periodic or reciprocating motions. Servos are commonly used in projects including radio-control airplanes, radio-control cars and robotics, among others.

The goal of the lab was to control the position of the servo using analog sensor values. To set up the breadboard, I connected power and ground on the breadboard to 5V power and ground on the Arduino. I connected an FSR sensor to Analog pin 0 on the Arduino, connected the yellow wire of the servomotor to digital pin 2, the red wire to power, and the black wire to ground.

Part I: Pulsing the servo with a digital pin to set the position of the servo. The first part of the lab involved figuring out the pulse length and position (the easier route would be the Servo library, which we’ll see in Part II). I controlled the Servo using an FSR sensor. To ensure that the servo moved through its full range of motion with the FSR, I first determined the range of values of the FSR and used the map(); function to adjust the servo code to fit.

Servo Lab Part I from katherine keane on Vimeo.

Part II: Controlling the servo using the Servo library. Part I was the hard part – because in Part II, we used the Arduino Servo library to take care of all the calculations.

Servo Lab Part II from katherine keane on Vimeo.

The sweet music of continuity

Soldering, I’ve discovered, is nothing like riding a bicycle. My initial attempts at soldering a DC power connector to a set of wires for a breadboard were alarmingly, embarrassingly awful. Entirely unreflective of previous hours spent in secondary school with soldering iron in hand (for constructive purposes, promise).

I attribute my initial failures to the “helping” hands, which wouldn’t seem to stay put, no matter how much I coaxed them to cooperate. After an hour or so, about six inches of solder, mass header pin destruction and multiple clippings of the red and black wires, I congratulated myself on my decision to use eight inches of red and black wire at the beginning of this project, rather than the suggested 4 inches.

Upon successful attachment of the wires to the header pins, everything appeared to be okay – but still, it was with trepidation that I dragged out the multimeter to test for continuity.

Center pin to red wire: BZZZZ!
Outside rim to black wire: BZZZZZ!

Understanding electricity is pretty, erm, fundamental for physical computing. I’ll admit that I don’t quite “get” it all yet. Therefore, it’s time for me to spend some serious hours in the shop playing around in an attempt to really figure it out. Circuits can’t be that hard, right?

This is but one of the things I'm planning to master...

Arduino is but one of the things I'm hoping to master...

CommLab readings

I’d always thought that the phrase “standing on the shoulders of giants” was coined by Isaac Newton. Ironically, I learned today that this term was first attributed to the twelfth-century philosopher, Bernard of Chartres.

Interconnectedness, influence, and standing on the shoulders of giants were all at the heart of this week’s CommLab readings. The four texts got me thinking about the nature of creativity and the challenge of building something new upon things that already exist, or things that we already know.

In Understanding Media: The Extensions of Man, Marshall McLuhan proposes the concept of the medium as the message. While I agree with McLuhan that a given medium has a profound effect on the content it delivers, I think it’s worth noting that for purposes of effective communication to an audience, we need to ensure we prioritize the content, rather than the medium itself.

McLuhan would likely favor Shepard Fairey over the Associated Press in reference to the Obama Hope Poster – he would likely claim that whether Fairey used the Clooney-Obama image or the solo Obama image is irrelevant, because the “personal and social consequences” of Fairey’s work have impacted society to a greater extent than those of the AP photographer, Mannie Garcia, who shot the image that Fairey referenced in his work.

Why is the AP pursuing Fairey so closely for his reference to the Obama image? Is it purely an attempt to get its share of the proceeds from this single image (a sum that was reinvested in further Obama, according to Fairey)? Is the AP grasping for straws in an attempt to recentralize/regain control over the news media? If Fairey loses this case, it will likely result in dire consequences for artists, as they will be reluctant to reference existing images in the future – thereby stifling artistic interpretation and freedom, and possibly creating a certain level of media censorship.

In a press statement issued on October 16, Fairey expressed regret at his attempt to conceal his “mistake,” and reminded us of the true focus of his case – “the right to fair use so that all artists can create freely. Regardless of which of the two images was used, the fair use issue should be the same.” According to the U.S. Copyright Act of 1976, the following four factors should be considered in determining what constitutes “Fair Use” of a copyrighted existing work:

  1. The purpose and character of the use, including whether such use is of commercial nature or is for nonprofit educational purposes
  2. The nature of the copyrighted work
  3. The amount and substantiality of the portion used in relation to the copyrighted work as a whole
  4. The effect of the use upon the potential market for, or value of, the copyrighted work

As Fairey’s defense has pointed out, the AP has “referenced” artwork on many occasions – and furthermore, profits from it. Regardless of which image Fairey used (in essence, whether or not George Clooney was cropped out of the image), the media is the message. The message conveyed by Fairey’s Hope poster conveys a far greater message, to a far greater audience, than the AP’s photo.

Regardless of which photo Fairey referenced in his work, I believe that his use of the Obama image constituted fair use. Really, what does it matter which image he used? Obama’s image was used commercially all over America, to the point where the White House legal department attempted to rein in commercial use of the President’s likeness. Obama Fries, anyone?

I found McLuhan’s discussion of “hot” and “cool” media somewhat baffling, given that any medium can potentially incorporate several layers of reference to other works. For instance, McLuhan defines “hot” media as those that appeal to a “single sense” in “high definition” and require little information to be filled in by the audience, and cites movies, radio and photographs as examples – but what if the artist/director includes visible and/or audible elements within the frame/soundtrack of the movie that refer to other works of art that precede it?

Jonathan Lethem prefaces his essay The Ecstasy of Influence: A Plagiarism with a quote from John Donne’s Meditation XVII in Devotions upon Emergent Occasions. Below is another excerpt from Donne’s Meditation that includes the phrase later used by Ernest Hemingway as a book title:

No man is an island entire of itself; every man
is a piece of the continent, a part of the main;
if a clod be washed away by the sea, Europe
is the less, as well as if a promontory were, as
well as any manner of thy friends or of thine
own were; any man’s death diminishes me,
because I am involved in mankind.
And therefore never send to know for whom
the bell tolls; it tolls for thee.

Donne’s Meditation no doubt also influenced Paul Simon when writing the song I Am A Rock, a song about loneliness and isolation in which he claims, “I am an island.” Many years after writing I Am A Rock, Simon looked back on this song and others and admitted, “I don’t believe in [these songs] as I once did. I have included them because they played an important role in the transition. It is discomforting, almost painful, to look back over something someone else created and realize that someone else was you. I am not ashamed of where I’ve been and what I’ve thought. It’s just not me anymore. It is perfectly clear to me that the songs I write today will not be mine tomorrow. I don’t regret the loss.”

Paul Simon understood the contextuality of his work and how things (and feelings) change over time.

Please don’t force others to reinvent the wheel – isn’t imitation the sincerest form of flattery? If we all held our intellectual property hostage, no progress would be made. All types of progress, including art, science, politics (everything!) require dialogue and historical context. Otherwise, you’re just banging your head against a wall. Similarly, humankind requires dialogue and context: we’re standing on the shoulders of giants, and no man is an island.

Ok – I think that just about covers all the cliches for one evening.

PComp lab: Serial output using Arduino

In this lab, data was sent from a sensor to a program running on the computer, and the output of the sensor was then graphed in Processing on the computer screen. The purpose of the lab (for me, at least!) was to gain a better understanding of how a sensor’s output corresponds to physical “events” as it senses them.

I connected the sensor to analog pin 2, and checked for the serial values using Arduino. I then opened Processing, imported the Processing library, checked the list of active serial ports and determined Port [0} as the appropriate serial port for use with the Arduino ("/dev/tty.usbserial-A6008e1S").

PComp Lab: Serial output using Arduino from katherine keane on Vimeo.

Experiment #1: I used a potentiometer as the sensor input, and set up the Processing sketch to draw a line for which the length corresponded to the byte value.

import processing.serial.*;

Serial myPort;        // The serial port
int graphXPos = 1;    // the horizontal position of the graph:

void setup () {
size(400, 300);        // window size

// List all the available serial ports
// I know that the first port in the serial list on my mac
// is usually my Arduino module, so I open Serial.list()[0].
// Open whatever port is the one you’re using.
myPort = new Serial(this, Serial.list()[0], 9600);

// set initial background:
background(50, 50, 250);
void draw () {
// nothing happens in draw.  It all happens in SerialEvent()

void serialEvent (Serial myPort) {
// get the byte:
int inByte = myPort.read();
// print it:
// set the drawing color. Pick a pretty color:
stroke(150, 200, 40);
// draw the line:
line(graphXPos, height, graphXPos, height – inByte);

// at the edge of the screen, go back to the beginning:
if (graphXPos >= width) {
graphXPos = 0;
// clear the screen:
else {
// increment the horizontal position for the next reading:

Experiment #2
I then experimented by using an FSR and a new Processing sketch to move a ball on a horizontal axis across the center of the screen as the FSR’s resistance changes. Below is the code I used:

import processing.serial.*; //import the serial library

Serial myPort;        // The serial port – makes a variable that we store methods from the serial library in
int ballPosition = 0;    // declares the the horizontal position of the ball

void setup () {
size(400, 300);        // create window size
println(Serial.list());   // print list of all the serial ports
myPort = new Serial(this, Serial.list()[0], 9600); //make a variable called myPort to store the serial library methods
noStroke(); //make it pretty
smooth(); //make it even prettier!

void draw () {
background (230, 180, 180); //create background color
fill(0); //make the ball black
ellipse(ballPosition, height/2, 30,30); //draw the ball and set it up to move back and forth across the page as the resistance changes.

void serialEvent (Serial myPort) {
// get the byte:
int thisByte = myPort.read();
// print it:

CommLab: Animation station

For this week’s Comm Lab assignment, Arturo and I created a stop-motion animation entitled “KK and AV’s Adventure.” Inspired by Michel Gondry/White Stripes’ ‘The Hardest Button to Button’ shown in class last week, we decided to create a music video.

After spending a few hours familiarizing ourselves with iStopMotion software, and then playing around with Sculpey and perusing toy offerings at Forbidden Planet in search of suitable characters to put in front of the camera, and ultimately decided to maximize points of subjects’ motion (and have fun!) by putting ourselves in front of the camera.

We set up the scene/lighting in Arturo’s studio, and used a camera, tripod and computer running iStopMotion, all mounted on a dolly, to capture about five minutes of footage. We used iStopMotion for most of the editing, and imported into Final Cut for the finishing touches.

I had minimal movie-making/animation experience prior to this project, but I was lucky to have a great teacher in Arturo – he’s a dab hand with the software, not to mention the performance side of things – and really helped me to gain the confidence and knowledge to feel like I could do this kind of project again.

‘Todo el Voltaje’ is the name of the sweet tune we used, from the album Picotero by Columbian electronic duo Monareta.

I think it’s pretty evident that we had fun with this one.

Reaction to Scott McCloud's "Understanding Comics"

When I was eight or nine years old, I enjoyed reading Buster, The Dandy, The Topper and the like, but I’ve barely glanced at a comic since. I wasn’t initially too enthused about reading Scott McCloud’s “Understanding Comics,” but once I opened the book, I found myself unable to put it down (well, until my arm was twisted to go to the pub).

Previously, I was aware that the comic was an art form, but I had little interest in learning more about comics because they dealt with fantasy characters, superheroes, action and other subjects that I have little interest in reading about, because I thought that superheroes are best seen on TV or in movies. I somehow managed to miss that comics are an ideal fusion of text and graphics, and I really wish I’d read this book earlier – perhaps around the time that I was studying literature and graphic design as an undergrad!

McCloud’s comic-style presentation was a refreshing change from lengthy texts, although it was perhaps denser in content than the reading assignments we’ve had so far. By dissecting the roles of text and graphics, and discussing the need for a balance between the two in effective storytelling, McCloud convinced me to reconsider how I present information and communicate ideas. I especially enjoyed McCloud’s discussion of how comics are set apart from other art forms in that they can appeal to all the readers’ senses, and of how readers are required to fill in the areas between comic frames.

“Understanding Comics” was an enjoyable and informative text that I expect will come in handy not only for Comm Lab, but also for any design or storytelling projects that we work on. The book proved an indispensable roadmap for the sequential images project that Peter Holzkorn and I worked on this weekend (currently being polished… stay tuned!).

Stupid Pet Trick: Attention-grabbing handbag

Every handbag makes a statement. I’m pretty sure that this shiny yellow one states something to the effect of, “Look at me! Look at me! Look at me!”

There are a lot of handbags out there, though – many of them larger, shinier and arguably more attractive than my awesomely bright and cheerful accessory. For our Stupid Pet Trick assignment, I decided to make some alterations to ensure that my yellow bag continues to stand out in any crowd. An LED on the bag ensures that it’s noticed at all times – but upon sight of any handbag that poses a threat, a squeeze of a black coin purse that discreetly houses an FSR sensor causes two other LEDs in the bag to turn on, depending on the amount of pressure exerted upon the coin purse.

The portability of the handbag was severely decreased due to wires, a breadboard and Arduino – which also filled the bag to capacity! – so next time, a breadboard-less, soldered circuit will be the way to go.

Stupid Pet Trick: Attention-grabbing handbag from katherine keane on Vimeo.

Reviewing the Stupid Pet Tricks in class, we discussed possible ways to push this project further. Suggestions included:

  • Use the magnetic snap on the bag as a switch to activate a security alarm, so the bag would light up when opened, OR add an audible alarm using the Tone library
  • Put the sensor on the bag’s shoulder strap, to require less input from the wearer to make the lights turn on. On the other hand, the in-hand FSR sensor provides the wearer with a greater degree of control to decide when more lights are necessary to keep the bag in the spotlight.
  • For a subtler effect, create a bag that “glows” upon the wearer’s command: construct the bag’s exterior from a less opaque material, and the lining from a material that diffuses light, and place LEDs between the exterior and lining.
  • Place the LEDs inside the bag, and use the magnetic snap as a switch to turn them on when the bag is opened, enabling the wearer to easily find specific contents within.

Below is the code I used:

int sensorPin = 0;     // select the input pin for the FSR
int ledPinOne = 9;  //select the digital output pin for the 1st LED
int ledPinTwo = 6; //select the digital output pin for the 2nd LED
int ledPinThree = 5; //select the pin for the 3rd LED
int sensorValue = 0;     // the analog reading from the FSR resistor divider
void setup() {
pinMode(sensorPin,INPUT); //set the sensor as the input
pinMode(ledPinOne,OUTPUT); //set 1st LED as output
pinMode(ledPinTwo,OUTPUT); //set 2nd LED as output
pinMode(ledPinThree,OUTPUT); //set 3rd LED as output
void loop() {
// read the value from the sensor:
sensorValue = analogRead(sensorPin);
//sensorValue = sensorValue/4;
Serial.println(sensorValue/4);      // print the sensor value in the debugger
if (sensorValue <=100) {
// turn on ledPinOne
analogWrite(ledPinOne, 127);
analogWrite(ledPinTwo, 0);
analogWrite(ledPinThree, 0);
else {
//if the sensor value is between 120 and 180
if ((sensorValue > 120)&&(sensorValue <=180)){
analogWrite(ledPinOne, 255);
analogWrite(ledPinTwo, 127);
analogWrite(ledPinThree, 0);
else {
//if the sensor value is between 181 and 255
if ((sensorValue > 181)&&(sensorValue <=255)){
analogWrite(ledPinOne, 255);
analogWrite(ledPinTwo, 255);
analogWrite(ledPinThree, 127);