The Fly
How the Code Works
The fly is always trying to follow you. You can use your hands to try to swat it out of the way.
Inspiration
After watching the Vice President debate in the U.S, I was inspired to create a fly filter. While watching, I was always on Twitter watching people live-tweet about Pence, and I found it interesting that the fly rendered all of what he said useless to viewers. Why is it that a small insect can steal the spotlight like that?
Background
The purpose of this experimental camera was exploration of face recognition. I wanted to mimic artificial reality in that you'd be able to interact with the fly. I chose to use PoseNet because it incorporates your entire body so you can actually use your hands. When I first initially started this project, I was adamant on creating a face filter, something that would be used on Instagram. But middway through, I figured that I shouldn't force a specific technology and should just explore the code in general.
Thought Process
Sketches here: https://www.dropbox.com/sh/grjammczgyu321o/AAD-Y_1tF1Q7Ys2ajrH38QJSa?dl=0
Even though the main subject during the Vice President debates were the people running for the Vice Presidency of the U.S, Pence and Harris, the fly stole the spotlight. It was one of the top-trending topics on Twitter, general news, etc. I was interested in exploring this idea because I saw a correlation between the subject and a smaller subject rendering the main subject unimportant. I can see this in my code as the frame of reference for how anything is done is related to the face, the main subject. The fly relies on where the face is in order to get to its position. But when you're looking at the camera, you're really focusing on the fly in order to swat it away. When do smaller subjects render the main subject unimportant? And when do the main subjects have the power to stay important despite lots of noise around the subject? Are there scenarios where we'd want to be intentional with rendering the main subject unimportant? Update: - PoseNet is able to detect wrists, so I wanted to use them as a way to push the fly out of the way. Basically, it's an if/then statement – if the location of the fly and wrist is both x and y, then the fly will move up by 50 pixels. I worked with Aarati and a tutor on this, and we concluded that PoseNet may be having an issue where it's not actually detecting the wrist. At first I thought it was because PoseNet was meant for full-body, that is it needs to see all of someone's body to know where to place the wrists. After some testing, this theory wasn't true. I'm still unsure why it's not detecting the wrist.