Research Projects

Skinput: Appropriating the Body as an Input Surface
We present Skinput, a technology that appropriates the human body for acoustic transmission, allowing the skin to be used as a finger input surface. In particular, we resolve the location of finger taps on the arm and hand by analyzing mechanical vibrations that propagate through the body. We collect these signals using a novel array of sensors worn as an armband. This approach provides an always-available, naturally-portable, and on-body interactive surface. To illustrate the potential of our approach, we developed several proof-of-concept applications on top of our sensing and classification system. To be published at CHI 2010.

Whack Gestures: Inexact and Inattentive Interaction with Mobile Devices
Whack Gestures seeks to provide a simple means to interact with devices with minimal attention from the user – in particular, without the use of fine motor skills or detailed visual attention (requirements found in nearly all conventional interaction techniques). For mobile devices, this could enable interaction without “getting it out,” grasping, or even glancing at the device. Users can simply interact with a device by striking it with open palm or heel of the hand. Published at TEI 2010.

Abracadabra: Wireless, High-Precision, and Unpowered Finger Input for Very Small Mobile Devices
We developed a magnetically-driven input approach that makes use of the (larger) space around a (very small) device. Our technique provides robust, inexpensive, and wireless input from fingers, without requiring powered external components. By extending the input area to many times the size of the device’s screen, our approach is able to offer a high C-D gain, enabling fine motor control. Additionally, screen occlusion can be reduced by moving interaction off of the display and into unused space around the device. Published at UIST 2009.

Stacks on the Surface: Resolving Physical Order with Masked Fiducial Markers
We present a method for identifying the order of stacked items on interactive surfaces. This is achieved using conventional, passive fiducial markers, which in addition to reflective regions, also incorporate structured areas of transparency. This allows particular orderings to appear as unique marker patterns. We discuss how such markers are encoded and fabricated, and include relevant mathematics. To motivate our approach, we comment on various scenarios where stacking could be especially useful. We conclude with details from our proof-of-concept implementation, built on Microsoft Surface. Published at ITS 2009.

Providing Dynamically Changeable Physical Buttons on a Visual Display
Physical buttons have the unique ability to provide low-attention and vision-free interactions through their intuitive tactile clues. Unfortunately, the physicality of these interfaces makes them static, limiting the number and types of user interfaces they can support. On the other hand, touch screen technologies provide the ultimate interface flexibility, but offer no inherent tactile qualities. In this paper, we describe a technique that seeks to occupy the space between these two extremes – offering some of the flexibility of touch screens, while retaining the beneficial tactile properties of physical interfaces. Published at CHI 2009.

Texture Displays: A Passive Approach to Tactile Presentation
Despite touch being a rich sensory channel, tactile output is almost exclusively vibrotactile in nature. We explore a different type of tactile display, one that can assume several different textural states (e.g., sticky, bumpy, smooth, course, gritty). In contrast to conventional vibrotactile approaches, these displays provide information passively. Only when they are explicitly handled by the user, either with intent to inquire about the information, or in the course of some other action, can state be sensed. This inherently reduces their attention demand and intrusiveness. We call this class of devices texture displays. Published at CHI 2009.

Where to Locate Wearable Displays? Reaction Time Performance of Visual Alerts from Tip to Toe
One way to draw a wearer’s attention is through vibrotactile stimulation, commonly employed in mobile devices to alert users (e.g., to an incoming call). Also popular, although not tied to any particular body location, are audio alerts. Visual displays offer an alternative notification method. However, there has been little research into their optimal body placement, despite being an unobtrusive, lightweight, and low-powered information delivery mechanism. Furthermore, visual stimuli have the added benefit of being able to work alone or in concert with conventional methods, including auditory and vibrotactile alerts. Published at CHI 2009.

Pseudo-3D Video Conferencing with a Generic Webcam
When conversing with someone via video conference, you are provided with a virtual window into their space. However, this currently remains both flat and fixed, limiting its immersiveness. We present a method for producing a pseudo-3D experience using only a single generic webcam at each end. This means nearly any computer currently able to video conference can use our technique, making it readily adoptable. Although using comparatively simple techniques, the 3D result is convincing. Published at ISM 2008.

Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile finger Input Surfaces
We present Scratch Input, an acoustic-based input technique that relies on the unique sound produced when a fingernail is dragged over the surface of a textured material, such as wood, fabric, or wall paint. We employ a simple sensor that can be easily coupled with existing surfaces, such as walls and tables, turning them into large, unpowered and ad hoc finger input surfaces. Our sensor is sufficiently small that it could be incorporated into a mobile device, allowing any suitable surface on which it rests to be appropriated as a gestural input surface. Published at UIST 2008.

Lightweight Material Detection for Placement-Aware Mobile Computing
Numerous methods have been proposed that allow mobile devices to determine where they are located (e.g., home or office) and in some cases, predict what activity the user is currently engaged in (e.g., walking, sitting, or driving). While useful, this sensing currently only tells part of a much richer story. To allow devices to act most appropriately to the situation they are in, it would also be very helpful to know about their placement – for example whether they are sitting on a desk, hidden in a drawer, placed in a pocket, or held in one’s hand – as different device behaviors may be called for in each of these situations. Published at UIST 2008.

3D Head Tracking With a Generic Webcam
Using OpenCV's Haar cascade face detection functionality, I was able to throw together a real time head tracking program. I coupled this with several 3D demos, including a Johnny Lee style targets demo. Performance is surprisingly good. On my 2.16 Ghz MacBook I am tracking at 25 frames per second - sufficient for real time interaction with no obtrusive reaction delays. Additionally, I've optimized the code so that only 50% of a single processor core is consumed (and that's on my relatively slow laptop).

Rethinking the Progress Bar
Numerous factors cause progress bars to proceed at non-linear rates (e.g. pauses, acceleration, deceleration). Additionally, humans perceive time in a non-linear way. The combination of these effects produces a highly variable perception of how long it takes progress bars to complete. Results from a user study indicate there are several factors that can be manipulated to make progress bars appear faster, when in fact they are not, or potentially even slower. Published at UIST 2007.

The Sling in Medieval Europe
The simple sling is often neglected when reviewing the long history of ranged warfare. Scholars typically focus on the simple thrown spear, atlatl, throwing axe, bow, and crossbow. However, in experienced hands, the sling was arguably the most effective personal projectile weapon until the 15th century, surpassing the accuracy and deadliness of the bow and even of early firearms. Published in the Bulletin of Primitive Technology.

Image Clustering
Using the RCBIS package developed by Stacey Kuznetsov and myself, Jeff Borden put together a image clustering package. The application was created using the Prefuse information visualization toolkit and a K-means clustering method. The latter used a distance metric the RCBIR engine calculates when given two images.

Search by Sketch
After several months of intense development alongside Stacey Kuznetsov, the Rapid Content-Based Image Search (RCBIS) engine was ready for use in user oriented programs. The most obvious and appealing project was to develop an image search engine which took not text, but a sketch as input.

CollaboraTV: Asynchronous Social Television
Television was once championed as the “electronic hearth” which would bring people together. Indeed, television shows provide a common experience, often affording even total strangers a social connection on which to initiate conversation. However, a fundamental shift in how we consume media is degrading such social interactions significantly – an increasing number of people are no longer watching television shows as they broadcast. Instead, users are favoring non-live media sources, such as Digital Video Recorders, Video-On-Demand services , and even rented physical media (e.g. Netflix). Published at UXTV 2008.

iEPG: An Ego-Centric Electronic Program Guide and Recommendation Interface
Conventional program guides present television shows in a list view, with metadata displayed in a separate window. However, this linear presentation style prevents users from fully exploring and utilizing the diverse, descriptive, and highly connected data associated with television programming. Additionally, despite the fact that program guides are the primary selection interface for television shows, few include integrated recommendation data to help users decide what to watch. iEPG presents a novel interface concept for navigating the multidimensional information space associated with television programming, as well as an effective visualization for displaying complex ratings data. Published at UXTV 2008.

Kronosphere: Temporal File Navigation
Hierarchical file systems mirror the way people organize data in the physical world. However, this method of organization is often inadequate in managing the immense number of files that populate users’ hard drives. Systems that employ a time-centric or content-driven approach have proven to be compelling alternatives. Kronosphere offers a powerful time and content-based navigation and management paradigm that has been specifically designed to leverage universal cognitive abilities.

Speech Enhanced Commenting
Ineffective and insufficient commenting of code is a major problem in software development. Coders find commenting frustrating and time consuming. Often, programmers will write a section of code and come back later to comment, potentially reducing the accuracy of the information. Speech Enhanced Commenting is a system that takes advantage of an unused human output - voice. This system allows users to speak their comments as they program, which is natural and can happen in parallel to the user typing the code.

Visual Hash
Checking passwords often requires a complex algorithm, but computation time is generally small. For users on their local machines, it's insignificant. However, if the password needs to be checked by a remote login server, things become more expensive. Secure connections have be negotiated and databases have to be queried. Throw in a few thousand users and your are looking at a serious load problem. Visual hashes can reduce the burden on login servers by eliminating some unnecessary login attempts.

Aphrodisias Regional Survey
During the 2006 and 2007 summers, I spent eleven weeks in Turkey assisting an archeological expedition at a Greco-Roman city named Aphrodisias. A small Geographical Information Systems (GIS) team worked on producing maps to assist the project, both in surveying the region and artifact collection. Creating site level contour maps was a major effort in the first season. I primarily worked on developing methods for predicting the location of ancient sites, such as iron mines and forts, and the routes of aqueducts and ancient roads.

'A' PPC Compiler
Compilers are quite possibly the programmer's most frequently used tool, and yet often overlooked. Over the course of 2 months, I coded up a compiler in C++ for a medium-sized language dubbed 'A'. The compiler supports all the goodies, including: arrays, pointers, function calls, recursion, type checking, static scoping, and more. The output: executable PPC assembly.

Gesture Based Process Scheduling
People place different levels of importance on applications they use. This importance is often transient. For example, when the user is in a rush and wants to quickly check their email before they leave, the email application should not have to wait on other applications. However, other times, the user might accept a longer launch time because they also value background processes, like a downloading a file or transferring pictures from their digital camera. Anticipating user needs is practically impossible. However, users provide a wealth of feedback in the form of gestures.

Enki
Over the 2005 summer, I had the pleasure of working at IBM's Almaden Research Center in San Jose, California, under the leadership of John Barton and Stephen Farrell. The development team consisted of Chris Parker, Meng Mao and myself. We worked on a unique application which blended aspects of personal information management, social networking, and content management. The project was dubbed Enki, after the Sumerian God of Knowledge.

A Comparative Analysis of Archaic Peoples and Early Dutch Settlers Living in The New York Region
In 1624, the Dutch established their first outpost about 150 miles north of Manhattan, up the Hudson River, called Fort Orange. It was located near the modern city of Albany, NY. Interestingly, the environment was similar to that of prehistoric Manhattan. The plant life was similar, with spruce, fir, birch, and poplar dominating the area around Fort Orange. Because archaic groups and a fledgling European outpost relied so heavily on the local environment to survive, there are many commonalities in how they lived. Using this similar environmental context, one can explore how each group, with their different technologies and social traditions, adapted and survived.

Temperature Regulated Charge Algorithms
This page is dedicated to the research I conducted while in my high school's science research class. I enjoyed much success in several science competitions and ultimately filed a patent for the technology I developed during my freshman year in college. This page is very out of date.

BaLiS (Bacterium Life Simulator)
BaLiS was my first attempt at designing an AI (high school). It uses a priority matrix which it builds by exploring. As it explores the 50x50 world, it encounters different items (such as food). The bacterium exhibits certain behaviors based on what it finds, and adjusts it's memory accordingly.