| 
Over
the 2005 summer, I had the pleasure of working at IBM's Almaden
Research Center in San Jose, California, under the leadership of
John Barton and Stephen Farrell. The development team consisted
of Chris Parker, Meng Mao and myself. We worked on a unique application
which blended aspects of personal information management, social
networking, and content management. The project was dubbed Enki,
after the Sumerian God of Knowledge.
The
initial goal was to produce a pen-aware, mobile application that
would provide on-the-go calendaring and meeting information. The
target audience were people that spend a significant portion of
their day in meetings: Executives, Doctors, Lawyers, etc. Many people
take handwritten or typed notes in these meetings, but have no easy
mechanism for associating these to the event. Handwritten notes
are either discarded or filed away. Ultimately the information is
either lost or inaccessible.
We considered
what types of "objects" people deal with in their business
life. This boiled down to three primitives:
- People
- Managers, government officials, family, visitors, co-workers,
etc.
- Events
- Meetings, lectures, phone conferences, dental appointments,
etc.
- Topics
- Bicycles, XYZ project status, financial results, bi-monthly
research roundtable, etc.
These
three things were interrelated. Events generally have a purpose,
and so they are associated with particular topics. People attend
meetings, which makes a clear association. Attaching topics to people
is harder to do automatically. One method we discussed was to assume
people were involved with topics based on the events they attended.
A flavor of this compromise is ultimately the route we chose.
Our
next thought was that once you have this complex graph of information,
we can use the handwriting and audio capabilities of the tablet
to let the user tag these object with their own data. Keywords could
be extracted from the data using handwriting and speech recognition
(both technologies we had access to at IBM). Something I thought
would be groundbreaking, but most people thought was of dubious
feasibility, was content-to-content relationships using text analytics
(perhaps something as simple as keyword matching). Assuming that
this could be pulled off, it would then be possible to make new
associations, perhaps ones the user did not even know about. For
example, a meeting about quantum computing is tagged with multiple
written notes about the subject. Elsewhere in Enki there is a person
that has a CV which included several references to quantum computing
(perhaps a list of publications). Even though the meeting and person
have no direct connection, a relationship could be inferred because
of similar content. If this feature ran in real-time, perhaps as
a user wrote notes or the tablet recorded audio pervasively, suggestions
about related content could be shown to the user on-the-fly. Mostly
because of time, this feature was dropped.
Enki
was designed to interface with the user's existing email and calendaring
application (pulling data only). The wireless connection on the
tablet PC meant that Enki could receive live data in a corporate
setting.

Left:
A graph of objects in Enki and their relationships. Right:
These object tagged with content. |

Using
text analytics, relationships between un-connected content
can be inferred. From this connection, Enki could infer relationships
between objects (people, topics, events) even though they
are not directly related. The dashed red line in the graph
to the right shows this inference.
|

 |
| A
screenshot of Enki's person view running on the tablet.
The lower half of the screen is reserved for written
notes about the person. The only thing absent from the
image is a toolbar along the bottom of the screen that
let you navigate to other functions. A button to provide
additional writing space was also provided. |
Person
View Features:
• Contact Information
Name,
title, location, etc.
• Communication
The
person's email was displayed, which launched the default email
client when clicked. Clicking on the person's phone number
fires up a VoIP softphone and calls the person on the tablet!
You can also run a Google or w3 (IBM internal) search with
a single click, which is pre-initialized with their full name.
•
Events
Tying
into Lotus Notes, we could find every meeting the user and
the selected person were both invited to. Only the four closest
meetings are shown (that's both forward and backward in time
from the current date). Users can click the blue "more"
button to see expanded meeting history. The first sentence
or so of the meeting description was displayed along with
the meeting name, date, and time. Clicking on an event forwards
the user to the event view. Hovering the pen over an event
shows the event's description.
•
Emails
This
section displayed the two most recent sent and received emails.
The "more" button expanded the email history, and
spit it into two columns: to and from. The first couple lines
of the email were shown under the email subject and timestamp.
Hovering over an email showed the body text of that email.
•
Files
This
section was intended to display files the user had associated
with the selected person. How great would it be if you could
pull up any person's resume, presentations, reports, etc.
and view it right there? However, a mechanism for doing this
was cut from the project due to time. Rather than scrapping
this useful feature entirely, we decided to pull files that
were sent to the user as emails attachments. The added bonus
was that this could be done and updated automatically. Clicking
on a file opened it.
•
Fringe Artifacts
Stephen
Farrell had been (and still is) working on a very clever social
networking engine called Fringe. This automatically fetched,
processed, and indexed information relating to people. Some
of the information it could show about a person included patents,
publications, blog entries, comments, projects/activities,
management chain, colleagues, pictures, and more. Enki tied
into Fringe, and displayed a few items (called artifacts)
about that particular person.

|
Event View Features:
• Event information
The
event name, time and location are shown in the header.
• People
This
showed all of the people invited to the event. Clicking on
a person takes the user to the person view of that person.
Hovering
the pen over a person's name or picture showed a abbreviated
translucent person-view.
•
Description
A
description of the event (if provided).
•
Files
Displayed
the files that were attached to the event. These are probably
things like meeting agendas and powerpoint presentations.
Clicking on a file launches opened it.

Month
View Features:
This
was primarily used as a navigation screen to jump between
days and months rapidly. Events that lasted all day were shown
inside the "day boxes". Notes could be written about
the month. Hovering with the pen over a day showed a mini
schedule for that day. Clicking on day forwards the user to
the day view.

Day
View Features:
All the events for the day were displayed (one per row). These
were sorted by time, earliest starting first. The title and
location of the event were shown, as well as the first 6 people
in the meeting. If there was more than 6 attendees, a small
"more attendees" button was displayed. Clicking
the latter button or the event title forwarded users to the
event view. Clicking a person would lead to the person view.
There
were several optional "markers" that could be attached
to each event. This was to allow people to color-code or iconize
their schedules. A few of the markers are shown below. Two
automatic markers were the attachment icon, which was present
if the meeting had attached files, and the ink icon, which
appeared if event had associated written notes. Hovering over
the ink marker showed a preview of the written text.

Log
View Features:
The log view was one of the last significant features to make
it into the Enki demo. Having built a relatively rich system
to capture written (and audio) data about people and events
(as well as months and days), a quick way to see all this
data needed to be developed.
The
result was the log view, which took all of the notes for a
single day, both written and audio, and presented them in
a chronological format. Now, at a glance, a user could scan
over all of their notes at the end of the day (or jump back
to past dates) and look for important items and remaining
to-dos.
Users
could jump back to the page the note was take on by clicking
the title (located above each note). From there, they could
make modifications (perhaps crossing out a to-do that was
completed).
Another
interesting thing Enki does is show the changes (deltas) between
notes on the same item. For example, on John Smith's page,
a user writes, "Knows a lot about computers". The
ink would be red in color. When the user returns to John's
page later that day, the previously written text would show
up in black. Now when a new note is added, perhaps "Ask
John about new laptops", only this shows up in red.
Although
a nice side effect is that it increases the readability of
the new text that is being added, the real importance is how
this manifests in the log view. The user can easily see when
notes were modified for a particular object; each change is
timestamped and highlighted in red. An mock example is shown
below.

|
 |
| Event
View |
 |
| Month
View |
 |
| Day
View |
 |
| Log
View: The colors and layout differ from the final Enki
version, but this graphic provides the basic idea. Audio
clips were embedded directly into the page and were
able to be played with a single click. |
|
Note:
Although
the Copyright Act does permit certain unauthorized reuse as fair
use under 17 U.S.C. Section 107, online availability of text and
images does not imply that they may be reused without the permission
of rights holders. No part of this page or files available for download
on this page may be used, quoted, or reproduced in any way without
the prior agreement and written permission of Chris Harrison and
The IBM Corporation. While every precaution has been taken in the
preparation of this material, Chris Harrison assumes no responsibility
for errors or omissions, or for damages resulting from the use of
the information contained in the available material.
Go
to Home Page
Go to Projects Page
Enki Logo by Meng Mao & co.
© IBM Corporation. All rights reserved.
|