In developing EventWeb, in addition to considering the infrastructure and information needs for the current events, one must also consider how past and future events should be represented. Today I thought of presenting some ideas about past events.
Let us assume that we have the infrastructure to capture what we want to do to realistically represent past events and provide a “better than being there” experience to visitors. Let us also assume that processing power, broadband communication infrastructure and storage networks are available to meet our needs. In other words our goal today is to dream about how we would like to experience past events, if we had all infrastructure available to us. This will also help us later in thinking what infrastructure will be required to present quality experience to visitors.
Suppose that an event is taking place in an area and we put several cameras, microphones, and other sensors to capture what is going on there. There could be RFID or similar sensors to identify people (or objects) participating in the event. There could be many other information sources – such as commentary, scripts – about the event. Similarly there could be databases capturing what went on in the event and updating statistics – particularly in sports. In summary there could be disparate sensors, databases, and other information sources. Most sensory data maybe collected at the time of the event, but some other data may become available only after the event. All the sensory data that is collected during the event, should be time synchronized. Time is the most important dimension in event capture. Location and “viewpoint” of the sensors will help in creating a spatiotemporal environment with relevant attributes at every point in this environment. A very important decision that the system designer will have to make is what important attributes of the environment are of interest to the visitor. These attributes will determine what sensors should be used, where and how they be placed, and how to assimilate data from these sensors to get the required attributes. For example, current sports events producers may decide that it is only the score that a user is interested – or that action new the ball is of interest and hence a video of that will represent what is going on. On the other extreme, an immersive presence offering a user any position on the field or around it to experience the event may be offered to provide better experience. The final experience will determine what data should be collected and processed.
Effectively using all data from disparate multifarious data sources, the system needs to create a “time machine” (TM) for the event. This TM is a database of all sources indexed using time and important parts, or subevents. We need to consider an event to be a hierarchical concept in which using several primitive or atomic events more compound events could be built. At atomic event is the event that can not be split into sub events. Again, this will be an important decision for designers to consider what should be atomic events. An atomic event may be determined both by the application context as well as the technology that is available to capture and represent those. Using these atomic events, one could build compound events. The process of building compound events is a process that could be extended to many levels. There should be a language – or grammar – to build compound events using atomic events. This is similar to the nature of languages, both natural as well as programming. The task of developing compound events from atomic events is also dependent on the application area and what resolution and granularity should be offered to a visitor. These are the decisions that will be made at the design time and will affect what could be presented to a visitor and what could be experienced by him.
If we offered all time and location indexed sensory data to a visitor and forced her to specify what she wanted to experience by specifying the sensors, then the application will be uninteresting to most people. What needs to be presented is a semantic interface for selected which part of the event needs to be experienced from which position. That means that a visitor to a past event should be able to specify a specific sub event from a specific position. An example would be, show me the three-pointer scored at the buzzer by Allen Iverson from the position behind him (or from the position of the coach). To do this the system should be able to compute which sensors should be used over which time interval. In some cases, the system may have created a complete attribute space for the event and it will select rendering techniques to present correct subsection of this environment over the selected interval.
In past events, the system could compute “highlights” that may be of interest to users and present those highlights to the user. A user can then select which highlights are of interest and from which viewpoint and could experience those. This may also be extended to preparing a personalized highlight reel for a user.
The database could also have a facet based on participants or objects in the event. Thus, a user could see all plays in which a player X participated. Or one could see only those parts of the wedding in which bride’s sister (or brother) were active.
What is being accomplished is to create this multidimensional database containing time indexed sensory data linked with other information and data assimilated from different sources and provide a user ability to slice and dice this database to access semantically the events of interest. The event experience will be recreated using the sensory data and will be augmented using other data and information sources to provide better than being there experience.
I am sure, I have covered only a subset of features, but this is intended not to be exhaustive but to be suggestive of how past events could be experienced and what should be the goal of systems to provide to users.
This entry about experiencing past events reminded me of a Japanese movie I recently saw titled \”After Life\”. The premise is that after you die, you can take one memory of your previous life with you to the next life, in the form of a video recording that recreates the memory you choose. Though fiction, it details the process people go through to select and recreate a memory. While I do not know anything about cog sci, it is important to establish that an event is the basis for a human memory, because it\’s memories you\’re trying to recreate, not just simple sensory descriptions.