All human civilizations share a common goal: the pursuit of knowledge. Most knowledge of the world has its roots in experience – recall Newton’s apple and Archimedes’ bathtub – and is not captured by current knowledge tools. Major advances in civilization have originated from the need to capture, produce, communicate, store, and disseminate the knowledge resulting from human sensory experiences. In our society, literacy is usually considered same as knowledge. Text is visual representation of a subset of sounds created using human vocal chord. Since this was the only technology to make human experience permanent, text has been popular for more than 5000 years, and became dominant after Gutenberg invented moveable print. Text based knowledge revolutionized human society and is behind much progress that we have seen. The WWW is dominated by text. Even photos and videos on WWW are organized and retrieved using tags and keywords. Most experts, including those involved in leading companies like Flickr (of Yahoo) and YouTube (of Google) know the limitations of tags in dealing with other media very well.

Progress in technology has now made it equally easy, or even easier, to capture, store, and disseminate knowledge in audio, photos, and video forms. But, our very limited understanding of the nature of human sensory experiences, has not yet allowed us to deal with non-textual knowledge effectively. Despite realizing the importance of experience, it was not possible to adopt a scientific approach to it due to lack of technology and formalisms. In the recent past, observation of experience in human brain using fMRI and similar approaches and better understanding of mind-body problem by cognitive scientists suggests that time is right to adopt a scientific approach to understand experience and develop formal and computational models for it to make further progress in understanding it and applying it. Another major transformative change in the last decade is the emergence of the Web of Knowledge and its active role in the creation and utilization of knowledge. The Web has demonstrated the power of exploring connections across objects, concepts, disciplines, space and time. By developing formal data and logical models for harnessing knowledge, similar and complementary to textual knowledge, from experiences of people collected using diverse sensing devices ranging from astronomical sensors to phone cameras, we can finally progress beyond Gutenberg Legacy.

Technical Thoughts, Experiential Computing | No Comments »

It is truly amazing how we the scientists behave like a person who has a hammer and (s)he wants to fix all the problems in the world using this hammer. Giver a problem, this person perceives that problem in a way that it becomes the head of a nail so the hammer could be used. Sometimes I feel that we scientists go one step further and if we can not perceive the problem as the head of a nail, we say lets assume that in many cases the problem can be considered the head of the nail.

This is so common that it is not even funny. People learn some tools that could be easily handled on — lets say on a powerful computer — and then they try to solve every problem using these tools. Now that is bad, but still OK. Recently I saw several reviews of several papers, only a small fraction of those had anything to do with me, that really was disappointing. In academia, reviews of papers are important because you could not publish your papers without good reviews. So reviews make or break papers and many times careers. Now the reviews that I called disappointing objected to author’s approach because they did not suggest or use the hammer. The reviewers failed to say why the hammer will be appropriate, they just objected to not using hammer because in their opinion the hammer will improve the solution.

Now it is entirely different that the authors whose paper is considered inadequate may also be using a hammer that is different than the hammer suggested by the reviewer.

It would appear that a nice trick to publish papers maybe to identify popular hammer(s) in a research community and then send your papers to the community that uses hammer similar to the one you like to use.

It may sound funny, but believe me, very few researchers pay attention to the problem that needs to be solved as much as how to handle the hammer and how to hit the head of the nail with it. Of course a good thing is that there are some people who care about solving a problem using approprite tools rather than using inly their favorite hammer.

Technical Thoughts | 1 Comment »

Digital cameras have changed our habits significantly. In early days, we wwaited for ‘kodak moments’ and used our camera thoughtfully. Now we use our camera all the time and take photos generously — photos are becoming our inexpensive secondary memory to preserve our experiences and memories.

A major problem has been how to use all the photos that we take. People take so many photos but how to organize them to retrieve them when needed has become a challenging problem. Flickr, iPhotos, Picasa and such are good starts but lack the flexibility and competence that will help consumers organize these experiences easily for visiting and sharing them. Tags are much hyped so far because who wants to take time to tag? And there is no easy way to tag them.

In computer vision, multimedia, and related communities people realized that most people organize and retrieve photos based on events (time and location). It would be great if automatic techniques could be developed to assign tags automatically to photos when they are loaded on a computer or a web site. SOme research has started in this area. Yesterday Bo Gong defended his doctoral thesis in this area In School of Information and COmputer Sciences on this topic. (Disclosure: I am his advisor.) This work shows how one can take some concrete steps in using context information — coming from EXIF data stored with each picture in modern cameras — in this direction. This is an early work, but is a strong indicator of things that could be done.
A good thing about such research is that it is needed to solve a pain point that the technology has created in the last few years.

Technical Thoughts | 3 Comments »

On May 15th I had knee surgery so it has been exactly one month now. This surgery is definitely a painful experience but as people say and I know first hand because of my left knee, this painful period is very well rewarded later. This is one of those surgery that makes a big difference in your life style.

In one month — well here is the progress. I can now walk with a cane — and without it also but it is a nice safety device to have and use for at least some more time. I can walk in the house without any thoughts of avoiding the walk and outside also at a time I have walked for more than about 200 yards. And that did not cause any serious pain — I was just concerned that I should not abruptly increase this much.

My physical therapist — Michele Straume — is a great therapist. She pushes you to limits that you can handle but does it in a very nice way so you don’t mind it. It has been a nice experience to work with her. She is definitely the best therapist I worked with in my last and this time. She motivates you and you know that you are pushing yourself to recover at your earliest.

I even drove a little bit yesterday. Since this time it is right knee, I need to use it all the time in driving. I drove for about 10-15 minutes and had no difficulty. I was warned not to drive long distances — obviously my knee is not ready to work for long time without building some more stamina.

This possibly was one of the last updates related to knee surgery — unless there is a specific reason. I love to share my experience hoping that somebody who has to go through similar experience may be looking for informtion and real world experiences from normal patients.

General Updates | 3 Comments »

Maybe I am just ignorant of this, but I feel that Multimedia research community does not have resource sites where one could go to find information, data, algorithms, research places, products and such.

If I need to run some experiments in clustering photos of the same event captured by multiple people (say Tom’s Wedding — where 25 cameras were used) — or just a collection of photos with EXIF data and some classification.
Or if I want to use somebody’s published algorithm — yes opencv is a step in a righ direction but is limited and is not that well maintained. I don’t even have an easy way to find all the places that are doing research in multimedia ontologies and such.

Why is it that such sites don’t exist? W have so many professional organizations. Should one of those take lead and build such a site?

Or is it that I feel the need for such a site, but there is no need for this.

Technical Thoughts | 3 Comments »