Ever-widening web

Larry Williams has an interesting article in Baltimore Sun. He starts out with interesting perspective and numbers.

Just when you think you have the Internet figured out, it evolves again, presenting fresh promises and complexity. That’s what’s happening in 2006 with an array of interesting developments. Among them:

An explosion of TV news entertainment and live sports offered free on the Web. Last month, millions of viewers hooked up with CBS SportsLine’s free online broadcast of the NCAA Basketball Tournament.

Some 18,000 computer servers were used to bring the games to the computers of basketball fans around the world. On March 16, the first day of the March Madness playoffs, Web content was provided for more than 102 billion Internet requests, with a peak rate of 2.3 million requests per second, according to executives at Akamai Technologies Inc., a Massachusetts company whose business of streamlining the flow of the basketball games and other content through the increasingly complex Internet is booming.

What is interesting is that this ever-widening gap is changing the complexion of the Web — in early stages of the web, 99+% of the data was text and hence all techniques assumed the data to be in the text form, Now soon — in less than 5 years — 80% data will be non-text and this trend will continue. Currently we are trying to solve all the problems using the same text-based thinking and perspective that was right when the Web started. What I find more intriguing is that some people — whose names I can, but will not, mention — who started out in image, video, and audio domains have started using text-based tools because they are good first step solutions. But they are also dead-end solutions for the long term. I am reminded of the days when people wanted to get to the Moon by climbing tallest mountains.

New challenges require new thinking. Granted, you should not ignore existing approaches, but also you don’t want to be the person with the hammer trying to solve all the problems that you will face using the same (text-based hammer in this case) hammer. I think this is a great chance for entrepreneurs and researchers to invent out of the box techniques for the new major challenges. And, definitely, researchers working in these challenging area also have to think about providing solutions to these emerging problems.

Having been in this area, I feel that just a few years ago, we were developing visual information retrieval technology and were interested in finding problems where we could apply our technology. Now, there are so many problems and these problems are looking for visual information retrieval tools that are not available. We really have a Pain Point, but there is no Pain-Killers available. What is interesting is that this pain is only increasing — with no pain killers in suight.