Google search guru Matt Cutts just happened to be in Seattle the day the company announced its new “Caffeine” Web indexing system that’s intended to provide 50 percent fresher search results.
“Fundamentally the change is as soon as a document gets crawled boom, it gets indexed,” Cutts said at the SMX Advanced search conference, where his audience Q&A is a highlight of the annual event.
Cutts told host Danny Sullivan that the upgrade enables Google to immediately index the results of its Web crawling, rather than updating the index in a staggered process.
“This essentially makes the entire index closer to real time,” he said.
He compared the change to going from a bus to a limo.
“Before you might have waited until a bus came by – if you were the document, you’d wait 20 minutes for the bus to come. Now its like the document walks out the front door and there’s a limo waiting for it.”
Here’s how Google’s announcement explained the change:
Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.
With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index. That means you can find fresher information than ever before — no matter when or where it was published.
Questions from the audience included, is there anything new with HTML5 and Caffeine?
Cutts said HTML5 is “completely unrelated” to caffeine. “But, as part of the switchover to the caffeine indexing system we are looking at ways to improve our HTLM parsing … and might be able to help with HTML5.”
Sullivan asked Cutts how the new interface “that Google stole from Bing” is working out.
“It’s working out well for me as a user,” Cutts said, noting the site’s redesign uses a lot of math to figure out the right information to display in its left column.
“What we have seen in all the testing we have done and the usage is it looks good.”
Sullivan prompted Cutts to talk about Google Buzz, which Cutts compared to Twitter in its early days when people weren’t sure what to do with the service.
“I think there’s this period where after awhile you get hooked,” Cutts said.
For several months Cutts didn’t do much with Buzz but he’s using it more, finding it “fits between something I just want to tweet like a link and a full-fledged blog post,” he said.
The two bantered about Bing here and there.
Mentioning the weather results that Bing returns, showing graphics that can push actual search results below the fold, Cutts said, “It seems like a lot of engines, they just want to show you the pretty stuff.”
“A lot of engines?” Sullivan interjected. “Who’s that? The other two…”
Cutts was also asked whether Google will get rid of the indents in search results, which Sullivan said could get more links higher in results.
“We’ve done tests,” Cutts said, adding that “It’s something we look at” but Google hasn’t yet decided it makes sense to get rid of indents.
The audience was mixed, with some booing at the suggestion indents be removed, sowing seeds of tech’s next great controversy.