Twing: Searching the Deep Web

The term “Deep Web” is shorthand for “stuff Google doesn’t bother spidering,” thus keeping it essentially invisible to the majority of web users, who use Brother Google as their sole guide to the web. A new search service named Twing shines a light into one vast, overlooked and easily derided shaft of the Deep Web: discussion forums.

Yes, we’re talking about those retro-web gathering spots, many designed during the early years of the Clinton Administration–back when blogs, social bookmarking and YouFaceMyTubeSpaceBook rumpus rooms were barely gleams in a delusional VC’s eye. Forums have the visual elegance of parking tickets. You often feel like you need a red-tipped cane to navigate them. Some even have little spinning logos.

And so I was startled to discover how many of these forums are going strong.

On Twing’s list of active forums I came across the following, all active as of this week and (per Twing’s calculations) growing:

• 3.2 million posts and 37,000 members in the Long Hair Care forum
• 2.5 million posts and 23,000 users in the Arofanatics forum (aquarium fish)
• 2 million posts and over 30,000 users in the OffshoreOnly forum (power boating)

Of course there are huge numbers of forums devoted to cars, guns, pets and games. And celebrities.

But let me remind you how ugly, dysfunctional and user-indifferent this forum software is. Here’s what the Long Hair Care forum looks like:

A good question is: Under what conditions might you want to use Twing as a search engine?

The answer, per Scott Germaise, Twing’s Director of Product Management [I'm paraphrasing from his e-mail response to my questions]: Whenever you’re looking not for information about the thing, but for conversations about the thing. I did a bunch of searches and can report that for many topics [one I tried: Bolshevism], Google returns basic high-value content pages. Twing delivers odd little snatches of conversation–people are still discussing Bolshevism!–that occur in active forums.

Google often surfaced active forums among search results. Twing delivered those only, and more of them.

Messing around with Twing reminded me of the early days of Yahoo, when I could spend hours muttering to myself, “who are these freaking people?” as the world’s remarkable obscurities slid across the screen. It also illustrated how. . .burnished and predictable and gamed the web has become. A lot of the loopiest, most strangely human content has sunk from view.

I have no idea whether there is a business here, or whether I will use Twing for any real-life searches that GooHoo can’t handle.

But Twing has shown me this: That whole creating communities-of-shared-interests thing? That whole power-of-collective wisdom thing? That whole long tail thing? You know, that whole web 2.0 thing?

It’s been under our noses all along. We just forgot where to look