TripAdvisor.com and the Wisdom of the Clowns

TripAdvisor.com, the aggregator of travelers’ opinions about hotels, resorts and such, has been posting the following warning on some hotel profiles.

This is an encouraging move.

Most people who have contributed or used web reviews of hotels, restaurants and books can smell a rat–or at least suspect one is usually around.

Wisdom-of-the-crowds opinion aggregators are laughably simple to game. How many e-mails have you received from your favorite restaurants begging for a vote in the annual “Best Of” web poll?

As this issue has circulated through the online travel sphere, things have heated up. In response to a few broader challenges to Trip Advisor’s overall integrity, the company’s chief Twitteur, April Robb, issued the following statement, which appeared on the blog Elliot.org:

We believe our nearly 25 million reviews and opinions are authentic, honest and unbiased, from real travelers, which is why we enjoy tremendous user loyalty. Also, the sheer volume of reviews we have for an individual property allows travelers to base their decisions on the opinions of many.

The integrity of TripAdvisor reviews is protected by three primary methods:

1. Every review is screened prior to posting and a team of quality assurance specialists investigate suspicious reviews

2. Proprietary automated tools help identify attempts to subvert the system

3. Our large and passionate community of more than 25 million monthly visitors help screen our content and report suspicious activity

When a review is suspected to be fraudulent, it is immediately taken down and we have measures to penalize businesses for attempts to game the system. Penalties are handled on a case by case basis.

Well, it’s hard not to smell a rat there too. “Every” review is pre-screened and a team of QA specialists “investigate”s suspicious reviews?

That’s a hell of a workload. Assume it takes 15 seconds to eyeball each of the 25 million reviews Trip Advisor says it has. This means that its screeners have spent. . .let’s see…4,340 around-the-clock days, or about 12 years of constant labor, vetting reviews since the site launched.

And that’s before the QA specialists step in to investigate the suspicious ones!

Even if this work is being done in Sri Lanka, that’s still a pretty high “contractor expense” in the ol’ budget.

You can read an excellent report on the Trip Advisor controversy at Elliot.org, the best-of-breed blog by online travel journalist Chris Elliot.

Also check out the 43 comments on his entry. It’s hard to tell without a dedicated team of QA specialists, but damn, I think I smell a rat there too.

It looks to me like several of the comments defending Trip Advisor’s integrity come from. . .wait for it. . .people with some undisclosed relationship with Trip Advisor.

InauguRate09: The Tweeple’s Balls

There were so many Tweets flying over D.C. for the past few days it’s surprising the Air Force didn’t scramble the F-15s to secure the air space. I won’t try to aggregate all the aggregations of Inaugural Tweets here. But this project caught my eye:

The InauguRate09 widget [note to lawyers: copyright that one for 2012!] was created by Thummit, a D.C. based ratings-and-recommendation service currently in beta.

It harvested Tweets about various inaugural events and did basic semantic analysis–parsing the language to determine whether a Tweet describing that event rated it a “thumbs up,” “thumbs down,” or something in between. It tallied the data and came up with the rankings above.

I suppose it’s not much of a surprise that the We Are One concert on the mall ranked higher than the Mid-Atlantic Ball ["So nice to see you, Madame State Senator!"].

But if the Thummit data can be trusted, Google’s hullabaloo was more fun than Al Gore’s Green fling–and both were rated higher than the high-gloss soiree thrown by perpetually reluctant interviewee Arianna Huffington.

This is all fun stuff, and a good window into how the Twitter ecosystem is being used by third-party developers to tell lots of different stories. Thummit allows users to rate local restaurants via mobile devices and the web, but it is working on other projects like this that parse other public commentary into yea/nay evaluations.

Just to check Thummit’s analytic work, I dug into the Tweetstream to check out that bottom-ranking “official” Youth Ball. It didn’t take much semantic analysis on my part to verify which way the thumbs were indeed pointed:

lol youth ball.what a joke. from twitter on 2009-01-21T06:40:36Z

Youth Ball FAIL. I got yelled at by cops and wasn’t allowed to see obama or kanye. Ditto for about 1000 others who paid $75 for tix. from twitter on 2009-01-21T06:28:51Z

youth ball, huh? is there a not quite middle age ball? from twitter on 2009-01-21T06:08:15Z

Disproof of Concept: Change.org’s Ideas for Change

Change.org is a social network where people who care about a variety of causes–global warming, hunger, gay rights, animal rights, worker rights and a bewildering range of others–find kindred spirits, action groups and all sorts of information. It also accepts donations on behalf of the non-profits in its network.

As a platform to help people connect and collaborate, it’s a classic–and admirable–use of social media.

The group’s Ideas for Change campaign, however, is another story. It’s a ripe demonstration of what happens when a “wisdom of the crowds” effort is overtaken by activists whose real agenda is self-promotion, not the public interest.

Here’s how Ideas for Change works: The site invited users to suggest ideas for change [over 7,700 submissions] which were then reduced [via over 280,000 votes] to a “short” list of 90. In round two, each site visitor is allowed 10 votes to distribute among the causes he or she feels most important. The final list of Top 10 causes will be “delivered” to the Obama Administration and Congress via a press conference at the National Press Club.

I suspect you know where this is heading.

As of this writing, the top two causes on Ideas for Change are. . .to legalize marijuana and end the war on drugs. Number 13 offers what could charitably be considered a slight variant, “decriminalizing” marijuana.

To be fair, other top vote-getters involve matters more central to the public conversation: the Patriot Act, marriage equality, universal healthcare and green initiatives. [All suggestions would likely be considered "liberal" responses to the issues. Nothing wrong with that.]

But there at Number 9 stands “Save Small Business from CPSIA“–a plea to free small toymakers from the strangling yoke of the Consumer Product Safety Improvement Act. [According to the site, the 2007 law is designed to ensure the products of large toymakers are certified free of lead, phthalates and other dangerous stuff, yet apparently captures small toymakers in its regulatory net.]

Number 9 is certainly a legitimate matter. So is the idea of decriminalizing weed.

But the thought that the tens of thousands of activists who participate in America’s civic life believe these to be among the most important issues to bring before the administration and Congress is fatuous.

I imagine the leaders of Change.org having a hard time presenting that list of issues, so redolent with pot smoke, at the National Press Club without humiliation. Yes, Mr. President, activist America has spoken, and we want our bongs back, dammit! And get the hell of small toymakers’ backs!

The explanation for this unfortunate outcome is so obvious I can hardly bring myself to type the words: A small number of activists organized their followers to vote for their cause without regard to what they really think are the most important issues facing America.

As the daily results of Digg have demonstrated over the years, opinion-aggregation sites can be gamed, and usually are. [Top Digg'd item of the last 30 days, as I write this: Digg this if your tired of power users stealing stories.] Ditto sites that review restaurants, hotels and fashion.

Applying a game-able system to serious civic matters just isn’t very wise-as the Obama Administration’s effort of an eerily similar name, Change.gov, is beginning to discover. [Change.org had its name first!]

Let’s all agree to this: There are many ways to use new media to involve citizens in the process of change. “Voting for your favorites” is not one of them.

Okay, so what can be done at this point to save Change.org from itself?

Why, game the system some more, of course!

Activists who really do care about improving the lot of this dysfunctional nation–or at least want to spare a worthy effort like Change.org public humiliation–should go to Change.org and vote in an intellectually honest manner.

Having a hard time? Look way at the bottom: “Create a permanent constituency to end genocide” needs some love. Only 37 votes so far.

NewsTrust: Wisdom of the . . .Few

From where I sit, NewsTrust is like Digg for thinking people–a way for an engaged, attentive usership to surface high quality journalism on a website for the world to see. Great premise. Solid execution. [Excellent usability!]

But. . .there are precious few contributors. The story that has the most user ratings–a widely-renowned, much circulated, 13-screen [!] New Yorker opus titled The Joshua Generation by editor David Remnick–has been reviewed 12 times. Most stories on NewsTrust have between 2 and 4 reviews.

NewsTrust is a non-profit organization, funded largely by grants and donations from, among others, the MacArthur Foundation. Its advisers and supporters include esteemed members of the digitelligencia.  It intends to become a self-sustaining business at some point. [Interest revealed: I used to work for PBS Engage, a NewsTrust partner, and have exchanged friendly e-mails with NT founder and digital pioneer Fabrice Florin.]

I’m sure there are ways to read the metrics–page views, stories reviewed, etc.–that make a case for the project’s value and success. I do not question that.

My interest is less with NewsTrust than with how it illustrates what I have come to consider an inviolable law of social networks: People will participate in a network only if they get more from it than they are required to give. Participation is premised on an implicit exchange of value, with the user coming out on top.

This may seem obvious–newsflash! people act in their own self-interest!. But many social networks fail to abide this rule–and fail.

Here’s a cost-vs.-benefit rundown on NewsTrust.

I submit a news story I think is particularly good. [This is easier when it's published on a NewsTrust partner site, such as The Washington Post, which includes a NewsTrust button along with the other save-and-share options such as del.icio.us, Digg, etc. ]

I rate the story, according to 6 criteria (”quick” review”), 10 criteria (”full”) or 13 (”advanced”). Ratings are from 1 to 5.

Doing this responsibly requires me to go back and re-read it carefully, to rate aspects like “sourcing” or “fairness” or completeness, which I probably hadn’t really thought about when I decided the story was really good. [I can skip anything other than a quick 1-to-5 quality rating if I like.]

Then I submit it.

My review gains value only if more than three other people rate the story I’ve submitted–and even then hardly rises to prominence where it can have much influence on site users, much less the broad reading public.

My contribution? At least 5 minutes of high-grade intellectual energy. [More like an hour if I actually read that whole damn Remnick thing online. I'll print it out. It sounds really good.] Another surfing distraction that keeps me from my paying work. A little more swelling in the ol’ carpal tunnel.

The value I’ve gotten back? Hmmm. . .a hope that a few people will read a story I think is unusually worthy? The mild satisfaction one gets from inflicting one’s opinion into a public forum? The recognition I get if I do a lot of reviews and people review my reviews highly?

Even though I’m a civic-minded guy–plus a former newshand who believes strongly in the public value of high-quality journalism–that’s too low a reward-to-effort ratio to earn my regular participation. I feel like I’m being asked to do a lot of homework for a class I’m just auditing.

A thriving community where thousands of people elevate the best journalism to the public’s eye would be an enormous national benefit. I hope NewsTrust can identify people who get the psychic payback they’d need to contribute regularly. Or that it can somehow alter the value proposition to the user’s benefit.

Maybe I’m too close to this situation to be objective.

As it happens, I am a contributor to a non-profit online effort called HealthNewsReview.  We rank health and medical news stories according to 10 criteria similar to NewsTrust’s. We also have a hard time getting users to offer their opinions on our bulletin board. We also struggle to build audience for this important effort beyond a core of folks who follow medical news very closely.

So why do I devote the time and energy to writing these long and difficult health and medical news reviews?

I wish I could say I made these contributions selflessly, in the name of the public good.

But, well. . .they pay me to do them.

Crowdsourcing a Restaurant

Fascinating story in the Washington Post yesterday [I still get the Sunday "hard copy" of the newspaper] about a Washington, D.C. group that is crowdsourcing a new restaurant.

Web 2.know-it-alls may sniff at calling this project “crowdsourcing” at all. But it’s an effort to harvest the best ideas of a group of enthusiasts and build a restaurant based on all that group input. The article claims it’s the first use of the crowdsourcing technique to build a restaurant. [I did a Google search and by that undependable measure the claim appears to be true.]

Linda Welch, 49, a serial entrepreneur, had gathered 386 Elements community members who have, the article reports, “helped develop the concept (a sustainable vegetarian/raw foods restaurant), the look (a comfortable gathering space with an open kitchen), the logo (a bouquet of colorful leaves) and even the name [Elements].”

“Most businesses are started because you have a great idea, and you take it out to the public to see if they like it,” Welch is quoted in the Post story. “This is the opposite. We’re finding out what people want and doing it.”

As for the genesis: The article continues,

“The Elements project began in February 2007 when Welch [49], who owns area several businesses in the District, purchased the business and liquor licenses of nearby Sparky’s, a coffee shop that had closed. Welch has helped launch 22 startups but has no restaurant experience. She didn’t know exactly what she planned to do with the licenses, other than open a small cafe. Around that time, Neil Takemoto, 40, another local entrepreneur who had worked with Welch, stopped by to chat. When Welch told him about her plans, Takemoto suggested crowdsourcing the restaurant.

“‘I said, ‘Great!’ ” Welch remembers. ” ‘What the hell is that?’ ‘”

Takemoto runs a business, CoolTown Studios, that helps companies use crowdsourcing and other social media techniques to support community development.

Here’s a schematic illustrating the collective developement process from the site his company created to support Elements:

The Elements project is a fascinating attempt at a proof-of-concept using “wisdom of the crowds” to build a real-life, carbon-based business from the ground up.

It’ll also be interesting to see what happens now that the effort has been publicized beyond the core group of enthusiasts and supporters. Since the article has appeared, about 30 people have signed up.

What happens to the wisdom ofthe crowds–and the value of their advice–as the crowd expands? Crowdsourcing theory says things will get better, as greater collective intelligence is tapped.

We’ll see. I, for one, am looking forward to the opening, sometime next year. Process is good. Product is vital for a restaurant.

Which is to say: I sure hope the food’s good.

iMedix: Social Search that Creeps Me Out

Oh, geez. Deb21 wants to chat again.

Here I am, trying to look up some information about tinnitus–a k a ringing in the ears, a condition which has recently afflicted a member of my family–and Deb21 [I've changed her handle to protect the innocent ] wants to chat. A little photo box pops up on my screen, with the icky solicitation “I’m online! Chat with me now!” There’s even an audible little ping whenever she implores me to spend some time with her.

Welcome to iMedix, a “social search” site in the personal health space.

In concept, social search is powerful: Combine the algorithmically valid but brain-dead health search results of a typical search engine with the “wisdom of the crowds”–the aggregated opinions of real humans who can validate the information they found worthwhile when dealing with the same issue. Add to that the ability to connect with those people, and (goes the theory) you’ve got something good.

Like any 2.0 community, iMedix faces the challenge of creating critical mass: A community with nobody home is in a death spiral from Day One. But building critical mass from scratch is no small task in mid-2008: Early adopters are oversubscribed to social networks and the mainstream hasn’t figured out what all the fuss is about. Every business based on network power needs people. A lot of them. Fast.

Which brings us back to Deb21. iMedix seems to be trying a bit too hard to get people to join the party, dispatching its youthful crowd to flag folks into the front door.

First it was Ann, a comely 29-year-old community manager interested in fitness and lifestyle. I acquiesced to her friend request but haven’t heard from her since.

I accepted friendship with a fellow calling himself neurosurgeon_55, figuring it’s never a bad idea to know a brain surgeon. But then I discovered he’s a 17-year-old guy in India, whose personal statement reads, in part:

Then we will ve a lots of chat (humourous)but valuable beniffitng both of us in the long run so what r u thinking of? Hmmmmmmmm..lets go ahead and chat.Yo man!!

An unsettling number of people who have set up profiles in iMedix are attractive and young and look, at least to these middle-aged eyes, like the happy-go-lucky group with cool haircuts and great teeth you see in ads for premium liquors.

Here is the problem: People with health problems have, well. . .health problems. They want to see that people like them, people who have something valuable to share, are in a community.

You will certainly find these people at iMedix: There’s a 53-year-old woman whose college age daughter has bipolar and is an abusive relationship. Good lord, the woman needs help. Call me too fast to judgment, but I don’t think neurosurgeon_55 is the guy to offer her support and guidance.

To be fair: iMedix is in beta. It appears they’ve seeded the site with the folks they have around–their young staff and (it appears) their social network contacts.

Building a 2.0 health community is hard. Not many people have gotten it right, and the very concept is fraught with danger. But social networks are based on the company they keep. And no matter who that company is, in the health space I’m not sure they should jump onto your screen saying “I’m online! Chat with me now!”

As for the search part of the social search: The information on tinnitus was really pretty good, better than what Brother Google served up on page one. Link number one was a direct hit.

Along the way I found the profile of someone named Niroo. She is 24 and says she has hearing loss and is interested in tinnitus. She lives in Iran. I sent her an e-mail. Haven’t heard from her yet. [#]

Punchline, added 5/19/2008

Seven days after writing the entry above, I received the following message in my iMedix mailbox.

Dearest One,
My name is Miss Ashandy,i am a single girl never marrie i saw your profile today at (www.imedix.com)Ashandy100@yahoo.com) and became intrested in you,i will also like to know you the more,and i want you to send an email to my email address so i can give you my picture for (i believe we can move from here.
I am waiting for your mail to my email address above.Miss Ashandy (Remeber the distance or colour does not matter but love matters alot in life
Yours Lover
Miss Ashandy Rolland

Next Page »