A post by Joel Spolsky announcing a new site he and Jeff Atwood are building (stackoverflow.com) covers the same theme as my recent Don’t believe everything you read, In the process of describing why he’s doing this, he nailed one of my pet peeves. In this passage, he’s talking about programmers, but he could just as easily be talking about newbie end users or IT professionals or Exchange administrators:
[T]hey happily program away, using trial-and-error. When they can’t figure something out, they type a question into Google.
And sometimes, the first result looks like it’s going to have the answer to their exact question, and they are excited, until they click on the link, and discover that it’s a pay site, and the answer is cloaked or hidden or behind a pay-wall, and you have to buy a membership.
And you won’t even get an expert answer. You’ll get a bunch of responses typed by other programmers like you. Some of the responses will be wrong, some will be right, some may be out of date, and it’s hard to imagine that with the cooperative spirit of the internet this is the best thing we programmers have come up with.
Joel’s too polite to mention the sites he’s talking about, but I’m not. I am sick and tired of looking up an error message or a troubleshooting term and getting taken to a link at Experts-exchange.com (I won’t include a link, because I don’t want to send them the traffic and it would only annoy you to see this in action). When you click on that link, you get to read the question someone asked, which is pretty close to the question you asked, but the answers are cloaked with text that says you have to pay for a membership before you’re allowed to read the replies. And it’s apparent from the structure of the replies that there’s no “expert” involved, just a bunch of other (paid) user/members.
Now, I have no argument with Experts Exchange trying to make a living with this economic model. If it works for them, good. But I do object to Google indexing these results and sending me to sites where the landing page is a teaser rather than the actual content I’m looking for. I’m not sure how to register that complaint, but I do know that the Google spider pays a visit here every so often, so maybe I just did.
(h/t to Dwight Silverman for the pointer)
Ed Bott 
AMEN! I couldn’t agree more. It’s hard enough finding answers to obscure scripting questions without having to do so much manual filtering of results. Thanks for bringing this up!
YES! I despise Experts Exchange. I do tech searches like that often many times a day, and those damn experts-exchange links are always cluttering up the search results with their “that was exactly the problem I was looking for” teasers.
If you click on the “cached” link, you can see all of the answers. So my guess is they give Google’s spider IP access to the site “for free” so it can crawl the content and then everyone else is supposed to pay. I am glad that the cached content is available, as I have been able to find answers there.
Graham, the cached results are cloaked for me, unfortunately. I’m not sure why you’re getting different results, but it’s not a solution in my case.
If you view the cached results from google, you can see the non-garbled answers, which I always thought was cloaking (http://en.wikipedia.org/wiki/Cloaking), but they still get a high page rank.
Still very obnoxious, but sometimes you need an answer…
If you scroll past the 10 pages of ads and scrambled stuff, the answers are usually down at the bottom.
I just use the Google options to remove a domain from the list. If you add
-site:experts-exchange.com
then you will get the results sans anything from them. Pretty soon I will be able to just do my search with
site:stackoverflow.com and not deal with any bogus hits from anyone else 🙂
For I don’t know how long, I’ve been able to see the solutions on experts-exchange.com by scrolling to the bottom of the page. It might be the proxy server I’m behind, but the answers are there without paying for membership or looking at the cached copy.
Hmmm… it’s always worked for me, but I just tested it again and I can’t see the answers now either. Sorry about that. I guess they finally fixed that. I’ve been doing that sort of cache viewing for over a year (even up until just a couple of weeks ago) and never had a problem until now. Bummer.
Yep – I’ve been doing the scroll down function for months – the answers are normally pretty good if you already have some background knowledge to weed out the obviously incorrect ones. What I’d like is the ability to ignore certain domains to appear from the results, customisable per user in my google preferences. That way I could choose to get expertsexchange or not if I was a paying subscriber. I could also remove all of the duplicate websites which are just usenet to web page scrapers.
Jason, Aaron and Andy are right. You have to scroll ALL…THE…WAY… down to the bottom of the page to see the answers. I just don’t get why people post on Experts Exchange at all. Post on Newsgroups, Windows IT Pro, the How To Geek, Neowin, MyITForum or ANYTHING that doesn’t require payment. I’d really like everyone in IT to agree that it makes no sense at all to PAY for a forum that hides the freaking thread at the bottom of 10 pages of garbage.
Aah this has been one of my pet peeves as well.
Dumb me. It never occurred to me that other people had the exact same experience I was having with the so-called expert’s exchange. Although it was a minor aggravation, I always quickly closed the tab without trying to look at the bottom.
Some simple questions seem to take an hour to answer, while a million complicated explanations are always within an arm’s reach.
The more disturbing trend is that outfits like Google could prevent progress from happening. I’m guilty of using Google almost exclusively. Every now and then I’ll force myself to use another search engine but I find them lacking.
It would be nice if there was a search site dedicated to programming questions.
My understanding is that it has to do with cookies. If you delete/reject all experts-exchange cookies you should get the answer (or not!) right at the bottom, but if you have accepted their tracking cookie, you don’t get to see.
(Sorry, but I’m not going to accept their cookie to test the theory!)
Yeah – block their cookies, and you can read the responses. Thanks for the tip. Now to see whether the responses are any good…!
Oh, I use google all the time for my programming needs and have HATED experts-exchange for such a long time! It’s good to see I’m no the only one.
They’re actually breaking some of Google’s SEO practices that could have all their rankings and indexing removed (check out http://www.prchecker.info/check_page_rank.php). They have 6 out of 10 which aint bad… I hope google puts it down to zero!
Ed,
It sounds as if you and your fellow Windows experts have a gripe with a particular site. However there are many subscription websites that provide very useful knowledge and expertise to their communities. There are many sites worth paying for because they provide unique, actionable and valuable information, frequently updated, from trusted sources. We know because at SubHub we provide the publishing platform for many of these sites and we see the communities that have grown around them.
There is no reason why these sites should not be findable via Google for those who would find them relevant. If Google were to remove them from its index then Google itself would be providing an incomplete picture of available information sources, and would be failing at its mission of indexing all of the world’s information — not just all of the world’s free information.
What is interesting is that despite the complaints your commenters seem to have about the specific site in question, many of the comments then go on to explain how to game the system using caching or blocking of cookies in order to get the information from the site for free. So it’s not that the information is not valuable, it’s that some people don’t want to pay for it.
I will leave it to others to comment on the ethics of doing this, but it strikes me as a bit dubious. If you don’t believe in paying for the information, fine, but don’t steal it. I imagine that those who are programmers would object if someone swiped their intellectual property — their code — so why would they be comfortable swiping someone else’s intellectual property — their content?
Kind regards,
Evan Rudowski
SubHub
Evan, paid sites are not normally indexed in Google. Go search for articles in the NY Times or in the EDGAR database, for example. They’re not there, nor should they be.
The reason that this one is in the Google index is because they’re misleading the search engine spider (and site visitors) by hiding the content at the bottom of the page, where most people won’t see it. That’s sleazy, in my opinion.
Ed,
Thanks for replying. What our sites do is use an article teaser to summarize the full article. The teaser gets indexed by Google (and also comprises the contents of the site’s RSS feed). Then when people click through from Google (or the RSS feed) to the page, they are asked to log-in, or register and pay, to see the full item.
Nobody is fooling Google, but this will be equally irritating to anyone who expects to see the full content for free.
I think the principle is the same; if someone feels the content is valuable, they should pay for it. If not, that’s fine too. It’s up to the site owner to make the content valuable and relevant enough to be worth paying for.
Kind regards,
Evan Rudowski
SubHub
Evan. we’ll have to agree to disagree. I go to a search engine to lead me to answers, not to be teased. And Experts Exchange, which uses your products, is one of the worst offenders. They’re wasting my time and my attention, and I’m tired of it.
Sorry, but your economic model fails for me.
Ed, thanks again for your reply. In our opinion, there’s no reason why paid content ought not to show up in results when it is relevant to the search. So we will have to agree to disagree.
If Google hypothetically offered you a chance to fund an account which would be debited each time you viewed paid content, but would then take you directly to the content from the search result, would you sign up for it? In other words, is your objection merely to the intervening page that is in the way of the paid content, or do you just flat out object to paying for content on the web? I wonder how some of your other commenters would answer.
Just to correct a point, Experts Exchange is not a customer of ours and never has been. If they were using our solution, it would not be possible to read their full content in the cache or by disabling cookies! 😉
Best wishes,
Evan Rudowski
SubHub
I was getting pretty tired of Experts-Exchange after the google cache trick stopped working for me. On a whim, I decided to log on to Experts-Exchange with an account I registered back in 2000 when it was free. Amazingly it still worked and I’ve been able to see the answers without paying.
“I think the principle is the same; if someone feels the content is valuable, they should pay for it. If not, that’s fine too. It’s up to the site owner to make the content valuable and relevant enough to be worth paying for.”
ExpertsExchange has absolutely no right to charge people to access answers that include information published free by Microsoft and others. I certainly am extremely displeased to find one of my articles linked from those scam artists.
I am slightly disappointed that experts-exchange now include a hyphen in their URL. They never used to and it was one of the most unintentionally funny things in existence.
Expert-Exchange? sounds like a st. Google should remove it permanently. even some hacking sites such Astalavista are much better helpful than that st
Sorry, missed one: to Evan Rudowski, his company and also E-E, do not ever play with many people like me for taking advantage by abusing free google search engine as your self interest. however your tricky way is so dirty and not deserve any reward at all. shame on you dude
Yes, absolutely, Google should seriously consider this. At minimum, the results should be very negatively weighted. I agree however, any site that forces you to pay for garbage should NOT be included in “PUBLIC SEARCH RESULTS!!!” get rid of that crap! c’mon google, you in cahoots? Don’t use auto-login uid/passwords to crawl sites! this is a simple rule and some bonehead has convinced you otherwise. Fire that guy and dump that behavior for “HEAVEN’S SAKE!”