Does Google have a Relevancy Problem?

It is just a matter of interpretation. I get very frustrated with Google, like many others, but the fact is Google can show whatever they want in results.

Every search engine has a relevancy problem that only rears its head on certain queries. No search engine is perfect. I'm sure someone with a little time and interest could find a similar query showing appropriate results on G, MSN, and AJ, but not on Yahoo. So what's the big deal?

I do think that G has a problem, but I think a better test would be to analyze something like some-resort-city hotels for the first 50 slots.

Counterpoint is well-put here: http://www.searchenginejournal.com/index.php?p=1567

Yeah, i don't think it is a particular big deal pleeker, though i really do tend to use Yahoo over G these days - it just seems to work better...

I did think that was a particularly good example though. Thanks for the link RC..

I would be more upset with Google internal decision for handling new websites or web site redesigns with this ridiculous "sandbox" waiting period than their sometimes illogical SERP's.

What I've noticed is I can go to nearly any search engine (even the babies) and find results that are equally useful. Maybe they aren't the same results, but all the algos seem virtually equal now. Sure, for single queries, one might be much better than another, but as a whole, everything is equal. I think that's why all of these guys are focusing on solving the Q&A problem, because what else can keyword search really do besides get less and less relevant with the more SEO sites that get created? I believe this is also why vertical search is getting hotter and hotter, because if a site caters to a niche, you know their results will probably be more relevant. Thoughts?

Maybe they aren't the same results, but all the algos seem virtually equal now.

I agree, at least for more routine/mundane serps that the average JohnQ is pursuing. For the last 6 or 8 months, it seems most of their firepower seems to have been redirected to ancillary services --local, maps, free business sites...

Maybe they aren't the same results, but all the algos seem virtually equal now

I would agree with you for the most part -- maybe routine/mundane queries as rc suggested -- but I just spent too much time poring over real estate related SERPs for a client and I can't call what I saw in Yahoo "useful". Searches for "nameoftown homes" or "nameoftown real estate" are filled with directories and junk like Google was 3 months ago. Most times, only 1-2 of the top 10 listings was an actual local real estate agent or agency site.

Google, OTOH, did much better with about 7-8 listings out of the first 10 being agents or agency sites with actual listings of homes and real estate. Much more relevent, much more useful. And a far cry from how Google was 3 months ago.

No doubt about it, real estate, like hotels, is a polluted category --not for those whose stomachs are easily upset. There has been much speculation about hand-tweaking in those 'money' categories, though I lean toward the "algo overlay" theory myself. At any rate, these battleground categories will probably show the most differences between SEs.

what else can keyword search really do besides get less and less relevant with the more SEO sites that get created?

What's this - text straight from the mouth of some 90s-retro search engine rep?

Sure there's lots of junk stuff in the SERPs, always was (even way before SEO was invented), though I seriously doubt it's all SEO driven. Assuming otherwise is mere hypothetical surmise, and prejudicial to a fault.

In any case, relevant SEO (and don't pretend it doesn't exist!) can actually level the playing field by making content accessible those dumb search engine spiders would never be able to cope with in the first place.

Just a quick note, if we can keep any examples to an "acedemic" or non-competitive nature it would be much appreciated...

thanks!

I would love to see someone take a stab at an independent search engine monitoring service, along the lines of www.nash-equilibrium.com.

I know, search is subject, there would be limitations, etc...

But surely there measures that can be tracked? Start with searches that most people agree has an obvious #1 result. Track a number of "proper name" searches for bigger sites, smaller sites, newer sites, etc.

Maybe also the number of results. Or the number of claimed results vs. the number displayed. Then, the mix of .com, .org., .edu., etc. in the top 1000 results on a variety of queries.

I dunno... perhaps if people can think of more useful trend data, this could be worthwhile.

There are many ways to do pretty accurate tracking of relevancy. I work with some very large European players that have done this for years and trended the data for internal use. And yes, Google is not improving relevancy as much as any of the other majors. Unfortnately this data is not publically available but I've seen some of it. Dosn't look nice for Google, I must say.

Perhaps it's myself being sentimental, but looking back, Google seems fixated on developing brilliant relevancy up until Florida - and afterwards spends its time trying to find different ways to skin the SERPs.

Mike Grehan writes about the goal of search is to produce the little old lady librarian, who finds what you need - but Google's librarian doesn't like to recommend new books, or anything outside of the biggest publishers. It's hard to see that as a good method of operation for any librarian.

As a surfer and SEO, I still think Google has the best relevancy overall - but as a surfer and SEO, I think they are hurting their relevancy more and more with the changes they implement - and providing Yahoo! and MSN with a much lower bar for real competition.

Barry has clarified the other part I have been thinking about, on the subject of user voting:

http://www.seroundtable.com/archives/001806.html

If you try searching msn for gmail, it does not give you gmail.google.com like most of the other search engines.

Are we talking about "relevant" results, or a "specific" result?

Now, many of us have heard Google spokespeople, whether GG in a forum or Matt Cutts at a seminar (the voice doesn't really matter), say that as long as G returns SERPs that are "relevant" to the search, well, they're pretty satisfied; "Specificity" is not a priority.

I remember ?Matt? at one PubCon arguing this point, something along the lines, "People ask for pages with this term, that's what we return."

The days of "I'm feeling lucky" are long gone.

and probably confused by many, at least if the ongoing debates on SE "relevancy" all over the place are anything to go by.

It may also define the limits of what we can expect from search engines within the conceivable future until they've really tackled Artificial Intelligence reliably. Simply delivering pages containing the keyword or search phrase (and more often than not they won't even do that if the links have been calculated by some ranking algo via linkage anchor tags, etc., being the virtual equivalent of mere hearsay) just won't do if you're targeting a modicum of precision (aka specificity). This requires a far more sophisticated "understanding" of content.

Are we talking about "relevant" results, or a "specific" result?

Both, depends on the keyword(s).

It's kind of like the 90's all over again where you used different search engines for different kinds of searches. I still use G primarily, but if I'm looking for something "new" that's not "news" per se, I know Yahoo is a better spot.

One of the areas where I see google being the most wonky is when you search for something with a number like "Alias Episode 3" the Yahoo results are much better. Google does find it but you'll also find lots of "season 3" results mixed in. Additionally it always throws in some bizzare forum posts, thinking the post count, thread number, or date number is relevant. If you scroll down a little you see star wars episode 3 and Harry potter episode 3. Google is just really wierd with number searches.

Maybe it's just that we notice it more, as most of us tend to search very specifically, where the public searches much more generally, and wants broader match.