Do you run rank checkers?

Yes but I run home grown ones that are well behaved and hide their signatures.

Yeah - Until Google decides it's worth their time to provide RSS results (even for a fee), we'll have to keep scraping privately, quietely, under the radar. There's no substitute for that kind of information, and a professional SEO really can't operate without knowing what's going on in the SERPs day by day.

Yep RSS SERPs from G would be most excellent.

Graywolf, why do you want RSS serps so badly? I doubt anyone would put Google serps in to their reader, at least not for long. To derive any value you have to analyse the serps with software, and RSS is no easier to scrape than any other of the XML outputs Google produces. Perhaps a guaranteed format would be nice, but I've been scraping Google for a while now, and some of the output formats have never changed.

FAO John Andrews: I would be interested in reading the exchange between you and Google over WPG, or perhaps a summary of their objections if you would be so kind?

Graywolf, why do you want RSS serps so badly?

IMHO it's just easier to run something like bloglines against the RSS of a SERP so you know when something changes, instead of scheduling programs to run on demand or a specified intervals. I have about 40 or so RSS news feeds from Google, and about 30 or so RSS SERPS from MSN.

it was long ago... I don't have the old emails. They simply said I was banned for running programs they didn't allow. When I pressed, and finally listed WP as a possibility, the guy said "yeah, that's one of them". He "made me" write an email saying I would never run it again, in order to get un-banned. It was a /29 network they banned (4 IPs), which represented our corporate DSL connections to the Internet at the time.

That sounds like Ray, alright. John, if you had seen the amount of queries we were getting from rank checkers and WPG back then, well, you might not agree, but our response would make a lot more sense.

Ahh, I remember those goldenrod-colored reports though..

Thanks Matt. Nothing like having people with long memories around to keep things honest ;-)

Since this seems so nostalgic, it might be fun to dig out that old email exchange... I'm not as committed to "organizing the world's information" as Google but I do know which DAT tape it is on :-)

What I recall most was the severity of the action especially given that WP had hardly been used at that time. Lucky for me the organization was far from "hi tech" and only a handful of people used Google besides the IT department. It also prompted me to assign someone to border gateway protocol (just emerging back then) to set up multi-homing. Competition is good :-)

Actually, Matt, why didn't Google see an opportunity in all that rank-checking traffic back then (or through modern times)?

I had the same problem when I first got my cable internet connection. I used to run one client's report each night, perhaps checking on 50 or 60 keyword phrases. One day I was forbidden to search at Google, as were all the computers networked in my house.

I emailed Google about it, and in a few days they emailed back that it was because I was running automated queries on them. They said if I promised not to do it any more, and I could give them my IP address they would unblock me. At the time, I had no idea how to even find out my IP address, and they provided me with some site that would tell it to me. But that site didn't seem to work, and it took me a week or so of back and forth with them to get it to them.

But once I did, they unblocked me, and I started checking rankings at AOL for my clients instead of Google. I've also tried to spread the word to others that checking rankings through automated means at Google was against their terms of service (whether you get caught or not).

Regarding the new version of WPG...I got the same email about my old version no longer going to be supported. I was debating whether I should get it or not as I don't do many ranking reports anymore. I do have a couple of legacy clients that I do a monthly report on, and I do often try to do an initial ranking report for clients I consult with. (Don't often have access to logs and stuff.) But I know it's not something I really need to provide, and I am mostly getting away from it.

In the end, mostly out of nostalgia, I decided to buy the new version. I like that I can put in my API key and actually get Google rankings now if I want also. Plus, it's the end of the tax year and I need as many deductions as I can get this year! (That was mostly my deciding factor!)

What about the most excellent rank checking tools that run with Google's API to check ranks? Aren't those legit?

Funny like John and Jill, I had WPG-2, and I have upgraded to WPG-4 recently.

Reason I upgraded, was that I tried and reviewed WebCEO and did not like that at all. I went back and dragged by old copy of WPG-2 out of retirementard, and decided to upgrade. As Jill says, the upgrade ain't that expensive.

Bit like members of the Antiquarians Society meeting up again.

Isn't the Google API using a search catalog that is older than the catalog you search in when you search from the www.google.com interface ?

for the same reason I am happy to sit impromptu at a client site (screen projected for all to see) and run queries against G or Y! or whomever. It's reality that counts, and the Google API is not reality.

First, it says "hey Google... this person us checking rankings on these terms/this website". How "typical" is that user behavior?

Second, as Dave suggested, if G has concerns about loading from rank-checking, why leave it against the live data centers? Seems obvious to have the API hit excess capacity (or planned capacity). I never inspected it to check.

Third, why expose yet another IP/subnet every time you hit via the API? Do something rom home... IP logged. From the office, IP logged. From the branch office, IP logged. From the client site...from Starbucks.... IP's logged. Pretty soon G is organizing the world's informaton about me and my travels. Not like I care, but G seems to care :-)

I don't get the bit about deductions, Jill. No matter how you slice it, you are paying upwards of XX% of that invoice...deductions are great because Uncle Sam pays a portion, but to seek out deductions? Maybe I didn't make enough last year to understand how that works ;-) Anyway, sounds like it's still a "free money" purchase.

Well, the way I understand it, since Uncle Sam is basically taking half my earnings these days, everything I buy as a business expense is like buying it for half price. (Not sure if that's exactly how it works, but that's how my husband has explained it to me.)

So if it's something I kinda sorta want, it feels better knowing it's half price. And I never know if next year I'll be in as high of a tax bracket (although I see no reason why I wouldn't be!)