Jenstar

Yahoo Search Marketing Bot goes wild with 83,000 visits to single page in 24 hours

As many publishers know, the Yahoo Search Marketing bot – also known by the “YahooYSMcm/2.0.0” user agent – is responsible for visiting publisher pages in order to provide targeted ads. However, after investigating a massive spike in my bandwidth for JenSense from June 14th and 15th, I discovered that the YSM bot apparently really liked my blog entry from last summer comparing the Yahoo Publisher Network and Google AdSense. So much so that it decided to visit 83,000 times over a 24 hour period (well, 23 hours and 38 minutes, to be exact).

Yes, that is right, the YSM bot came as often as three times a second, but mostly every second. The longest break it seemed to take once it hit its groove was simply leaving a single second between visits. And it made for a grand total of 82985 visits by this hungry bot. And that makes for a nasty 3 GB of bandwidth that this bot used just on this single page.

It first showed up at 6:01 am PST on June 14th and the non-stop visits continued through to 5:39 am PST the following day.

It is worth noting that the visits came from three separate IPs – 63.163.102.180, 63.163.102.181 and 63.163.102.182, meaning that it wasn’t a single bot having a bad day, but it was three of them. And those bots do trace back to Yahoo Search Marketing (as Overture Services).

Here is a bit of the raw logs a minute or so before it stopped (logs are EST):

63.163.102.180 – – [15/Jun/2006:08:38:40 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:41 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:41 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.182 – – [15/Jun/2006:08:38:42 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:43 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.182 – – [15/Jun/2006:08:38:45 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:45 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.182 – – [15/Jun/2006:08:38:46 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:47 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:47 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:48 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:50 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.182 – – [15/Jun/2006:08:38:51 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:52 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:52 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:53 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:54 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:56 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:56 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:57 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:58 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.181 – – [15/Jun/2006:08:38:58 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

63.163.102.180 – – [15/Jun/2006:08:38:59 -0400] “GET /archives/2005/08/womensfinanceco.html HTTP/1.1” 200 33712 “-” “YahooYSMcm/2.0.0”

I do have to wonder how no one from Yahoo could have noticed that the bots were going crazy on this particular page, especially on a page that generally only has a handful of page views per day. I never heard anything about this from Yahoo, I only discovered it when investigating the source of the massive traffic spike.

However, it is possible they noticed it, realized whose site they did it to, said a few colorful adjectives, and then crossed their fingers and hoped I wouldn’t notice 😉 If I were in their shoes, I would probably do the same!

However, this problem could definitely affect publishers who might not have the money or hosting arrangement to allow for the increased bandwidth the YSM bot used. Hopefully this was an isolated incident and I was the (un)lucky one to be on the receiving end of it.

Share this with others!
  • Twitter
  • Digg
  • Sphinn
  • StumbleUpon
  • del.icio.us
  • Reddit
  • Technorati
  • Mixx
  • Google Bookmarks
  • Facebook

2 comments to Yahoo Search Marketing Bot goes wild with 83,000 visits to single page in 24 hours

  • I’ve blocked or throttled various Yahoo bots on my sites because of incidents like this that have happened to me. Not nice.

  • Another reason that I am glad when YPN gave me a hard time over the phone back in its early days, I dumped them. Would my few tiny blogs have gained a few cents had I submitted to their rudeness? Perhaps, but I’m currently YPN free and glad of it. Google treats me as a publisher/partner, not a doormat.

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>