Resubmitting changed sitemaps to search engines

Sitemaps are a great way of submitting the content in your site to search engines. The only problem is that if you don't want to wait for them to find the changes you've made to your sitemaps themselves, you need to do the work of submitting them.

Seeing as I'm lazy I figured I'd write something that would do this for me. It's just a small python script that accesses the sitemap ping URL for each search engine and checks that they output the right response.

If it doesn't get the expected response it will try again, but only once. I may also update it so that it caches a copy of each sitemap and only resubmits them if the current copy has changed since it last resubmitted it.

The easiest way to use it is to add it to a cron job and have it run once a week or however often you expect to update your site.

Here's the script:

search_sitemap_update

#!/usr/bin/env python
# Search Sitemap Resubmission
# Resubmits changed sitemaps to Google, Yahoo!, Ask, and MS Live Search
import urllib
import time
from urlparse import urlparse

#List your sitemaps here.
sitemaps = ['http://domain1.org/sitemap.xml', \ 
'http://domain2.org/sitemap.xml', \
'etc.']

pingurls = [['http://webmaster.live.com/webmaster/ping.aspx?siteMap=', \ 
'Thanks for submitting your sitemap.'], \ 
['http://submissions.ask.com/ping?sitemap=', \ 
'Sitemap submission successful'], \ 
['http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=', \ 
'Update notification has successfully submitted.'], \ 
['http://www.google.com/webmasters/sitemaps/ping?sitemap=', \ 
'Sitemap Notification Received'], \
['http://api.moreover.com/ping?u=', \ 
'Thank you for your ping.']]

for pingurl in pingurls:
   print "Starting submissions to "  urlparse(pingurl[0])[1] + "."   
   for sitemap in sitemaps:
      print "Resubmitting sitemap " + sitemap + " to " +  \ 
      urlparse(pingurl[0])[1] + "."
      response = str(urllib.urlopen(pingurl[0] + sitemap).read())
      if response.find(pingurl[1]) == -1:
         print "There was a problem. Will try again in 1 minute."
         time.sleep(60)
         response2 = str(urllib.urlopen(pingurl[0] + sitemap).read())
         if response2.find(pingurl[1]) == -1:
            print "Hmmm. Doesn't seem to have worked again."
            print "I'm not trying again though."
         else:
            print "It seems to have worked this time."
            print "Sitemap " + sitemap + " successfully resubmitted \ 
            to " + urlparse(pingurl[0])[1] + ".\n"      
      else:
         print "Sitemap " + sitemap + " successfully resubmitted \ 
         to " + urlparse(pingurl[0])[1] + "."
   print "Finished submissions to " + urlparse(pingurl[0])[1] + ".\n"
print "Finished sitemap submissions."