New uses for those robots.txt files

So, I was cruising some SEO forums, trying to read up on some new concepts and topics, when I came across something interesting about robots.txt files. As a SEO site optimization coder and analyst (I’m still very much a “rookie” in the industry), I frequently build on page and XML formatted sitemaps as part of my company’s marketing packages. As I’m sure anyone who knows about this field is concerned, that XML file can come in pretty handy. After reading a discussion forum, I found a topic regarding a fairly recent report regarding robots.txt files. Apparently, in accordance with some proprietery extension for Google, adding a single line of code to the robots.txt files pointing the spider in the direction of the on-page sitemap, effectively eliminates the need to manually submit your site. According to the newsletter cited in the forum thread, this went into effect sometime arround April 11th, and was also mentioned in Matt Cutts‘ blog too. Hey, if it makes my job that much easier, I’m not gonna complain.