Whitehouse.gov Submarines Their Robots.txt and Fails to use 301 Redirects
Unless you work in web development you probably have no idea what that title means, but this made me laugh out loud and I'm sure other web geeks will get a kick out of it as well.
If you haven't seen it yet, the White House website was redesigned to usher in the new president, Barack Obama. The design is nice, not overdone, and incorporates many new features including, most notably, a White House blog. However when the web development team flipped the switched on the new website they made a couple mistakes - they nuked the 2,400 line robots.txt file and they failed to implement any redirects for the old pages.
For all the non-web geeks out there, a robots.txt is like a security guard for your website that tells search engines where they can and can not go, what they can and can not include in their indexes. A 301 redirect is like a "We've Moved!" sign on the window of a business, it tells you the new location of that page, but instead of you having to drive across town it will automatically take you there.
Now upon investigation the robots file doesn't include paths to anything super secret, mostly just text versions of regular pages so it's not a huge deal, but they apparently don't care that all the new pages could be indexed by the search engines including useless search results pages. I would say the real blunder here is not redirecting the old pages to alternative or closely matching pages. So that means that every website in the entire internet that linked to any of those pages, approximately 3 million, now has a broken link. Visitors trying to follow those links will be taken to a "Page Not Found" message thereby causing confusion and making them wonder what they may have done wrong to get to this page. Most of them will not take the time to read the nice little note about the brand new site. They also lose any ranking value those old pages may have held.
Proper 301 redirects are basic elements when it comes web development, something most developers learn in the first year of developing websites, but apparently the new government web team has no regard for the basics. This could have been easily fixed with an archives section and pointing all the old web addresses to the new archives pages. Now they're left having to explain why they nuked the official online record of the George Bush administration.
On a side note do we really need four w's at the beginning of the web address? Pretty sure they could've dropped the "www". Then again this crackpot team couldn't get the other basics right, so the possibility is slim to none.
Tags: government clustrfck
