User-agent: * # All "spiders" take note Disallow: /nbcnews # This changes every week, why create bad links? Disallow: /guestbook # No need to search this Disallow: /tech # Tech notes are also private # # This file is read by Web crawlers. This is our chance to exclude portions # of our site from the Internet search engines. Why? Some of our links # (example -- NBC News) don't last very long. Other sections are private. #