Effective Measures To Save Bandwidth

For some websites short of bandwidth, frequent visits by search engine crawlers can be problematic since they can disrupt revenue generating visitor traffic by choking the limited bandwidth. For this, search engines recommend enabling If-Modified-Since HTTP header on your server. This is particularly true of websites which change their existing pages quite rarely. This feature allows your web server to tell SEs whether your content has changed since we last crawled your site.  This data can also be mentioned in <lastmod> tag in XML sitemap. Also use robots.txt file to block pages and directories which are of no use to visitors. For e.g. CSS, JavaScript files, internal usage sections etc. These measures can save a lot of bandwidth.