Google Search has updated its crawl budget documentation, emphasizing the importance of link consistency between mobile and desktop versions of websites. This update addresses potential indexing delays for large websites maintaining separate HTML versions for different devices.

Key Documentation Changes

The update focuses on a new best practice recommendation for websites with separate mobile and desktop HTML:

• Mobile versions should maintain all links present in desktop versions
• Missing mobile links can slow new page discovery
• Alternative solution includes comprehensive sitemap implementation

Technical Implementation Guidelines

Google provides specific recommendations for maintaining efficient crawling:

• Include complete link sets in mobile HTML versions
• Ensure sitemap coverage for any omitted mobile links
• Consider that Google primarily indexes mobile versions
• Maintain simple URL structures for improved crawling

Impact on Page Discovery

The documentation highlights several factors affecting page discovery:

• Standard crawl times extend beyond three days
• Exceptions exist for news sites and time-sensitive content
• Link disparity between mobile and desktop can delay indexing
• Crawl budget efficiency depends on link accessibility

Best Practices

The update reinforces key practices for optimal crawling:

• Use news sitemaps for news content
• Implement lastmod tags in sitemaps
• Provide standard, crawlable a tags
• Maintain consistent link structure across versions