MERJ and Vercel conducted a comprehensive study to understand how Google handles JavaScript throughout the indexing process. This research aimed to debunk common myths and provide empirical evidence on Google's rendering capabilities.
Evolution of Google's Rendering Capabilities
- Pre-2009: Limited support for JavaScript, primarily indexing static HTML.
- 2009-2015: Introduction of the AJAX crawling scheme, requiring HTML snapshots for dynamic content.
- 2015-2018: Early JavaScript rendering using a headless Chrome browser.
- 2018-Present: Modern rendering capabilities with an up-to-date Chrome version, supporting universal rendering, stateless rendering, and asset caching.
Methodology
The study analyzed over 100,000 Googlebot fetches using Vercel’s infrastructure and MERJ’s Web Rendering Monitor (WRM) technology. Data was collected from sites like nextjs.org, monogram.io, and basement.io, focusing on Googlebot's behavior.
Myth 1: "Google Can't Render JavaScript Content"
Findings:
- Google successfully rendered 100% of HTML pages, including those with complex JavaScript interactions.
- Dynamically loaded content via API calls was indexed.
- Next.js framework and React Server Components (RSCs) were fully rendered.
Myth 2: "Google Treats JavaScript Pages Differently"
Findings:
- Google renders pages with or without JavaScript equally.
- Pages with 200 status codes were rendered, while those with 3xx, 4xx, and 5xx errors were not.
- Noindex meta tags in the initial HTML prevented rendering, regardless of JavaScript content.
- No significant difference in rendering success rate based on JavaScript complexity.
Myth 3: "Rendering Queue and Timing Significantly Impact SEO"
Findings:
- Median rendering delay was 10 seconds, with 75th percentile at 26 seconds and 90th percentile at ~3 hours.
- URLs with query strings experienced longer rendering delays.
- Frequently updated sections had shorter rendering times.
Myth 4: "JavaScript-Heavy Sites Have Slower Page Discovery"
Findings:
- Google discovered and crawled links in fully rendered pages, regardless of rendering method.
- Links in non-rendered JavaScript payloads were discovered.
- Server-rendered pages had a slight advantage in immediate link discovery.
- Updated sitemap.xml reduced time-to-discovery differences.
Overall Implications and Recommendations
Implications:
- Google can render and index JavaScript content effectively.
- No fundamental difference in processing JavaScript-heavy pages vs. static HTML.
- Rendering queue impact is less significant than believed.
- JavaScript-heavy sites are not disadvantaged in page discovery.
Recommendations:
- Use JavaScript frameworks but prioritize performance.
- Implement error boundaries in React applications.
- Use server-side rendering for critical SEO elements.
- Ensure critical resources are not blocked by robots.txt.
- Update sitemap.xml regularly.
- Monitor using Google Search Console.
Moving Forward with New Information
Different rendering strategies have varying impacts on crawl efficiency, discovery, rendering completeness, and indexing. However, Google will index sites effectively regardless of the rendering method, emphasizing the importance of performance and user experience. Rendering Strategies:
| Feature | Static Site Generation (SSG) | Incremental Static Regeneration (ISR) | Server-Side Rendering (SSR) | Client-Side Rendering (CSR) |
|---|---|---|---|---|
| Crawl Efficiency | Excellent | Excellent | Very Good | Poor |
| Discovery | Excellent | Excellent | Excellent | Average |
| Rendering Completeness | Robust | Robust | Robust | Might Fail |
| Rendering Time | Excellent | Excellent | Excellent | Poor |
| Link Structure Evaluation | After Rendering | After Rendering | After Rendering | After Rendering, links might be missing if rendering fails |
| Indexing | Robust | Robust | Robust | Might not be indexed if rendering fails |
Focus on creating performant web applications that benefit users, as page speed is a ranking factor and improves user experience.
By focusing on performance and adhering to best practices, developers can ensure their applications are effectively indexed by Google. For more details, check out the full analysis on Vercel's website below.
