Screaming Frog's Major Update: Custom JavaScript, OpenAI Integration, and N-gram Analysis

The new custom JavaScript execution feature in Screaming Frog allows users to execute custom JavaScript during the crawl of a website. Users can set up a JavaScript library to run in conjunction with the crawl, enabling tasks such as analyzing page sentiment, creating text for category pages, or generating alt text for images. This feature is designed to perform these actions in-crawl, providing a powerful tool for SEOs to enhance their website analysis and optimization.

To integrate OpenAI with Screaming Frog, you need to connect your OpenAI API key to the tool. Once connected, you can leverage ChatGPT-powered crawling, which allows ChatGPT to perform actions on your content. This includes creating meta descriptions, image alt text, title tags, and more, utilizing the AI capabilities of ChatGPT to automate and enhance content optimization tasks.

Analyzing N-grams in content with Screaming Frog enables you to examine the N-gram distribution of a given page. The new N-gram tab in Screaming Frog will display commonly used terms and keywords within your site's content. This analysis helps identify prevalent phrases and can inform content strategy and keyword optimization efforts.

N-Gram analysis can assist with internal linking strategies by identifying specific N-grams used in the site's content that have unlinked text. This feature allows SEOs to quickly find internal linking opportunities using terms that already exist within the site's content, potentially improving the site's SEO by enhancing the internal link structure and relevance.