Gary Illyes from Google advises website owners to disallow the crawling of action URLs, such as "add to cart" and "add to wishlist," to save server resources and improve efficiency. Crawlers do not interact with these URLs in a meaningful way, and their crawling can unnecessarily consume server resources.
Illyes suggests adding disallow rules for these URLs in the robots.txt
file or converting them to the HTTP POST method, though some crawlers can still make POST requests. Examples of such URLs include:
https://example.com/product/scented-candle-v1?add_to_cart
https://example.com/product/scented-candle-v1?add_to_wishlist