It’s easy to think of the web as a green medium. After all, it allows us to explore the world without boarding a car, boat, or plane. And online platforms like Zoom now routinely replace in-person meetings.
But the reality is more complicated. The web’s infrastructure consumes large volumes of energy. Everything from a massive server farm to an individual visiting a website requires electricity. And with fossil fuels still a prevalent source of power, there’s an environmental impact to consider.
There’s a movement afoot to make websites more environmentally friendly. As a market leader, WordPress has an opportunity to play a key role.
Among the latest efforts is the Yoast SEO crawl optimization feature. The company identified an area where it could make an impact: unnecessary bot traffic that crawls websites. By allowing website owners to turn off specific WordPress features, it reduces crawling and a site’s carbon footprint.
Previously a feature of their premium product, Yoast has decided to implement crawl optimization into its free version. By their estimates, the 13+ million websites running the plugin will be able to take advantage. That could be a significant win for the web – and the environment.
A Look at Yoast SEO Crawl Optimization
Let’s take a closer look at what Yoast SEO’s crawl optimization feature does and how it came about. To help, The WP Minute spoke with Taco Verdonschot, Head of Relations at Yoast.
Our talk has been lightly edited for clarity and brevity.
The WP Minute: It seems like more people are becoming aware of the web’s carbon footprint. But it may not be common knowledge just yet. How did the issue come to your attention?
Taco Verdonschot: The online world has become more and more aware of this, but for us, it really came down to a talk given by our founder, Joost de Valk. He discovered the scale of the problem when looking at his father-in-law’s website. He first talked publicly about it at WordCamp The Netherlands 2022, a presentation you can see here: https://wordpress.tv/2022/11/17/joost-de-valk-improve-the-environment-start-with-your-website/.
TWPM: How much of a role does bot traffic play in energy usage?
TV: A big role. Because every visit to a website costs energy. Bots are responsible for the far majority of hits to a website. In the presentation mentioned before, Joost shows that the website of his father-in-law got 950 times more hits than one would expect based on the number of actual visitors. A big part of that is bot traffic.
TWPM: The Yoast SEO plugin seems uniquely equipped to address unnecessary crawling. Tell us about your team’s approach to developing a solution.
TV: What Joost identified is that the crawl problem is a compound problem. It’s not a single cause, but a combination of search engines crawling too often, hackers constantly going around the web to find vulnerable websites, and the fact that sites have way more URLs than they realize…
And while the first two are beyond our control, the latter is within our scope. We can help sites reduce the number of links they create, either knowingly or unknowingly. So, that’s where we’ve focused our efforts. Lowering the number of (unnecessary) URLs is the feature we’re bringing to 13 million websites through Yoast SEO.
TWPM: Yoast’s crawl optimization feature was initially added to the premium version of the plugin. What type of impact has it made so far? How has user feedback been?
TV: Yes, we’ve first built this for Yoast SEO Premium as a feature to optimize your crawl budget. Later we realized that when doing this at scale, it could have a huge positive environmental impact. That’s why we’ve now moved these features to our free plugin.
TWPM: What led to your decision to add crawl optimization to the free version of Yoast SEO?
TV: At Yoast, we care for the world around us. That’s why we are sponsoring diversity initiatives, are planting trees, have solar panels on our buildings, and have an electric car colleagues can use. So, when we saw that the crawl optimization features are actually having a positive impact on both your SEO and the environment, we decided to make these features available to all 13+ million websites running Yoast SEO.
Because by doing so, we can have the biggest positive impact on the environment.
TWPM: For users who want to take advantage, do you have any advice as to what crawl optimization settings will make the most impact?
TV: Enabling them all has the biggest impact. The fewer links on your website, the less power it consumes. However, you’ll have to be smart about it. Enable the settings one by one and continuously assess if your website still works as expected.
TWPM: Is there anything users should be aware of, in terms of how the feature could impact their website’s functionality?
TV: Yes. If your website is relying on specific feeds, or if you the REST API needs to be exposed in the source for an app to work, you don’t want to accidentally disable those.
TWPM: On a related note, we noticed that some Yoast team members were involved in a WordPress Eco-Mode project at the recent CloudFest. Can you share any details?
TV: In line with our green efforts, a group of our developers came up with a project to save the planet through saving energy by throttling the outgoing requests.
How Small Steps Can Make a Difference
Experimenting with Yoast’s crawl optimization brings a couple of thoughts to mind. First, it’s an easy way to reduce bot traffic. Doing so is as simple as toggling a few options on your screen.
It’s also a reminder that there are little things we can all do to reduce energy usage. And awareness may be the biggest challenge we face. Here’s hoping for more discussion of the topic so that more progress can be made.
If you’d like to view your website’s carbon footprint, try the Website Carbon Calculator. Run a test before and after making changes to see what sort of impact you’ve made. Every bit counts.
Many thanks to Taco Verdonschot for speaking with us!
That’s it for today’s episode, don’t forget to share share share this episode with others and jump on the mailing list 👇
Consider Supporting The WP Minute
Buy us a digital coffee for as little as $5 OR better yet! Join our community of WordPress newsies, get access to our Slack server, private podcast, or purchase a classifieds listing!
Thanks to our Pillar Sponsors
