Advanced seo 74da3a3
Cyrus Shepard

New SEO Experiments: A/B Split Testing Google's UGC Attribute

The author's views are entirely his or her own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.

When Craig Bradford of Distilled reached out and asked if we'd like to run some SEO experiments on Moz using DistilledODN, our reply was an immediate "Yes please!"

If you're not familiar with DistilledODN, it's a sophisticated platform that allows you to do a number of cool things in the SEO space:

  1. Make almost any change to your website through the ODN dashboard. Since the ODN is a cloud platform that sits in front of your website (like a CDN) it doesn't matter how your website is built or what CMS it uses. You can change a single page — or more likely — entire sections.
  2. The ODN allows you to A/B split test these changes and both measure and predict their impact on organic traffic. They also have a feature called full-funnel testing allowing you to measure impact on both SEO and CRO at the same time.

When you find something that works, you see a positive result like this:

DistilledODN Positive Result

SEO experimentation is great, but almost nobody does it right because it's impossible to control for other factors. Yes, you updated your title tags, but did Google roll out an update today? Sure, you sped up your site, but did a bunch of spam just link to you?

A/B split testing solves this problem by applying your changes to only a portion of your pages — typically 50% — and measuring the difference between the two groups. Fortunately, the ODN can deploy these changes near-instantly, up to thousands of pages at a time.

It then crunches the numbers and tells you what's working, or not.

Testing Google's UGC link attribute

For our first test, we decided to tackle something simple and fast. Craig suggested looking at Google's new link attributes, and we were off!

To summarize: Google recently introduced new link attributes for webmasters/SEOs to label links. Those attributes are:

  • rel="sponsored" - For paid and sponsored links
  • rel="ugc" - For links in user-generated content (UGC)
  • rel="nofollow" - Remains a catch-all for all followed links

On the Moz blog, all comments links are currently marked "nofollow" — following years of SEO best practices. Google has stated that using the new attributes won't give you a rankings boost. That said, we wanted to test for ourselves if changing these links to "ugc" would impact the rankings/traffic of our blog pages.

To be clear:
 We are not testing if the pages we link to change rankings, but instead the source page that hosts the link — in this case, the blog pages with comments.

Here's an example of a comment the ODN modified.

UGC Comment

After we set the test running, 50% of blog posts had comments with "ugc" links, while 50% kept their original "nofollow" attributes.

Experiment results

We expected a "null" test — meaning we wouldn't see a significant impact.

In fact, that's exactly what happened.

DistilledODN Null Results

If we detected a significant change, the probability cone at the bottom right would have pointed more dramatically up or down.

In fact, at a 95% confidence interval, the test predicted traffic would either fall 26,000 visits/month or gain 9,300 visits/month.

Hence, a null result.

This validates Google's statements that using the "ugc" attribute won't give you a ranking boost.

What should Moz test next?

While "null" tests aren't as fun as a positive result, we have a lot of cool A/B SEO testing ahead of us.

The great thing is we can now test out changes with the ODN, and when we find one that works, pass that to our developers to make the changes permanently. This cuts down on needless development work and stops the guessing game.

We have a Trello board set up for test ideas, and we'd love to add some community ideas to the mix. The ODN is currently running on the Moz Blog and Q&A, so anything in these site sections is fair game.

We're also looking at experiments where we use Moz data to inform these decisions. For example, a Moz Pro crawl identified that the Moz Blog titles currently use H2 tags instead of H1. Google recently indicated this likely shouldn't impact rankings, but wouldn't it be good to test?

Missing H1 Tags

What wild/clever/ridiculous/obvious SEO things should we test? With each good test, we'll publish the results. Leave your ideas in the comments below.

Big thanks to the Distilled Team, including Will Critchlow and Tom Anthony, for embarking on this journey with us.

And if you'd like to learn more about DistilledODN and SEO split testing in general, this post is highly recommended.

About Cyrus Shepard —

Cyrus Shepard is the founder of Zyppy SEO, an SEO consulting and software company. He writes/tweets about Google ranking signals, SEO best practices, experiments, tactics, and industry updates.

For the latest, follow Cyrus on Twitter, or check out more of his posts on Moz.

With Moz Pro, you have the tools you need to get SEO right — all in one place.

Read Next

Google Autocomplete: More than a Free Keyword Research Tool

Google Autocomplete: More than a Free Keyword Research Tool

In Defense of Spam Score and the Concept of a Toxic Link

In Defense of Spam Score and the Concept of a Toxic Link

New Competitive Research Suite: Actionable Data to Drive Real Results

New Competitive Research Suite: Actionable Data to Drive Real Results

Comments

Please keep your comments TAGFEE by following the community etiquette

Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.