Handling User-Generated & Manufacturer-Required Duplicate Content Across Large Numbers of URLs

We know that Google tends to penalize duplicate content, especially when it's something that's found in exactly the same form on thousands of URLs across the web. So how, then, do we deal with things like product descriptions, when the manufacturers require us to display things in exactly the same way as other companies?

In today's Whiteboard Friday, Rand offers three ways for marketers to include that content while minimizing the risk of a penalty.

For reference, here's a still of this week's whiteboard!

Video Transcription

Howdy Moz fans, and welcome to another edition of Whiteboard Friday. Today I'm going to be chatting a little bit about a very specific particular problem that a lot of e-commerce shops, travel kinds of websites, places that host user-generated and user-review types of content experience with regards to duplicate content.

So what happens, basically, is you get a page like this. I'm at BMO's Travel Gadgets. It's a great website where I can pick up all sorts of travel supplies and gear. The BMO camera 9000 is an interesting one because the camera's manufacturer requires that all websites which display the camera contain a lot of the same information. They want the manufacturer's description. They have specific photographs that they'd like you to use of the product. They might even have user reviews that come with those.

Because of this, a lot of the folks, a lot of the e-commerce sites who post this content find that they're getting trapped in duplicate content filters. Google is not identifying their content as being particularly unique. So they're sort of getting relegated to the back of the index, not ranking particularly well. They may even experience problems like Google Panda, which identifies a lot of this content and says, "Gosh, we've seen this all over the web and thousands of their pages, because they have thousands of products, are all exactly the same as thousands of other websites' other products."

So the challenge becomes: How do they stay unique? How do they stand out from this crowd, and how can they deal with these duplicate content issues?

Of course, this doesn't just apply to a travel gadget shop. It applies broadly to the e-commerce category, but also to categories where content licensing happens a lot. So you could imagine that user reviews of, for example, things like rental properties or hotels or car rentals or flights or all sorts of things related to many, many different kinds of verticals could have this same type of issue.

But there are some ways around it. It's not a huge list of options, but there are some. Number one, you can essentially say, "Hey, I'm going to create so much unique content, all of this stuff that I've marked here in green. I'm going to do some test results with the camera, different photographs. I'm going to do a comparison between this one and other ones. I'm going to do some specs that maybe aren't included by the manufacturer. I'll have my own BMO's editorial review and maybe some reviews that come from BMO customers in particular." That could work great in order to differentiate that page.

Some of the time you don't need that much unique content in order to be considered valuable and unique enough to get out of a Panda problem or a duplicate content issue. However, do be careful not to go way overboard with this. I've seen a lot of SEOs do this where they essentially say, "Okay, you know what? We're just going to hire some relatively low quality, cheap writers." Maybe English isn't even their first language or the country of whatever country you're trying to target, that language is not their first language, and they write a lot of content that just all sits below the fold here. It's really junky. It's not useful to anyone. The only reason they're doing it is to try and get around a duplicate content filter. I definitely don't recommend this. Panda is built even more to handle that type of problem than this one, from Google's perspective anyway.

Number two, if you have some unique content, but you have a significant amount of content that you know is duplicate and you feel is still useful to the user, you want to put it on that page, you can use iframes to keep it kind of out of the engine's index, or at least not associated with this particular URL. If I've got this page here and I say, "Gosh, you know, I do want to put these user reviews, but they're the same as a bunch of other places on the web, or maybe they're duplicates of stuff that happened on other pages of my site." I'm going to take this, and I'm going to build a little iframe, put it around here, embed the iframe on the page, but that doesn't mean that this content is perceived to be a part of this URL. It's coming from it's own separate URL, maybe over here, and that can also work.

Number three, you can take content which is largely duplicative and apply aggregation, visualization, or modifications to that duplicate content in order to build something unique and valuable and new that can rank well. My favorite example of this is what a lot of movie review sites, or review sites of all kinds, like Metacritic and Rotten Tomatoes do, where they're essentially aggregating up review data, and all of the snippets, all of the quotes are coming from all of these different places on the web. So it's essentially a bunch of different duplicates, but because they're the aggregator of all of these unique, useful pieces of content and because they provide their own things like a metascore or a Rotten Tomatoes rating, or an editorial review of their own, it becomes something more. The combination of these duplicative pieces of content becomes more than the sum of its parts, and Google recognizes that and wants to keep it in their index.

These are all options. Then the last recommendation that I have is when you're going through this process, especially if you have a large amount of content that you're already launching with, start with those pages that matter the most. So you could go down a list of the most popular items in your database, the things that you know people are searching for the most, the things that you know you have sold the most of or the internal searches have led to those pages the most; great, start with those pages. Try and take care of them from a uniqueness and value standpoint, and you can even, if you want, especially if you're launching with a large amount of new content all at once, you can take these duplicative pages and keep them out of the index until you've gone through that modification process. Now you sort of go, "All right, this week we got these 10 pages done. Boom, let's make them indexable. Then next week we're going to do 20, and then the week after that we'll get faster. We'll do 50, 100, and soon we'll have our entire 10,000 product page catalog finish and completed, all with unique, useful, valuable information that will get us into Google's index and stop us from being considered duplicate content."

All right everyone, hope you've enjoyed this edition of Whiteboard Friday. We'll see you again next week. Take care.

Video transcription by Speechpad.com