Blocking links from being crawled
-
For my blog, should I block category, "read more" and author links from being crawled by search engine robots? Is it good for SEO?
Thanks
-
Hi, sorry... I just meant if you did have a few other spammy links from other sites, those could hurt you. I doubt the woorank ones are hurting, but perhaps the other 3 spammy sites you mentioned could have been.
-
Thanks Dan. Just to be clear, what do you mean by this "It may not be those links, but if there were even a few others, it could potentially hurt the site." You mean blogroll links in other sites? I have 82 links from Woorank. I did not ask them to put me there and I didnt even know who they were until I saw them on webmaster tools. They're unrelated to my niche but they're not a bad quality site. Could they hurt my site?
I did have spammy unrelated links from 3 other websites. But that's it. I have a few links from general topic websites and most of my links are from relevant sites. I'm also on another blogroll from a relevant site, just not as big as the one I mentioned.
-
Thanks for the details. It may not be those links, but if there were even a few others, it could potentially hurt the site. I had a client who had done some spammy link building in 2011ish with another agency and just those few links hurt the site. I would try removing or disavowing anything you can find that's spammy, and not helping, and see if they helps anything.
The types of things to de-optimize on your own site would be more like keyword stuffing in the titles, URLs, headers and internal anchors - not so much blocking pages from being crawled. Blocking crawling it typically only if you want to improve crawl efficiency on really large sites or you just don't want Google to page attention to that content, but it won't fix a Penguin issue if it's on-site.
-Dan
-
Alicia,
I was hit by penguin 2.0. There's no question about it
-
Dan,
The website's topic is very similar to mine. The link was part of a blogroll of about 15 other sites. The anchor text was my website name. For example, if my website were www.cleanbluecars.com, the anchor was Clean Blue Cars. Maybe Google is considering that to be an exact match commercial anchor text?
The websites on that blogroll were not part of a link exchange, but some of the websites on that blogroll mention the main website on their blog. The main website hosts a blog carnival every Monday so people naturally mention it. The main website was who put me on its blogroll because I normally participate on their weekly blog carnival. The carnival links are and have always been no-follow.
-
agree with Dan. If it's not any of the above and you are sure none of the other links are spam, you might not have been hit with penguin 2.0, but some other penalty.
I would check with other tools as well. Use as many tools you have at your disposal and create a comprehensive list of backlinks.
-
It depends. If the sitewide links were;
- off topic from the site it was on, not as relevant
- among a giant list of other "blogroll" links
- on a site that may have a penalty against it
- exact match commercial anchor text <--big one
- if it's part of a "link exchange"
Those issues could hurt you coming from one site. If the links were exact match commercial anchors I'd see about changing them to brand or domain name anchors. If the site linking to you is a random topic, or you're among a big giant list of links, or part of a link exchange get the links removed or disavow.
-Dan
-
I used Google Webmaster tools and I used Site Explorer to try and find the spam. From Webmaster tools was where I found that 50% of all my backlinks came from just 1 site. But this also happens to be the strongest relevant site I could get backlinks from. Is this something that Penguin penalizes you for? The fact that most links are coming from just 1 website?
-
I agree with David here. Someone made a custom category plugin for WordPress which will do what David is suggesting. I would not block anything, but rather noindex what you don't want in the index, which is typically for me tags, subpages, author and date archives.
-Dan
-
Penguin 2.0 is definitely from webspam and backlinks. I suggest using Google Webmaster Tools as your backlink dataset. They provide links that are more likely to be causing problems. It's extremely unlikely the internal links are hurting you if it is Penguin 2.0.
-
I agree with Alice I would not block them. If the content is unique, they can't hurt you. Category sections show up an SERP's all the time and will increase your CTR (click thru rate)
-
I'm concerned because my website lost 50% of traffic from Penguin 2 and I'm trying to find out why. My website only has about 150 posts. According to the Moz Campaign Link Analysys, I have 168 followed external links and more than 11,000 internal links, which I find to be odd. I thought that may be hurting my rankings
I've read a bunch of articles on Penguin about spam and my website had less than 1% of those types of links. Most of my links are from relevant websites. The only link that I think might have hurt me is from a PR4 relevant website that had me on their blogroll. Over 50% of the links to my site came from just that one website because I was on their blogroll. But this still doesn't make sense to me because that link was 100% natural.
-
Category, you can block it, if you see they are creating duplicate content to the website.
Read More: Why block this? I mean it is link to your post and this is very much ok so I personally will not recommend you this!
Author Link: Hello No, Infect making it more interactive can allow you inspire new readership!
-
If possible add unique content to your category pages, and use unique post extracts. Then no need to block.
Don't block "read more" as these link to full posts.
-
Hi,
I wouldn't block these pages from being crawled by search engines. Category pages are great for making sure more link juice flows to your deeper blog pages and making sure they get indexed. I believe author links give authorrank to the corresponding blog post too. I'm not sure about what you mean by 'read more' links. May I ask why you are concerned that these pages hurt SEO?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help Me With These Horrible Crawl Errors
I am currently using Opencart to run a fairly popular ecommerce website. I have done quite well in terms of rankings so far. However, the Opencart platform only goes so far. There are endless amounts of crawl errors, from duplicate content down to missing meta tags. There is no easy way to add canonical tags. The pages that are being listed in the crawl errors seem to be mainly search URL's that i have no idea how Moz found them. An example of the page is: .....com/index.php?route=product/search&tag=home+office+flooring Which has ten duplicate pages all missing meta tags. I have used Google webmaster tools and htaccess to deny access to these pages with the slug "route=product/search" but it doesn't seem to work. So my question to you guys is as follows: Is it worth trying to fix these errors and do they have an effect on SEO? If so, how do I prevent these errors that seem to grow at every crawl? Cheers, Danny
Search Behavior | | DannyHoodless0 -
What should go in a "Link Juice" cocktail?
The office just got some of those fancy Google Partner bottles, and I want to fill them up with a tasty cocktail that is link juice themed. Any ideas on how to tie Google and/or SEO into a fancy cocktail?
Search Behavior | | WilliamKammer1 -
Personalised Geo-targeted results - How does Google pass link juice?
Hello, Many websites now serve specific home page offers based on the location of the customer, my question is, how does link juice flow around a site when the links (this case from the homepage) are served up based on a visitors location? Internal links from your homepage are valuable for ranking that product well in the SERPs so how does Google deal with this? So, for example, a car hire website based in the UK. If you arrive on the care hire website sat in Manchester (Northern UK city), on the homepage the website serves offers of car hire deals in Manchester, Leeds, London and international destinations. If you arrived on this website from London (Southern UK City), you would not see the Manchester link at all but London, and other cities in the South. In this case, when Google crawls the car hire website, it will see internal links but a)which version and b) is there any way of sharing this link value around? Basically, we want to understand if Manchester in this case will get the benefit of an internal homepage link from Google even though we only show Manchester to people FROM Manchester, OR, do Google only give juice based on one version of the website, a generic UK version? Or to put it another way, is there any way of cashing in on both geo-targetting the customer based on their location AND getting link juice from those geo-specific home page links? Perhaps there is some code or way of telling Google that people from Manchester (a certain % of our visitors) will see a homepage internal link for Manchester that will pass some small % link value?
Search Behavior | | xoffie0 -
What to expect after removing external and sitewide links?
Hello, I've been working on removing some sitewide links and some directory links and I would like to know what to expect to see on my rankings based on your own experience if you've dealt with this before. Cheers 🙂
Search Behavior | | mbulox0 -
Is putting/removing Adsense ads on the site affects crawling for seo?
Because I noticed when we totally remove Adsense ads on our site, the pages crawled per day on the google webmastertools suddenly dropped into noticable amount and we tested again to turn it on for a singe day, it jumped up again. So if Adsense affects crawling then, if we have adsense ad on all pages of the site the more chances it gets crawled by the bot? WS76R8w
Search Behavior | | CruiseControl0 -
Is it my back link profile that is affecting my rankings ?
my website www.funktiongolf.co.uk had been ranking well until January and i'm nearly sure its due to low quality links and poor back-link profile i.e over-optimized money keywords can someone guide me? thanks for anyone that replies. I really do appreciate the
Search Behavior | | funktiongolf
help.0 -
Gathering Data on Link Removal and Disavow Tool
Hi, There is so much opinion out there about Link Removal and Google's Disavow Tool, but it's really hard to get hold of reliable data about what people are actually doing in this area. Through my involvement in rmoov, I've become really interested in participation and cleanup rates when it comes to Link Removal campaigns. So, I've put together a quick survey aimed at getting a real indication of what is happening out there. The survey is here The recent introduction of Google's Disavow Tool has opened up another area I'd really like to understand more, so there is a question on this included. I'd love to pull together a blog post with a decent snapshot of participation, success or failure for those trying to remove links and hope Moz Members might help out by answering this quick survey. Sha P.S. I'll update this post once results are available for those that are interested.
Search Behavior | | ShaMenz1 -
Google Webmaster Tools notice of detected unnatural links to our website
Hi, We have been using SEOmoz for a good few months now and have found it incredibly useful. Unfortunately however I think we may have slipped up a little bit as we have just received the following message in our Google Webmaster Tools: Google Webmaster Tools notice of detected unnatural links to http://www.refreshcartridges.co.uk/ I have a funny feeling I might know what the problem is but I'd like a confirmation before I potentially go off on a wild goose chase. We also own the domain name www.computerarticles.co.uk and recently started introducing the author at the bottom of the post along with both a link to the website homepage and a link to a deep, popular page on our website. For example, check out http://www.computerarticles.co.uk/twitter-time-saving-tools/ We genuinely didn't realise this was a bad thing and thought it would just provide Google with a way of deep linking straight in to popular areas of our site. I would really appreciate advice as to whether you think it might be these links that are causing the problem or whether there is something else that could be causing the problem. In the event it is these links that are causing the issue would you recommend removing the entire 'about the author' page (as we have published around 500 pages on that site) and simply put a link in the blog roll or simplify it to just link to the homepage. I appreciate in advance any help you could give. Regards Chris
Search Behavior | | ChrisHolgate0