Site Crawl: Step By Step
What's Covered?
This guide acts as a workflow for tackling Site Crawl issues and tracking your progress as you work through identified issues.
Quick Links
Getting Started with Site Crawl
Whether it’s your first Site Crawl report or your 50th, sometimes figuring out how to work through the issues can be overwhelming! In the below workflow we’ll offer a framework for tackling Site Crawl issues and tracking your progress.
Before getting started though, be sure to have a Campaign setup in your Moz Pro account with a Site Crawl completed.
If you need help setting up a new Campaign, be sure to check out our guide on how to track a site with Moz Pro.
1. Identify Top Priority Issues
The first step when it comes to working with your newly acquired Site Crawl data is to identify your top priority issues. This step relies heavily on your site’s and/or company’s specific SEO strategy and may vary.
That being said, we have a few hints to help get you started if you’re having trouble identifying your first few tasks.
Moz’s Subject Matter Expert and Marketing Scientist, Dr. Pete Meyers has an excellent blog post available called Site Crawl, Day 1: Where Do You Start? which offers great insight into where to begin.
Additionally, the Site Crawl section of your Moz Pro Campaign offers some suggestions of issues to tackle first. There are 2 places to check from within your Site Crawl Overview. The first is within the breakdown of All Issues. Issues marked with a red exclamation mark are considered critical issues and may be greatly impacting your site. This includes issues like 4xx errors, 5xx errors, and redirect to 4xx issues.
The next place to look is the Moz Recommended Fixing section of the Site Crawl Overview. This section of your Site Crawl will update and change based on the issues found in the most recent Site Crawl for your site.
2. Implement Changes On Your Site
Now that you’ve identified your top priority issues, it’s time to implement changes on your site to resolve them. This step may involve reaching out to your web development team or marketing team. Or you may be implementing these changes yourself depending on your company’s structure and your skillset.
If you’re looking for help learning how to implement these changes on your site or you have questions about the best practices associated with each issue category you’re tackling, we have some great resources to help.
3. Mark Issues as Fixed
As you’re working through issues surfaced in your Site Crawl, it can be helpful to mark issues as Fixed to better keep track of your open tasks. You can do this right within the Site Crawl section of your Moz Pro Campaign!
Marking an issue as Fixed will indicate to our crawler that you have taken action on your site to resolve the issue. During the next crawl of your site, the issue will be removed from the interface if the crawler does not encounter the same issue again.
To mark issues as Fixed, first head to the Issue Category for the issues you’re working to resolve. Then select the issues you’ve completed using the checkboxes on the left and click Mark as fixed on the top right.
4. Ignore Issues Out of Scope
As you work through your Site Crawl data, you may encounter issues which are outside of the scope of your current SEO strategy or which you do not plan to resolve. These may include pages intentionally marked as noindex or URLs marked as too long which would be better off not being shortened. These types of issues can be marked as Ignore in your Site Crawl interface.
Marking an issue as Ignore will indicate to our crawler that you are aware of the issue but are not planning to take action to resolve it. The crawler will not include the issue in upcoming crawls as a result. You can Unignore issues at any time from your Campaign Settings.
To mark issues as Ignore, first head to the Issue Category for the issues you’re working to resolve. Then select the issues you’ve completed using the checkboxes on the left and click Ignore on the top right.
5. Recrawl Your Site
Once you’ve worked through some issues or you’ve completed your to-do list, it’s time to recrawl your site! Just a reminder that Moz will automatically crawl your site weekly and update your Site Crawl data accordingly. New Site Crawl data will replace the previous crawl so if you’re in the middle of working through issues, you may see your Campaign update. If you’re concerned about your current list of tasks being overwritten, you can export your Campaign data to keep on hand.
If you have a medium level subscription or higher, you have the ability to launch a Recrawl of your site between weekly Campaign updates. To do so, head to the Site Crawl section of your Campaign and click Recrawl my site.
Please note: If a Site Crawl is already in progress for your Campaign, the Recrawl my site button will be greyed out and unclickable. You are not able to launch a new crawl while one is already in progress.
6. Review Site Crawl Data for Unresolved Issues
When your new crawl completes and the data is populated in your Campaign, you can review the progress you made by reviewing issue counts and looking out for issues marked as Not fixed.
For issues marked as Fixed, if our crawler does not encounter those issues in the next crawl they will be removed from your list of site crawl issues. If the issue is still found, it will be marked as Not fixed.
For issues marked as Ignore, our crawler will not report those issues in your next crawl. They will be removed from your list of site crawl issues. You can Unignore issues at any time.
In the main Site Crawl Overview section of your Campaign you will see issue counts by category along with a delta value for how those counts changed since the previous crawl.
A green arrow indicates there was a decrease in issues for that issue category while a red arrow indicates our crawler found more issues in that category. Reviewing these numbers can give you a good idea of the progress you’ve made and where you may want to focus next. We also offer guides to help if you’re seeing an unexpected spike in Site Cawl issues or a fluctuation in your total pages crawled.
Related Articles
Was this article helpful?
Yes! Amazing!
Yes! It was what I needed.
Meh. It wasn’t really what I was looking for.
No, it wasn’t helpful at all.
Thanks for the feedback.