Why do i get multiple variations of my url with ?order=asc and ?view=list at the end of it in my crawl report?
-
I just did a crawl for one my clients to validate any error in the structure. Next thing I know is that the website have multiple variation of the same url with query like ?order=asc and ?view=list at the end of it.
I am wondering why these url variations appears in the crawl I just did since bots aren't suppose to go further thant the ? normally.
Just to show you a couple of url's of my crawl test.
<colgroup><col width="484"></colgroup>
| https://test.com/exemple/?per_page=9 |
| https://test.com/exemple/?per_page=15 |
| https://test.com/exemple/?per_page=30 |
| https://test.com/exemple/?orderby=popularity |
| https://test.com/exemple/?orderby=date |
| https://test.com/exemple/?orderby=price |
| https://test.com/exemple/?orderby=price-desc |
| https://test.com/exemple/?order=asc |
| https://test.com/exemple/?order=desc |
| https://test.com/exemple/?view=list |Thank you Guys
-
Thank you Samantha your answer is very useful !
-
Hey there!
Sam from Moz's Help Team here! As far as I'm aware, Google and other crawlers do crawl past the '?', unless certain parameters are disallowed within the robots.txt. If the URL is: https://test.com/exemple/?per_page=9, a search engine will see something like test com exemple 'search' etc. Google recommends blocking all Internal Search Results in the Robots.txt file - for Rogerbot, it would look something like this
User-agent: Rogerbot
Disallow: ?utmHere is a great resource about the robots.txt file that might be helpful: https://moz.com/learn/seo/robotstxt
I'd recommend checking your robots.txt file in this handy Robots Checker Tool once you make changes to avoid any nasty surprises
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Trying to duplicate Screaming Frog report
I am trying to use the crawl report to generate the Screaming Frog report shown below. I want to use it to calculate the internal page rank as Paul Shapiro outlined in this article. Google is not seeing my site the way I want them to and I want to work on the site heirarchy and structure. I am not seeing where: Type, Anchor, and Alt are listed in the crawl report - is it called something else? Is there another report I can run for the whole site to generate the data above? Does the Moz PA already calculate the internal PR of pages? Any additional resources or people that could help me with this? Thanks, SAM
Moz Bar | | SammyT0 -
Learn how to examine and analyze links using MozBar: Get your Daily Fix!
Hello again! We have another tutorial for The Moz Daily SEO Fix video series--tips and tricks with Moz tools in two minutes or less. In today's Daily SEO Fix, Abe shows you how to use MozBar to examine and analyze SERPs and access keyword difficulty scores for a given page--in a single click. Watch The Moz Daily Fix: New MozBar Features to Help You Examine and Analyze SERPs to learn how. And, if you don't have MozBar, you can download it for free here: http://mz.cm/1Be9wjj To view more videos like this, be sure to check out The Moz Daily SEO Fix playlist on YouTube.
Moz Bar | | kellyjcoop3 -
Why is 410 (Gone) being classed as a high priority issue in crawl diagnostics?
Are high priority issues have suddenly soared by over 100 because Moz is classing 410s as high priority.
Moz Bar | | Melissabraz
Google doesn't class these as so serious, so we were wondering if anyone knows why Mos does?0 -
Keyword Difficulty Report - Any way to have it send weekly?
I need the SERP data found in the Keyword Difficulty research tool each week for a search phrase. We are not tracking any specific URL - just really need the SERPS page 1 for the term weekly. Is there a way to have this report sent to use each week automatically? We have RSS feeds for the search but the end result is not visually appealing. If not this, does anyone know of any service that could provide this? Thanks
Moz Bar | | S2RSolutions0 -
Did the Crawl Test tool go away or was it replaced
I loved that tool as it provided me with all of my URLs and it was easy to catch all errors at once. I had it booked marked but now I am just going to the regular tools page.
Moz Bar | | KJ-Rodgers0 -
Rank Report color coding
As I recall the weekly Rank Report used green and red to indicate an increase or decrease in page rank. Now, there isn't even a minus sign for a drop in page rank. If the current page rank is 10 and it dropped 5, all I see is 10 and 5. How am I supposed to know at a glance that it dropped 5 without having to pull up the historical graph? Best,
Moz Bar | | ChristopherGlaeser
Christopher0 -
Rel Can notice issue on my SEOMoz reporting
Need some help understanding this report... I have 17 notices for Rel Can on my campaign. Then, it lists all the links. But what is this report actually telling me? Is it telling me that Rel Can's are listed on these pages? The are all blog posts...our blog was redirected when the site was recently rebuilt. I just need to understand what the report is really telling me to do/not do. Or is it ok to ignore this "notice"?
Moz Bar | | cschwartzel0