Why do i get multiple variations of my url with ?order=asc and ?view=list at the end of it in my crawl report?
-
I just did a crawl for one my clients to validate any error in the structure. Next thing I know is that the website have multiple variation of the same url with query like ?order=asc and ?view=list at the end of it.
I am wondering why these url variations appears in the crawl I just did since bots aren't suppose to go further thant the ? normally.
Just to show you a couple of url's of my crawl test.
<colgroup><col width="484"></colgroup>
| https://test.com/exemple/?per_page=9 |
| https://test.com/exemple/?per_page=15 |
| https://test.com/exemple/?per_page=30 |
| https://test.com/exemple/?orderby=popularity |
| https://test.com/exemple/?orderby=date |
| https://test.com/exemple/?orderby=price |
| https://test.com/exemple/?orderby=price-desc |
| https://test.com/exemple/?order=asc |
| https://test.com/exemple/?order=desc |
| https://test.com/exemple/?view=list |Thank you Guys
-
Thank you Samantha your answer is very useful !
-
Hey there!
Sam from Moz's Help Team here! As far as I'm aware, Google and other crawlers do crawl past the '?', unless certain parameters are disallowed within the robots.txt. If the URL is: https://test.com/exemple/?per_page=9, a search engine will see something like test com exemple 'search' etc. Google recommends blocking all Internal Search Results in the Robots.txt file - for Rogerbot, it would look something like this
User-agent: Rogerbot
Disallow: ?utmHere is a great resource about the robots.txt file that might be helpful: https://moz.com/learn/seo/robotstxt
I'd recommend checking your robots.txt file in this handy Robots Checker Tool once you make changes to avoid any nasty surprises
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL country
My url says we are in Germany but we are actually in the US. We do not have offices Germany either. How can I fix my url so it says US?
Moz Bar | | heidiodonnell0 -
Crawl-test not doesn't finish
Hello, I have used this crawl-test on 2 website 3 days ago, and it hasn't finished yet. I'm wondering if the crawler is on an infinite loop, or has crashed without sending back an error. I could re-launch the test, but if it's really still crawling, I don't want to loose any work in progress. Is there any way to check the status of a crawl?
Moz Bar | | Nobody16116145880332 -
Sorry, but that URL is inaccessible.
am using moz panel in there is an option on page grader where i use my portal name www.bookmyticket.com and keyword is flight ticket booking but getting error Sorry, but that URL is inaccessible. how i can fix this ??
Moz Bar | | tejaschudasama0 -
Learn how to use Rank Tracker to analyze rank progress for any given keyword. Get your Daily SEO Fix!
Moz's Rank Tracker tool retrieves search engine rankings for pages and keywords, and stores them for easy comparison later. With it, you can analyze and track rank progress for a keyword or set of keywords that you may not want to include within a Moz Analytics campaign. In today’s Daily SEO Fix, James dives into the tool to show you how to use Rank Tracker to start keeping tabs on the ranking history of a given keyword. This video is part of The Moz Daily SEO Fix tutorial series--Moz tool tips and tricks in under 2 minutes. To watch all of our videos so far, and to subscribe to future ones, make sure to visit the Daily SEO Fix channel on YouTube.
Moz Bar | | kellyjcoop1 -
"Sorry! We weren't able to find that page when we crawled your site." Please help!
Can someone please explain whey I am getting this error for this link "http://lensoutloud.com/san-antonio-real-estate-photography/" when I attempt to perform an on page SEO grading? The link is indexed and ranking very well but for some reason Moz says it can't find the page when it crawled my site. This has also happened when I attempt to grade other pages on my site. Thanks in advance!
Moz Bar | | AndreGant0 -
Can we export branded reports from Moz Analytics Beta yet?
Hey guys, Just looking around the new Moz-alytics (droooool) and it is superb, but are we able to export and build reports from it yet? Or is this something that will be in the final release? Apologies if this has already been asked. Regards! Joe Sturgess
Moz Bar | | Sturge
Twitter: @Sturge871 -
Why do the crawl diagnostics indicate duplicate page content among blog postings hosted by WordPress?
Does anyone know why the crawl diagnostics indicate duplicate page content regarding the blog we are hosting on WordPress? And does anyone know how to fix this issue? The content is not, or does not appear to be duplicate.
Moz Bar | | AndreaKayal0 -
Duplicate Page Content Report on MOZ
Hi, I am just wondering as to the accuracy of this report - does it pick up all the duplicate on page content? Or is there a limit? We have an ecommerce store with a lot of copied and pasted descriptions - just wondering if there is a limit on how much the moz crawler picks up? In other words, once we fix what MOZ has detected, will there be more detected because it is limited to display say up to 200?? Hope you understand what I mean. Thanks
Moz Bar | | bjs20100