Seomoz crawling filtered pages
-
Hi,
I just checked an seo campaign we started last week, so I opened seomoz to see the crawl diagnostics.
Lot's of duplicate content & duplicate titles showing up, but that's because Rogerbot is crawling all of the filtered pages as well. How do I exclude these pages from being crawled?
/product/brand-x/3969?order=brand&sortorder=ASC
/product/brand-x/3969?order=popular&sortorder=ASC
/product/brand-x/3969?order=popular&sortorder=DESC&page=10
/product/brand-x/3969?order=popular&sortorder=DESC&page=11 -
So if the site has a structure like this:
and the filter on this page has several options like /?order= , ?brand=, .... I have to rel-canonical them al to www.xyz.com/overview
-
I'd rel-canonical if you can, as theres still nothing stopping links to them being indexed. It might stop Rodger/Google from crawling them, but the potential indexation issues won't go away. Otherwise perhaps no-index them.
I'd usualy go as far to do re-prev and rel-next for the paginated searches as well.
-
Rel-canonical will be an immense job, would it be ok using robots.txt like this
disallow: /?order=
disallow: /?sortorder=
...
?
-
Last I checked, Rodger obays robot.txt and meta-robot commands as he trys to simulate what google would crawl. If he can crawl those pages, google probably is as well.
I think if you rel-canonical them properly, or no-idex them properly e.t.c. It should show as a normal page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What SeoMoz tool am I thinking of?
A few months ago I found a tool on Moz that did keyword link research. It is not keyword analysis either. It took a word and turned it into links such as dir:cabinet. I can't find it or remember or it. Does anyone else know? (I know this description sucks, sorry about that)
Moz Pro | | EcommerceSite0 -
Only One page crawled..Need help
I have run a website in Seomoz which have many URLs with it. But when I saw the seomoz report that showing Pages Crawled: 1. Why this is happen my campaign limit is OK Tell me what to do for all page crawling in seomoz report. wV6fMWx
Moz Pro | | lucidsoftech0 -
Duplicate content in SEOMOZ report
Hi guys, The SEOMOZ report shows there is duplicate content on my Magento ecommerce: footdistrict.com Example: http://footdistrict.com/nike-air-royalty-386169602.html?___store=footdistrict_en Duplicate content shown on the report: http://footdistrict.com/marcas/puma.html?___store=footdistrict_en
Moz Pro | | footd
http://footdistrict.com/new-balance-m400rk.html?___store=footdistrict_en
http://footdistrict.com/new-balance-gm500mbn.html?___store=footdistrict_en
http://footdistrict.com/new-balance-m400nnb.html?___store=footdistrict_en My guess is that this is due to the fixed footer that we have set where modal windows pop up with delivery info and so on. As such, all the content within it is repeated through all the pages What do you recommend me to remove this duplicate content? I have read about duplicate content issues but they don't usually deal with div tag duplicate issues, modal windows and so on. Thanks Regards0 -
Why would the SEOMoz Page analysis pick up exact keywords used in page title and text?
Hi, I am trying to optimise this URL : www.adaptiveconsultancy.com/ecommerce/features/advanced-ecommerce with the keyword being 'advanced ecommerce' With the 'On-Page Report Card' from SEOMoz that the exact keyword isn't featured in the page title or text, but it is in there. Why would this not be picked up? Thank you in advance,
Moz Pro | | adaptiveconsultancy
M0 -
SEOMoz Crawl Warnings, do they really hurt rankings?
SEOMoz reports 250 crawl warnings on my site. In most cases its too long title tags, with 4 of them its missing meta description. SEOMoz says it will hurt my rankings? However, I'm sure a recent whiteboard Friday contradicted this. So what is it?
Moz Pro | | sanchez19600 -
SEOmoz crawl error questions
I just got my first seomoz crawl report and was shocked at all the errors it generated. I looked into it and saw 7200 crawl errors. Most of them are duplicate page titles and duplicate page content. I clicked into the report and found that 97% of the errors were going off of one page It has ttp://legendzelda.net/forums/index.php/members/page__sort_key__joined__sort_order__asc__max_results__20 http://legendzelda.net/forums/index.php/members/page__sort_key__joined__sort_order__asc__max_results__20__quickjump__A__name_box__begins__name__A__quickjump__E etc Has 20 pages of slight variations of this link. It is all my members list or a search of my members list so it is not really duplicate content or anything. How can I get these errors to go away and make search my site is not taking a hit? The forum software I use is IPB.
Moz Pro | | NoahGlaser780 -
Campaign Not Crawling
I set up my first 5 campaigns and one is not crawling beyond one-page. It's been over 48 hours. This site has nearly 3.500 pages the others much less, however, this shouldn't make any difference. I searched for the problem and couldn't find it so I hope this question isn't redundant. Comments and advice would be appreciated.
Moz Pro | | JavaManOne0 -
Why is Roger crawling pages that are disallowed in my robots.txt file?
I have specified the following in my robots.txt file: Disallow: /catalog/product_compare/ Yet Roger is crawling these pages = 1,357 errors. Is this a bug or am I missing something in my robots.txt file? Here's one of the URLs that Roger pulled: <colgroup><col width="312"></colgroup>
Moz Pro | | MeltButterySpread
| example.com/catalog/product_compare/add/product/19241/uenc/aHR0cDovL2ZyZXNocHJvZHVjZWNsb3RoZXMuY29tL3RvcHMvYWxsLXRvcHM_cD02/ Please let me know if my problem is in robots.txt or if Roger spaced this one. Thanks! |0