How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why do crawlers still track meta keywords if it is not needed in my site?
I have crawled three sites already and it returns more than 5000 errors most of which are MIssing Meta Keywords tags. The sites are on Wordpress and using my SEO plugin I can easily edit the meta keywords of each page, but I am having second thoughts. Well should I?
Moz Pro | | jernest0020 -
Organic search visits in SEOMoz reporting
hi Are the organic search visit figures in SEOMoz traffic reporting unique or total visits ? cheers dan
Moz Pro | | Dan-Lawrence0 -
No Info on My On-page Optimization Scan Reports
I'm not getting any information on On-page Optimization reports. I.E. All the Grade A - F reports are empty. How do I get On-page Optimization reports?
Moz Pro | | lbohen0 -
False Pro reporting of duplicate titles
I am testing Pro. About 250 pages of content at my website. Pro says ALL of my pages have duplicate titles., but when I click on details, they display as unique titles. Ie: first page of results of Pro is as follows. While the content of my website is on one major topic the title meta tags are NOT identical. Is this an issue with Pro, or is Pro looking at something other than the title meta tags? Please advise ? Fiance Visa Help What is Adjustment Of Status from K1 Visa Adjustment of Status support Taiwan US Consulate Visa Interview Adjustment of Status Order Form How to Choose between K1 Fiancee or CR1 Marriage Visa Removal of Conditions on Residence support US Embassies + Consulates that process Fiancee and Spousal Visas | | | | |
Moz Pro | | microonae
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| |0 -
Will Social be Incorporated into the New Reports Interface
Just wondering if there are plans to integrate the social media reporting section with the new "Reports" section. Having a social section in the new Reports interface that could show weekly/monthly/yearly Facebook and Twitter interactions would be awesome to have all in one PDF report with the rest of the reporting options.
Moz Pro | | noBulMedia0 -
On raking reports
Hello. I was wondering why doesn't SEOMoz update ranking reports daily as opposed to weekly. Rankings to change overnight and since other services do it (e.g sescout.com) I can't really why not this one.
Moz Pro | | phaistonian0 -
Open Site Explorer CSV reports
Hello, Small question about the Open Site Explorer. on August 24th I've been trying to create several CSV reports which I knew would take a few days. But now, almost a week later when I check Open Site Explorer > Recent CVS reports it tells me it's still saving data. It's been saving data for 4 days now. Am I missing something, is this a bug, do I have to do something or should I wait a little longer? Thanks in advance, Dennis Tappij Gielen Retail&Clicks JFwMt.jpg
Moz Pro | | RetailClicks0 -
On the Crawl Diagnostics Summary, its reporting over 100 "Title Missing or Empty" issues, but they all check out fine?
Wondering if there Is a bug with the crawler or known timeout issues? Site speed is fast, but we do run a couple of large cron jobs out of hours, which may be the cause of any timeouts, but shouldn't the crawler report that, rather saying no title tags on 100 pages, when there are? SEOmoz newbie, so still finding my feet 🙂
Moz Pro | | sjr4x40