How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Forward slash on URL on Duplicate Content Report
Hi I'm new to this whole Moz thing, so needing help from some kind people! I've just looked at my Duplicate Page Content report and there are loads of URLs in there which are the same but are just differentiated by adding / at the end of the URL, e.g. http://youngepilepsy.org.uk/news-and-events/events http://youngepilepsy.org.uk/news-and-events/events/ Is this be a canonical issue? I can't understand why though as these aren't at the root. However when we add inline text links within the page HTML, there are some URLs with / and some without, could that be the reason? Thanks for your help! Jackie
Moz Pro | | YoungEpilepsy1 -
No follow links also been reported in SEOmoz crawl diagnostics
Hi, Why does SEOmoz reports links which has been marked as 'nofollow'. I am getting 'Overly-Dynamic URL' reports on links which I have designated as nofollow which means Google will discount them. So why does SEOmoz still report them. Thanks.
Moz Pro | | malpani0 -
Page Rank Report says #6 in Google but I can't find the page anywhere
So SEOMoz says that I've consistently ranked #6 for a certain keyword. But when I search I'm no where to be found. I've done regular searches, incognito and some non-seomoz reports and all come up with nothing in Google. I noticed it a week or two ago, but didn't think it would continue. This is no bueno. I wouldn't be surprised if I got penalized (luckily my homepage relatively well for similar keywords), an old seo consultant used very spammy tactics. I recently removed them, but not before I started to notice that I fell off the map. Why would SEOMoz not recognize this, and continue to say I'm ranking well? The keyword is bpi building analyst the page is http://www.cleanedison.com/courses/bpi-building-analyst
Moz Pro | | CleanEdisonInc0 -
Why do the Ranking Report emails differ from the acutal Ranking Report?
I have been noticing that the Rankings listed in the emails are far different from the ones in my report once I log in. Any idea what this is about? Often the rankings in the email are much better then in the report. And I am talking about on the same day as the email was sent out, not a week later.
Moz Pro | | 2CDevGroup0 -
Keyword Difficulty Tool Full Reports - Are they correct?
Hello SEOMOZ Community I hope you can answer my following question. I’ve used the Keyword Difficulty and SERPS to analyse why my competitors seem to be higher than me in the SERPS for a certain keyword. The only problem I can see when I’ve done the full report is that our website has no internal links to the page we are trying to rank for, however when I go into Google webmasters tools it’s stating we have over 99,000 links. Can you please let me know why this is showing in the report? Many Thanks Matt
Moz Pro | | Steve250 -
Are organic site links excluded in the rankings report?
If I'm pulling a rankings report for a specific subdomain and a different subdomain appears at the main result and the specific subdomain is a site link beneath it, does that show up in the results? It doesn't seem to in my report.
Moz Pro | | mattiasantin0 -
How long does opensiteexplorer take to generate exportable reports
I tried exporting a few reports out of opensiteexplorer. They have been pending for more than 36 hours. Progress bar appears to be stuck. Is this normal? How long does it take to get the reports generated?
Moz Pro | | ibwork0 -
SEOMOZ reports the statistics.
SEOMOZ reports the Statistics, but where do i manage & improve??? Simply Statistics is all about SEOMOZ??
Moz Pro | | webicers0