How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How are you reporting specific KW ranking metrics these days?
Does Moz report average ranking for the month? What is your best advice for setting expectations for clients about keyword ranking data and reporting?
Moz Pro | | MeridianGroup0 -
Moz reporting incorrect ranking
Moz is reporting no ranking on one of my keywords, but when I search that keyword on Google we are on the first page. What would cause Moz to report that incorrectly?
Moz Pro | | SpearOne0 -
Rankings Report Tool
Hi, One of my campaigns gets updated every Thursday, but the keywords ranking hasn't been updated yesterday. The last update was made on 28th February. Is there anything wrong with this tool? Thanks in advance.
Moz Pro | | cvissi0 -
Why the discrepancy in organic traffic - monthly report vs analytics
I've been going through my monthly reports, and noticed something odd in the SEO Moz report section showing the visits from organic (non-paid) search engine results. When checking this data against the same data in Google Analytics, everything for the most part seems to line up with the exception of two dates: Jan 09 and Jan 23 of the SEO Moz report show 0 visits, but Analytics shows I do have visits on those days. Any ideas why this would be in the report or what I'm missing? IsePEAS hWBASOd
Moz Pro | | EEE30 -
Claim Universal Placement Media in Rankings Report
There are several videos of ours on 3rd party sites (YouTube, Vimeo) that are in universal placements but show as not ours. Since they are not on our domain is there a way we can claim them in the rankings report to make the report more accurate?
Moz Pro | | BWG0 -
Where does the crawler find the urls?
The SEO Moz crawler has found a number of 500 error pages, and 404s etc which is very useful 🙂 however some of the urls are weird/broken formats we don't recognise and nobody remembers ever using - not weird enough to imply hacking, but something broken in the CMS Is there anyway to find out where the crawler found these urls? I can patch up and redirect the end result as best I can but I would prefer to fix plug the leak thanks 🙂
Moz Pro | | Fammy1 -
Confusion about how SEOMOZ crawler works...
So according to my SEOMOZ dashboard, I'm ranking between #3-4 for one of my keywords. My keyword is 'Boston Wedding Photographer'. My site is http://www.symbolphoto.com I show up in google places, true. But i was wanting to rank organically. Am i right in the assumption that Google Places and Google Organic are not the same thing? SEOMOZ claims 3,4th but not organically(Assuming they aren't the same thing) I get pretty good traffic right now being in Places, but i can't help but feel that organically ranking would bring more traffic. Any suggestions or advice is greatly appreciated. TIA! -Brendan
Moz Pro | | symbolphoto0 -
On Page Report Card... with or w/o local modifiers?
Hey all! So I am curious how you recommend using the "on page report card" (which is really helpful) along with the concept of local modifiers. IE, here is a term I am going after: business forums but really I care about a specific location: business forums | Greensboro NC So the word I hear is typically to do your keyword research & page optimization FOR the primary term, but then tack on your local modifiers after. So which do you run reports on? Probably both is the best answer, eh? Obviously my local sites won't have a shot at ranking nationally/internationally for such a broad term as "business forums", especially with some monster sites out there with some serious clout. This is more of a best practices question. Thanks dudes.
Moz Pro | | nsmcseo20