How to set the crawler or reports to ignore
-
I have a mobile version of a site with a URL string that disables the mobile view on smartphones (view full site), string is like this example.com/page-name.html?mobile=off
I need the seomoz pro reports or crawler to ignore it because the crawler visits both versions of the site then reports them as duplicate content. Is there a setting page I haven't visited yet that will set this?
-
Thanks for the confirmation. I hadn't wanted to put that in the robots file because if inquisitive, non-robot traffic if you get my meaning, but in it goes; can't stand seeing errors.
-
Got it, thanks for jumping in so quick.
-
Hey Str8,
Donford is correct. You would need to specifically block our crawler from the mobile pages in your robots.txt file. Unfortunately, we don't currently have a way to disregard specific pages or errors in the web app, but we are looking to add that function sometime in the future.
I hope this helps. Please let me know if you have any other questions.
-
Hi Str8
**user-agent: rogerbot disallow: /*mobile=**
I think may do the trick... somebody may need to confirm this.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Campaign KW data source thats auto populated in campaign set up?
HI I'm just setting up a new project/client and see Moz has auto-populated the keyword field in the new project setup form. Where does it get these from? Since I'm looking at the client's site through the Moz browser and cant see any of these KW in the usual/ obvious places? Whats the data source for this auto-population? All Best
Moz Pro | | Dan-Lawrence
Dan0 -
Can you export a keyword rankings report with the keywords listed in order of ranking ?
My keyword ranking report shows the results all jumbled. I can put them in order when viewing the report online, but when I export it , the results go back to all jumbled rankings.
Moz Pro | | jlane90 -
How can i export a historical ranking report which contains keywords with special characters.
How can i export a historical ranking report which contains keywords with special characters? Previously it only turned the keywords into jumbled format (forgot the tech term) in excel i.e. онлайн чат which are in this case russian characters. The way i got around this was to import it into google docs but that also is now converting it into this format. Due to this i 1 do not know what the keywords are and 2 all my formulas do not work.
Moz Pro | | ColumK0 -
SEO moz poor rankings reporting
How are we supposed to send ranking reports to clients if the rankings are constantly reporting false metrics This is not the first time this has happened, but the rankings are WAAAAAAY off. What is the point of reporting rankings that are not accurate? What is the possible cause of this. Is there a solution? What is SEOmoz doing to improve their ranking programs. Our client actually ranked #2 for "northern Virginia HVAC contractors" when moz had us not in the top 50! How are we supposed to bank on information that is so consistently unreliable? northernvirginiahvaccontractors.png northernvirginiahvaccontractorsrank.png
Moz Pro | | imageworks-2612900 -
Reset Crawler
Hello, Does anyone know how to reset the crawler? We recently uploaded our new website and deleted the current campaign but it seems the crawler is caching our old websites data and not the new so every time we try to create a new campaign with the same details, it's just pulling everything from cache it seems. Thanks
Moz Pro | | ForzaHost0 -
I have a Rel Canonical "notice" in my Crawl Diagnostics report. I'm presuming that means that the spider has detected a rel canonical tag and it is working as opposed to warning about an issue, is this correct?
I know this seems like a really dumb question but the site I'm working on is a BigCommerce one and I've been concerned about canonicalisation issues prior to receiving this report (I'm a SEOmoz pro newbie also!) and I just want to be clear I am reading this notice correctly. I presume this means that the site crawl has detected the rel canonical tag on these pages and it is working correctly. Is this correct?? Any input is much appreciated. Thanks
Moz Pro | | seanpearse0 -
Sorting On-page Optimization Reports
I know there are several pre-set ways to sort the on-page optimization reports, but how can I custom sort them into my own order?
Moz Pro | | WillWatrous0 -
Automated SEO Report for Agencies
I have a lot of SEO reports to send out, SEO Moz is awesome but it's not designed for this kind of work. I would like an automated report that does an on page optimization report on the potential client's site, finding errors etc it does not have to include fixes for each. This is just a basic report but it must look nice and be user friendly. I have looked at Raven and it looks ok however it does not have any decent On Page Optimization reports. I know I could crawl and then delete the campaign in SEO Moz but that is very time consuming and far from ideal. If anyone knows of a solution please let me know 🙂
Moz Pro | | SEOKeith0