Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

How to no index / no follow CAD files .dxf .dwg

Technical SEO

472

Jon_Pearce last edited by

Hi,

I have a new Wordpress site with a number of CAD files (.dxf& .dwg) downloadable straight from the site.

These have been flagged in MOZ as warnings with everying from No Title/Description to duplicate content.

Does anybody now how I would no index these type of files?

Many thanks.
1 Reply Last reply
Reply Quote 0
Jon_Pearce @LoganRay last edited by

Awsome, thank you very much!
1 Reply Last reply
Reply Quote 0
LoganRay last edited by

Hi Jon,

If your objective is to prevent Moz from reporting crawl issues from these file types, you don't want to nonidex/nofollow them, you simply want to prevent Moz's User Agent (rogerbot) from crawling these file types.

Add these 3 lines to the top of your robots.txt file:

User-agent: rogerbot

Disallow: /*.dxf$

Disallow: /*.dwg$
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Blogs Not Getting Indexed Intermittently - Why?

indexation indexed urls

Over the past 5 months many of our clients are having indexing issues for their blog posts.
A blog from 5 months ago could be indexed, and a blog from 1 month ago could be indexed but blogs from 4, 3 and 2 months ago aren't indexed. It isn't consistent and there is not commonality across all of these clients that would point to why this is happening. We've checked sitemap, robots, canonical issues, internal linking, combed through Search Console, run Moz reports, run SEM Rush reports (sorry Moz), but can't find anything. We are now manually submitting URLs to be indexed to try and ensure they get into the index. Search console reports for many of the URLs will show that the blog has been fetched and crawled, but not indexed (with no errors). In some cases we find that the blog paginated pages (i.e. blog/page/2 , blog/page/3 , etc.) are getting indexed but not the blogs themselves. There aren't any nofollow tags on the links going to the blogs either. Any ideas? *I've added a screenshot of one of the URL inspection reports from Search Console alt text
Technical SEO | | JohnBracamontes

0
Sudden Indexation of "Index of /wp-content/uploads/"

Hi all, I have suddenly noticed a massive jump in indexed pages. After performing a "site:" search, it was revealed that the sudden jump was due to the indexation of many pages beginning with the serp title "Index of /wp-content/uploads/" for many uploaded pieces of content & plugins. This has appeared approximately one month after switching to https. I have also noticed a decline in Bing rankings. Does anyone know what is causing/how to fix this? To be clear, these pages are **not **normal /wp-content/uploads/ but rather "index of" pages, being included in Google. Thank you.
Technical SEO | | Tom3_15

0
Indexed, but not shown in search result

Hi all We face this problem for www.residentiebosrand.be, which is well programmed, added to Google Search Console and indexed. Web pages are shown in Google for site:www.residentiebosrand.be. Website has been online for 7 weeks, but still no search results. Could you guys look at the update below? Thanks!
Technical SEO | | conversal

0
Hey all -- ever seen a client with URLs that keep repeating the domain? Something like: client.com/client.com/client.com/subfolder-name. Any idea what glitch could cause that?

Hey all -- ever seen a client with URLs that keep repeating the domain? Something like: client.com/client.com/client.com/subfolder-name. Any idea what glitch could cause that?
Technical SEO | | TDC_SEO

0
Google indexing staging / development site that is redirected...

Hi Moz Fans! - Please help. We had a acme.stagingdomain.com while a site was in development, when it went live it redirected (302) to acmeprofessionalservices.com (real names redacted!!) no known external links to staging site although staging site url has been emailed from Google Apps(!!!) now found that staging site is in the index even though it redirects to the proper public site. and some (but not all) of the pages are in the index too. They all redirect to the proper public site when visited. It is convenient to have a redirect from the staging site to the new one for the team, Chrome etc. remember frequently visited sites. Be a shame to lose that. Yes, these pages can be removed using webmaster tools.
But how did they get in the index to start with? And if we're building a new site, and a customer has an existing site is there a danger of duplicate content etc. penalties caused by the staging site? We had a similar incident recently when a PDF that was not linked anywhere on the site appeared in the index. The link had been emailed through Google Apps, and visited in Chrome, but that was it. So 3 questions. Why is the staging site still in the index despite the redirects? How did they get in the index in the first place? Will the new staging site affect the rank of the existing site, eg. duplicate content penalties?
Technical SEO | | mozroadjan

0
No-follow for article directory?

My clients pull from a central article directory on our server (medical directory), as the information is about standard medical issues. This said, the MOZ analytics is showing these articles for each client as indexed and duplicate in content, descriptions, titles, etc. Would it be better to use a no-follow for these articles to avoid looking like duplicate content, or should I consider overhauling the resource section into static pages and making each article unique to each client-considering the latest updates in Google? Any help/insight would be greatly appreciated!!!!! Thanks
Technical SEO | | lfrazer

0
Canonical needed after no index

Hi do you need to point canonical from a subpage to main page if you have already marked a no index on the subpage, like when google is not indexing it so do we need canonicals now as is it passing any juice?
Technical SEO | | razasaeed

0
Google Off/On Tags

I came across this article about telling google not to crawl a portion of a webpage, but I never hear anyone in the SEO community talk about them. http://perishablepress.com/press/2009/08/23/tell-google-to-not-index-certain-parts-of-your-page/ Does anyone use these and find them to be effective? If not, how do you suggest noindexing/canonicalizing a portion of a page to avoid duplicate content that shows up on multiple pages?
Technical SEO | | Hakkasan

1