Remove false urls
-
Hi,
we are using expressionengine as our CMS. Every url that gets tracked, which does not lead to a real page, gets directed to the home page. This gives us a lot of duplicate content. For every wrong link, we get +1 in duplicate content.
Is there a way to list the false urls, so Moz does not use them anymore in the statistics, or that Google does not crawl them anymore?
Where would be the best place to do this?
All help appreciated,
Michel -
Hi,
thank you for taking the time to respond. The issues with broken paths are fixed, but those pages seem to be still alive in Google/Moz. I guess adjusting the robots file will be the best option here.
Thanks,
Michel -
It's been a few years since I was good at EE, but this problem plagued me as well.
There's no easy workaround to my knowledge, but erhaps the most direct way is to exclude those URLs via your robots.txt file, and then use Google's URL removal tool to wipe them from the index.
Adding them to your robots.txt file should take care of your duplicate content problems in Moz, and the removing them from Google's index should help with the rest.
Another option is to redirect these through your .htaccess file via 301. I won't get into the technicalities here, but it's another route to consider.
-
Hi Michel,
I guess my first suggestion would be to fix the issue at the source - so you no longer have the broken linking in the first place.
If that is not an option and you can access the broken pages in EE, you could use the rel="canonical" to specify the preferred version of the duplicate page or use "noindex" on those pages to tell Moz and Google to not crawl those pages anymore.
If you cannot access the broken pages, you could set up 301 redirects so that the homepage content isn't being duplicated on other URLs... but that the broken links just permanently redirect to the homepage URL.
Does that make sense?
Mike
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I add a URL to one of my tracked keywords?
In other words, if one of my tracked keywords doesn't currently have a URL associated with it / assigned to it, how do I add a URL to it?
Getting Started | | AllChargedUp0 -
901 error code showing url back to back in crawl
Hi Everyone, I'm absolutely dumbfounded about this 901 issue (showing pages with our url back to back). Our site is hosted on Big Commerce: https://www.santabarbarachocolate.com When I look for these pages being crawled I don't find them. I've called BC for help and I can't seem to find a solution or where to turn as to how to fix the issue at hand or even if it matters. Please see below what the Moz crawl shows. Could this be related to Yotpo or some app we have running? Or does this even matter and does it have any influence on rank? Do you have recommendations or ideas? Thanks so much. Pages with Crawl Attempt Error as of Mar 3 URL Page Authority Linking Root Domains Status Code | Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/100-percent-pure-cacao-unsweetened-baking-chocolate -- -- 901 Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/buy-wholesale-bulk-chocolate -- -- 901 Error Code 901: DNS Errors Prevented Crawler from Resolving Hostname http://www.santabarbarachocolate.comhttp/www.santabarbarachocolate.com/organic-chocolate-wholesale | -- | -- | 901 |
Getting Started | | santabarbarachocolate0 -
Remove moz team conversation bubble
That little notification bubble on the bottom right corner of my account has finally gotten well beyond the point of irritation. Is there any way that I can permanently disable this?
Getting Started | | cslattery1 -
New campaign - invalid URL
Hi, Invalid URL when I try to set up a new campaign. I'm trying http://www.rogerspictures.com Thanks, Paul
Getting Started | | rogerspictures0 -
I'm setting up a new campaign and getting the error "This does not appear to be a valid URL" What's wrong?
I've tried multiple times (over 2 days) with every variation of the URL with no luck. Any ideas for why the URL does not seem to be working?
Getting Started | | jgrammer0 -
Not a valid URL?
When ever I type in my website its tells me its not a valid URL . I have bought the URL www.buykickzonline.com. This was on my 90 day trial and it is not a good way to start my trial. nlGDl3f
Getting Started | | Teddy_Corl0 -
In Open site explorer the page title and Url show in the left hand column. Why do some of my pages have no data for page title?
I am a first time user. Newly updated site using Drupal and having lots of SEO problems. Under site explorer, several pages list NO DATA for the page title. This doesn't seem right. Any suggestions on what this means?
Getting Started | | IV-Debbie0