Crawl Tool Producing Random URL's
-
For some reason SEOmoz's crawl tool is returning duplicate content URL's that don't exist on my website. It is returning pages like "mydomain.com/pages/pages/pages/pages/pages/pricing" Nothing like that exists as a URL on my website. Has anyone experienced something similar to this, know what's causing it, or know how I can fix it?
-
The same thing is happening for one of my campaigns, specifically for a 302 redirect to the homepage. My guess is I need to update it to a 301, but I'm not 100% sure if that would solve the issue?
-
Well we have our website setup to where if you type something after mydomain.com/ that is not a valid URL it will take you to the site map. For example if I typed in mydomain.com/ljlksdfsdfkjsdlfjsflj it would take me to the site map. The same holds true for mydomain.com/pages/pages/pages/pages/pages/pricing. So to answer your question, no it does not take you to the correct page, but it doesn't give you a 404 error either. It takes you to the site map.
-
I had this weird issue too but it was down to how our developers created the website. Does "mydomain.com/pages/pages/pages/pages/pages/pricing" still take you to the correct page or does it show you a 404 error?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
404's being re-indexed
Hi All, We are experiencing issues with pages that have been 404'd being indexed. Originally, these were /wp-content/ index pages, that were included in Google's index. Once I realized this, I added in a directive into our htaccess to 404 all of these pages - as there were hundreds. I tried to let Google crawl and remove these pages naturally but after a few months I used the URL removal tool to remove them manually. However, Google seems to be continually re/indexing these pages, even after they have been manually requested for removal in search console. Do you have suggestions? They all respond to 404's. Thanks
Technical SEO | | Tom3_151 -
Specifying Your Organization's Logo Schema Required If Corporate Contacts Schema is in Place?
Does anyone know if specifying the organization's logo schema is required if corporate contacts schema is in place? I have the corporate contact schema in place on my site but not the second one. The site is http://www.cobaltrecruitment.com/ Thanks,
Technical SEO | | the-gate-films0 -
Is putting a manufacturer's product manual on my site in PDF duplicate content
I add the product manuals to our product pages to provide additional product information to our customers. Is this considered duplicate content? Is there a best way to do this so that I can offer the information to my customers without getting penalized for it? Should they be indexable? If not how do I control?
Technical SEO | | merch_zzounds0 -
What's Worse - 404 errors or a huge .htaccess file
We have changed our site architecture pretty significantly and now have many fewer pages (albeit with more robust content and focused linking). My question is, what should I do about all the 404 errors (keep in mind, I am only finding these in Bing Webmaster tools, not Moz or GWT)? Is it worse to have all those 404 errors (hundreds), or to have a massive htaccess file for pages that are only getting hits by the Bing crawlbot. Any insight would be great. Thanks
Technical SEO | | CleanEdisonInc0 -
Duplicate title tags being caused by upper case and lower case version of urls'
Hi GWT is reporting lots of duplicate titles for a clients new site. Mainly these are due to 2x different versions of the url, one with words starting with an upper case character and the other all lower case. Clients dev saying this has something to do with windows server and is ok! Is this correct or should i be telling them to delete and then 301 redirect all upper case versions to the lower case (since lower case better practice) and that will deal with the reported dupe titles ? All Best Dan
Technical SEO | | Dan-Lawrence0 -
Should I change by URL's
I started with a static website and then moved to Wordpress. At the time I had a few hundred pages and wanted to keep the same URL structure so I use a plugin that adds .html to every page. Should I change the structure to a more common URL structure and do 301 directs from the .html page to the regular page?
Technical SEO | | JillB20130 -
Why are my URL's changing
My rankings suddenly dropped and when trying to understand why I realized that nearly all images in Google's cached version of my site were missing. In the actual site they appear but in the cached version they don't. I noticed that most of the images had a ?6b5830 at the end of the URL and these were the images that were not showing. I am hoping that I found the reason for the drop in rankings. Maybe since Google cannot see a lot of the content it decided not to rank it as well (particularly since it seems to happen on thousands of pages). This is a cached version of my site I am using the following plugins that might be causing it: Yoasts SEO plugin, W3 total cache. Does anyone know what is causing ?6b5830 to be added to the end of most of my URL's? Could this be the reason for the ranking drop? Thanks in advance!
Technical SEO | | JillB20130 -
Google webmaster tool doestn allow me to send 'URL and all linked pages"
Hello! I made a lot of optimization changes in my site ( seo urls, and a lot more ) , I always use Google Webmaster tools, fetch as Google Bot to refresh my site but now it doesnt allow me to 'Send URL and all linked pages' check the attachment Thank you
Technical SEO | | matiw0