Crawl reveals hundreds of urls with multiple urls in the url string
-
The latest crawl of my site revealed hundreds of duplicate page content and duplicate page title errors. When I looked it was from a large number of urls with urls appended to them at the end.
For example:
http://www.test-site.com/page1.html/page14.html
or
http://www.test-site.com/page4.html/page12.html/page16.html
some of them go on for a hundred characters.
I am totally stymied, as are the people at my ISP and the person who talked to me on the phone from SEOMoz.
Does anyone know what's going on?
Thanks So much for any help you can offer!
Jean
-
I couldn't find exactly your exact problem mentioned above, but I downloaded XENU and crawled your site and I strongly suggest you hire a SEO expert to help clean your site.
You have URLS like this:
and thousands of other strange and duplicate URL's
-
-
Please share your site and I will help analyze.
-
I forgot to say - this is the SEOMoz crawl.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved URL dynamic structure issue for new global site where I will redirect multiple well-working sites.
Dear all, We are working on a new platform called [https://www.piktalent.com](link url), were basically we aim to redirect many smaller sites we have with quite a lot of SEO traffic related to internships. Our previous sites are some like www.spain-internship.com, www.europe-internship.com and other similars we have (around 9). Our idea is to smoothly redirect a bit by a bit many of the sites to this new platform which is a custom made site in python and node, much more scalable and willing to develop app, etc etc etc...to become a bigger platform. For the new site, we decided to create 3 areas for the main content: piktalent.com/opportunities (all the vacancies) , piktalent.com/internships and piktalent.com/jobs so we can categorize the different types of pages and things we have and under opportunities we have all the vacancies. The problem comes with the site when we generate the diferent static landings and dynamic searches. We have static landing pages generated like www.piktalent.com/internships/madrid but dynamically it also generates www.piktalent.com/opportunities?search=madrid. Also, most of the searches will generate that type of urls, not following the structure of Domain name / type of vacancy/ city / name of the vacancy following the dynamic search structure. I have been thinking 2 potential solutions for this, either applying canonicals, or adding the suffix in webmasters as non index.... but... What do you think is the right approach for this? I am worried about potential duplicate content and conflicts between static content dynamic one. My CTO insists that the dynamic has to be like that but.... I am not 100% sure. Someone can provide input on this? Is there a way to block the dynamic urls generated? Someone with a similar experience? Regards,
Technical SEO | | Jose_jimenez0 -
Google crawl drop
the crawl request of my company site: https://www.dhgate.com/ has dropped nearly over 95%, from daily 6463599 requests to 476493 requests at 12:00am on 9th, Oct (GMT+8). This dramatic dropping trend not only showed in our GSC crawl stats report but also our company's own log report. We have no idea what’s going on. We want to know whether there is an update of google about crawlling, or is this the issue of our own site? If something is wrong with our site, in what aspects would you recommend us to check, analyze and accordingly optimize?
Technical SEO | | DHgate_20140 -
Is it Detrimental to Repeat a Word in Our URL?
Hey guys! We run a tour company in Barcelona. Our company name is Barcelona Experience. We're customizing our URL's to include keywords which can be found in all the important areas on the page (title tage, meta descp., etc).
Technical SEO | | BarcelonaExperience
We want to change "www.barcelonaexperience.com/bike-tours" to "www.barcelonaexperience.com/barcelona-bike-tours"
We're worried the repetition of "barcelona" could be a bad thing. True, or not true? Thanks!0 -
Best way to deal with these urls?
Found overly dynamic urls in the crawl report. http://www.trespass.co.uk/camping/festivals-friendly/clothing?Product_sort=PriceDesc&utm_campaign=banner&utm_medium=blog&utm_source=Roslyn Best way to deal with these? Cheers Guys
Technical SEO | | Trespass0 -
Website not crawled
i added website www.nsale.in in add campaign, it shows only 1 page crawled. but its working fine for other sites, any idea why it failed ?
Technical SEO | | Dhinesh0 -
.htaccess: Multiple URLs catches filename
Hi, I have the following line in my .htaccess:
Technical SEO | | rasmusbang
RewriteRule privacy stdpage.php?slug=privacy [L] So if you go to the www.mysite.com/privacy it takes the stdpage.php with the argument above. But if you go to www.mysite.com/privacysssssssss catches the same file. How can I prevent this? It will give me multiple URLs with the exact same content. I have a 404 page which i would like to show instead when the match is not 100%. -Rasmus0 -
How do I use only one URL
my site can be reach by both www.site.com and site.com. How do I make it only use www?
Technical SEO | | Weblion0
