Getting Rid of Duplicate Page Titles After URL Structure Change
-
I've had all sorts of issues with google when they just dropped us on our head a few weeks ago. Google is crawling again after I made some changes, but they're still not ranking our content like they were so I have a few questions.
I changed our url structure from /year/month/date/post-title to just /post-title and 301 redirected the old link structure to the new. When I look I see over 3000 duplicate title errors listing both versions of the url.
1. How do I get google to crawl the old url structure and recognize the 301 redirect and update the index?
2. Google is crawling the site again, but they're not ranking us like they were before. We're in a highly competitive category and I'm aware of that, but we've always been an authority in our niche. We have plenty of quality backlinks and often we're originators of the content which is then rewritten by a trillion websites everywhere. We're not the best at writing and titles, but we're working on it and this did not matter much to google previously as it was ranking us pretty highly on the front page and certainly ranking us over many sites that are ranking above us today.
Some backlinks http://www.alexa.com/site/linksin/dajaz1.com
A few examples - if you google twista gucci louis prada you'll see many of the sites who trackbacked to us since we premiered the song rank much higher than us. 3 weeks ago we were ranking above them.
http://dajaz1.com/twista-gucci-louis-prada/
google search jadakiss consignment mixtape 3 weeks ago we were ranking higher than all 4 sites ranking above us. The sites ranking above us even link to us or mention us, yet they rank above us now.
original content here
http://dajaz1.com/watch-jadakiss-confirms-cosignment-mixtape-2012-schedule/
I could throw out a ton of examples like this. How do we get google to rank us again.
It should be noted that I'm not using any SEO plugin's on the site. I hand coded what's in there, and I know I can probably do it better so any tips or ideas is welcome. I'm pretty sure that our issues were caused by the Yoast SEO Plugin as when I search site:dajaz1.com the pages and topics that display were all indexed while the plugin was active. I've since removed it and all calls to it in the database, but I'm pretty nervous about plugins right now. Which brings me to my third and final question
How do I get rid of the page category and topic pages that were indexed and seem to be ranking higher than the rest of our content?
I lied one more. For category url I've set it to remove the category base so the url is dajaz1.com/news or dajaz1.com/music
is that preferable or is this causing me issues?
Any feedback is appreciated.
Also google is crawling again (see attached image) but the Kilobytes downloaded per day hasn't. Should I be concerned about this?
-
Thanks so much for your response.
Honestly I'm still not sure why or what happened.
Essentially the domain has a bit of history. It's been up since 2008 and is very well known authority figure in the niche. The site was taken down by the Govt in their domain seizure crap and held for 13 months. They had to give it back because they had no probable cause, and we were innocent. It's been a big issue in the media, especially when all the SOPA madness was going on but not sure if that 13 month disruption caused any issues for us.
When we came back up we were doing really well, ranking very high and doing about 5k visitors a day from google. We had our highest google traffic day on March 4th and then for some reason we did a massive free fall. Google just left. When I searched site:dajaz1.com it showed paginated pages, category, tags, search pages ranking, it stopped ranking our post pages.
I started going through the site to try and figure out what was wrong. I set everything that's not the home page or the actual post as noindex, follow, we started working on writing more, better titles, better seo urls, removed the pagination plugins etc. We're often the originators of content and sourced as such so it's not necessarily an original content issue.
Google bot came back, but it's still not ranking us like it was and now ranking people above us who link to us, source us, or copy our content. When I check site:dajaz1.com now it's at least pulling posts, but it's pulling posts with numbers in the url, or very old urls. Our newer posts are not ranking. We were getting 5k-8k unique visitors a day organically, now we're getting less than 1000. I'm still at a loss as to why and how to correct.
-
Hi Melissa,
Let me try to answer your question, with the caveat that I'm missing part of the back story. Specifically, it sounds like Google wasn't crawling your site for awhile - any idea why?
If Google is indexing duplicate pages for much of your content, then in all likelihood you won't reach your ranking potential for that content. It sounds like this is because you changed your URL structure, and Google has crawled the new URLs without crawling/de-indexing the old URLs. Does this sound about right?
One solution, and probably your best bet, it to maintain a sitemap of the old URLs. This way, Google is more likely to crawl those URLs, discover the 301 redirect, and de-index those pages much faster. Ideally this is setup before the change in URL structure. At this point, it may be a major pain... if you catch my drift.
(there are a couple ways you could accomplish this, both require a lot of work. You could rewrite the URLs using Excel magic, or write a script to generate an old sitemap. Or you could very briefly switch back to your old url structure, generate a new sitemap to save, and then switch back)
Another method would be to use the URL removal tools in Google Webmaster tools. If your old URLs all used a unique directory structure (that differed from your new directory structure) you can remove entire directories from Google's index with the click of a button.
Regardless, whatever way you choose to remove those URLs, doing so should help improve your rankings.
Hope this helps! Best of luck with your SEO.
-
This is what I have
The reason I have it set to noindex, follow for category is because it's ranking my category pages over my actual blog posts. I wouldn't have an issue with having the categories indexed, but I do have an issue with posts going no where and my category pages showing up in search results.
I keep getting conflicting information on this so I'm not sure what I should with that.
What concerns me is that it seems to be crawling everything, but it's not dumping the old url structure for the new. I realize that it loses a small amount of link juice on the old urls, but on the newer urls it shouldn't and those are ranking lower than were. I can't figure out for the life of me why.
-
It's known that when you 301 a page, you loose some of the power it once had. That being said, if you're basically 301'ing your whole site, Google will have to reevaluate everything.
For your category pages, add the robots tag and Google will not index them.
if you're using wordpress, here's a quick snippet.
if (is_category()) {
//Allow indexation of category URLs
echo ‘’. “\n”; } elseif (is_404() || is_search() || is_author() || is_archive()) {
//Do not allow indexation and crawling of 404, search, author and archived pages
echo ‘’. “\n”; } else {
//For other URLs, not mentioned above such as post URLs, Page URLs - allow indexation and crawling.
echo ‘’. “\n”; } ?>
-
We are also doing a URL rewrite soon....I am looking forward to seeing responses. It is probably not relevant but I read on here somewhere about making sure you old site map and new site map or on the page for a while until google starts making the connections.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL Structure
Hi, Hope you are all well. On our website we have a 'blog' and a 'news' section. The blog is located on "/blog" - but when you click on a post the url structure changes to /name-of-article and the blog subdomain isn't included. Would it be better to have "blog/name-of-article as this would then make the blog perform better in search results? Also, if our news page is under /news - but when you click on an article it changes to /news-article/name-of-article Wouldn't it be better to have /news/name-of-article Thanks a lot!! 🙂
Technical SEO | | National-Homebuyers0 -
Duplicate content or titles
Hello , I am working on a site, I am facing the duplicate title and content errors,
Technical SEO | | KLLC
there are following kind of errors : 1- A link with www and without www having same content. actually its a apartment management site, so it has different bedrooms apartments and booking pages , 2- my second issue is related to booking and details pages of bedrooms, because I am using 1 file for all booking and 1 file for all details page. these are the main errors which i am facing ,
can anyone give me suggestions regarding these issues ? Thnaks,0 -
Duplicate Titles on Wordpress blog pages
Hi, I have an issue where I am getting for duplicate page titles for pages that shouldn't exist. The issue is on the blog index page's (from 0 - 16) and involves the same set of attachment_id for each page, i.e. /blog/page/10/?attachment_id=minack /blog/page/10/?attachment_id=ponyrides /blog/page/11/?attachment_id=minack /blog/page/11/?attachment_id=ponyrides There are 6 attachment_id values (and they are not ID values either) which repeat for every page on the index now what I can't work out is where those 6 links are coming from as on the actual blog index page http://www.bosinver.co.uk/blog/page/10/ there are no links to it and the links just go to blog index page and it ignores the attachment_id value. There is no sitemap.xml file either which I thought might have contained the links. Thanks
Technical SEO | | leapSEO0 -
Container Page/Content Page Duplicate Content
My client has a container page on their website, they are using SiteFinity, so it is called a "group page", in which individual pages appear and can be scrolled through. When link are followed, they first lead to the group page URL, in which the first content page is shown. However, when navigating through the content pages, the URL changes. When navigating BACK to the first content page, the URL is that for the content page, but it appears to indexers as a duplicate of the group page, that is, the URL that appeared when first linking to the group page. The client updates this on the regular, so I need to find a solution that will allow them to add more pages, the new one always becoming the top page, without requiring extra coding. For instance, I had considered integrating REL=NEXT and REL=PREV, but they aren't going to keep that up to date.
Technical SEO | | SpokeHQ1 -
Advice on Duplicate Page Content
We have many pages on our website and they all have the same template (we use a CMS) and at the code level, they are 90% the same. But the page content, title, meta description, and image used are different for all of them. For example - http://www.jumpstart.com/common/find-easter-eggs
Technical SEO | | jsmoz
http://www.jumpstart.com/common/recognize-the-rs We have many such pages. Does Google look at them all as duplicate page content? If yes, how do we deal with this?0 -
Content and url duplication?
One of the campaign tools flags one of my clients sites as having lots of duplicates. This is true in the sense the content is sort of boiler plate but with the different countries wording changed. The is same with the urls but they are different in the sense a couple of words have changed in the url`s. So its not the case of a cms or server issue as this seomoz advises. It doesnt need 301`s! Thing is in the niche, freight, transport operators, shipping, I can see many other sites doing the same thing and those sites have lots of similar pages ranking very well. In fact one site has over 300 keywords ranked on page 1-2, but it is a large site with an 12yo domain, which clearly helps. Of course having every page content unique is important, however, i suppose it is better than copy n paste from other sites. So its unique in that sense. Im hoping to convince the site owner to change the content over time for every country. A long process. My biggest problem for understanding duplication issues is that every tabloid or broadsheet media website would be canned from google as quite often they scrape Reuters or re-publish standard press releases on their sites as newsworthy content. So i have great doubt that there is a penalty for it. You only have to look and you can see media sites duplication everywhere, everyday, but they get ranked. I just think that google dont rank the worst cases of spammy duplication. They still index though I notice. So considering the business niche has very much the same content layout replicated content, which rank well, is this duplicate flag such a great worry? Many businesses sell the same service to many locations and its virtually impossible to re write the services in a dozen or so different ways.
Technical SEO | | xtopher660 -
Should i Change On Page Optimization ?
Hi, PC monitoring and computer monitoring software are our targeted keywords. Around 5 weeks ago, We created a page for pc monitoring software (home/pc-monitoring-software) and did some bookmarking and guest posts targeting PC monitoring software keyword. Now we are in Top 15 on Google for PC monitoring software keyword . Initially we were thinking to change content of around 2 year old home page to adjust computer monitoring software keyword and do SEO for this keyword. But few days ago, we noticed that our pc-monitoring-software page is already ranking in early fourties for computer monitoring software keyword as well. May be Google is giving advatage of being synonym of PC . Now we are thinking that we should optimize the PC monitoring software page for both computer and PC software keywords like adding "computer monitoring software" in addition to existing "pc monitoring software" in title and similalrly do other on page related work for Computer Mnitoring Software. We are also thinking of doing 301 redirect of existing pc-monitoring-software page to new computer-monitoring-software page which will be optimized both for PC and Computer. Please suggest me if it will help to get good ranking for both PC and Computer Monitoring software if we make above mentioned changes or we should not change the existing pc-monitoring-software page and shall stick to earlier plan of changing the home page to adjust for computer monitoring software.? I'm new to SEO, so want to make wise decision with your help instead of learning with failures. Thanks, shahzad
Technical SEO | | shaz_lhr0 -
URL Structure
Hi Guys, I'm in the process of creating a very exciting startup aimed at the baby industry. It's essentially a social commerce question where parents can shop for products, create lists of products and ask questions. The challenge I'm facing is how best to structure my URLs from an SEO standpoint. For example a common baby topic such as "feeding", can sit in all three categories: Shopping category aggregates all products related to feeding List category aggregates all lists related to feeding Question category aggregates all question and answers on feeding So for that keyword "feeding" you have 3 potential landing pages. What I was wondering is what is the most effective way of doing it? I was thinking of something along these lines: /shopping/feeding /baby_list/feeding /ask/feeding Would love to hear your points of view on this. Thanks! Walid
Technical SEO | | walidalsaqqaf0