Need help with Robots.txt
-
An eCommerce site built with Modx CMS. I found lots of auto generated duplicate page issue on that site. Now I need to disallow some pages from that category. Here is the actual product page url looks like
product_listing.php?cat=6857And here is the auto generated url structure
product_listing.php?cat=6857&cPath=dropship&size=19Can any one suggest how to disallow this specific category through robots.txt. I am not so familiar with Modx and this kind of link structure.
Your help will be appreciated.
Thanks
-
I would actually add a canonical tag and then handle these using the Parameters section of Search Console. That's why it's there, for exactly this type of site with exactly this issue.
-
Nahid, before you use the robots.txt file's disallow for those URLs, you may want to reconsider. You may want to use the canonical tag instead. In the case where you have different sizes, colors, etc. we typically recommend using the Canonical Tag and not the disallow in robots.txt.
Anyhow, if you'd like to use the disallow you can use one of these:
Disallow: /?
or
Disallow: /?cat=
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Branding and Page Titles - Please Help
Hello, I have a question about page titles. How important is branding here? I'm not referring to the company name, but rather the terminology that's used as "branding language" for a company. For example, let's say that the it would be a good idea to target the keyword "Restaurant Coupons" based on search volume and competition. However, our branding adheres to the language "Dining Offers". Is it considered a bad idea to use "Restaurant Coupons" in the page title? Or is that considered inconsistent branding? Basically, I'm just trying to figure out the correct balance between the SEO value of words and adhering to a company's branding. Any help is appreciated! Thanks,
Intermediate & Advanced SEO | | atmosol
Nick1 -
Please help with some content ideas
I was reading this post http://www.clambr.com/link-building-tools/ about how he had basically outreached to experts in the field and each one had shared this post with their followers. I am wondering how this could translate to our small business marketing and design blog I am really struggling for content ideas that will work in regards to popularity and link building.
Intermediate & Advanced SEO | | BobAnderson0 -
Help needed for a domain
I have a small translation agency in Brazil (this website), totally dependent on SEM. We are in business since 2007, and we were on top position for many relevant keywords until the middle of 2011, when the ranking for the most important keywords started dropping. In that time, we believed that we needed to redesign the old static website and replace it by a new modern one, with fresh content and with weekly updates, which we did, and it's now hosted on Squarespace. I took care to keep the old links working with 301 redirections. When we made the transfer from the static site to Squarespace (Mar/2012, see the attachment), the ranking dropping became even more serious. Today, we have less than 50 unique visitors per day, in a total desperate situation! To make things worse, we received an alert from Google on 23/September/2012 talking about unnatural inbound links, but Google said that "As a result, for this specific incident we are taking very targeted action on the unnatural links instead of your site as a whole", so we thought we didn't need to worry about. Google was correct, I worked many hours to register our website in web directories, I thought there would be no problem since I was doing this manually. My conclusions are: Something happened prior to Mar/2012 that was making us losing territory. I just don't know what! The migration to Squarespace was a huge mistake. I lost control over the html, and squarespace doesn't do a good job optimizing the pages for SEO. We also were also blasted by Penguin on September, but I believe this is not the main cause of the drop. We were already running very badly at this time. My actions are: a) I generated a DTOX report and I'm trying to clean up the links marked as toxic. That's a hard work! After that I will submit a reconsideration request. b) I'm working on the site: Improving internal link building for relevant keywords Recently I removed a "tag cloud" which I believe was hurting my SEO. Also, I did some redirections that were missing. c) I trying to generate new content to improve link building to my site. d) I'm also considering to stop putting all my coins on this domain, and maybe start a fresh new one. Yes, I'm desperate! 🙂 I would appreciate a lot to hear from you guys, expert people! Thanks a lot, MWcEdPa.png?1
Intermediate & Advanced SEO | | rodrigofreitas0 -
My site falls more than 50 in two days, help me!
My site appeared in the top 10 for that link (http://www.vipgoldrj.com/paginas/ensaios.html) and not by this (http://www.vipgoldrj.com), was well at 2 months, and he suddenly disappeared, I wanted to know if he had been penalized and Google told me it was not. What should I do? The. Sorry my English, I am Brazilian and I'm using Google translator. Warning from SEOmoz staff: this is an escort site with full frontal nudity and is not safe for most workplaces.
Intermediate & Advanced SEO | | WebMaster0210 -
Help choosing titles
Good evening guys, I changed my titles last month, in preparation for the over optimisation penalty and the result was an instant and quite dramatic loss in traffic. I believe the reason is, the change resulted in a lot of duplicate titles. My website is similar to deviant art, but for mobile phones. So the titles include the brand of mobile phone for example. The titles were: Upload name + Brand + Content type - 3 tags - FILEID So an example would be Black Nokia wallpaper - black, abstract, grey - 12345 I changed them to Black Nokia wallpaper by artist name on domain name. But this resulted in thousands of duplicate titles and a dramatic loss in traffic. For example a user could upload 20 black wallpapers. With this in mind, I need to change my titles and fast. But I don't want to make another mistake. The one I am quite keen to try is: Black Nokia Wallpaper - Tag1, tag2 wallpapers - on domain name. So the main variable would be the name of the upload and then the 2 tags, to mix things up a little. Another option would be to throw the file ID in there somewhere? As that will always be unique. Perhaps the file ID could be in the place of the "wallpapers" after the two tags? I'd like to keep the domain name, for branding reasons. Any other suggestions are warmly welcomed. Thanks a lot.
Intermediate & Advanced SEO | | seo-wanna-bs0 -
I am not a SEO expert , need a gentleman to help me
I have been hit by Latest google update . dont have enough money to pay seo consultants , just wondering if anyone expert can off a free analysis of my site and can point problems. Its not a adsence site . My living depends on this site so cant just keep on doing experiments , by the looks I have been hit hard by negative seo by competitor . I give url in private . Is there any one kind here who can help in private . Would really appreciate .
Intermediate & Advanced SEO | | HateDoingSEO0 -
Redirect Help - Domain Change and Website Redesign
Hi there, I've redesigned a website for a client, but we are also changing domains and I'm trying to figure out the best way to set up the redirects from their old domain to the new one. 95% of their search engine traffic originally came through brand related keywords that landed on their homepage, and most of the remaining 15% landed on 3 other pages. The new site has pages to replace these 3 main SEO pages, and I'm about to set 301 redirects from their old domain, but I can't figure out the quickest/best way to do it. Is it possible to set up a specific redirect for the 4 main pages (Home + plus the 3 others) then a "catch all" type of thing for the rest of the pages, that redirect either to the homepage, or some sort of "Check out our new Site" landing page. How do you do this, or is there a better way to set it up? Thanks!
Intermediate & Advanced SEO | | timscullin0 -
Google +one button - help needed
I have installed Google plus button on my homepage. My GA report is being displayed like this - 7 total events were recorded Total Events Unique Events 7 2 Event action Total Events on 4 off 3 My questions are - 1 ) Is someone clicking on Google plus button considered an Event ? 2 ) What is meant by 2 Unique Events 3 ) Why does GA report shows on 4 and off 3
Intermediate & Advanced SEO | | seoug_20050