Question about robots.txt
-
Solved!
-
Just a friendly reminder. Please don't delete your question after it's been answered. It's very likely that someone in the future could have the same question and they would have been able to find the answer if you hadn't deleted the question.
-
Consider deleting all of this:
Disallow: /&limit
Disallow: /?limit
Disallow: /&sort
Disallow: /?sort
Disallow: /?route=checkout/
Disallow: /?route=account/
Disallow: /?route=product/search
Disallow: /?route=affiliate/
Disallow: /?marca
Disallow: /&manufacturer
Disallow: /?manufacturer
Disallow: /?filter
Disallow: /&filter
Disallow: /?order
Disallow: /&order
Disallow: /?price
Disallow: /&price
Disallow: /?filter_tag
Disallow: /&filter_tag
Disallow: /?mode
Disallow: /&mode
Disallow: /?cat
Disallow: /&cat
Disallow: /?product_id
Disallow: /&product_id
Disallow: /?route=affiliate/
Disallow: /*?keywordThose rules are telling Google not to crawl domain.com/EVERYTHING(then the URL parameter). This could be where the issue stems from. If you're worried about URLs with these things ranking, consider implementing canonical tags instead to point to the proper pages
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Meta Robots information
Hi, I have a question about the Meta Robots information Accoarding to the Moz bar, our page uses the meta robots noodp and noydir. Our competitor uses
On-Page Optimization | | AdoBike
INDEX,FOLLOW I read that noodp and noydir are dated and not used anymore. Is it wise to use INDEX FOLLOW instead for better SEO? Thanks in advance!1 -
Another anchor text question
Hi, We are updating a site and creating new separate pages for each coworker (app. 50 coworkers). Each page will include picture, short description, cv and also a subtitle/"keyword list" called: "areas of expertise". Each keyword (area of expertise) has it´s own in depth page on the subject, would it be wise to anchor text -link these? or is this something that is likely to be picked up by google as spam? Thanks!
On-Page Optimization | | Agguk
/Anders0 -
2 Canonical questions
QUESTION 1
On-Page Optimization | | Marketing_Today
I'm working on a site where a canonical link is implemented as That's not the normal format that I am used to, as I would have written it as Is that a problem? QUESTION 2
Why do so many sites these days have a sitewide canonical tag which refers to itself as the canonical?
For example: http://www.site.co.uk/page has a canonical of0 -
Disavow Question
Hey, In webmaster tools i have 520 domains linking to my site and i disavowed some today, when i was done it said "you successfully uploaded a disavow links file with 75 URLs and 0 domains". First question, did i do this right? Second question if a site is linking to me on multiple pages on their site do i have to disavow each link or just the domain is enough. When i download the all links to my site it was around 5,000 from the 520 domains. I appreciate the help.
On-Page Optimization | | benjaminmarcinc0 -
To redirect or not to redirect...That is the question
We have old product data that is being updated (approx 30k sku's). Most of the old pages have been indexed and drive the majority of our online sales. But, the updated products have different titles, H1's, etc and have been re-formatted for the better in terms of usability. So, should we redirect thousands of pages to the new pages? Will Google be ok with that?...or should we reformat the current url's with the new data? And, if we reformat current url's with new data...the fear is that the on-page elements that the pages are being pick up for will get wiped out, (be it the H1 in some cases or the title or alt tag for another, etc) because it's simply too much to retain them during the import. Please share
On-Page Optimization | | nisvision0 -
Site restructure question
Our site was deigned years ago to target customers in specific cities, now we've grown beyond this and I believe it is time to change the site structure.
On-Page Optimization | | PM_Academy
Ignore the 302 from the root page. Current structure: (assuming you've never been to our site before) projectmanagementacademy.net 302->/select-location.php /select-location.php -> /city-name/pmp-training.php This page was meant to be a "homepage" for each city, pointless page really /city-name/pmp-training.php -> /ciy-name/product-name.php These pages are for each individual product My suggested site structure: /city-name/pmp-training.php becomes projectmanagementacademy.net no more redirect /city-name/pmp-training.php gets removed and 301 to root page. /product-name.php each product's page and you would select a location when necessary (some products are online only) would 301 each /city-name/product-name to corresponding product page /product-name/city-name.php could add these pages if we still wanted the city name in url for city specific products My thoughts here are /product-name.php would receive a higher % of link juice because there are fewer page between 2 vs 4 if you came to the root page. and 2 vs 3 if you came from the select-location page. Also instead of being split between over 50 locations, all these would be together on one page. Your thoughts? Would this change improve our SERP for those product pages? Would we see a drop off in traffic if we did this? How long, if done correctly, would it take to see the recovery of rankings and traffic? Could we 301 /select-location.php to the root page? Thanks in advance for your insights to this. Any answer is a good answer. Trenton0 -
Couple of noob questions
Hi, couple of easy questions for you guys... Im a web designer and my #1 keyword is web design huddersfield. I own the domain huddersfield.com, and i will have a page titled web design. Now do i target the web design page or the index page as i don't want keyword cannibalization, but is it better for my index page to be the landing page? #2 Considering that huddersfield.com is my domain should i set the title to: web design huddersfield or just web design? please help.
On-Page Optimization | | klsdnflksdnvl0 -
Question about Multi-national Websites
I am about to work on a multi-national site and need some more information about what I should consider regarding: content keyword research anything else My biggest question is regarding content. The company would like a UK version of the site with a different URL, but plan to keep the content essentially the same, with the exception of a few minor details. In this case, would duplicate content still be an issue? If so, any suggestions for working around this? Any strategy information on multi-national sites would be really helpful. Thank you! Erin
On-Page Optimization | | HiddenPeak0