Question about robots.txt
-
Solved!
-
Just a friendly reminder. Please don't delete your question after it's been answered. It's very likely that someone in the future could have the same question and they would have been able to find the answer if you hadn't deleted the question.
-
Consider deleting all of this:
Disallow: /&limit
Disallow: /?limit
Disallow: /&sort
Disallow: /?sort
Disallow: /?route=checkout/
Disallow: /?route=account/
Disallow: /?route=product/search
Disallow: /?route=affiliate/
Disallow: /?marca
Disallow: /&manufacturer
Disallow: /?manufacturer
Disallow: /?filter
Disallow: /&filter
Disallow: /?order
Disallow: /&order
Disallow: /?price
Disallow: /&price
Disallow: /?filter_tag
Disallow: /&filter_tag
Disallow: /?mode
Disallow: /&mode
Disallow: /?cat
Disallow: /&cat
Disallow: /?product_id
Disallow: /&product_id
Disallow: /?route=affiliate/
Disallow: /*?keywordThose rules are telling Google not to crawl domain.com/EVERYTHING(then the URL parameter). This could be where the issue stems from. If you're worried about URLs with these things ranking, consider implementing canonical tags instead to point to the proper pages
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Url permalink structure question!
Hello! I just read the "15 SEO Best Practices for Structuring URLs" but I have still a question: My A version bellow "20 accessoires" has no meaning in french. If I add "voyage" (version B), then is it considered as spammy? I mean the "voyage" keyword repetition? A) http://www.lytchee.com/preparer-son-voyage/20-accessoires/ B) http://www.lytchee.com/preparer-son-voyage/20-accessoires-voyage Thanks for advices! Sylvain
On-Page Optimization | | lytcheetv0 -
To redirect or not to redirect...That is the question
We have old product data that is being updated (approx 30k sku's). Most of the old pages have been indexed and drive the majority of our online sales. But, the updated products have different titles, H1's, etc and have been re-formatted for the better in terms of usability. So, should we redirect thousands of pages to the new pages? Will Google be ok with that?...or should we reformat the current url's with the new data? And, if we reformat current url's with new data...the fear is that the on-page elements that the pages are being pick up for will get wiped out, (be it the H1 in some cases or the title or alt tag for another, etc) because it's simply too much to retain them during the import. Please share
On-Page Optimization | | nisvision0 -
A Hummingbird Question
So with Google SERPs looking for more customer intent and especially to direct questions eg, How much does a x cost? Where can I find x etc - is it good practice to create a bank of Q&A's on site or is it sufficient to have good quality content on the subject matter stating the costs of x and where you can find it written in a customer friendly manner? Will a bank of Q&As add any value if they directly answer the most likely search questions? Thanks Ash
On-Page Optimization | | AshShep10 -
Internal linking question
I have a Q&A site which I am struggling to get ranking well in Google. I had a few questions 1. Should I be doing some internal linking and if so how is it best to do this? Should I go back into users questions and change certain keywords to links if applicable? 2. Should I be nofollowing the index pages ie /p1 /p2 of discussions? 3. If anyone has chance to look at the site and let me know if there is anything else I am doing wrong I would be really grateful. Thanks
On-Page Optimization | | oliboy0 -
Site restructure question
Our site was deigned years ago to target customers in specific cities, now we've grown beyond this and I believe it is time to change the site structure.
On-Page Optimization | | PM_Academy
Ignore the 302 from the root page. Current structure: (assuming you've never been to our site before) projectmanagementacademy.net 302->/select-location.php /select-location.php -> /city-name/pmp-training.php This page was meant to be a "homepage" for each city, pointless page really /city-name/pmp-training.php -> /ciy-name/product-name.php These pages are for each individual product My suggested site structure: /city-name/pmp-training.php becomes projectmanagementacademy.net no more redirect /city-name/pmp-training.php gets removed and 301 to root page. /product-name.php each product's page and you would select a location when necessary (some products are online only) would 301 each /city-name/product-name to corresponding product page /product-name/city-name.php could add these pages if we still wanted the city name in url for city specific products My thoughts here are /product-name.php would receive a higher % of link juice because there are fewer page between 2 vs 4 if you came to the root page. and 2 vs 3 if you came from the select-location page. Also instead of being split between over 50 locations, all these would be together on one page. Your thoughts? Would this change improve our SERP for those product pages? Would we see a drop off in traffic if we did this? How long, if done correctly, would it take to see the recovery of rankings and traffic? Could we 301 /select-location.php to the root page? Thanks in advance for your insights to this. Any answer is a good answer. Trenton0 -
Using Transcription Service For Videos - Have Question Around Search and Spiders
Hi All, So I have put together a weekly video series on security topics. I have read an SEOmoz post around how you can boost SEO by adding the transcription to the page, which makes perfectly good sense. My question is, can I include the first couple of paragraphs and then have a "read the full transcription" so when the user clicks, the rest of the content appears? Do the search engine spiders only crawl the first two paragraphs in this instance or do they crawl the whole thing even though the entire content is not on the page? I dont mind making the page longer and including the entire transcription if it is easier for SEO but if there is no difference, than I think the first option would be the best user experience. Thanks for the help Pat
On-Page Optimization | | PatBausemer0 -
Blog Question
Hello! I try to add as much fresh content to my site weekly, I do also have a blog that I update 3 times per week. Will google crawl frequently if I am updating info on my wordpress hosted blog when my blog name is www.blog.mywebsite.com? and the sites name is www.mywebsite.com? Will they treat them as separate domains? Thank you!!
On-Page Optimization | | TP_Marketing0 -
Quick question about bold italics keywords in today's SEO world
Hello guyz do you think that , **or **tags still help you in ranking better for some keyword or this method has become obsolete?****
On-Page Optimization | | ksbnok0