Why is Amazon crawling my website? Is this hurting us?
-
Hi mozzers,
I discovered that Amazon is crawling our site and exploring thousands of profile pages. In a single day it crawled 75k profile pages.
- Is this related to AWS?
- Is this something we should worry about or not? If so what could be a solution to counter this?
- Could this affect our Google Analytics organic traffic?
-
@jacombo Amazon requires sellers to offer a minimum 30-day return window, accept returns of new, unopened items, and process refunds within two business days. Sellers must cover return shipping costs for errors or defective items. Compliance with safety and security standards is also essential.
To ensure your return policies meet these standards:
Stay Updated: Regularly review Amazon’s guidelines.
Clear Communication: Clearly state your return policy on listings.
Quality Control: Implement a rigorous quality control process.
Amazon Consulting Service: Consider using an Amazon consulting service to audit and improve your policies. -
What are the basic guidelines and requirements for third party sellers' return policies on Amazon, and how can I ensure they meet safety and security standards for buyers?
-
My website https://www.avenuessouthresidences.com also has traffic coming from Amazon bots. I wonder if it's my competitors doing something funny or it's Amazon itself doing it.
-
this is not about moz
-
- Is this related to AWS? Yes, most likely a customer on AWS scraping your site.
- Is this something we should worry about or not? If so what could be a solution to counter this? Monitor it. If the crawling get's ridiculous, you could have site performance issues. Could deny IP addresse(s) in .htacces or try via Robot.txt file.
- Could this affect our Google Analytics organic traffic? If your site performance suffers, could affect organic position and decrease traffic.
Good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Single topic website or as part of a multiple topic website?
I have content sitting on a site here - https://www.pfizerpro.co.uk/product/xeljanz/rheumatoid-arthritis - domain authority 25 page authority 18 - the pages went live three months ago and the website was launched 18 months. We now have the option to use a brand new domain www.xeljanz.co.uk Which is the better option to stick with the www.pfizerpro.co.uk as it is a larger multiple topic site that should attract more links or to start a new single topic site which google may view as the better source as it is dedicated to the topic? Thanks
Intermediate & Advanced SEO | | Kate_team_DM0 -
Should I delete 100s of weak posts from my website?
I run this website: http://knowledgeweighsnothing.com/ It was initially built to get traffic from Facebook. The vast majority of the 1300+ posts are shorter curation style posts. Basically I would find excellent sources of information and then do a short post highlighting the information and then link to the original source (and then post to FB and hey presto 1000s of visitors going through my website). Traffic was so amazing from FB at the time, that 'really stupidly' these posts were written with no regard for search engine rankings. When Facebook reach etc dropped right off, I started writing full original content posts to gain more traffic from search engines. I am starting to get more and more traffic now from Google etc, but there's still lots to improve. I am concerned that the shortest/weakest posts on the website are holding things back to some degree. I am considering going through the website and deleting the very weakest older posts based on their quality/backlinks and PA. This will probably run into 100s of posts. Is it detrimental to delete so weak many posts from a website? Any and all advice on how to proceed would be greatly recieved.
Intermediate & Advanced SEO | | xpers1 -
Would you rate-control Googlebot? How much crawling is too much crawling?
One of our sites is very large - over 500M pages. Google has indexed 1/8th of the site - and they tend to crawl between 800k and 1M pages per day. A few times a year, Google will significantly increase their crawl rate - overnight hitting 2M pages per day or more. This creates big problems for us, because at 1M pages per day Google is consuming 70% of our API capacity, and the API overall is at 90% capacity. At 2M pages per day, 20% of our page requests are 500 errors. I've lobbied for an investment / overhaul of the API configuration to allow for more Google bandwidth without compromising user experience. My tech team counters that it's a wasted investment - as Google will crawl to our capacity whatever that capacity is. Questions to Enterprise SEOs: *Is there any validity to the tech team's claim? I thought Google's crawl rate was based on a combination of PageRank and the frequency of page updates. This indicates there is some upper limit - which we perhaps haven't reached - but which would stabilize once reached. *We've asked Google to rate-limit our crawl rate in the past. Is that harmful? I've always looked at a robust crawl rate as a good problem to have. Is 1.5M Googlebot API calls a day desirable, or something any reasonable Enterprise SEO would seek to throttle back? *What about setting a longer refresh rate in the sitemaps? Would that reduce the daily crawl demand? We could set increase it to a month, but at 500M pages Google could still have a ball at the 2M pages/day rate. Thanks
Intermediate & Advanced SEO | | lzhao0 -
Website losing rank positions after https
Hi there.. we were having a good increase in rankings since using Moz and then we purchased an ssl certificate so our website would run under https... to make sure our website is only accessed through https, we have used a method on .htaccess (appache) to make sure the website is only reached through https, using: RewriteEngine on
Intermediate & Advanced SEO | | mszeer
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}/$1 [R,QSA] after doing this, from our last crawl perspective, we lost several positions that we were increasing.. what have we done wrong ? we have to fix this asap, please help us...0 -
Worth it to redirect .swf during a website migration?
Recently one of our clients switched web providers. We've been implementing 301 redirects to the new URL structure for the most important pages. We noticed upon doing a site search, Google has indexed a .swf URL on page 3 of search results. Please see attached image. 1. Should I implement a redirect for this? There are no links to this filetype.
Intermediate & Advanced SEO | | EEE3
2. There are three others (.swf) that are indexed I would think the best thing to do is to have the new website return a 404 for this page, as it's currently doing, and it will eventually drop out of the index. Any other suggestions? Thanks! g2tur4r0 -
100% links within the website - could it rank?
Hi, If I create blog posts inside my website and link it back to my website, Would it still rank? I understand that its better to get links from the established domain. But I just wonder what kind of impact would my site have if the blog posts within my site link back to itself for ranking. Please let me know. thanks
Intermediate & Advanced SEO | | zsyed0 -
Is it Penguin or is it just a shoddy website?!
Right, I'm struggling and could really do with a different perspective on a website I am working on http://www.data-contracts.co.uk/ This is an Analytics image for March through to July of organic unique visitors http://imgur.com/Y2zaekP The page for concrete repair is struggling in the rankings and is currently residing somewhere like page 15 on Goooooogle. Is there anyway back??? Y2zaekP
Intermediate & Advanced SEO | | Hughescov0 -
What is the value of Google Crawling Dynamic URLS with NO SEO
Hi All I am Working on travel site for client where there are 1000's of product listing pages that are dynamically created. These pages are not SEO optimised and are just lists of products with no content other than the product details. There are no meta tags for title and description on the listings pages. You then click Find Out more to go to the full product details. There is no way to SEO these Dynamic pages This main product details has no content other than details and now meta tags. To help increase my google rankings for the rest of the site which is search optimised would it be better to block google from indexing these pages. Are these pages hurting my ability to improve rankings if my SEO of the content pages has been done to a good level with good unique Titles, descriptions and useful content thanks In advance John
Intermediate & Advanced SEO | | ingageseo0