Crawl rate drop
-
Hi Guys,
I have a crawl rate drop in webmastertools and can't figure out way. In the last month I removed a lot o duplicate pages that we don't need anymore, there were at least 1.5 million pages. Can this be a motive?
-
Hi there.
Of course it can! The crawl rate is in pages per day, so if you remove pages (especially 1.5 million), there won't be as much to crawl
Also it can happen, if you have the same static pages and the crawl bot has crawled them all. Google doesn't crawl all pages all the time, they have limited resources. So, if you have launched or updated website recently and now not really updating it, you can see the change in crawl rate.
However, you can change crawl rate for 90 days, if you need them to crawl your website constantly (usually good for websites in process of reconstruction). Here: https://support.google.com/webmasters/answer/48620?hl=en
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Two high ranking pages instantly dropped from index - no manual penalty notification
We are facing an issue where two of our major rankings pages have just completely disappeared from search results. This has happened in the last 24 - 48 hours and there has been no changes made to the site. From what we can tell, it's only impacted two pages (but two very important category pages). I have double and triple checked all standard indexing protocols - Search Console URL inspection says the pages are fine to crawl and index. URLs have been requested to re-index but nothing has worked. This would have me to believe it could be a manual action yet there are no notifications in Search Console and we are listed as 'No issues detected' in all versions of our web property. Can anyone else think what could be the reason?
Intermediate & Advanced SEO | | Vuly0 -
Site dropped in search for 95% after 18-19 feb
Hi people. I have a client which holds a escort agency website operating in Amsterdam the Netherlands. Please take note that Escort Agency is legal and holland is a liberal country with permit and all, so just trying to point out that it's a perfectly legal crafted business. To the point: we are seeing a 95% drop in traffic (basicly back to barely 10 clicks a day) after 18 to 19 feb. We've inspected the incoming links, and we did see some things going on that where not quite healthy. At first a banner was bought on a dutch advertising website. On their behalf, this sitewide banner got published with a follow on roughly 132k of pages. That was strike 1 (i think). After tossing this in disavow for temporarily basis and informed the advertisal website to put any sitewide link to nofollow in the first place, nothing changed. We found a 2nd site doing the same mistake. Frankly the banner got exposed on roughly 100k of pages on which some of 'm where barely 2 days old. Strike 2. Solved this by putting it in disavow for the time being and asking politely to put the banner again, nofollow. We cleaned out any incoming potential spammy links by using disavow. The data we obtained was a mix of google webmaster itself and moz profiling. However we're one month further now, and the graph is still a big phat flatline. What is going on? I've noted that, one other sites, which share the same brand, but completely different websites / subnets / content and all, has the same threatment going on showing a huge drop from 18th of feb 2019 and is unable to recover. We cannot see anything 'bad' actually going on @ webmasters and there is no manual action taken. So we're kind of stuck now on a site that was my project but now completely fell into oblivion and hurting someone's business. The url is https://www.qualityescort.nl/ - anyone has a reasonable advice to this issue? full.jpg full.jpg
Intermediate & Advanced SEO | | Jvanderlinde0 -
Ranking drop for "Mobile" devices category in Google webmaster tools
Hi, Our rank dropped and we noticed it's a major drop in "Mobile" devices category, which is contributing to the overall drop. What exactly drops mobile rankings? We do not have any messages in search console. We have made few redirects and removed footer links. How these affect? Thanks,
Intermediate & Advanced SEO | | vtmoz
Satish0 -
Google crawling different content--ever ok?
Here are a couple of scenarios I'm encountering where Google will crawl different content than my users on initial visit to the site--and which I think should be ok. Of course, it is normally NOT ok, I'm here to find out if Google is flexible enough to allow these situations: 1. My mobile friendly site has users select a city, and then it displays the location options div which includes an explanation for why they may want to have the program use their gps location. The user must choose the gps, the entire city, or he can enter a zip code, or choose a suburb of the city, which then goes to the link chosen. OTOH it is programmed so that if it is a Google bot it doesn't get just a meaningless 'choose further' page, but rather the crawler sees the page of results for the entire city (as you would expect from the url), So basically the program defaults for the entire city results for google bot, but for for the user it first gives him the initial ability to choose gps. 2. A user comes to mysite.com/gps-loc/city/results The site, seeing the literal words 'gps-loc' in the url goes out and fetches the gps for his location and returns results dependent on his location. If Googlebot comes to that url then there is no way the program will return the same results because the program wouldn't be able to get the same long latitude as that user. So, what do you think? Are these scenarios a concern for getting penalized by Google? Thanks, Ted
Intermediate & Advanced SEO | | friendoffood0 -
Can spiders crawl javascript navigation now?
I was reading Danny Dover's book and decided to try some websites and so far everyone I have looked at has had navigation that does not work with disabled javascript. Is this still as important as it was at the time of publish (2011)? Thanks!
Intermediate & Advanced SEO | | Sika220 -
Rich Snippets Ratings For Q&A Discussions, Articles,
Hi, I'm looking for how I can use a star rating for a q&a discussion or article/blog post to achieve a rich snippets search result. I'm thinking about a user rating for "Was this helpful?" 1 to 5 stars. As I look at schema.org and do and other reading on it, it looks like it's possible to rate only a set group of content types, blogs and discussions not included. However, I've seen rich snippets ratings in SERPs for blog posts, like this example https://www.google.com/search?q=erp+implementation+challenges&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a#q=panorama+consulting+blog&client=firefox-a&hs=gId&hl=en&rls=org.mozilla:en-US:official&ei=QmCBUYLLCOfwiwKHhIAQ&start=20&sa=N&bav=on.2,or.r_cp.r_qf.&bvm=bv.45921128,d.cGE&fp=eb2f15e2a98a4631&biw=2144&bih=995 On page, it looks like they used some simple span tags. So, my question is, which content type category does that fit into for rating and is that strategy safe enough going forward? Also, are there more steps to making this work? It it is okay to have users rate the helpfulness of a discussion or article and get rich snippets, I'd kinda like to do it. Best... Darcy
Intermediate & Advanced SEO | | 945010 -
Can a home page penalty cause a drop in rankings for all pages?
All my main keywords have dropped out of the SERPS. Could it be that the home page (the strongest) page has been devalued and therefore 'link juice' that used to spread throughout the site is no longer doing so. Would this cause all other pages to drop? I just can't understand how all my pages have lost rankings. The site is still indexed so there's no problem there.
Intermediate & Advanced SEO | | SamCUK0 -
Dropped 50 places for Page Title Change
Hey Guys, I've got a keyword set that is really annoying me. You see on the 19th July I changed my page title for the home page to what I thought would be something more competitive and instead of climbing from rank 9/10 I dropped out of the top 50. So I waited 2 weeks to see if it changed and it just stayed out of the top 50. After two weeks I changed it back again and I've been out of the top 50 ever since. I havnt made any real significant changes to the site as I've held off waiting for the keyword ranking to return but I dont think it will. I was trying to improve the keyword phrase "bahamas all inclusive" for the website www.smallhope.com. I'd managed to get on to page 1 rank 9/10 depending. I changed the title; from "Bahamas All Inclusive - Andros Bahamas Hotel - Bahamas Vacation - Small Hope Bay Lodge All-Inclusive" to "Bahamas All Inclusive Vacation - Small Hope Bay Lodge All-Inclusive" This is really bad for us and I'm not sure what to do. I had a whole plan in place starting with the Title and moving on to the H1 tags, but now I'm not so sure. I've put everything on hold. If this was you what would you do to get back to page 1? Thanks, Adam
Intermediate & Advanced SEO | | NaescentAdam0