Google Shopping Feed being blocked by robots.txt
-
I had created a manual Google Shopping Feed that was working fine, and then someone well meaning put a block in my robots.txt file so Google couldn't read the images folder. because of this, Google now won't accept my feed.
I changed the robots.txt file to allow them to read the images again, but it's been 3 days now and I'm still getting the error saying my products are disallowed because the robots.txt file won't let them scan for images.
Does anyone know how long it will take for Google to see it again?
-
When I manually fetch, the /images folder isn't on the list anymore.
For now I have just reuploaded the images in to a new folder and I'm creating my feed by hand, so that should get around it for now.
Thanks for your help.
-
Did you try manually fetching as Googlebot the robots.txt file via the webmaster tools? If not, do that, then click submit to index. Then do the same thing for some of the images or better if you have a file linking to all images (like an image sitemap). Once that is done, let Google recrawl everything for a few days (even a week) and try again.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Google Ads - Subdomain on Sitelinks & Composition Change for Strategy Status
I have a basic query but could not find a definite answer on the internet. I am currently running a campaign for the main website of a big education brand and they also have a secondary learning website on subdomain, and I want to add sitelinks of subdomain to the campaign, but I am not sure whether it is allowed or not. The brand I am running ads for is https://www.rauias.com/ and the secondary website is https://compass.rauias.com/ branded slightly different in a subdomain, so should I add the sitelinks of Compass to the main campaign? Also one more silly question My Max Conversion search campaign gave me this status today. "Learning (composition change): Campaigns have been added to or removed from the bid strategy. Google Ads is now adjusting to optimize bids. 5 days left for learning" What does this mean exactly? And Why does it reenter the learning phase whenever I make a small change?
Paid Search Marketing | | rauoff0 -
How Google Search works
I watched this video over the weekend. If you haven't yet, this is a great documentary every SEO client should watch. watch?v=tFq6Q_muwG0
Paid Search Marketing | | WebDaytona0 -
Impressions data: Google Webmaster Tools VS Google AdWords
Dear community, I was doing some research within Google Webmaster Tools (WMT) keyword data when I noticed that the impressions within this tool are quite different compared to the keyword impression data provided by Google AdWords (ADW). Some specs about the situation: Date: December 2014 (31 days). Keyword: brand + main keyword* (e.g. amazon shoes, if Amazon would have been the client) Visitors come from: the Netherlands (>97%). Search volume: 3,8 K for this branded keyword last year (December 2013), of which >99% came from the Netherlands (source: Google Keyword Planner). Search query data: "impressions" for Google WMT (organic keyword impressions) and “impressions” for Google ADW (paid keyword impressions). Of course the ADW keyword is [exact]. So, what's wrong? The data doesn’t match, while I expected approximately the same amount of impressions assuming the keywords are: both [exact] keywords; (check! ✓) within the same period of time; (check! ✓) the Ad is being displayed "all the time" (check! ✓) the domain/page is being indexed "all the time" (check! ✓) How much of a difference is there? Organic impressions: 7,4 K (source: Google WMT) Paid impressions: 2,1 K (source: Google ADW) Note that the search volume according to Google Keyword Planner of last year is somewhere in between: 3,8K. The search volume from this tool of last December is not available just yet (but I don’t expect much difference here since Google Trends shows a steady search volume). If the difference would have been 10-20%, I wouldn't be surprised at all, but this is huge. **What could explain the differences? ** If a lot of people were using AdBlockers (they do, but not nearly as massively (around 10%)). If we would have made mistakes in AdWords: budget, bidding, targeting etc. - This is not the case, got this confirmed by the manager who double-checked the data and settings. Also: since it’s a branded keyword it’s really cheap for us and easy to get high quality scores. If we would have made mistakes regarding indexing/crawling that would have caused an extreme loss in domain visibility in the SERP's: possibly caused by robots.txt, a noindex-tag, server problems etc. This was not the case and Google WebmasterTools says the average position was 1,0 during the complete month. "Something else" went wrong during that specific period of time with this specific domain. I don’t think so because I checked multiple months and multiple other domains of other clients. These gave me the same relative results (okay, some were a bit closer: 30 K paid impressions vs 62 K organic impressions for instance, still a big difference). What other possibilities are left? The impressions from Google WMT and Google ADW are not the same, even though they are called the same and therefore suggest they should be (about) the same. AdWords just randomly fails to display and/or measure some branded ads (even though there is plenty of budget, bidding is fine and QS is 10/10). Definitions of "impressions" for both tools according Google: AdWords definition of "impressions" WebmasterTools definition of "impressions" Hope someone has some more suggestions or useful links! Thanks in advance! Ektor Tsolodimos
Paid Search Marketing | | BlueMango0 -
Why does my google analytics show a massive discrepancy from facebook's reported website clicks?
We're running a Facebook news feed ad that is pointing at our homepage. Facebook says that for yesterday there were 47 website clicks. Google analytics shows 15 total visitors from facebook with 3 of them landing on the homepage. I understand that there is likely going to be some discrepancy with users accidentally clicking and clicking back before the page loads, but this seems a little insane. I tested the ad using a page that pulls the Analytics cookie data using php and it is working properly so I don't understand what's happening. The url isn't tagged with utm parameters, which is going to be fixed. Anyone experience this or have any insight as to what could be this issue? Is this click fraud? Edit: For more clarification I was checking on my completely unfiltered google analytics profile/view.
Paid Search Marketing | | spencerhjustice0 -
A heads-up about problems with the new Google Adwords Keyword Planner tool
I noticed that when I was using this tool, the data in the tables would change, often by a lot (up to 50%) when I sorted the columns. (Did nothing else, just sorted.) Since I was starting a new campaign and really needed data, I contacted Google about this and was told: "I understand that the estimates are changing in the Keyword Planner tool when you sort the data. I reproduced this on my end as well and was able to confirm that this is a known technical issue on our end." "Unfortunately the information in the Keyword Planner Tool is currently not the best source of data for your analysis." They then went on to suggest I just test things blindly. So, what do you do to get information about keyword traffic and ppc estimates?
Paid Search Marketing | | Linda-Vassily0 -
Google Adwords Book
Does anyone have a suggestion on a very recent Google Adwords book?
Paid Search Marketing | | phogan0 -
Your Google AdWords account has been permanently suspended for repeated violation of AdWords or Landing Page and Site policies in this or a related account.
My client nor I received any warning. We even had a google adwords team optimize the account and my rep does not yet know the reason for the ban. Not sure if its related but their google organic rankings dropped significantly at the same time. https://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=164786 Any advice here? Do these Questions get indexed by google? I will ask my client if I can disclose the domain. Is there any way around a permanent ban? They were spending 50K per month. Is this enough to have any clout?
Paid Search Marketing | | webbroi0 -
Paid Comparison Shopping Engines
I am looking into starting to use the comparison shopping engines. Any ideas on where to start wtih this? Is there a good place to go that would manage all the different sites?
Paid Search Marketing | | Shopadorn0