Google Shopping Feed being blocked by robots.txt
-
I had created a manual Google Shopping Feed that was working fine, and then someone well meaning put a block in my robots.txt file so Google couldn't read the images folder. because of this, Google now won't accept my feed.
I changed the robots.txt file to allow them to read the images again, but it's been 3 days now and I'm still getting the error saying my products are disallowed because the robots.txt file won't let them scan for images.
Does anyone know how long it will take for Google to see it again?
-
When I manually fetch, the /images folder isn't on the list anymore.
For now I have just reuploaded the images in to a new folder and I'm creating my feed by hand, so that should get around it for now.
Thanks for your help.
-
Did you try manually fetching as Googlebot the robots.txt file via the webmaster tools? If not, do that, then click submit to index. Then do the same thing for some of the images or better if you have a file linking to all images (like an image sitemap). Once that is done, let Google recrawl everything for a few days (even a week) and try again.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Merchant Center product reviews
My googlefu is not good enough to put the words together to find an answer for this, but I am sure there is one out there. I am making a review feed for a client's site to submit their product reviews to google merchant center. But one problem we are having is that the reviews are generalized, like how amazon does it. A sample product would be Whey Protein, this product is available in 4 different sizes and has 4 different GTIN's (one for each size). But the reviews we collect are just for the base product of Whey Protein. What I am trying to figure out, is if I can assign a parent GTIN, then correlate the reviews to that GTIN and then let them populate for the child GTIN's some how.
Paid Search Marketing | | LesleyPaone0 -
Looking at google shopping results from other country
Hi, I run shops in several languages out of London. One of our key revenue drivers is google shopping. It is important for me to look the the shopping search results for example in germany. Recently Google changed something so when I want to look at the german shopping results from here, eg. http://www.google.de/shopping it always shows me the english ones with prices in pounds. Is there a trick to still get the foreign results? Thanks in advance Dieter
Paid Search Marketing | | Storesco1 -
Site penalized by google
I noticed for a while that our competitors don't rank for any keywords organically and they are using paid ads to gain traffic. They have been in the industry for over 6 years and we are only new and have ranked above them already. How can we find out if a competitor has been penalized? Thanks
Paid Search Marketing | | edward-may0 -
Google URL Builder / Campaign Tracking on two Different Domain using the Same Analytics Code
Hey Everyone, I think I know the answer to this but I'd like to get some confirmation. I currently have a landing page at "www.xyz.com", it's a separate domain in which only the landing page exists and not a vanity URL which redirects. However, the navigation and all the links on "www.xyz.com" actually link out to "www.abc.com". The domain / landing page "xyz" has the same analytics tracking code as domain "www.abc.com". My question is this, if I use Google URL builder to create custom URL's to track for each ad that I'm running in Adwords, will this data show up in the analytics of "abc" even though it's a separate domain because it has the same analytics code? In other words, does campaign data show only if the domain and the google analytics code line up, or does the domain not matter and as long as you have the same analytics code (despite two separate domains) that campaign data (built through Google URL builder) will show? My hunch and best guess it that as long as the analytics code is the same (regardless of a separate domain) that the data in campaign will show with the custom URL's I build. I'm aware that I can test this and I will but I'd like to get an idea from the community first to make things easier. Anybody have experience with this? Answers greatly appreciated! Thanks!
Paid Search Marketing | | EvansHunt0 -
CPC or CPM for Google Display Network?
HI all I'm setting up my first Remarketing campaign on the display network. I'm targeting those that have visited a specific product page at the moment and therefore there won't be a massive amount of traffic to remarket to - roughly no more than 50 clicks/day. My question is - what is the best bidding option for a campaign like this? Not sure whether to go for CPM or traditional CPC. I have previously found CPM much better for Facebook but obviously thats very different to Google. All help appreciated!
Paid Search Marketing | | SamMaley0 -
What is a good CTR for a Google AdWords Remarketing banner campaign?
Hello there, given that in the banners we offer a promotion with "some bonus if you sign up", what is from your experience a good CTR for a Google AdWords Remarketing banner campaign? Many thanks to everyone that answers. YESdesign
Paid Search Marketing | | YESdesign0 -
SIte wide links by google display / adsense
hey guys, I had a quick question about my back links that might be coming from adsense. I am trying to help my father our on his site and help him recover from penguin. unfortunately his company hired some shady SEOers to do "SEO". little did my dad know the BLs he were getting might be spammy, thats what I am checking now. When I ran a BL report on him ( i use two tools, SEO spyglass and OSE ) i found that there were many "sitewide" links, but i dont think there were placed there by the webmaster, i think they come from adsense. I know he has a PPC campaign where the daily spend is 3k/ day so I know its very possible that this might be the case. If my tool is picking up BLs from these sites ( which are niche related ) does google count these links for ranking purposes? and might this hurt my rankings if google just sees these as "sitewide links"
Paid Search Marketing | | david3050 -
Where are the URLs in Google top ads coming from?
I'm not sure if this new but I noticed it today: Some of the ads that are displayed on top of the SERPs have a title, and then their web address, separated by a vertical bar. For example (see image): Coffee Beans Online | CoffeeBeansDirect.com The same ad, when displayed in the side bar (see 2nd image) doesn't contain the web address. I expected that because if you count the letters in the ad, including the 'coffeebeansdirect.com' it exceeds the amount allowed for a google ad. My question is: How did it get there? There's no room to distinguish between top ads and side ads when you write a new a new ad in adwords so where did it come from? Plus, notice that both CoffeeBeanDirect.com and ROASTe.com use caps that are not used in the green link at the bottom. How did Google know to present the url that way? 549sR 4tAnb
Paid Search Marketing | | 5225Marketing0