Welcome to the Q&A Forum

OK I just checked it and it is now updated to the correct preview.

So it took up to approx 19 hours to change. Although bear in mind I wasn't checking it all the time, could be faster. Also note, if you have started highlighting the old version, when you go back into the saved one it will still have the old preview, so you need to start over again.

I can also confirm the preview in the data highlighter is not connected to Google's cache of the page in the index, as the old version of the page is still cached.

Thanks for the tips Thomas. I had considered doing it 'manually' but wanted to experiment with the data highlighter tool.

I'll keep on eye on it and report back the time it took Google to update to the correct preview.

Hi Catherine,

Although the page is live and can be accessed, it is actually sending a 404 Not Found status.

You can test it here:

http://www.webconfs.com/http-header-check.php I'm afraid cant help identify why it's doing this.

I want to mark up a business address for Google Local, so I thought I would use the data highlighter in WMT. However I only just added the address to the the bottom of the home page and when using data highlighter iit is giving me the old version of page to mark up without the address on.

Rather frustrating, does any body have any experience on the time frame until Google updates the page in the data highlighter?

According to this thread it's not even related to the page re caching:  Data Highlighter: Start link is pulling an old version of page

Could the pages have been blocked in robotos.txt in the past?  It can take an age for Google to re-index pages a proper after having been blocked previously.

Just to add, you don't need a page to physically exist to add a 301 redirect Kristin, so you can set the redirects from these old pages fine.

Hey Gary,

Back in the day when when I wanted Google to notice links I had built I used to ping them and it seemed to work.

Matt Cutts also touches on pinging services such as PubSubHubbub in this video where he is talking about getting Google to notice pages faster: http://www.youtube.com/watch?v=4LsB19wTt0Q#t=77 I used to use https://pingler.com/ and  http://pingomatic.com/.

Also in that vid Matt talks about links on places such as Twitter. You could perhaps build links to these pages elsewhere, but weather they would get crawled any faster depends on the site I guess.

You could also submit the URL to Google. I don't know weather it would speed crawling up, worth a go though.

Hey Mak,

One thing to bear in mind is that the canonical tag should be used on pages with the same content, if there is extra content on Page 2 that doesn't appear on Page 1, then Google could ignore the canonical tag al together:

_The rel="canonical" attribute should be used only to specify the preferred version of many pages with identical content (although minor differences, such as sort order, are okay).

https://support.google.com/webmasters/answer/139394?hl=en_

Just edited it.

Google just posted 5 videos on site hacking that might be of interest:

Google Web Master Videos

Hey Dan,

If you are correctly using rel="alternate" hreflang= meta tags or sitemaps, then it shouldn't matter if you have mostly similar content. One of Google's examples is just like your question:

Your pages have broadly similar content within a single language, but the content has small regional variations. For example, you might have English-language content targeted at readers in the US, GB, and Ireland. Now, your question on IP sniffing is a slightly different kettle of fish. Google contradict themselves in their own guidlines!

On this page, Multi-regional and multilingual sites, they say:

Avoid automatic redirection based on the user’s perceived language. These redirections could prevent users (and search engines) from viewing all the versions of your site. Yet here, Introducing "x-default hreflang" for international landing pages, they say:

The homepages of multinational and multilingual websites are sometimes configured to point visitors to localized pages, either via redirects or by changing the content to reflect the user’s language. And then there is this recent video from Matt Cutts: Is redirecting users based on their location spam?

Ahh yes, <lastmod>.  How did I forget this, I'm usually such a stickler on sitemaps.   Do you have any further info on 2% tolerance Alan, or is that just your personal findings? </lastmod>

You can actually host sitemap.xml files on external domains, you just need too verify them all in Web Master tools.

Guidelines here from Google: Sitemaps for multiple sites

Bear in mind Patrick, you need to be careful with keyword cannibalisation with titling inner pages like this.

If many of your pages have 'City Web Design' in the title there could be conflict with the page that you are targeting that term for (usually the home page).  I would avoid this myself on pages such as About Us, Contact, News, Blog, etc. and probably  just something like use:

• About Us | Company Name

Hey Allan,

You only need to update your sitemaps if you are creating new pages; if you are simply updating the content on pages that already exist in the sitemaps, you do not need to update them.

Hey Ron,

I had a quick look at your site and I couldn't see an obvious reason why it might not be indexed. The page is linked to via your sites navigation and at the 1st level (meaning its linked from your top level pages such as home page), it’s present in your sitemap.xml file, isn't blocked either in robots.txt or at a page level via robots meta tags and it doesn't have a different canonical URL defined.

Might just be one of those mysteries’ things, I've worked on a site before that had some pages that weren't indexed, and we could never fathom out why either.

I did notice something, you have a strap line in your header and footer which could be seen as keyword stuffing by Google. Although this wouldn't result in a page getting indexed, its not good.

its the ‘Titanium-Jewelry.com - The largest selection of titanium rings, tungsten rings and men's wedding bands on the internet!’

And

Really amazing jewelry. And really cool people. We sell titanium rings, tungsten rings, mens wedding bands and other awesome stuff.

Lines I’m talking about.

You have also got some duplicate titles on your news pages you should look at changing.

You might want to carrying out an in depth on-site SEO audit, you may have more issues if I noticed these at a quick glance.

Your slightly actually incorrect Devanur, the reason the wrong page is cached is because the page previously had a canonical tag referencing the other page.

If you look at the cache of http://www.thinkdigit.com/top-products/Laptops-and-PCs/top-10-laptops-124.php  ::::  Click Me

You will see in the source code a canonical tag for the other page:

http://www.thinkdigit.com/top-products/Ultrabooks/top-10-ultrabooks-153.php" />

And the info at the top of cache page confirms Google is counting the one page as the other (see attachment)

oLdv2dF.png

Hey Thomas,

I can actually see your site on searches for both "styleblueprint" or "style blueprint", but I have just noticed you have made a terrible error.

You have blocked search engines from indexing your site by adding the 'noindex,nofollow' robots meta tag to every page.  This meta tag instructs search engines not to index the page. You need to remove this tag ASAP as you site is probably being dropped from the SERPS and that's why you cannot find your site and probably why Moz cant crawl the site.

The code to remove looks like this in your source code:

name='robots' content='noindex,nofollow' /> The reason this code might be on your pages could be your settings on wordpress.  On the left menu in wordpress dashboard click 'Settings', then click 'Reading' and see if the following box is checked: 'Discourage search engines from indexing this site'.  If it is, uncheck it then check all your pages to see if the code has gone.

Hi Ron,

If you had some example URLs that would be helpful, we can check for any obvious issues.

Hi ColumK,

The number of characters in the title is actually defined on pixels as opposed to character limited. I think it’s around 520 pixels wide. As a rule of thumb you can fit approx. 70 latin characters within the space, so that why many people recommend that.

So of course you're right, with non-latin characters you can fit a differing amount of characters in that pixel space. For instance, Thai characters are very small and you can fit many in there.

There is a great tool here, the Google SERP Snippet Optimization Tool. The tool simulates Google's search engine results pages and truncates off approx. where Google should (although it’s not always 100% spot on). You can also visualise the meta descriptions, URL, rich snippets etc. It’s an invaluable tool I use daily. There is also some great humorous SERP research on the site.

You can also set up an excel sheet to truncate the title length at the same amount of pixels as Google does. Set column width to 520px, set columns to wrap text, and font to Arial 12pt. Type in your Title, and bold the main keyword. If the line breaks, your Title tag will truncate. (credit to Vudu Marketing for this tip)

Hi Sean,

The problem with Magento is that out of the box its not particularly that SEO friendly. There are quite a few canonical issues you get using just out of the box, caused by a variety of things such as sort= pages, duplicate category/product pages, duplicate home pages etc.

For example you have a duplicate home page which is indexed in Google here : http://jrdunn.com/home

There are duplicate pages caused by pages such as sort pages:

http://jrdunn.com/designers/marco-bicego.html?mode=grid&p=4 A duplicate of http://jrdunn.com/designers/marco-bicego.html (just sorted different)

Also having a quick look at your site in the index you have hundreds of duplicate home pages indexed. Some of these look like they are being caused by redirects from the old site and the fact that 404's are not correctly configured on the site, and others from search parameters on the new site.

Click for Google Result

The 2nd result from the Google search above is http://store.jrdunn.com/SearchResults.asp?searching=Y&sort=5&search=basha+pink&show=240 

Which is one of the old site URLs which redirects (via a chain of redirects) to a page on the new site:

http://jrdunn.com/?searching=Y&sort=5&search=basha+pink&show=240 Which is a duplicate of the home page.  This looks like non existing  page and should return a 404 status, but you don't have 404's correctly configured, as any URL with a question mark in is a live version of the home page:

e.g:  http://jrdunn.com/?TypeAnyThingHere
Also if you look on page 2 of that Google result above, you will see some URLs that are caused by the sort/search function of Magento and again are duplicates of the home page:

http://jrdunn.com/?searching=Y&sort=5&search=tacori+diamond&show=160 This is really on scratching the surface of the problems that Magento can cause. You really need to carry out an extensive onsite audit, or perhaps consult with a Magento specialist.

Also you need to check the 301 redirects you have set from the old site, as you have all those invalid redirects from the old site that are going to non existing pages on the Magento site.

I hope this is clear and helps give you a starting point for your troubleshooting.

In addition to Neenor and Craigs great tips you also have internal & external duplicate content  and canonical issues.

Click for Google result

The link above highlights an example of internal duplicate content caused by your paginated pages. Ideally you should be using the robots 'noindex, follow' meta tag so the pages are removed  from Google's index, but the product links on the pages are still followed.

Click for Google result The link above highlights an example of external duplciate content where you have used content from Lost & Founds website verbatim. You should make sure all you content is unique.

Also looking at the first link above you will see that some of your URLs are being indexed via secure https URLs while others are just via regular http. It appears that the site isn't utlising canonical tags, so you could potentially be having duplicate page issues where pages are indexed twice, once under https and once more under http.

I'd suggest you do an in-depth onsite audit, as If I noticed these couple of issues in 2 mins, you could potentially have issues elsewhere.

I've had a look at the page and like Martijn said it seems fine to me. I also crawled it as Google bot user agent and it returned a correct 200 status.

It could perhaps be a temporary server issue and the page/site is going down intermediately? You could check your server logs to see if there has been any issues? You could also try a free up time checker such as one of these: 10 Free Services to Monitor Your Site's Uptime.

If the error keeps happening and the page isn't actually going down, might be best to raise the question on Google's support forums; unless of course someone comes along here with the answer!

I don't think there is really a right or wrong answer to this and I think you should be tackling it from a user experience angle, as opposed to an SEO one. It's generally thought of that links in the main navigation are there for a good reason, and shouldn't have an adverse on duplicate/diluting content. (within reason)

Do you think your customers would prefer the ease of navigation that keeping the same linking architecture consistent through out the site would bring? I know as a user I would, so that is how I implement it when adding blogs to eCommerce site whenever possible.

Say you had an article about Wool Socks. Your user might see the Wool Site menu link in the main navigation and might expect another dropdown so they can easily check the different types of wool socks without loading another page. I like to try and keep page clicks down to a minimum when a user is looking for pages.

But as I said, doing it either way is fine and many sites do, below are some examples:

Same linking architecture:
Main site: http://www.overstock.com/
Blog: http://www.overstock.com/blogs

Different linking architecture:
Main Site: http://www.tripadvisor.com/
Blog: http://blog.tripadvisor.com/

If that's the case, then yes I would also go with folders as they are my preferred way to do it.

I've had a couple of Trespass jackets in my time!

Hi Tresspass,

Before thinking about using subfolders and sub domains you should think about using the county top level domains (TLDs) you already have, as country specific TLDs are (generally) a much favored way to Geo target websites.

Trespass.com for US

Trespass.co.uk for UK

However this could become a stumbling block if you wanted one that isn't available. such http://www.trespass.de/

Here is a great post on International and Multilingual sites  and here is another one The Ultimate Guide to Multilingual and Multiregional SEO

Just to add, if you block URLs in robots.txt they wont actually get deindexed. They will be for all intents and purposes be blocked (wont cause duplicate content issues etc) but they will drop into the omitted results:

_In order to show you the most relevant results, we have omitted some entries very similar to the 13 already displayed._If you like, you can repeat the search with the omitted results included. And will look like this in the SERPS (see attachment).If you want them removed from the SERPs you will need to use the robots NOINDEX meta tag, or use GWMT as William advised.

The disallow entry you posted will block these pages, as long as they all start with that way. Although you don't actually need the trailing wild card as that gets ignored, you can just leave it open. Google robots.txt specs

QqCCH1J.png

Hey Elchanan,

http://www.barracuda-digital.co.uk/panguintool/ offers a very similar tool which overlays Google updates over your organic traffic.  Gives a nice insight on how any updates may have affected your site.

I see. I kinda jumped the gun a little with that replay!

It's definitely an interesting case, the problem is usually the reverse of yours. Made even more intriguing by these other results.

The reason you most likely aren't ranking in the UK is you don't appear to have any company address on the site (in the UK or other wise), but the site is hosted on a US IP address.  So if the only clue to your location is your IP address, Google will use that. So if Google is thinking your site is in the US, there really isn't much reason for it to rank it in a UK based search.

The usual thing to suggest to get you ranking in the UK would be to Geo target your domain to the UK, but would you want to sacrifice your ranking in .com? As I'm pretty sure this would effect your .com ranking.

So to keep your ranking in .com and get ranking in UK searches, one approach would be to create an version of your site for the UK -  like you have currently have for your Chinese, German, French and Japanese pages.

You could created a sub folder such as:

http://www.translationzone.com/uk/    (such as Apple do )

Add a version of the site in there to target the UK and you can Geo Target just that folder to the UK in Google Web Master Tools.  Make sure to mark up the site using the correct rel="alternate" hreflang="x" language and locale attributes so Google knows its for the UK and how it relates to the rest of the site.

More reading can be done here on mult lingual sites guidelines.

No I always leave categories free to be indexed on eCommerce sites. (unless perhaps there is some keyword cannibalisation going on with another page). The first page any way, paginated pages get noindexed and sort pages get canonicaled/blocked.

The reason for this is  that (usually) wordpress category/archive pages are just a list of articles with no other value. But with category pages on eCommerce sites I always add content on the page about the particular category and the products within it, making the page a useful page for visitors.

Also there is generally less duplication on eCommerce categories, as I only every use the product title and price, where as on a wordpress category there is going to be a snippet of a paragraph or two, or even the entire article that gets duplicated.

Categories pages can be very important on eCommerce sites, but i don't view wordpress categories as much so.

Again, people might argue about about whats right, and I don't think there is really a right or wrong answer; but i do know what works for me.

Out of all the rank trackers I have found it the fastest and least likely to fail when checking a large amount of keywords. Well, approx 120 any way.

Perhaps this is too obvious, but can you not change the 302 to 301's?

As you've pointed out, people argue until the cows coming home weather to noindex or not to noindex tags, categories, date and author archives; but using the canonical tag is definitely wrong in this case.  The canonical tag should be used on pages that have the same content, perhaps just differing in order, but this might not be the case if you canonical a tag page to  the blog start page, as there will be articles on that page that don't appear on the tag page.

So you are are left with just to noindex, or index these tag pages. Here is a great post on the subject:

No Indexing WordPress Taxonomies: Do or Don’t

To summarise, he found that if he noindexed the tag and category archives and installed a better pagination plugin, his traffic increased by 30% in two weeks.

Yes this is what I thought, but wanted some second opinions.

Although I wouldn't actually need a wild card after BTS, as just leaving it open is the same as using a wildcard:

/fish*..........  Equivalent to "/fish" -- the trailing wildcard is ignored. https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt Thanks for the link, I'll take a look

Do you need the entire preceding path in robots.txt for it to match?

e.g:

I know if i add Disallow: /fish to robots.txt it will block

/fish
/fish.html
/fish/salmon.html
/fishheads
/fishheads/yummy.html
/fish.php?id=anything

But would it block?:

en/fish
en/fish.html
en/fish/salmon.html
en/fishheads
en/fishheads/yummy.html
**en/fish.php?id=anything

(taken from Robots.txt Specifications)** I'm hoping it actually wont match, that way writing this particular robots.txt will be much easier!

As basically I'm wanting to block many URL that have BTS- in such as:

http://www.example.com/BTS-something
http://www.example.com/BTS-somethingelse
http://www.example.com/BTS-thingybob

But have other pages that I do not want blocked, in subfolders that also have BTS- in, such as:

http://www.example.com/somesubfolder/BTS-thingy
http://www.example.com/anothersubfolder/BTS-otherthingy

Thanks for listening

Hey Jarrod,

I'm afraid there isn't anything you can actually do to tell Google you are the original author of your content, other than the tips Remus mentioned.

However, there is a service that you can use to help you identify sites that are duplicating your content.  It's called Copysentry and it automatically scans the web to check for content duplication. You could use this, in conjunction with DMCA take down requests (as mentioned in Remus's post) to help to defend against this in future.

Cuterank is a great ranking checker that will check up too 100 results and any amount of keywords even in their free version. The free version is limited to a single site however.

it should be added to the end of your robots.txt and be proceeded by 'Sitemap', like:

Sitemap: http://www.exmaple.com/sitemap1.xml
Sitemap: http://www.exmaple.com/sitemap2.xml

Hi Irene, can you explain what you mean by category view?

Perhaps you mean one of the views on the screen shot below, on the left or right?

SHAuIpW.png

You don't actually need to use a sitemap index file to use multiple sitemaps. You can list and submit them separately in robots.txt file and Google Web Master Tools.

Yes this is fine, from Google:

Whether you list all URLs in a single Sitemap or in multiple Sitemaps (in the same directory of different directories) is simply based on what's easiest for you to maintain. We treat the URLs equally for each of these methods of organization. More info can be found here multiple sitemaps in same directory 

There is a great plugin for chrome called Nofollow which outlines nofollow links  and also detects nofollow and noindex robots meta tags on webpages.

It automatically adds a dotted line round any links which are no followed, as you can see in the screen shot below on Gary's link.

KtQafrk.png

Both of these examples posted have rather terrible duplicate content issues, both internally and externally. As duplicate content is possibly one of the biggest onsite SEO issues with eCommere site, these aren't particular great examples in that regard.

e.g:

theiconic.com.au

zappos.com

Could this possibly be connected to Google's Rank-Modifying Spammers Patent?

These are caused by the links to your language pages.  If you click one of the language links from within the source code (not on the page) it redirects to a URL with '?q=/index.php/Web-Pages/binary-options-platforms.html' added. Then if you click the same language link on that page it again redirects to another page with previous URL added to the end:
?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html.

e.g:

On the example page view source, search for German and click the link below:

http://de.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html

This link 301 redirects too:

http://www.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html Then if you view source, search for German and click the link again:

http://de.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html

This link 301 redirects too:

http://www.binary-options.biz/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html?q=/index.php/Web-Pages/binary-options-platforms.html

So basically every time a web crawler follows a language link, new URLs are being created with the previous URL added to the end, causing a never ending crawl as an infinite amount of new pages will always be created.

I don't think this is connected with the Joomla SEF as Chris pointed out, as your URLs are already SEF.

However it's not an easy thing to identify how to fix the issue with the language links. You should probably speak to the developer who implemented it and/or the creator of the plugin if it is a plugin.

Also do you even need this functionality? As none of the language links work, they just redirect back the main site.