Hi,

We are currently in the process of getting our app developed. From an SEO point of view I want the app to get indexed and am currently trying to get a step by step guide for it.

I've come across this helpful guide: https://moz.com/blog/how-to-get-your-app-content-indexed-by-google but need to go a bit deeper now.

Questions I have is related to the "apple-app-site-association" file

As far as I could find out the JSON code of that file has the following syntax:

{_    “applinks”: {__        “apps”: [],__        “details”: [__            {__                “appID”: “9JA89QQLNQ.com.my.bundle.id“,__                “paths”: [ “/myPath1”, “/myPath2/*” ]__            },__        ]__    }_

}

My question is related to the key "appID"

I'm wondering how the entry would look like if I have 10 country subdomains and would like to make sure  that the deep link clicked in the browser is going to the right subdomain page within the app.

Current website structure:
uk.example.com/english-path/
de.example.com/german-path/

Many Thanks in advance

Hi Dan,

Yes, the URL would change in both cases (AJAX request as well as for a complete refresh).

Thanks

Hi Dan,

Although it doesn't address the specific question I had I think your link is a good start for me to dig into the topic in more detail.

Many Thanks,

Hi,

I got a question in regard to webpages being served via AJAX request as I couldn't find a definitive answer in regard to an issue we currently face:

When visitors on our site select a facet on a Listing Page, the site doesn't fully reload. As a consequence only certain tags of the content (H1, description,..) are updated, while other tags like canonical URLs, meta noindex,nofollow tag, or the title tag are not updating as long as you don't refresh the page.

We have no information about how this will be crawled and indexed yet but I was wondering if anyone of you knows, how this will impact SEO?

We have the same issue however we have image XML sitemaps on each country subdomain's XML Index which point to the image files on images.domain.com.

Example:

https://uk.domain.com/image-sitemap1.xml

https://us.domain.com/image-sitemap1.xml

These 2 files are the same.

We also don't have a homepage on images.domain.com and it currently responds with a 404.

Do you think we need to create a landing page on the homepage and host the image XML sitemap at https://images.domain.com/images-sitemap1.xml rather than in each sub-domain?

Thanks.

What are your thoughts around using OnScroll() function because it seems the search engines don't crawl this function easily and therefore uploading the content in that way might be safer.

http://googlewebmastercentral.blogspot.co.uk/2014/02/infinite-scroll-search-friendly.html

Hi,

We are considering the impact when you want to deliver content directly on the product pages. If the products were manufactured in a specific way and its the same process across 100 other products you might want to tell your readers about it. If you were to believe the product page was the best place to deliver this information for your readers then you could potentially be creating mass content duplication. Especially as the storytelling of the product could equate to 60% of the page content this could really flag as duplication.

Our options would appear to be:1. Instead add the content as a link on each product page to one centralised URL and risk taking users away from the product page (not going to help with conversion rate or designers plans)2. Put the content behind some javascript which requires interaction hopefully deterring the search engine from crawling the content (doesn't fit the designers plans & users have to interact which is a big ask)3. Assign one product as a canonical and risk the other products not appearing in search for relevant searches4. Leave the copy as crawlable and risk being marked down or de-indexed for duplicated contentIts seems the search engines do not offer a way for us to serve this great content to our readers with out being at risk of going against guidelines or the search engines not being able to crawl it.How would you suggest a site should go about this for optimal results?

Hi,

Does anyone have a list of (major) Search Engines that subscribe to the Ajax Crawling Scheme? (https://developers.google.com/webmasters/ajax-crawling/)

Specifically interested in major international Search Engines such as Bing/Yahoo, Baidu & Yandex - if anyone knows, please let me know!

Thanks in advance

Thank you Chiaryn, I will post on the Feature Request section

Thanks again

Dean

As we've been fixing errors on our website, the crawl diagnostic graphs have been showing great results (top left to bottom right for errors).

The problem is the graphs themselves aren't very pretty. I can't use them in my internal reports (all internal reports are standardised colours/formats).

Is there anyway of exporting the top level summary with historic data so the graphs can be recreated in company colours? I don't want the detailed CSV breakdown of what errors occurred, but rather than on X date there were Y errors, the next month Z errors and so forth.

The data must already be in the SEOMoz system in order to create the graphs themselves - I was hoping this can be made available to us if it isn't already?

Does anyone know if there is already a way of doing this? I've tried to 'inspect element' and find the underlying data in the source code but to no avail, and can't see any exports that would do this.

Thanks in advance

Dean

Further update to this. Ran into a problem with option 3... this solution works really well when navigating the site internally, however a user landing on one of these URL's directly (bookmark, social share etc) would have a slow loading page as (for non-default product variations) the page will load after the 1st request, then a 2nd request to the server is needed to pull in the image via AJAX.

Loading the other images, stock information, prices, copy etc into an array and doing the work on the client side wasn't an option as the page would get too heavy. So option 3 ruled out.

Ultimately the goal was to reduce duplicate content of product pages and none of the 3 options above do this whilst not affecting page loading times. I did look to fall back on using canonical tags however I've just now found that Facebook are using this tag, so if a user wanted to share a 'red apple' when the canonical is 'green apple' - Facebook would show an image of the 'green apple'.... so at the moment that is ruled out also.

I'll start a new thread on product page duplicates and the best solution - but if anyone has any ideas then please do let me know.

Thanks

Dean

Thanks for the response Dana. Option 3 did feel like the best option and that is the one I'm choosing to go with.

Point 2 (with the hash) provides the desired result of Search Engines only seeing the clean URL as the parameters behind the hash will never be seen, but the browser will use them to power the breadcrumbs. In the end it was a toss-up between 2 & 3 but 3 is the most maintainable and quickest for users.

Thanks again

Dean

Hi,

Our products are listed in multiple categories, and as the URLs are path dependent (example.com/fruit/apples/granny-smith/, example.com/fruit/green-fruit/granny-smith/ and so forth) we canonicalise to the 'default' URL (in this case example.com/fruit/apples/granny-smith/).

For mainly crawling bandwidth issues I'm looking to change all product URL's to path neutral so there is only ever one URL per product (example.com/granny-smith/), but still list the product in multiple categories.

If a user comes directly to example.com/granny-smith/ then the breadcrumbs will use the default path "Fruit > Apples", however if the user navigated to the product via another category then I'd like the breadcrumbs to reflect this. I'm not worried about cloaking as it's not based on user-agent and it's very logical why it's being done so I don't expect a penalty.

My question is - how do you recommend this is achieved from a technical standpoint? Many sites use path neutral product URL's (Ikea, PCWorld etc) but none alter the breadcrumbs depending upon path.

Our site is mostly behind a CDN so it has to be a client side solution. I currently view the options as:

• Store Path to product in a cookie and/or browsers local-cache
• Attach the Path details after a # in the URL and use Javascript to alter breadcrumbs onload with JQuery
• When a user clicks to a product from a listing page, use AJAX to pull in the product info but leave the rest of the page (including the breadcrumbs) as-is, updating the URL accordingly

Do you think any of these wouldn't work? Do you have a preference on which one is best? Is there another method you'd recommend?

We also have "Next/Previous" functionality (links to the previous and next product URLs) on the page so I suspect we'd need to attach the path after a # and make another round trip to the server onload to update the previous and next links.

Finally, does anyone know of any sites that do update the breadcrumbs depending upon path?

Thanks in advance for your time

FashionLux

Thanks Lewis. I think this video is commenting more about whitespace outside tags than inside them, but I think the same principle applies.

Unless I hear that it does cause issues, for now I'll leave it in and fight other battles with the dev team.

Thanks

Hi,

I understand that 'whitespace' in source code is absolutely fine and is stripped out. For example the following code is fine:

Red Apples

some text

However, how is whitespace interpreted INSIDE html tags such as H1's? My Dev team have instructions to add H1's to a page, however they have done so like this:

Red Apples

(37 characters long)

rather than this:

Red Apples

(10 characters long)

Do you think this extra space will be harmful? The browser renders it fine, however if you use something like the mozbar plugin is shows the H1 length as 37 characters.

I know the 10 character H1 is 100% relevant to the search term "Red Apples", however is the 37 character H1 only 27% relevant? (10/37)

I've made the request to the Dev team to remove this whitespace because I'd rather err on the side of caution, but its been knocked back because the HTML spec specifies consecutive white-space should be interpreted as a  single space and all browsers build the DOM by trimming a tags value - and they imagine search bots do the same so don't want to mess with the compiler.

Anyone have experience of this? I've never had whitespace in a H1 before so don't know. Happy to leave the whitespace in if it's not going to be an issue.

Thanks in advance

Hi,

Our site is very good for Javascript-On users, however many pages are loaded via AJAX and are inaccessible with JS-off. I'm looking to make this content available with JS-off so Search Engines can access them, however we don't have the Dev time to make them 'pretty' for JS-off users.

The idea is to make them accessible with JS-off, but when requested by a user with JS-on the user is forwarded to the 'pretty' AJAX version. The content (text, images, links, videos etc) is exactly the same but it's an enormous amount of effort to make the JS-off version 'pretty' and I can't justify the development time to do this.

The problem is that Googlebot will index this page and show a preview of the ugly JS-off page in the preview on their results  - which isn't good for the brand. Is there a way or meta code that can be used to stop the preview but still have it cached?

My current options are to use the meta noarchive or "Cache-Control" content="no-cache" to ask Google to stop caching the page completely, but wanted to know if there was a better way of doing this?

Any ideas guys and girls?

Thanks

FashionLux

Thanks Casey, that's good to know

Hi,

Back in Jan I asked "Why has SEOmoz added G+ code to multiple pages?" - I wanted to double check something so looked at your source code but see that you've now removed the rel="publisher" code from all of your pages...

Could you please let me know why its been removed? I don't want to add it to my site and find out 2 weeks later that you've done a big test and shown it hurts rankings or something like that.

Thanks

FashionLux

solved via GWT thread: https://groups.google.com/a/googleproductforums.com/forum/#!topic/webmasters/WlUzNLFQB54/discussion

Have you had any joy with this yet?

Please forgive duplicating this question on the SEOMoz & Webmaster Tools forum but I'm hoping to hit both audiences with this question...

A few days ago I noticed that our US homepage (us.burberry.com) had dropped from PR5 to PR0, and the page has been deindexed by Google. After checking Webmaster Tools I also received the following message:

http://us.burberry.com/: Big traffic change for top URL      April 2, 2012Search results clicks for http://us.burberry.com/ have decreased significantly.Message ID: 593f1ceb2d67.We're not doing any link building at all (we've enough on-site issues to deal with). The only changes I have made are adding Google Analytics to the website, uploading sitemaps via Webmaster Tools (it's not linked to from robots.txt yet), and setting the burberry.com and www.burberry.com geo-location settings to 'unlisted' (we want uk.burberry.com appearing in the UK results, us.burberry.com appearing in the US results etc rather than www.burberry.com).I've reversed the geo-location settings but I doubt this would have caused this. We've duplicate copies of our homepage (such as us.burberry.com/store//) from typos in inbound links (and bad programming that allows them to work rather than 404'ing) but I don't think any of this is new. What I don't understand is (a) why this is happening now and (b) why is this just affecting our US homepage? We've ~40 different duplicates of the homepage (us, uk, ca, pt, ro, sk etc etc) so why is the US site being affected and not the others? Does anyone know if this is due to an algorithm change by Google or something else all together? Background:Our website www.burberry.com has 46 subdomains such as uk.burberry.com, ca.burberry.com and us.burberry.com. There is a lot of duplicate content on each subdomain (including basic things like tracking parameters in URLs) and across subdomains (uk.burberry.com/store & us.burberry.com/store are exactly the same), there's very little text on the site (its nearly all images), as well as poor redirects, inaccessible content (AJAX/Flash) and a whole host of basic SEO things that aren't being done correctly. I've joined the company in the last few months and have started addressing these issues but I've got a LOT of work to do yet.One thing that we have in our favour is a link profile that is as clean and natural as they come - there was only ever one link building campaign performed (which was before my time) and I had all of those links removed as soon as I joined the company.Any help would be greatly appreciated! Thanks for your timeDean RoweEdit: us.burberry.com 301 redirects to us.burberry.com/store/ as explained on the webmaster tools forum, but I don't believe this is the cause as its the same across all subdomains.

Hi Highland/Dr Pete,

My apologies I wasn't very clear - fixing the duplicate problem... or rather stopping our site from generating further duplicate content isn't an issue at all, I'm going to instruct our developers to stop generating dupe content by doing things like no longer passing variables in the URL's (mysite.com/page2?previouspage=page1).

However the problem is that for a lot of instances duplicate URL's work and they need to work - for example if a user types in the URL but gets one character wrong ('1q' rather than '1') then from a usability perspective its the correct thing to serve the content they wanted. You don't want to make the user have to stop, figure out what they did wrong and redo it - not when you can make it work seamlessly.

My question relates to 'once my site is no longer generating unnecessary duplicate content, what should I to do about the duplicate pages that have already made their way into the Index?' and you have both answered the question very well, thank you.

I can manually set-up 301 redirects for all of the duplicate pages that I find in the index, once they disappear from the index I can probably remove those 301's. I was thinking of going down the noindex meta tag route which is harder to develop.

Thanks guys

FashionLux

Hi,

I'm in the process of instructing developers to stop producing duplicate content, however a lot of duplicate content is already in Google's Index and I'm wondering if I should bother getting it removed... I'd appreciate it if you could let me know what you'd do...

For example one 'type' of page is being crawled thousands of times, but it only has 7 instances in the index which don't rank for anything. For this example I'm thinking of just stopping Google from accessing that page 'type'.

Do you think this is right?

Do you normally meta NoIndex,follow the page, wait for the pages to be removed from Google's Index, and then stop the duplicate content from being crawled?

Or do you just stop the pages from being crawled and let Google sort out its own Index in its own time?

Thanks

FashionLux

Thanks Kris - but it is adding unneccessary code to the page? and slowing site speed? (although marginally)

Hi,

If you look at SEOmoz's Blog pages, you'll see they have added the G+ rel="publisher" code.

On http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1708844 Google says:

"A rel="publisher" link from the main page of your website to your Google+ page."

Is there any benefit to putting this code into the sites template (adding it to every page of the site) when it could be added to just the homepage?

Not a criticism, just curious to know how I should implement the code on my own site (concerned that adding it to the site template is a kind-of link out on every page of my site)

Cheers

FashionLux

"If it is a very basic pure html/css site, you can pretty much achieve your goal." - Yes this is exactly what I need, I don't want the site to be functional and allow users to place orders (which could happen for non-JS users who don't see the notice that fills the entire screen). I don't want to do anything apart from rehost the site and put a big message up that says "THIS SITE WAS A SCAM - BEWARE OF OTHER SCAM SITES" and cannot be closed down.

"Do you obtain control over just the domain?" - Yes only the domain, not the hosting. We go through legal proceeding to prove the site is illegally selling counterfeit goods and obtain the blank domain.

"I understand your intentions are good, but the method is not complaint with Google's Guidelines." Fair point, but Google shouldn't rank these sites in the first place - they have no genuine links and should be banned already. Google aren't spotting this, so I have to fix Google's **** up. If the site gets banned I couldn't care less. Whilst they rank they serve a genuine purpose of (a) showing users that there are counterfeit sites and they need to be wary and (b) new sites have to better the SEO ability of the old ones in order to rank on page 1.

"Your goal is purely to manipulate search engine results which makes these activities black hat and subject to penalty." Yes but it doesn't matter if the domain is banned, it's not my genuine website and has no links back to my genuine site. I'm not going to host the sites on the same server as our genuine site so no risk to the company. Really I couldn't care less if it gets banned - the counterfeit sites are ranking due to black hat techniques - its in my interest for Google to eventually work it out and fix their algo as it will stop the hundreds of other counterfeit sites from ranking too.

"You can use every social media page, etc. If you put in the time and effort, these pages will rank very well in SERPs."

Yes I could, by building links to social media pages for the hundreds of search terms currently dominated by counterfeiters but this is not a good idea for two reasons:

1. Trying to rank social media sites for irrelevant terms isn't a good thing - you wouldn't do it for users if this situation wasn't happening. As you said already, this is a form of trying to manipulate SERPs and I wouldn't want to risk these genuine SocMed pages getting banned because of this.

2. There are hundreds of search terms to optimise for, and 8 remaining slots on Google to fill for many of these. These sites are also powerful in their SEO strength - 17 counterfeit sites made it into Majestic's top 1million sites by links - these sites have literally tons of scummy, comment box spammed links pointing at them and they are ranking (shame on Google).  Competing against these isn't possible via white hat methods and I'm not a black hat kind of guy.

My thought process is - Why try and compete against these sites (and waste A LOT of time and effort) trying to bump them down the rankings when they've already done the hard work of optimisation and link building for these terms? I could simply 're-use' them for a genuine purpose (making our customers beware of ordering from unofficial websites).

The previous owner won't sue us for re-using their content - that involves making themselves known to authorities and they'd get arrested in turn for their illegal activities.

I'm happy to debate it more as its an interesting subject and I don't want to waste time going down the wrong route, but I think re-using the sites is the best option - I just need to get copies of them so they LOOK the same to Google and hopefully keep their SERPs.

If you know of any software or techniques to crawl and download multiple (html) pages and images of a site then please let me know.

Thanks for all of the responses

FashionLux

Thanks for the response.

"you can download the the html but not the files themselves" - the html is all I need. I don't want the site to actually work so having only the html files is perfect.

I can go to the homepage and manually save it, and go through 100+ pages and manually download them - I just wanted to ask if there was any software that would do this for me and save some leg work.

Thanks again

Hi Ryan,

Thanks for the reply. To clarify, the site is deleted prior to me gaining control of it - by the time it comes into my hands it's completely blank, so FTP'ing isn't an option.

The site owners are essentially scamming members of the public by charging hundreds of dollars for goods that are never delivered. We've seized hundreds of sites through legal proceedings, but more keep popping up the moment we get hold of them.

These sites rank for hundreds of popular search terms (some have hundreds/thousands of spammy inbound links), so bumping them off page 1 for all SERPs isn't achievable.

By seizing the sites, keeping the content, but making the site non-functioning (imagine a popup image that fills the screen and can't be escaped) it will hopefully mean we own these SERPs and new counterfeit sites have to try and outrank them.

In turn we'll seize those sites, so the next wave of counterfeit sites have to do even more link building - eventually (maybe years) they'll realise its not worth it and give up.

Manually downloading individual webpages isn't an option, so I'm wondering if theres any programmes that can download all html files for a website so I can then just upload them via ftp once the site has been seized and add my javascript image

Thanks for all of the responses

FashionLux

Hi all,

I work for a large retail brand and we have lots of counterfeit sites ranking for our products. Our legal team seizes the websites from the owners who then setup more counterfeit sites and so forth.

As soon as we seize control of a website, the site content is deleted and subsequently it falls out of the SERPs to be immediately replaced by the next lot of counterfeit sites.

I need to be able to download a copy of the site before it is seized, so that once I have control of it I can put the content back and hopefully quickly regain the SERPs (with an additional 'counterfeit site' notice superimposed on that page in JS).

Does anyone know or can recommend good software to be able to download an entire website, so that it can be easily rehosted?

Thanks

FashionLux

(Edited title to reflect only wanting to download html, CSS and images of site. I don't want the sites to actually be functional - only appear the same to Google)