Welcome to the Q&A Forum

This is long time question. And answer is as fast you forget about that "count" the seamless migration will be.

Because many people was stopped migration awaiting an miracle about moving their counts to HTTPS version. And years was passed - and no solution for that.

Forcing HTTPS -> HTTP -> HTTPS is risky because sooner or later something can be changed and crawling issues can happens. Also maintain both sites for HTTP/HTTPS infrastructure can have some effect on your bills.

You need to simplify it by providing only one - HTTPS version.

Peter

And actual answer is "it depends". For example linking a "contact us" or "about us" is important. But on other side linking "Privacy Policy" or "Terms of Service" isn't somehow important. Except if you're in Germany, Austria, Switzerland where people are looking for Imprint pages.

One of easiest ways to fix this is to make links to be non-crawlable:
https://youtu.be/PFwUbgvpdaQ?t=600
because bot didn't click on web page.

I hope that this will helps you!

Peter

Canonical and hreflangs are always full url and you can't use relative addresses:
https://developers.google.com/search/docs/advanced/crawling/localized-versions?hl=en
"Alternate URLs must be fully-qualified, including the transport method (http/https), so: https://example.com/foo, not //example.com/foo or /foo"
https://developers.google.com/search/docs/advanced/crawling/consolidate-duplicate-urls

So you need to remake your example with full URLs like:
protocol://domain (with or withour WWW.)/path/to/some/folder/

Also avoid geotargeting for IP.  This can went you on deep issues since Googlebot is crawling mostly from US IP. Therefore Google NEVER can see your UK links. Source:
https://developers.google.com/search/docs/advanced/crawling/locale-adaptive-pages?hl=en
"If your site has locale-adaptive pages (that is, your site returns different content based on the perceived country or preferred language of the visitor), Google might not crawl, index, or rank all your content for different locales. This is because the default IP addresses of the Googlebot crawler appear to be based in the USA. In addition, the crawler sends HTTP requests without setting Accept-Language in the request header."

Regards,

Peter

Best practices is to remove and noindex such parameters. One of latest projects was Ecommerce shop with 5k products and over 220k indexed pages from categories.

So you need to AVOID indexing such ?results=16 or ?order=asc or ?search=test and to have PURE and only pagination.

Example:
https://www.jamestowndistributors.com/product/epoxy-and-adhesives
https://www.jamestowndistributors.com/product/epoxy-and-adhesives?page=2
https://www.jamestowndistributors.com/product/epoxy-and-adhesives?page=3
https://www.jamestowndistributors.com/product/epoxy-and-adhesives?page=4
are ONLY valid categories pages.

If you have infinite scroll situation is described here:
https://developers.google.com/search/blog/2014/02/infinite-scroll-search-friendly?hl=en

I hope that this will help you!

Peter

Can you explain your issue please?

Peter

I think that i was answer you on other forum...

%5B is [ 
and %5D is ]
%20 is " " (space)

So technically speaking something add this to the URL - "[null id=43484]" without quotes. You need to find that and fix it.

Is code on .htaccess works?

It should be on top because that file is executed from top to bottom. And if some rule must be executed then execution flow can stop so next rows can't be executed.

Hello there!

In GA you tracks site by "code ID". So any site that have code will be tracked.

Few months ago i migrate site from XYZ.machine to XYZmachine.com without changing GA code. And GA still collecting data for sites without changing code at all.

Peter

Both issues are on ALL WordPress sites.

xmlrpc.php return 405 because it's work under HTTP POST but bot crawl under HTTP GET.
And error 405 mean switch from GET to POST. But bot can't do that.

admin-ajax.php return 400 because it's not designed for direct use. You must make call with some parameter and it will return some information.
But as name says - it's responsible for AJAX calls and it's not for direct use that's why it return 400 when it's called w/o parameters.

So - that issues are not related to your issue.
For example mine WP sites also return same error codes to the bot.

Peter

Just as EGOL describe it.

If you're on Amazon AWS then you can use their CloudFront as CDN. But also you can observe source of traffic. Could coming from one country, one IP range or one user-agent. There should be some kind of pattern and you should investigate it.

Then just need to make rule to block that traffic or just redirect them to one static "hello world" page.

I was also victim of such traffic, but was from humans trying to depleting an AdWords daily budget. Once budget it over ads was stopped showing, after few hours they recalculate clicks, some funds was returned, ads are shown again, they click it, budget is over... and so on.

Too bad.

Probably best way is to contact someone from webspam team and sent your observations there directly. 
Other way is to report same sites as hacked because they're definitely hacked.

And will take time.

Right now we're updating SEOSpyder ( http://www.mobiliodevelopment.com/seospyder/ ) for rendering pages but i can't give you timeframe when will be done.

So far memory requirements isn't too high and was crawl 250k site with 8G ram machine.

Yes - but there is one thing that you should know. Once you enable RegEx ALL your parameters in funnel should be regexed.

https://support.google.com/analytics/answer/1116091?hl=en#matchTypes

"Match Types: Begins with, Equals to, & Regular Expression
There are three diferent match types that define how Analytics identifies a URL for either a goal or a funnel. The match type that you select for your goal URL also applies to the URLs in the funnel, if you create one."

Well there are few "loadings website" parameters - TTFB (time to first byte), initial rendering or document complete.

It's great if TFFB is under 1s including network connection and DNS resolving. 
It's fantastic if initial rendering is closer to 1 sec too.
Would be great if document complete is near 3-4-5 secs.

But this won't help you with ranking... well just little bit even insignificant.
But that would help you in CRO because users want to visit faster sites.

Nadav,

this could happen in restrictive robots.txt where there are some enabled parts and other parts disabled for indexing.

But you help you could you sent link to that file so we can give you best advise for your case?

Well... i have two news for you. First - good one... theme is good looks fantastic.

Now bad news - i look it on mobile phone and you can see result in attached file. Yes - i have iPhone SE because i love it's form-factor.

I'm so sorry but it's year of 2018 and everyone today works with smartphone. Probably something custom should works perfect for you. Because it's 2018 and there is also AMP that you should check that too:
https://moz.com/blog/amp-digital-marketing-2018

NlC9VVk

I totally agree on everything you said.

I think that you're talking about WordPress, right? So it's tuff situation and i'm in same. So i will explain what i'm doing now, but note that this is because i'm dev and i have experience with WP like 10+ years.

So i have site (link in profile) that it's built using one purchased WP theme and works good. Built in 2012 because previous version was on .html files so maintaining become too time consuming. So i get theme and built site. Still works, but i was seeing in process of building site that something isn't right.

First - theme didn't seeing all items correctly as in demo due some issue with pages and posts. Second we're using "tab" interface with all pros and cons. Thirds - theme uses too much CSS/JS code. After month we was found how to make content for all sections. After another month one of plugins broke all site due JS malfunction code and tab interface didn't works at all for month or two. So site went in optimization process and now works much faster than before in 2014.

But it's 2018 and this isn't enough. That's why i start making brand new theme for that site with few goals:

1. CSS code less than 30k. I found that on site i was use between 5% or 10% of all CSS code. So mine next goal is less than 30k.
2. Optimization with CSS delivery. ATF CSS code will be inlined into , rest of code will be put in bottom of HTML. Just to not count as "render blocker".
3. Minimal JS. I have custom JS code for menu (1k) and will use gallery code as Photoswipe. And CSS slider, yes i'm not kidding. But that's not all - using custom PHP code that extra codes will be included just on pages where they're needed. I.e. there will be not gallery on homepage or 'contact us' so that code will be not included into that pages.
4. Remove most of plugins. Most of them make situation from bad to worse. I wish to live with no more than 5 plugins from reputable authors.
5. Fixing WP to generate small and clean code where is possible. Have you seen your head section? That's what i'm talking about - "Less is more!"
6. Testing "statification" of site. I'm not posting each week, there are no valuable comments. Honestly for 6 years i didn't have not received even single comment in posts that isn't spam. If i can turn page into static one then i can solve hosting issues (i don't have it!) by using one of those serverless hostings.

So as you can see this will solve all mine issues for next 5 years.

For you it will be good if you can start make changes with child themes, but in long term probably custom one will be suitable. I found that many themes comes with 999 features but in real life situation people used just few of them.

I hope that this give you rough idea of situation.

It's hard to be explained but "Less is MORE!" in general for that numbers.

Examples - redirectors. Redirects can overkill your site specially on mobile users. For that even simply site redirect can took second or two. Example www.example.com -> 301 -> m.example.com; looks simple isn't? But in reality after client took 301 redirect he must make new domain resolving (for m.exmaple.com) and then new connect to new server (m.example.com). And this is simply case... if you have 2 or even 3 redirects mobile users wait for 5 seconds before see anything. Hint - that's why i won't click on most bit.ly, ow.ly, goo.gl links in Twitter, Facebook, G+ when i'm on mobile. Because they first pass via t.co redirect then redirect that i can see and sometime even 3rd redirect. I know that marketers want to see "clicks", but isn't good for mobile users.

Server connection is also need to be less. But this mean that server need to be closer to user. Best example is Australia. There even simply DNS resolving + connection took one second. And client doesn't receive single byte from server yet... You can see WebPageTest.org (there are Australian servers). But of course providing single server there is expensive, so you need to have deep pockets to make servers there. That's why most of companies providing CDN support. Since CDN endpoint is closer to user it make things little bit faster for them. And if CDN is setup correct should be much faster.

So - idea is "Less is More!". The best is if you use WPT to benchmark your site from all over the world. And also setup Analytics to count speed. Because it's different speed when your site is on perfect conditions in datacenter than in real world.

BrightonSEO + SMX London maybe? Here is full list of events:
http://www.stateofdigital.com/2016-digital-events-conferences/

I didn't use GMB in Twitter but i have some issue with AdWords. And GAW Twitter account was very useful for me answering me exactly what i need. And even they answer me for other questions in DM.

Overall their support forum strange since in 99% of cases you receive help from non-employees and some of answers are not very useful at all. Of course there are spectacular examples where someone receive help from employees:
https://productforums.google.com/forum/#!msg/webmasters/R0MGhtF3IDs/6UghGfcXCAAJ
but they're rare. Very rare.

Their phone support is IVR (interactive voice response), their mail support was scaled with AI (i can't find article but i read in past someone complaining about that with case of Nexus ordering and epic mail correspondence with them). So we shouldn't scary that Joy trying their Twitter for helping.

There are many plugins for schema.org markup in WordPress directory. But this one:
https://wordpress.org/plugins/schema-creator/
is extremely useful.

As alternative - you can use JSON-LD and put it anywhere in HTML.

We just talk about this today here:
https://moz.com/community/q/default-robots-txt-in-wordpress-should-i-change-it

And here are Yoast configuration settings:
https://yoast.com/wordpress/plugins/meta-robots-wordpress-plugin/
https://yoast.com/prevent-site-being-indexed/

Since WordPress robots.txt is "virtual" file you need to create new file in root folder of site (usually this is same WP folder) and create that file. This is only where you need to edit it.

Yes - 301 to destination could be the problem. Can you do test w/o redirects putting end URL into rel=alternate?

Also you must check correct linking from Android to web:
https://developers.google.com/app-indexing/android/app

In ideal linking both must be linked vice versa. Probably one of linking now is broken as far as we see.

As long as you return 404 this is good. But the question is "why search results in CMS are indexed?". And answer of this is "Joomla". I will keep them as 404.

But there are other 404 pages there:
http://www.ethicaredental.co.uk/download.php?f=tombolis-article.pdf <- http://www.ethicaredental.co.uk/download.php?f=tombolis-article.pdf
http://www.ethicaredental.co.uk/download.php?f=ethicare-dental-ad-april-2009.pdf <- http://www.ethicaredental.co.uk/download.php?f=ethicare-dental-ad-april-2009.pdf
http://www.ethicaredental.co.uk/download.php?f=ethicare-dental-ad-may-2008.pdf <- http://www.ethicaredental.co.uk/a-one-stop-shop-for-oral-health-needs/
http://www.ethicaredental.co.uk/download.php?f=how-i-did-it.pdf <- http://www.ethicaredental.co.uk/how-i-did-it/

Few domains that didn't exist anymore:

file:///Z:/Clients/Current%20Clients/Ethicare%20Dental/Articles/JULY%20ARTICLES/Is%20the%20new%20best%20toothpaste%20in%20the%20world%20all%20its%20cracked%20up%20to%20be.docx <- http://www.ethicaredental.co.uk/is-the-new-worlds-best-toothpaste-all-its-cracked-up-to-be/
http://www.hypnotherapydunstable.co.uk/img/dentist.jpg <- http://www.ethicaredental.co.uk/taking-the-fear-out-of-a-trip-to-the-dentists-chair

Few 301 that i would skip:
http://www.ethicaredental.co.uk/orthodontist -> http://www.ethicaredental.co.uk/orthodontist/
http://www.ethicaredental.co.uk/hygienist -> http://www.ethicaredental.co.uk/hygienist/
http://www.ethicaredental.co.uk/finance -> http://www.ethicaredental.co.uk/finance-and-guarantee/
http://www.ethicaredental.co.uk/zoom-tooth-whitening -> http://www.ethicaredental.co.uk/tooth-whitening/
there are much more i just show 3-4 examples.
And few more small issues. This is result of direct crawl now on site. If there was link as:
http://www.ethicaredental.co.uk/blah-blah
that i can't see it now anymore and it return 404 now. I would 410 it to prevent deleting them over and over from SearchConsole.

In addition of Martijn here is mine robots.txt:

User-agent: *
Disallow:

Sitemap: http://peter.nikolow.me/sitemap_index.xml

But using Yoast - categories, tags, most of archives and other generated pages are disabled for indexing.

No. Because HTML text will be same and altered with JS.

And bots using HTML text for indexing and ranking.

Same here. This is when you put "share bar" on top somewhere in pages. And repeating "share" many times is also causing this.

Probably solution is to put share as side bar or in bottom and definitely minimize texts as "share" or "facebook" in text of articles.

There is bug on rel=alternate. It should be as:

only.

Anyway - get this URL and try in your SearchConsole to "Fetch and Render" probably this will help you fixing your web error.

TL;DR - NO.

There are few ways for translation and they're described there:
https://support.google.com/webmasters/answer/182192?hl=en

You also must implement hreflang and link sites each other.

Tuff situation. Why? Browser didn't sent # and everything behind it to the server.

So if you trying to get url as http://www.example.com/#!my-super-duper-url
Browser will sent to the server request for http://www.example.com/ and server will process it. But full url that browser want is also included #! fragment. This mean that you can't make .htaccess redirect, nor some server side redirects for the moment.

So same hurt also all bot and crawlers (Including Moz Roger!). And there was solution:
https://developers.google.com/webmasters/ajax-crawling/docs/specification?hl=en
but later this solution was deprecated:
https://googlewebmastercentral.blogspot.bg/2015/10/deprecating-our-ajax-crawling-scheme.html

And this make things complicated. For now they still support old solution so will be OK for bots. Probably for some users that comes from bookmarks, emails and/or other traffic sources can have hard times. Because will be redirected to "homepage". So maybe combination of both methods (JS redirector + your actual method) can save the day for humans and bots.

Same here - normal traffic till mid Dec. Then drop for Xmas and NY. And this is normal for holidays. Then up in beginning of Jan, from 24-25 drop little bit (back to normal traffic).

So there is also change called "core update":
https://moz.com/google-algorithm-change
http://mozcast.com/

Yes, it's possible. I have example that on same host i host few sites that can rank on same keywords in SERP.

But you should watch for some spam sites on same IP:
https://www.youtube.com/watch?v=AsSwqo16C8s
there is example 26k vs. 1.

Update:
So few sites can rank. But when you try to rank much more then you can trigger some algorithmic filter (i can't find hangout where this is explained) because ranking engine will start to demote you. Since you have time to invest in few sites you should focus on single site. It's like quantity vs quality debate.

Well - go in SearchConsole and reevaluate site from scratch. As long as you return 404 to hacked pages its' OK for you and for Google. But what they told you in "reconsideration request"? Because "this site may be hacked" isn't helpful for your CTR.

PM me site please.

It's normal. I think that there is change in Google Play and their rich snippets are broken:
https://play.google.com/store/apps/details?id=com.mobilio.seoggestor&hl=en
just see structured data:
https://developers.google.com/structured-data/testing-tool?url=https%253A%252F%252Fplay.google.com%252Fstore%252Fapps%252Fdetails%253Fid%253Dcom.mobilio.seoggestor%2526hl%253Den

So - reviews are there but due warning and errors they wasn't showed in SERP. Your competitor also can lost his rich snippet soon.

Like mine SEOggestor? But it's for one keywords.

The trick is that for large set of keyword they will can detect your robot activity and temporary stop you with captcha filling.

Disavow w/o any doubts. Also report for scrapper and hosting weird redirects (and maybe malware soon!):
http://www.webpagetest.org/result/160202_M1_GH/
just see waterfall diagram and watchout for iTunes, Alibaba, Bing, Codecanyon, Walmart, and few more.

Starting from today Google also sent notes about spam:
http://www.thesempost.com/google-sending-confirmation-notices-for-spam-reports/
so for now every report count.

PS: I just make another test here:
http://www.webpagetest.org/result/160202_19_Y1/
this time with "save results body". Seems not so trivial! This site is trying to add someone in mine list of CodeCanyon. Also trying to manipulate Groupon + increase visits of Indeed job offer.

I see errors with Googlebot:
https://www.google.com/webmasters/tools/mobile-friendly/?url=http%3A%2F%2Flesliekays.com
and Mobile Friendly test uses Googlebot.

You need to go in Search Console -> Crawl -> Crawl errors just to see what results are return to Googlebot.

PM me link to blog please.

And this would be $1M question... even $1B.

As i said - it's core algo changes that's out of our control. I believe that top SEO experts (include Moz one's!) are trying to disassemble what's changed. So far there wasn't news that someone share it.

You also can use JSON-LD to add this in site w/o messing with HTML. Just refer to their documentation because it can be placed everywhere - in HEAD or in BODY but wasn't visible.

Well - this is easy win. But you need to ask yourself - should you put products in homepage?

In Woo you can easy duplicate /shop page with / (homepage). And when i go in some site i want to know about their company, best (best-of-best) products, etc. This is huge niche for A/B tests. For example - you can put some promo on some products and drive traffic to them. Or company innovation, or company news, etc.

Imagine Amazon (just example) if you visit them did you get ALL of their billion products on their homepage? Or if you visit newspaper did you get ALL their news on their homepage?

Only change between /shop and / is one image in homepage in ATF.

So - yes, you can canonical them, but then you will miss much better opportunity in CTR or CRO.

I don't know what kind of Woo + WordPress + theme you use. But you can download everything into separate folder and use tool to find text in all files. You just need to look who is responsible for "schema.org/Offer" adding into HTML.

Here is shoot in the dark:
https://webdesires.co.uk/blog/add-schema-to-woocommerce-products/
you just need to reverse this change.

As i said - it's just guessing. Can be also in plugin somewhere.

Yes! I have tested them in this one:
https://www.youtube.com/watch?v=5ip03xNfpwg

CTR was 6.36% and closes are 9.09%. I know that they're not "best-of-best" annotations, but definitely helps. But video is strange because have long intro and one of note is for jump inside of video where action begins. I'm trying to implement also "cards" in YouTube videos to see their performance too.

Well - you can see on monday traffic from US, VA, Ashburn that spike in Monday.

Go in GA, site, Audience, Geo, Location, US, VA, Ashburn. Then add secondary dimension - Browser and later Browser version. And start looking for monday patterns. Once you see spike you need go back in your http web log and find full user agent.

For now there is one good news - if bot can be seen in GA then this mean that bot execute JS for sure. Because there are bots that can't execute JS but make "parasite" traffic. There are also bots that didn't visit your site but make "fake" visit only in GA too.

Here is example of Rich Snippet w/o offer (with price):

ACME
  Executive Anvil
 
  Sleeker than ACME's Classic Anvil, the
    Executive Anvil is perfect for the business traveler
    looking for something to drop from a height.

Product #: 925872
  4.4 stars, based on 89 reviews

As you can see this code can be validated and works OK. Now you need to go back in Woo and make "patch" to remove http://schema.org/Offer where is price located. OR temporary disable them.

The usual suspect is "Amazon Datacenter":
https://aws.amazon.com/about-aws/global-infrastructure/

Since this datacenter is hosting many bots (Even Roger - MOZ bot was there!) you must identify bot user agent and see why this happens. Most bots follow robots.txt protocol so you can disable them scanning your site.