Screaming Frog returning both HTTP and HTTPS results...
-
Hi,
About 10 months I switched from HTTP to HTTPS. I then switched back (long story).
I noticed that Screaming Frog is picking up the HTTP and HTTPS version of the site. Maybe this doesn't matter, but I'd like to know why SF is doing that. The URL is: www.aerlawgroup.com
Any feedback, including how to remove the HTTPS version, is greatly appreciated.
Thanks.
-
No problem! I'm not too familiar with secure migrations in WP, but I bet there is some kind of plugin for that.
-
Hi,
Thank you! Fixing that link prevents SF from finding the HTTPS pages.
I use WP for this site. Is there a quick way to redirect HTTPS to HTTP? As opposed to forwarding page by page? Ideally, there is a way in WP (either in WP directly or via some plugin) that I can tell WP to automatically 301 redirect all HTTPS to HTTP. I use WPEngine for my hosting, if that's at all relevant.
Thank you again for your excellent answer.
-
Hi,
Found the source of your issue: this URL (https://www.aerlawgroup.com/sex-crimes.html) has a link pointing to it from a HTTP URL (http://www.aerlawgroup.com/sex-crimes/new-allegations-emerge-in-sexual-misconduct-case.html), which is probably how SF found all the other secure URLs.
All of your HTTPS page return 200. If you want to go back to non-secure, those secure URLs will need to be redirected back to HTTP, the same as you probably did when you went to secure.
Also, looks like you've still got a couple HTTPS pages indexed: https://www.google.com/search?q=site%3Aaerlawgroup.com+inurl%3Ahttps&oq=site%3Aaerlawgroup.com+inurl%3Ahttps&gs_l=serp.3...5577.15088.0.15655.24.24.0.0.0.0.150.1864.19j5.24.0....0...1.1.64.serp..3.0.0.bf5ZiosfHMI
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Dates on Google Search Results
Hello, I manage htts://globalrose.com When I search on Google for "Yellow Roses", "Yellow Roses Globalrose", or any search that might bring up one of our pages, sometimes our search results appear with dates right before the description. Does anyone know what this mean? Why they appear on some and not other pages? Here is a search result for example: Example Google Search Can someone please help clarify this for us?
Intermediate & Advanced SEO | | globalrose.com0 -
HTTP HTTPS Migration Gone Wrong - Please Help!
We have a large (25,000 Products) ecommerce website, and we did an HTTP=>HTTPS migration on 3/14/17, and our rankings went in the tank, but they are slowly coming back. We initially lost 80% of our organic traffic. We are currently down about 50%. Here are some of the issues. In retrospect, we may have been too aggressive in the move. We didn't post our old sitemaps on the new site until about 5 days into the move. We created a new HTTPS property in search console. Our redirects were 302, not 301 We also had some other redirect issues We changed our URL taxonomy from http://www.oursite.com/category-name.html to https://www.oursite.com/category-name (removed the .html) We changed our filters plugin. Proper canonicals were used, but the filters can generate N! canonical pages. I added some parameters (and posted to Search Console) and noindex for pages with multiple filter choices to cut down on our crawl budget yesterday. Here are some observations: Google is crawling like crazy. Since the move, 120,000+ pages per day. These are clearly the filtered pages, but they do have canonicals. Our old sitemaps got error messages "Roboted Out". When we test URLs in Google's robots.txt tester, they test fine. Very Odd. At this point, in search console
Intermediate & Advanced SEO | | GWMSEO
a. HTTPS Property has 23,000 pages indexed
b. HTTP Property has 7800 pages indexed
c. The crawl of our old category sitemap (852 categories) is still pending, and it was posted and submitted on Friday 3/17 Our average daily organic traffic in search console before the move was +/-5,800 clicks. The most recent Search Console had HTTP: 645 Clicks HTTPS: 2000 clicks. Our rank tracker shows a massive drop over 2 days, bottoming out, and then some recovery over the next 3 days. HTTP site is showing 500,000 backlinks. HTTPS is showing 23,000 backilinks. I am planning on resubmitting the old sitemaps today in an attempt to remap our redirects to 301s. Is this typical? Any ideas?0 -
Not showing up in search results for non-branded terms
Hello! Can anyone see any glaring reasons why this post: "98 Book Marketing Ideas That Can Help Authors Increase Sales" isn't on page one of Google — or even page 10! — for the term "book marketing ideas"? Many other sites with lower domain and page authority — even ones linking to this article — are ranking on the first ten pages for this term, and I can't figure out why we're not appearing anywhere. The same thing is happening for ALL of our other blog posts, and the keywords they're optimized for. According to GA, the only terms we're getting clicks from are branded keywords. This subdomain is now 2 years old, and the domain bookbub.com has been around for 5 years. Our domain authority is 61. We have the Yoast SEO plugin installed and are following all the standard SEO best practices. We have enough external links to at least be ranking within the first 10 pages of this Google search. I feel like there's something technically wrong, maybe in the code or backend, but nobody here can figure it out, and our hosting provider WP Engine has no ideas. Moz is returning crawl errors on our site, mainly "Error Code 804: HTTPS (SSL) Error Encountered" and "Error Code 803: Incomplete HTTP Response Received." I have confirmed with WP Engine that everything is set up correctly on our end, and that this is a known Moz issue. I've reached out to Moz's support team about this, and am awaiting a response. But what else am I missing? There's got to be something — I've been blogging for 10 years for different companies and my own personal websites, and I've never come across anything like this before. I'm completely stuck! I'd appreciate any insights you can offer. Thanks in advance! 🙂 EDIT: I heard back from Moz on those errors. The 804 errors are a Moz-side issue — their crawler isn't equipped to be able to handle SNI. They're looking into a resolution, and this wouldn't affect search engine crawlers. Regarding the 803 error: "When you see an 803 error, that means your site closed its TCP connection to our crawler before our crawler could read a complete HTTP response. You don't see this error when you go to the page in your browser because content-length is an outdated component for modern browsers and they will disregard this error, but the intention of our crawler is to report any errors that might be occurring. So the crawler is configured to detect and report such errors." The only thing I can think to do here is go back to WP Engine with this information, but other than that, I'm not sure what this could mean or how to fix it, or if this might be the underlying technical issue keeping us from ranking.
Intermediate & Advanced SEO | | bookbubpartners1 -
Http resolving to https - why isn't it doing that?
Hi everyone I've just been looking at a few https websites and noticed the http urls weren't redirecting to their https equivalents - why would a website owner not bother redirecting? As an example: http://www.marksandspencer.com I look forward to your feedback. L
Intermediate & Advanced SEO | | McTaggart0 -
How to dominate first page results?
see this https://www.google.com/search?q=podiatrist+edgewater+nj&rlz=1C1KMZB_enUS592US598&oq=p&aqs=chrome.4.69i60l3j69i61j69i59j69i60.2731j0j1&sourceid=chrome&es_sm=93&ie=UTF-8 Dr. Gehegan ; Progressive Podiatry How I can appear my main keyword 3-4 times on first page results?
Intermediate & Advanced SEO | | Green.landon0 -
Google Search Results...
I'm trying to download every google search results for my company site:company.com. The limit I can get is 100. I tried using seoquake but I can only get to 100. The reason for this? I would like to see what are the pages indexed. www pages, and subdomain pages should only make up 7,000 but search results are 23,000. I would like to see what the others are in the 23,000. Any advice how to go about this? I can individually check subdomains site:www.company.com and site:static.company.com, but I don't know all the subdomains. Anyone cracked this? I tried using a scrapper tool but it was only able to retrieve 200.
Intermediate & Advanced SEO | | Bio-RadAbs0 -
Help with https// redirects
Hey there
Intermediate & Advanced SEO | | Jay328
I have a client who just moved from a self hosted CMS to Adobe Catalyst (don't ask!)
The problem: Their url indexed with google is https://domain.com, Adobe Catalyst does not support third party SSL certificates or https domains. Now when people google them https://domain.com shows up in search, HOWEVER it does not have a trusted certificate and a pop up window blocks the site. They are a mortgage company so SSL is really not needed. What can I do to get google to recognize the site at http: vs. https? Would this be something in GWMT? Thanks!0 -
Any way to make my page on top of results again?
Hello Everyone, For a long time now I've had many of my website pages ranked on first page of results for some popular searches. On friday I noticed the traffic coming from google dropped dramatically and I realized that I was not on first page anymore. My SEO is focused for Google Spain. Here are some examples: http://msn-messenger.hispazone.com/ This page was on position 9 (first page) if we perform the search "Descargar msn" on Google. Now it's on page 3. http://http://adobe-acrobat-reader.hispazone.com/ This was position 5 (first page) for the search "descargar adobe". Now it's also page 3. Any suggestions? Best Regards.
Intermediate & Advanced SEO | | HispaZone0