I should mention that this is a WordPress site and, with that, I may have answered my own question. Perhaps WordPress generates a robots.txt dynamically when the setting is active at Settings > Privacy?
Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Best posts made by joshcanhelp
-
RE: Invisible robots.txt?posted in Technical SEO
-
RE: Should you get a new Google Analytics account if your site has a new domain after a site redesign/new development?posted in Reporting & Analytics
Theresa,
I'm going to support the two answers here: creating a new account is not a good use of time.
The main reason, in my mind, is just that you wouldn't have anything to compare the new site with. If you have new code, new content, new everything, it's important to compare the changes with the old site to make sure you made positive changes.
The best example I can give of a scenarios where you'd want that historic data is with any page on the site that already ranks well. After the page changed, did traffic increase? Decrease? Time of page? It's good to know how these were changed.
I'll also add, historic analytic data can help you find problems caused by the move. Disallowed robots.txt from the test site? Structure issues? Many of these can be found out from looking at your analytic data and having a baseline is important.
Hope that helps!