Crawl questions
-
My first website crawl indicating many issues. I corrected the issues, requested another crawl and received the results. After viewing the excel file I have some questions.
1. There are many pages with missing Titles and Meta Descriptions in the Excel file. An example is http://www.terapvp.com/threads/help-us-decide-on-terapvp-com-logo.25/page-2
That page clearly has a meta description and title. It is a forum thread. My forum software does a solid job of always providing those tags. Why would my crawl report not show this information? This occurs on numerous pages.
2. I believe all my canonical URLs are properly set. My crawl report has 3k+ records, largely due to there being 10 records for many pages. These extra records are various sort orders and style differences for the same page i.e. ?direction=asc.
My need for a crawl report is to provide actionable data so I can easily make SEO improvements to my site where necessary. These extra records don't provide any benefit.
IF the crawl report determined there was not a clear canonical URL, then I could understand. But that is not the case. An example is http://www.terapvp.com/forums/news/ If you look at the source you will clearly see
Where is the benefit to including the 10 other records in the Crawl report which show this same page in various sort orders? Am I missing anything?
3. My robots.txt appropriately blocks many pages that I do not wish to be crawled. What is the benefit to including these many pages in the crawl report?
Perhaps I am over analyzing this report. I have read many articles on SEO, but now that I have found SEOmoz, I can see I will need to "unlearn what I have learned". Many things such as setting meta keyword tags are clearly not helpful. I wish to focus my energy and I was looking to the crawl report as my starting point. Either I am missing something, or the report design needs improvement.
-
Those are commends added to the code. My site has is part of the effort to rid the internet of IE6 browsers. You can read more about it http://www.theie6countdown.com/join-us.html. There is a simple script placed in the site which detects IE 6 & 7 browsers and asks users to update their software.
TYNT is a SEO tool http://www.tynt.com/. It allows webmasters to track any information which is copied and pasted from their site. It creates links back to the site, and tracks all activity on those links.
Both of those scripts are working normally and neither should negatively impact crawls. My Google WMT show my site being crawled normally, no errors. I have multiple pages ranked #1. With that said, there are plenty of opportunities for me to improve, which is why I am here.
The position of the Title and meta tags within the head should not be a factor at all. I did hear Matt Cutts share once that webmasters could move the information to the top to help if there are other issues such as a page where the tag is not properly closed, but that is not really relevant in this case.
-
Your source code looks different to me, if not .... abnormal. What's with doing above the description anyway. I don't know if that code is blocking the crawler from that code to . I'm no code monkey, but all my metas are right next to each other. Yours are completely separated from the title. Take out that code and see if it happens again. OR better yet, go to GWT and FETCH AS GOOGLEBOT. For sure that will tell you what Google bot is seeing.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Suggested Screaming Frog configuration to mirror default Googlebot crawl?
Hi All, Does anyone have a suggested Screaming Frog (SF) configuration to mirror default Googlebot crawl? I want to test my site and see if it will return 429 "Too Many Requests" to Google. I have set the User Agent as Googlebot (Smartphone). Is the default SF Menu > Configuration > Speed > Max Threads 5 and Max URLs 2.0 comparable to Googlebot? Context:
Intermediate & Advanced SEO | | gravymatt-se
I had tried NetPeak SEO Spider which did a nice job and had a cool feature that would pause a crawl if it got to many 429. Long Story short, B2B site threw 429 Errors when there should have been no load on a holiday weekend at 1:00 AM.0 -
Portfolio Image Landing Page Question/Issue
Hello, We have a client with a very image heavy website. They have Portfolio pages with a large number of images. We are currently working on adding more copy to the site but wanted to confirm we are taking the right approach for the images on the site. Under the current structure each image has its own landing page (with no copy) and is fed in (or generated on) to a Portfolio Page. While we know this is not ideal as it would be best to have the images on the Portfolio Page directly or even fill out the landing pages with copy; due to the amount of images and the fact these are only images (and not a 'targeted' page) that would not really be feasible. Aside from the thin content concern these individual landing pages were being indexed so they are showing hundreds of pages on their sitemap.xml and in GSC even though they only have a few actual pages. In the meantime we went into each image-page and placed a canonical tag back to the main Portfolio Page (with the hopes to add content to that page and have it as the ‘overarching’ page). Would this be the right approach? – We considered ‘noindex-follow’ tags but would want the images to be crawled; the issue is because the pages are not on the actual page are we canonicalizing these images to nothing? Any insight would really be appreciated. Thank you in advance.
Intermediate & Advanced SEO | | Ben-R0 -
Sitemap and content question
This is our primary sitemap https://www.samhillbands.com/sitemaps/sitemap.xml We have a about 750 location based URL's that aren't currently linked anywhere on the site. https://www.samhillbands.com/sitemaps/locations.xml Google is indexing most of the URL because we submitted the locations sitemap directly for indexing. Thoughts on that? Should we just create a page that contains all of the location links and make it live on the site? Should we remove the locations sitemap from separate indexing...because of duplicate content? #
Intermediate & Advanced SEO | | brianvestSitemap Type Processed Issues Items Submitted Indexed --- --- --- --- --- --- --- --- --- 1 /sitemaps/locations.xml Sitemap May 10, 2016 - Web 771 648 2 /sitemaps/sitemap.xml Sitemap index May 8, 2016 - Web 862 730
0 -
Metatags on drupal question
Hi Im quite inexperienced on drupal (normally an umbraco user!) and im having some difficulty with the Metatags on the CMS. I have been applying Meta Title and descriptions to the individual pages however they only appear when i preview the page and not when the page is saved. When i go into the metatag section located at /admin/config/search/metatags i am given a list of settings including Global: Front Page and Node. Im sure the reason it keeps defaulting the metatags back is to do with this but im not sure what to change to apply my own Thanks in advance
Intermediate & Advanced SEO | | TheZenAgency1 -
Question For Anyone
Hi All, Would you be able to answer one small question If you go to Australian Google - www.google.com.au and search for "loans" on positions number # 38 you will see the following site paydayloansyouknow.com.au . It has only 3 pages , 0 links, PA 1,and DA 1 How it's possible to archive such results? This is the print screen in case you dont see what i am asking about
Intermediate & Advanced SEO | | Webdeal
( http://www.freeimagehosting.net/oa75d Will appreciate any answer?0 -
Multilingual sites: Canonical and Alternate tag implementation question
Hello, I would like some clarification about the correct implementation of the rel="alternate" tag and the canonical tag. The example given at http://support.google.com/webmasters/bin/answer.py?hl=en&answer=189077 recommends implementing the canonical tag on all region specific sub-domains, and have it point to the www version of the website Here's the example given by Google. My question is the following. Would this technique also apply if I have region specific sites site local TLD. In other words, if I have www.example.com, www.example.co.uk, www.example.ca – all with the same content in English, but prices and delivery options tailored for US, UK and Canada residents, should I go ahead and implement the canonical tag and alternate tag as follows: I am a bit concerned about canonicalizing an entire local TLD to the .com site.
Intermediate & Advanced SEO | | Amiee0 -
What Questions Should I Be Asking?
I just read a discussion that was originally posted by Steve Ollington on May 22, 2011 where he states that many people are asking the wrong types of questions on this forum. He said that he wonders if he will see a shift from people asking questions on "how to rank" to questions dealing with "how to work out the best KPIs" (Key Performance Indicators - yes I had to google it). I was once told we learn more by asking questions about a topic than by just listening. I've also been told that sometimes the right question to ask is, "What questions should I be asking?" So here is my question, what types of questions should I be asking to be better at SEO? Perhaps these are some of them: Is it possible to be good at SEO when it is not a full-time job? It is very tempting to look for easy answers when you only have limited time. What are considered KPI's? Are they different for every industry? How do you know what is junk information vs what is truly good SEO advice? Is it just simply trial and error? It seems to me that if people find truly good SEO information, they aren't going to be sharing it so easily. It's the whole, "You get what you pay for". Maybe some of you can tell me more of the questions I should be asking.
Intermediate & Advanced SEO | | kadesmith1 -
Redirect on exact match domain to Brand domain question :)
Hi, If I have a website with the domain crazysocks.co.uk and a title tag 'black socks' would I see any benefit redirecting blacksocks.co.uk to crazysocks.co.uk, to give my keyword 'black socks' a boost in the SE's from the EMD. I see it loads where an EMD is indexed for its term but when you click the result it redirects to a branded domain. I personally cant see this being true but wanted to double check.
Intermediate & Advanced SEO | | activitysuper0