Unsolved Crawling only the Home of my website
-
Hello,
I don't understand why MOZ crawl only the homepage of our webiste https://www.modelos-de-curriculum.comWe add the website correctly, and we asked for crawling all the pages. But the tool find only the homepage. Why?
We are testing the tool before to suscribe. But we need to be sure that the tool is working for our website. If you can please help us.
-
@Azurius
Certainly, I understand your concern about MOZ only crawling the homepage of your website despite adding it correctly and requesting a full crawl of all pages. It's frustrating when testing a tool before subscribing, and not getting the expected results can raise doubts about its functionality. To address this issue, I recommend double-checking your website's settings within the MOZ platform. Ensure that the website URL provided matches the actual structure of your site and that there are no typos or errors in the input. Additionally, review MOZ's documentation or contact their customer support to confirm if there are specific settings or configurations needed for a comprehensive crawl. Sometimes, minor adjustments in the tool's settings or the website's structure can make a significant difference in the crawling process. I hope this helps, and I wish you success in resolving this matter and making an informed decision about your subscription. -
Crawling only the home page of your website is a common practice in web indexing and search engine optimization. This approach allows search engine bots to focus on your site's main landing page, ensuring that it's properly indexed and ranked. Here's how you can specify that only the home page should be crawled:
Robots.txt File,
Canonical Tag
Sitemap.xml,
Noindex Tags,
Meta Robots TagBy implementing these methods, you can direct search engine crawlers to focus primarily on your website's home page, ensuring that it receives the most attention in terms of indexing and ranking. This can be particularly useful if your home page is the most important and relevant page for your website's SEO strategy.
https://www.clippingnext.com/ -
@Azurius i have the same issue,. I think the answer here is quite helpful
-
Hello,
There could be several reasons why MOZ is only crawling the homepage of your website, https://www.modelos-de-curriculum.com. Here are a few possibilities:
Robots.txt file: Check your website's robots.txt file to ensure that it's not blocking MOZ's web crawlers from accessing other pages. Make sure there are no disallow rules that could restrict access to certain areas of your site.
Nofollow tags: Ensure that your website doesn't have "nofollow" tags on internal links that may be preventing MOZ from following and crawling those links.
JavaScript: If your website heavily relies on JavaScript for content rendering, MOZ may face difficulty crawling the content. Ensure that important content is accessible without JavaScript.
Canonical tags: Check for canonical tags in your website's HTML. If you have specified the homepage as the canonical page for all other pages, this could limit MOZ's ability to crawl additional pages.
Site structure: MOZ may have trouble crawling pages with complex or unconventional site structures. Ensure that your website follows standard navigation and linking practices.
Crawl settings: Double-check the settings in your MOZ account to confirm that you have requested a full site crawl, not just the homepage.
If you've verified all these aspects and MOZ is still not crawling your website correctly, you may want to reach out to their support team for further assistance. They can provide more specific guidance based on your account and settings.
Testing the tool before subscribing is a wise approach to ensure it meets your needs. I recommend contacting MOZ's support for personalized assistance in resolving the crawling issue with your website. (PMP Exam Prep) (Canada PR) (Study abroad) (Project Management Life Cycle)
-
Hello,
The reason Moz is only crawling the homepage of your website could be due to various factors. Here are a few possibilities:
Robots.txt File: Check your website's robots.txt file to ensure that it doesn't block search engine crawlers from accessing specific pages.
Meta Robots Tags: Make sure there are no "noindex" meta tags on your internal pages that might prevent them from being indexed.
Crawl Restrictions: Moz may not have had enough time to crawl all the pages on your website. It might take some time for the tool to explore and index your entire site.
Sitemap: Ensure that your website's sitemap is correctly submitted to Moz. A sitemap helps search engines find and index all the pages on your site.
Internal Linking: Check if there are internal links from your homepage to other pages. A lack of internal links can make it harder for search engines to discover and crawl your site.
Access Permissions: Make sure there are no access restrictions or password protection on certain pages.
To resolve this issue and ensure Moz crawls all the pages of your website, consider checking and addressing these factors. If the problem persists, it might be helpful to reach out to Moz's customer support for specific assistance with your subscription trial.
(Canada PR) (Study abroad) (PMP Exam Prep) (Reference Letter for Canada PR) -
check your .htaccess if there is something that is blocking crawlers.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to index e-commerce marketplace product pages
Hello! We are an online marketplace that submitted our sitemap through Google Search Console 2 weeks ago. Although the sitemap has been submitted successfully, out of ~10000 links (we have ~10000 product pages), we only have 25 that have been indexed. I've attached images of the reasons given for not indexing the platform. gsc-dashboard-1 gsc-dashboard-2 How would we go about fixing this?
Technical SEO | | fbcosta0 -
Unsolved Strange "?offset" URL found with content crawl issues
I recently recieved a slew of content crawl issues via Moz for URL's that I have never seen before For example:
Moz Pro | | HannahPalamara
Standard URL: https://skilldirector.com/news,
Newly identified URL: https://skilldirector.com/news?offset=1469542207800&category=Competency+Management). Does anyone know where the URL comes from and how to fix it?0 -
Dynamic Canonical Tag for Search Results Filtering Page
Hi everyone, I run a website in the travel industry where most users land on a location page (e.g. domain.com/product/location, before performing a search by selecting dates and times. This then takes them to a pre filtered dynamic search results page with options for their selected location on a separate URL (e.g. /book/results). The /book/results page can only be accessed on our website by performing a search, and URL's with search parameters from this page have never been indexed in the past. We work with some large partners who use our booking engine who have recently started linking to these pre filtered search results pages. This is not being done on a large scale and at present we only have a couple of hundred of these search results pages indexed. I could easily add a noindex or self-referencing canonical tag to the /book/results page to remove them, however it’s been suggested that adding a dynamic canonical tag to our pre filtered results pages pointing to the location page (based on the location information in the query string) could be beneficial for the SEO of our location pages. This makes sense as the partner websites that link to our /book/results page are very high authority and any way that this could be passed to our location pages (which are our most important in terms of rankings) sounds good, however I have a couple of concerns. • Is using a dynamic canonical tag in this way considered spammy / manipulative? • Whilst all the content that appears on the pre filtered /book/results page is present on the static location page where the search initiates and which the canonical tag would point to, it is presented differently and there is a lot more content on the static location page that isn’t present on the /book/results page. Is this likely to see the canonical tag being ignored / link equity not being passed as hoped, and are there greater risks to this that I should be worried about? I can’t find many examples of other sites where this has been implemented but the closest would probably be booking.com. https://www.booking.com/searchresults.it.html?label=gen173nr-1FCAEoggI46AdIM1gEaFCIAQGYARS4ARfIAQzYAQHoAQH4AQuIAgGoAgO4ArajrpcGwAIB0gIkYmUxYjNlZWMtYWQzMi00NWJmLTk5NTItNzY1MzljZTVhOTk02AIG4AIB&sid=d4030ebf4f04bb7ddcb2b04d1bade521&dest_id=-2601889&dest_type=city& Canonical points to https://www.booking.com/city/gb/london.it.html In our scenario however there is a greater difference between the content on both pages (and booking.com have a load of search results pages indexed which is not what we’re looking for) Would be great to get any feedback on this before I rule it out. Thanks!
Technical SEO | | GAnalytics1 -
Unsolved How do I cancel this crawl?
The latest crawl on my site was the 4th Jan with a current crawl 'in progress'. How do i cancel this crawl and start a new one? I've been getting keyword ranking etc but no new issues are coming through. Screenshot 2022-05-31 083642.jpg
Moz Tools | | ClaireU0 -
Solved Why is MOZ crawl taking so long?
I began my site crawl on November 3rd and now it is November 7th and it is still "in progress". Why is this happening?
Product Support | | CarisaS_Wenda0 -
Crawl test
I used to use the crawl test tool to crawl websites and it presented the information in a really useful hierarchy of pages. The new on-demand crawl test doesn't seem to do this. Is there another tool I should be using to get the data?
Product Support | | Karen_Dauncey0 -
Link to Moz Pro home 404s, and then I stumbled upon Moz's staging site...
I'm wary of giving more information as it's probably best that little to no one be able to access Moz staging for one reason or another. I should also mention that this was done very unintentionally on my part. That said, from the 404 from Moz Pro home, I tried to access "My QA", and that's when I realized that everything I did thereafter was within staging.moz.com (and posted a question similar to this one). Does Moz permit access to their staging site, or did I stumble upon a mistake - or Moz-stake, if you will?
Product Support | | Lumina0 -
MOZ Crawl help
Our MOZ report says it crawled 1800 pages so it reports a lot of errors based on those pages. We don't have that many pages on our site. What is MOZ crawling? I updated the profile to make sure it crawls the filtered page section of Google Analytics.
Product Support | | JessiK0