Site Maps
-
I have provided a site maps for google but although it craws my site www.irishnews.com at 6:45AM the details in the site map are not seen on google for a few days - any ideas how to get this feature working better would be great.
example
<url><loc>http://www.irishnews.com/news.aspx?storyId=1126126</loc>
<priority>1</priority>
<lastmod>2012-01-23</lastmod>
<changefreq>never</changefreq></url>thanks
-
Hi Liam,
There is always a natural delay between crawling and indexing, and it's rarely instantaneous. Although I can see why you'd want a news site to be getting indexed pretty quickly.
The one thing that stands out is from the example is the <changefreq>tag, which you've got set on 'never'. This is essentially for archived pages, and tells crawlers that it's low importance (even though you've given it high priority). Even if you're not intending to change the article, I'd still recommend giving it a change frequency of 1 month so you're inviting the crawlers to come and check it more often. Saying that, this doesn't mean that if you set the frequency to hourly that the crawlers would come back every hour, as they'd soon figure out that nothings changing.</changefreq>
Really I'd have your home and catergory pages on daily, the articles on monthly, and the static pages on monthly or yearly.
In terms of getting them indexed quicker after the crawl, this is just a case of establishing trust and importance from the search engines. They need to know that you have news content that requires a quicker indexed. You can gain this trust by providing regular, high quality pages. When their crawlers pick up that there are new pages going live daily you will see the index get quicker.
Depending what CMS you are using, it is also worth getting a script or plugin that pings the search engines with an updated site map every time a new page or post is added. I've done this through Wordpress and articles get indexed within 4 hours. Again though, it didn't start this way and it only got this quick once I was putting up 4 posts a week like clockwork.
Good luck.
Cheers
David
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Are bloggs published on blog platforms and on our own site be considered duplicate content?
Hi, SEO wizards! My company has a company blog on Medium (https://blog.scratchmm.com/). Recently, we decided to move it to our own site to drive more traffic to our domain (https://scratchmm.com/blog/). We re-published all Medium blogs to our own website. If we keep the Medium blog posts, will this be considered duplicate content and will our website rankings we affected in any way? Thank you!
White Hat / Black Hat SEO | | Scratch_MM0 -
If our site hasn't been hit with the Phantom Update, are we clear?
Our SEO provider created a bunch of "unique url" websites that have direct match domain names. The content is pretty much the same for over 130 websites (city name is different) that link directly to our main site. For me this was a huge red flag, but when I questioned them and they said it was fine. We haven't seen a drop in traffic, but concerned that Google just hasn't gotten to us. DA for each of these sites are 1 after several months. Should we be worried? I think yes, but I am an SEO newbie.
White Hat / Black Hat SEO | | Buddys0 -
HELP - Site architecture of E-Commerce Mega Menu - Linkjuice flow
Hi everyone, I hope you have a couple of mins to give me your opinion. Ecommerce site has around 2000 products, in english and spanish, and around only 70 hits per day if that. We have done a lot of optimisation on the site - Page Titles, URL's, Content, H1's, etc.... Everything on page is pretty much under control, except I am starting to realise the site architecture could be harming our SEO efforts. Once someone arrives on site they are language detected and do a 302 to either domain.com/EN or domain.com/ES depending on their preferred language. Then on the homepage, we have the big MEGA MENU - and we have
White Hat / Black Hat SEO | | bjs2010
CAT 1
SubCat 1
SubsubCat 1
SubsubCat 2
SubsubCat 3 Overall, there are 145 "categories". Plus links to some CMS pages, like Home, Delivery terms, etc... Each Main Category, contains the products of everything related to that category - so for example:
KITCHENWARE
COOKWARE BAKINWARE
SAUCEPANS BOWLS
FRYING PANS Kitchenware contains: ALL PRODUCTS OF SUBCATS BELOW, SO COOKWARE ITEMS, SAUCEPANS, FRYING PANS, BAKINGWARE, etc... plus links to those categories through breadcrumbs and a left hand nav in addition to the mega menu above. So once the bots hit the site, immediately they have this structure to deal with. Here is what stats look like:
Domain Authority: 18 www.domain.com/EN/
PA: 27
mR: 3.99
mT: 4.90 www.domain.com/EN/CAT 1
PA: 15
mR: 3.05
mT: 4.54 www.domain.com/EN/CAT 1/SUBCAT1
PA: 15
mR: 3.05
mT: 4.54 Product pages themselves - have a PA of 1 and no mR or mT. I really need some other opinions here - I am thinking of: Removing links in Nav menu so it only contains CAT1 and SUBCAT1 but DELETE SUBSUBCATS1 which represent around 80 links Remove products within the CAT1 page - eg., the CAT 1 would "tile" graphical links to subcategories, but not display products themselves. So products are only available right at the lowest part of the chain (which will be shortened) But I am willing to hear any other ideas please - maybe another alternative is to start building links to boost DA and linkjuice? Thanks all, Ben0 -
Redesigning my site, and not sure what is best for seo, subfolders or direct .html links?
,I have 4 examples to choose from, what is best:? http://hoodamath.com/games/dublox/index.html http://hoodamath.com/games/dublox.html http://hoodamath.com/dublox/index.html http://hoodamath.com/dublox.html
White Hat / Black Hat SEO | | hoodamath0 -
Are back links from audio sites any good?
In light of G's view of links from directories and other sources I have heard that links for audio sites like soundcloud.com can be beneficial. Has anyone had any positive experiences building likes from sources like this?
White Hat / Black Hat SEO | | Aikijeff0 -
I think my site is affected by a Google glitch...or something
Although google told me No manual spam actions found i had not received an unnatural link request notice i figured it would be a good idea to clean these up so i did. So i have submitted 3 reconsideration requests from google. They all came back with the same response: No manual spam actions found. I really doubt that anyone at google really checked those out.You will notice that i don't even appear on page 1-10 at all...its clearly google filtering the site out from the results(except for my brand terms), but i have no idea what for.What do you guys think it is? If you see anythign let me know so i can have it fixed.This has been going on for 2 months now...my company has been around for a long time...i dont understand why suddenly im not showing up in searches for the keyword si used to rank for...
White Hat / Black Hat SEO | | CMTM0 -
Should We Pull The Plug On This Site?
I am helping a retailer out with their site. They were hit hard with the Penguin update, and traffic has dropped by about 75%. Here are the stats: It is fairly new, has been up for about 3 years. Has partial match domain name Is nearly fully indexed with over 4K pages Has NOT received an unnatural link message from Google, so no manual penalty. Has had most keywords BURIED in the search results. Link profile: Has done about 50-100 blog comments, 500 directory submissions, 800 social bookmarks, 5-6 press releases, 300 article submissions (most removed), about 30-50 guest blog posts. I am thinking it may have just been hit because of aggressive use of anchor text as opposed to massive spamming. Then again, the site has never really added great content and the product pages have no unique content. Any thoughts?
White Hat / Black Hat SEO | | inhouseseo1 -
Why are these sites so high with poor relevant links...
Hello, Keyword: TV Stands. I have been researching competitors for a client and we seem to be unable to understand why certains pages are ranking on page 1 of Google UK for keyword TV Stands. eg: http://www.furnitureinfashion.net/plasma-TV-stand.html (Google UK 8 - TV Stands) http://direct.tesco.com/q/N.1999542/Nr.99.aspx (Google UK 9 - TV Stands) The furniture in fashion has links from sites like: http://www.ummah.com/forum/ and http://www.muslimco.com/ which is totaly irrelevant to the site. Any ideas on other things as the tesco.com site does not have direct links to it. Cheers
White Hat / Black Hat SEO | | JohnW-UK0