Site Maps
-
I have provided a site maps for google but although it craws my site www.irishnews.com at 6:45AM the details in the site map are not seen on google for a few days - any ideas how to get this feature working better would be great.
example
<url><loc>http://www.irishnews.com/news.aspx?storyId=1126126</loc>
<priority>1</priority>
<lastmod>2012-01-23</lastmod>
<changefreq>never</changefreq></url>thanks
-
Hi Liam,
There is always a natural delay between crawling and indexing, and it's rarely instantaneous. Although I can see why you'd want a news site to be getting indexed pretty quickly.
The one thing that stands out is from the example is the <changefreq>tag, which you've got set on 'never'. This is essentially for archived pages, and tells crawlers that it's low importance (even though you've given it high priority). Even if you're not intending to change the article, I'd still recommend giving it a change frequency of 1 month so you're inviting the crawlers to come and check it more often. Saying that, this doesn't mean that if you set the frequency to hourly that the crawlers would come back every hour, as they'd soon figure out that nothings changing.</changefreq>
Really I'd have your home and catergory pages on daily, the articles on monthly, and the static pages on monthly or yearly.
In terms of getting them indexed quicker after the crawl, this is just a case of establishing trust and importance from the search engines. They need to know that you have news content that requires a quicker indexed. You can gain this trust by providing regular, high quality pages. When their crawlers pick up that there are new pages going live daily you will see the index get quicker.
Depending what CMS you are using, it is also worth getting a script or plugin that pings the search engines with an updated site map every time a new page or post is added. I've done this through Wordpress and articles get indexed within 4 hours. Again though, it didn't start this way and it only got this quick once I was putting up 4 posts a week like clockwork.
Good luck.
Cheers
David
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hlp with site setup
Hi there and thanks for the great information, certainly lots to take in. Can anyone suggest the best way to setup product / category url structure for a store? At the moment we have something like domainname.com/parentcategory/subcategory/product name.html As the product url, we edited url structure using a plugin, we don't use default WooCommerce url settings. domainname.com/parentcategory/subcategory/product name.html. this can sometimes be long But when you click on the product the url changes to the following. domainname.com/product name.html. This can shorted the url by 40% and still have keyword in url Is there any benefit in doing his? Re canonical urls, I only have about 15 products that are selected in many categories.the other 200 are under once category only. Product pages don't have many backlinks at the moment. Thanking you so much.
White Hat / Black Hat SEO | | IvanaDaulay0 -
Paid Link/Doorway Disavow - disavowing the links between 2 sites in the same company.
Hello, Three of our client's sites are having difficulty because of past doorway/paid link activity, which we're doing the final cleanup on with a disavow. There are links between the sites. Should we disavow all the links between the sites? Thank you.
White Hat / Black Hat SEO | | BobGW0 -
Site dropping in rank even through there are more backlinks being added
Hello, One of my client's sites is ranking lower than he should. This happened when we took off backlinks (20 little blogs, several site-wide paid links. It really dropped the site, but it had to be done. Since then we've increased his # of root domains by 10% through white hat link building in his non-competitive niche, and rankings are still poor. I know that's not much in the way of added backlink value, but we're working on it. My question is, how have the recent (and coming) updates possibly effected us. We want to take the remaining problem areas off right away, but another drop in traffic is not a good idea. Even though the blogs (see below) have no backlinks of themselves, they cause drops when taken off) He still has -20 little blog backlinks w/ a quarter of them being exact match anchor text.
White Hat / Black Hat SEO | | BobGW
-1 sitewide paid link - an image, exact match alt tag anchor text
-1 non-site-wide paid links that is an image near the footer, exact match alt tag anchor text.
-3 links on a domain, this one looks fairly editorial, but there are a bunch of paid links on that page. Changing to non-exact-match anchor text
-2 links on two domains that look completely editorial with no other paid links on that page. non-exact-match anchor text -70 backlinks total with about 1/3 being problematic. How does this site look in regards to updates and when to take links off without tanking our site even more? Thanks.0 -
Do some sites get preference over others by Google just because? Grandfathered theory
So I have a theory that Google "grandfathers" in a handful of old websites from every niche and that no matter what the site does, it will always get the authority to rank high for the relevant keywords in the niche. I have a website in the crafts/cards/printables niche. One of my competitors is http://printable-cards.gotfreecards.com/ This site ranks for everything... http://www.semrush.com/info/gotfreecards.com+(by+organic) Yet, when I go to visit their site, I notice duplicate content all over the place (extremely thin content, if anything at all for some pages that rank for highly searched keywords), I see paginated pages that should be getting noindexed, bad URL structure and I see an overall unfriendly user experience. Also, the backlink profile isn't very impressive, as most of the good links are coming from their other site, www.got-free-ecards.com. Can someone tell me why this site is ranking for what it is other than the fact that it's around 5 years old and potentially has some type of preference from Google?
White Hat / Black Hat SEO | | WebServiceConsulting.com0 -
A site is using their competitors names in their Meta Keywords and Descriptions
I can't imagine this is a White Hat SEO technique, but they don't seem to be punished for it by Google - yet. How does Google treat the use of your competitors names in your meta keywords/descriptions? Is it a good idea?
White Hat / Black Hat SEO | | PeterConnor0 -
HELP - Site architecture of E-Commerce Mega Menu - Linkjuice flow
Hi everyone, I hope you have a couple of mins to give me your opinion. Ecommerce site has around 2000 products, in english and spanish, and around only 70 hits per day if that. We have done a lot of optimisation on the site - Page Titles, URL's, Content, H1's, etc.... Everything on page is pretty much under control, except I am starting to realise the site architecture could be harming our SEO efforts. Once someone arrives on site they are language detected and do a 302 to either domain.com/EN or domain.com/ES depending on their preferred language. Then on the homepage, we have the big MEGA MENU - and we have
White Hat / Black Hat SEO | | bjs2010
CAT 1
SubCat 1
SubsubCat 1
SubsubCat 2
SubsubCat 3 Overall, there are 145 "categories". Plus links to some CMS pages, like Home, Delivery terms, etc... Each Main Category, contains the products of everything related to that category - so for example:
KITCHENWARE
COOKWARE BAKINWARE
SAUCEPANS BOWLS
FRYING PANS Kitchenware contains: ALL PRODUCTS OF SUBCATS BELOW, SO COOKWARE ITEMS, SAUCEPANS, FRYING PANS, BAKINGWARE, etc... plus links to those categories through breadcrumbs and a left hand nav in addition to the mega menu above. So once the bots hit the site, immediately they have this structure to deal with. Here is what stats look like:
Domain Authority: 18 www.domain.com/EN/
PA: 27
mR: 3.99
mT: 4.90 www.domain.com/EN/CAT 1
PA: 15
mR: 3.05
mT: 4.54 www.domain.com/EN/CAT 1/SUBCAT1
PA: 15
mR: 3.05
mT: 4.54 Product pages themselves - have a PA of 1 and no mR or mT. I really need some other opinions here - I am thinking of: Removing links in Nav menu so it only contains CAT1 and SUBCAT1 but DELETE SUBSUBCATS1 which represent around 80 links Remove products within the CAT1 page - eg., the CAT 1 would "tile" graphical links to subcategories, but not display products themselves. So products are only available right at the lowest part of the chain (which will be shortened) But I am willing to hear any other ideas please - maybe another alternative is to start building links to boost DA and linkjuice? Thanks all, Ben0 -
Will my association's network of sites get penalized for link farming?
Before beginning I found these similar topics here: http://www.seomoz.org/q/multiple-domains-on-same-ip-address-same-niche-but-different-locations http://www.seomoz.org/q/multiple-domains-on-1-ip-address We manage over two dozen dental sites that are individually owned through out the US. All these dentists are in a dental association which we also run and are featured on (http://www.acedentalresource.com/). Part of the dental associations core is sharing information to make them better dentists and to help their patients which in addition to their education, is why they are considered to be some of the best dentists in the world. As such, we build links from what we consider to be valuable content between the sites. Some sites are on different IPs and C-Blocks, some are not. Given the fact that each site is only promoting the dentist at that brick and mortar location but also has "follow" links to other dentists' content in the network we fear that we are in the grey area of link building practices. Questions are: Is there an effective way to utilize the power of the network if quality content is being shared? What risks are we facing given our network? Should each site be on a different IP? Would having some of our sites on different servers make our backlinks more valuable than having all of our sites under the same server? If it is decided that having unique IPs is best practice, would it be obvious that we made the switch? Keep in mind that ALL sites are involved in the association, so naturally they would be linking to each other, and the main resource website mentioned above. Thanks for your input!
White Hat / Black Hat SEO | | DigitalElevator0 -
Could a sitewide footer EXACT MATCH anchor text link hurt or potentially penalize a site?
I am pretty sure this would hurt rankings yet I just want another's opinion on it. Would a sitewide footer link with exact match keyword anchor text to the page you want to rank for your main keyword hurt you? Basically if it were a link to the homepage, yet you wanted to make the anchor text your main objective keyword, would it hurt to have this in the footer along with the logo link at the top of a page that is just "home" anchor text?
White Hat / Black Hat SEO | | jbster130