Tuesday, January 26, 2010

Re: Website Crawling Issue

Anita,

1 - The sitemaps are just some accelerators/ facilitators for the
robots to spider your site. Then your pages are seen by the spiders
does not guarantee however that they will all be indexed: each search
engine will index them "as usually", ie as if they have discovered
them directly. Whether they index the page or not, and how they rank
it, is not directly related to the sitemap.

2 - Sitemaps are highly useful though, most specially if your internal
links are not very visible to spiders (eg: flash-only or javascript-
only navigation). So they are a must for most serious SEO work. In
fact, they should be part of the "normal life" of your site.
If your site is highly dynamic, you might consider having several
sitemaps: one for "permanent" pages, the main structure; and one or
several for the "dynamic pages", eg automatically generated, news,
etc.
You would then use a "metamap" which would point to these different
maps

3 - Most serious spiders/ search engines are now using the sitemap.
Although submitting your sitemaps to GYM (Google, Yahoo, Microsoft/
Bing) is an easy step and should be done at least once, you need to
simply place a permanent link to your map (or metamap) in your
robots.txt: place a line such as

Sitemap: http://mysite.com/sitemap.xml
(note that the URL must be complete with http://)
This line must be present only once, so if you have several maps you
should make this line point to your "metamap"
Placing this permanent link ensures that main robots will regularly
reload your sitemaps, even if you have made no recent submission


On Jan 8, 8:29 am, Anita <alwaysonlinepro...@gmail.com> wrote:
> I am facing crawling problem with my website. I have already made xml
> site with help of free xml site map generator tool and submitted in
> google via webmaster tools.
>
> Website most of pages indexed but some pages not indexed yet. I have
> already my site map 2-3 times, but still those pages not crawled.
>
> Any body any solutions about this issue...

--
You received this message because you are subscribed to the Google Groups "Google Search Engine Optimization SEO Google - MSN - Yahoo" group.
To post to this group, send email to seo1@googlegroups.com.
To unsubscribe from this group, send email to seo1+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/seo1?hl=en.

No comments:

Post a Comment