well, the spider found this page, also have the right to grab the page, it can grab is a big problem, we all know that spiders crawl this page will be from the domain name to the DNS server to the web space, so any part of a problem, spiders are not normal. First of all, domain name DNS record, then to normal, if there is a problem, the domain name is unable to access the IP address of the website, the website does not open, the spider to grab the page.
most of the time you update the search engine is not going to grab included, how to let the spider know your update? In addition to regular updates, to love Shanghai Webmaster Platform submit URL or submit the site map, also can make the search engine passive collection, this is through the chain let the spider know you have a lot of time ", the spider came to your site, but you can not find the updated link, because the web link structure is too complex or within the chain structure, so that the search engines climbed to the depths of the website pages in the crawl time class does not, which also caused the spider don’t know which page exists. So anyway, the first step we should ensure that the page can be seen spiders on this point, you can view web log to determine whether the spider visited.
website has been included in the optimization process is a either a rookie or veteran are very worried, and even included website ranking is not directly related, but the ranking and included the relationship known as love Shanghai algorithm is constantly updated, for the love of Shanghai recorded standards are increasingly high, how to do a good job included the matter is a problem of Adsense or enterprise are very concerned about, then why not to talk about the website included problems.
1. web search engine can’t find the entrance of
2. does not let the search engines
in the server space is often mentioned in the long term, the server can not open that is certainly not grasping, we all know this. And if the server is card or unstable, it will affect the spider, the spider is not very good to grab the page or >
on the page included of course we must know this page, spiders don’t know what this page included? This is mainly divided into 2 categories: Web search engine can’t find the entrance and not let the search engines crawl.
many sites directly in the robots file naturally or half unconsciously blocked love Shanghai part of the page or website, but to find the reasons why not all day in the spider to grab my page, can this strange love Shanghai? You don’t let others in love, what Shanghai is included in your web page? So it is necessary to check from time to time robots file is normal.
two, whether can normal grab
is aware of the existence of this page