登录 | 注册 | FAQ
Anonymous

robots.txt

+ 发表回复

6 篇帖子 分页: 1 / 1


robots.txt

iDana » 周五 5月 20, 2011 4:04 pm

Hello,

I know about robots.txt but can anyone give me any more information on it?

Thanks.
头像

iDana

  • 帖子: 60
  • 注册: 周二 10月 26, 2010 10:05 am
--
iDana

Re: robots.txt

iDana » 周五 5月 20, 2011 4:28 pm

Thank you!:)
头像

iDana

  • 帖子: 60
  • 注册: 周二 10月 26, 2010 10:05 am
--
iDana

Re: robots.txt

nigerianprince » 周二 5月 24, 2011 7:53 pm

thank you as well! I was looking for robot.txt info before but the linked site is definitely a better resource than what i was referring to before
头像

nigerianprince

  • 帖子: 1
  • 注册: 周二 5月 24, 2011 7:44 pm

Re: robots.txt

markomako79 » 周五 8月 05, 2011 4:30 am

I added a sitemap to my wordpress blog and when I ran an page analyzer it told me that robots.txt was now present. Does this term have something to do with the sitemap, or is it totally unrelated. It would be great if someone could tell me if having robots.txt benefits my website when it comes to SEO?
头像

markomako79

  • 帖子: 11
  • 注册: 周二 8月 02, 2011 6:08 pm
--
Creating a survival seeds website, and need some basic HTML help.

Re: robots.txt

HTML.net » 周五 8月 05, 2011 6:48 am

Robots.txt is a file you can upload to your web site to guide search bots (Google etc.) how to index your site.

For example, if you write in a file robot.txt:

User-agent: *
Disallow: /

no page will be indexed.

You can read more at the link above: http://www.robotstxt.org

- Andreas, HTML.net
头像

HTML.net
Site Admin

  • 帖子: 1194
  • 注册: 周六 9月 11, 2010 11:24 am
--
Show some love for HTML.net on Twitter, Facebook and Google: Use the buttons on top of all pages.

Re: robots.txt

Preeju M S » 周五 6月 09, 2017 4:22 am

Robots.txt file is a text file for restricting bots (robots, search engine crawlers ) from a website or certain pages on the website. Using a robots.txt file and with a disallow direction, we can restrict bots or search engine crawling program from websites and or from certain folders and files.
头像

Preeju M S

  • 帖子: 6
  • 注册: 周四 6月 08, 2017 7:40 am
--
Data Entry Outsourcing Company - http://www.onlinedataentryoutsourcing.com


+ 发表回复

分页: 1 / 1