Uso y creación de Robot.txt: Mejora SEO y guía robots

  • Autor Autor ditable
  • Fecha de inicio Fecha de inicio
D

ditable

Zeta
Verificado por Whatsapp
Buenas a todos y todas! :encouragement:

Tengo varias dudas sobre Robot.txt. -sireve para decir a google lo que debe indexar y lo que no....mejora el seo, facilita trabajo a robots..

¿Para crear Robor.txt. primero debo crear SISTEMAP??

Este es un ejemplo: User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$


User-agent: Googlebot-Image
Disallow:
Allow: /*

Sitemap: http://www.AQUI-PONE-TU-WEB.com/sitemap.xml
Disallow: /?s=
Disallow: /search

Allow: /feed/$
Disallow: /feed


¿¿¿¿¿¿¿Para crear Robor.txt. primero debo crear SISTEMAP??? en lo codigos viene tambien sistemap

- - - Actualizado - - -

o con solo= mi web.com/sistemap.xml sobra ??
 
Puedes tener robot sin tener sitemap y viceversa. Aunque te aconsejo crear las dos herramientas.
 
Un sitemap no es necesario para paginas con pocos puntos de menú.
En el robot text pongo todito que no quiero indexado.
 
no hace falta, solo borra esa linea de codigo y listo... aunque con plugins te podes crear el sitemap en segundos.
 
frN13 dijo:
no hace falta, solo borra esa linea de codigo y listo... aunque con plugins te podes crear el sitemap en segundos.
Hacer clic para expandir...

me puede facilitar algun plugin para crear sistemap?

- - - Actualizado - - -

Esta bien este robot.txt?
HTML: 
# Disallow parsing individual post feeds, categories and trackbacks..
Disallow: /trackback/
Disallow: /feed/

User-agent: *
# disallow all files in these directories
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contacto/
Disallow: /tag/
Disallow: /trackback/
Disallow: /*?*
Disallow: */trackback/

Sitemap: http://www.bebogeek.com/sitemap.xml

User-agent: Googlebot
# disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.cgi$
Disallow: /*.wmv$
Disallow: /*.png$
Disallow: /*.gif$
Disallow: /*.jpg$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: */trackback*
Disallow: /*?*
Disallow: /z/
Disallow: /wp-*
Allow: /wp-content/uploads/


# allow google image bot to search all images
User-agent: Googlebot-Image
Allow: /*

# allow adsense bot on entire site
User-agent: Mediapartners-Google*
Disallow: /*?*
Allow: /z/
Allow: /about/
Allow: /contact/
Allow: /wp-content/
Allow: /tag/
Allow: /manual/*
Allow: /docs/*
Allow: /*.php$
Allow: /*.js$

Allow: /*.inc$
Allow: /*.css$
Allow: /*.gz$
Allow: /*.cgi$
Allow: /*.wmv$
Allow: /*.cgi$
Allow: /*.xhtml$
Allow: /*.php*
Allow: /*.gif$
Allow: /*.jpg$
Allow: /*.png$



User-agent: *
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /about/
Disallow: /contact/
Disallow: /tag/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contact
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/

User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$

Disallow: /*?*

User-agent: Googlebot-Image
Disallow:
Allow: /*

User-agent: Mediapartners-Google*
Disallow:
Allow: /*

# Disallow Google from parsing indididual post feeds and trackbacks..
Disallow: */feed/
Disallow: */trackback/
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$
Disallow: /*/*/feed
Disallow: /*/*/*/feed
Disallow: /*/*/feed/
Disallow: /*/*/*/feed/
Disallow: /*/*/*/*/feed
Disallow: /*/*/*/*/feed/

# Disallow all files with ? in url
Disallow: /*?*
Disallow: /*?

Sitemap: http://www.bebogeek.com/sitemap.xml.gz

- - - Actualizado - - -

Esta bien este robot.txt?
HTML: 
# Disallow parsing individual post feeds, categories and trackbacks..
Disallow: /trackback/
Disallow: /feed/

User-agent: *
# disallow all files in these directories
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contacto/
Disallow: /tag/
Disallow: /trackback/
Disallow: /*?*
Disallow: */trackback/

Sitemap: http://www.bebogeek.com/sitemap.xml

User-agent: Googlebot
# disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.cgi$
Disallow: /*.wmv$
Disallow: /*.png$
Disallow: /*.gif$
Disallow: /*.jpg$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: */trackback*
Disallow: /*?*
Disallow: /z/
Disallow: /wp-*
Allow: /wp-content/uploads/


# allow google image bot to search all images
User-agent: Googlebot-Image
Allow: /*

# allow adsense bot on entire site
User-agent: Mediapartners-Google*
Disallow: /*?*
Allow: /z/
Allow: /about/
Allow: /contact/
Allow: /wp-content/
Allow: /tag/
Allow: /manual/*
Allow: /docs/*
Allow: /*.php$
Allow: /*.js$

Allow: /*.inc$
Allow: /*.css$
Allow: /*.gz$
Allow: /*.cgi$
Allow: /*.wmv$
Allow: /*.cgi$
Allow: /*.xhtml$
Allow: /*.php*
Allow: /*.gif$
Allow: /*.jpg$
Allow: /*.png$



User-agent: *
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /about/
Disallow: /contact/
Disallow: /tag/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contact
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/

User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$

Disallow: /*?*

User-agent: Googlebot-Image
Disallow:
Allow: /*

User-agent: Mediapartners-Google*
Disallow:
Allow: /*

# Disallow Google from parsing indididual post feeds and trackbacks..
Disallow: */feed/
Disallow: */trackback/
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$
Disallow: /*/*/feed
Disallow: /*/*/*/feed
Disallow: /*/*/feed/
Disallow: /*/*/*/feed/
Disallow: /*/*/*/*/feed
Disallow: /*/*/*/*/feed/

# Disallow all files with ? in url
Disallow: /*?*
Disallow: /*?

Sitemap: http://www.bebogeek.com/sitemap.xml.gz
 
Hay que estar conectado o registrado para responder aquí.

Temas similares

D
Guía Robots.txt para optimizar Wordpress SEO
2 3
Respuestas
51
Visitas
7K
Alee12
Alee12
Atrás
Arriba