# Allow all user agents to access everything User-agent: * Disallow: # Allow Facebook's crawler User-agent: facebookexternalhit Allow: / # Allow other common crawlers (optional) User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / # Disallow access to specific directories or files (optional) Disallow: /wp-admin/ Disallow: /wp-login.php