# Allow all search engines and social media crawlers User-agent: * Allow: / # Explicitly allow social media crawlers (in case of CDN/firewall rules) User-agent: facebookexternalhit Allow: / User-agent: Twitterbot Allow: / User-agent: LinkedInBot Allow: / User-agent: WhatsApp Allow: / User-agent: TelegramBot Allow: / User-agent: Pinterest Allow: / User-agent: Slackbot Allow: /