User-agent: * Allow: / Disallow: /WEB-INF Disallow: /META-INF Disallow: /polopolydevelopment/ Disallow: /status/ Disallow: /search-results/ Disallow: /poll/ Disallow: /logger/ Disallow: /cm/ Disallow: /cmlink/ Disallow: /error/ Disallow: /errorpages/ Disallow: /content/ Disallow: /captcha/ Disallow: /membership/ Disallow: /blogimageupload/ Disallow: /263 Disallow: /json Disallow: /webpage Disallow: /tabjson Disallow: /home Disallow: /sponsored/ Disallow: /section/ User-agent: Googlebot-News Allow: / User-agent: Googlebot-Mobile Allow: / User-agent: Googlebot-Image Allow: /image/ User-agent: Twitterbot Allow: / User-agent: ia_archiver Allow: / User-agent: proximic Disallow: / User-agent: grapeshot Disallow: / #Sitemaps Sitemap: https://www.mathrubhumi.com/sitemap.xml # http://linkfluence.net/ User-agent: Kraken Disallow: / #http://napoveda.seznam.cz/en/seznambot-intro/ User-agent: SeznamBot Disallow: / #https://awario.com/bots.html User-agent: AwarioRssBot Disallow: / User-agent: AwarioSmartBot Disallow: / User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: /