Smartertrack robots.txt blocked resources reported on Google Searchconsole
Problem reported by Jade Benson - March 8, 2016 at 2:50 AM
Submitted
I'm busy running through a list of issues affecting one of the domains that I manage and Google Search console is unable to correctly index and render pages where we have the live chat script enabled.
Google is complaining about the following resources being blocked
  • ST.ashx?scriptonly=true
  • ChatScript.ashx
I reviewed the contents of the generated robot.txt file generated by Smartertrack and it is blocking these resources, how ever I need to override this file in order to allow the above files to be indexed by google in order to avoid being penalized 
 
The thread below suggests placing a robots.txt file into the main directory, how ever this does not work.
portal.smartertools.com/community/a620/indexed-by-google-help.aspx
Any suggestions on how I can go about editing or replacing the robots.txt file?
 
 
 
Copy of existing robots.txt file
Sitemap: //*********************:443/Sitemap.ashx
User-agent: *
Disallow: /About/
Disallow: /App_Data/
Disallow: /App_Themes/
Disallow: /bin/
Disallow: /CaptchaImageHandler.ashx
Disallow: /ChatLink.ashx
Disallow: /ChatScript.ashx
Disallow: /Common/
Disallow: /frmError.aspx
Disallow: /GettingStarted.html
Disallow: /HttpHandlers/
Disallow: /Login.aspx
Disallow: /m/
Disallow: /Main/frmMySettings.aspx
Disallow: /Main/frmTicket.aspx
Disallow: /Main/frmVerifyAccount.aspx
Disallow: /Main/Login.aspx
Disallow: /Management/
Disallow: /Mobile/
Disallow: /Ping.aspx
Disallow: /ScriptResource.axd
Disallow: /Services/
Disallow: /Services2/
Disallow: /ST.ashx
Disallow: /WebResource.axd
Disallow: /WhosOnLink.ashx
 

1 Reply

Reply to Thread
0
In the web.config is a couple of lines with robots.txt telling the software to use code to answer the request. Make a backup of your web.config file and then remove those lines two lines. IIS will then respond as normal and you can put a robots.txt of your choosing in there. For instance, the ChatLink is pretty required on our site, but has cost us a lot of money since Google thinks we don't want them searching the site because of it.

Reply to Thread