Smartertrack robots.txt blocked resources reported on Google Searchconsole

I'm busy running through a list of issues affecting one of the domains that I manage and Google Search console is unable to correctly index and render pages where we have the live chat script enabled.

Google is complaining about the following resources being blocked

ST.ashx?scriptonly=true
ChatScript.ashx

I reviewed the contents of the generated robot.txt file generated by Smartertrack and it is blocking these resources, how ever I need to override this file in order to allow the above files to be indexed by google in order to avoid being penalized

The thread below suggests placing a robots.txt file into the main directory, how ever this does not work.

portal.smartertools.com/community/a620/indexed-by-google-help.aspx

Any suggestions on how I can go about editing or replacing the robots.txt file?

Copy of existing robots.txt file

Sitemap: //*********************:443/Sitemap.ashx
User-agent: *
Disallow: /About/
Disallow: /App_Data/
Disallow: /App_Themes/
Disallow: /bin/
Disallow: /CaptchaImageHandler.ashx
Disallow: /ChatLink.ashx
Disallow: /ChatScript.ashx
Disallow: /Common/
Disallow: /frmError.aspx
Disallow: /GettingStarted.html
Disallow: /HttpHandlers/
Disallow: /Login.aspx
Disallow: /m/
Disallow: /Main/frmMySettings.aspx
Disallow: /Main/frmTicket.aspx
Disallow: /Main/frmVerifyAccount.aspx
Disallow: /Main/Login.aspx
Disallow: /Management/
Disallow: /Mobile/
Disallow: /Ping.aspx
Disallow: /ScriptResource.axd
Disallow: /Services/
Disallow: /Services2/
Disallow: /ST.ashx
Disallow: /WebResource.axd
Disallow: /WhosOnLink.ashx

Jade

https://absolutehosting.co.za

Richard Watson Replied

12/9/2016 at 5:23 PM

Apparently, they aren't talking.

Richard Watson Replied

12/9/2016 at 5:32 PM

In the web.config is a couple of lines with robots.txt telling the software to use code to answer the request. Make a backup of your web.config file and then remove those lines two lines. IIS will then respond as normal and you can put a robots.txt of your choosing in there. For instance, the ChatLink is pretty required on our site, but has cost us a lot of money since Google thinks we don't want them searching the site because of it.

2 Replies

Reply to Thread

Related Knowledge Base Articles

2 Replies

Reply to Thread

Tags

Related Knowledge Base Articles