Main
16
February
2014

500 Server Error Only for Googlebot in Joomla

 

Unreachable URL from GooglebotWhile trying to set up a new landing page for our maintenance services, I ran into a problem where the page would load just fine for me and on 3rd party tools like BrowserStack, but it was returning a 500 server error for Googlebot.  It would also return a 500 server error for W3C Validation and Spider View checks.  I would have never even noticed it had Google Adwords not disapproved the page URL repeatedly.

This was completely perplexing to me and took several hours to troubleshoot.  In researching the issue, the only possible leads I found were some older forum posts mentioning both the core Joomla { loadposition } plugin for modules and the NoNumbers Modules Anywhere plugin.  I was not using the Joomla loadposition and disabling Modules Anywhere did not change the error code.  

I contacted our hosting company HostGator who found nothing from their end. In systemically disabling items, I found the culprit to be RSForms Pro by RSJoomla.  It did not matter whether our signup form was loaded as a menu item, module, or in the content plugin (like { rsform 4 }), it would give the 500 server error for Googlebot.  I have no idea why this is and would love feedback in the comments.  

To solve this, I was able to disable RSForms, and load our ZenDesk form in a RokBox instead, but I think we could have better conversions if the user could see how short the form is.  Again, I would love any alternative solutions that anyone can think of.

EDIT 2/19/2014: The problem within RSForms was a bad script in the PHP script section.  I had pasted in sample code to use the ZenDesk API and then forgotten about it. 

 

Here are some of the lessons I learned:

 

 

How to check whether Googlebot can access your site:

1. Login to Google Webmasters for that particular site.

2. From the left menu, click on CRAWL, then FETCH AS GOOGLEBOT.

3. Add the URL and hit FETCH.  

4. Google will either tell you "Success" or "Unreachable."  If unreachable, it will show you the error messages.

 

 

Make sure URL is not Blocked by Robots:

1. Login to Google Webmasters for that particular site.

2. From the left menu, clickon CRAWL, then BLOCKED URLs.

3.  Google will show you the currently loaded robots.txt file.  Below it, is a part for "URLs: Specify the URLs and user-agents to test against."  Add the URL you want to test to that textarea, then hit TEST.

4. It will show the "Test Results" below.

 

 

How to View HTTP Live Headers for a website:

1. There is a Google Chrome extension that makes this easy. Go to https://chrome.google.com/webstore/detail/live-http-headers/iaiioopjkcekapmldfgbebdclcnpgnlo to download and install. 

2. Once installed, click on the extension icon (little blue cloud), open a new tab, and then put in the URL you want to test.

 

 

How to add event tracking with Google Analytics (since I couldn't just track the "Thank you" page as a Goal URL):

see separate post

Author: Alexis Priddy Categories: Joomla Extensions

About the Author

Alexis Priddy

Alexis Priddy

Alexis moved to Waco in 2008 after five years in the United States Army as a military police officer.  Web design was a hobby that grew into a passionwhile she was in the military.  She is the founder of ArmyToolbag.com, a reference website for Army personnel that averages over 25,000 unique visitors a month.  When she sold that website to a national marketing firm, she decided to venture out and start YellowWebMonkey Web Design in 2009.  She has a Bachelor of Science degree in Psychology and Computer Science Engineering from the United States Military Academy at West Point, NY.

blog comments powered by Disqus

Sitemap

Get Started with YellowWebMonkey

Invalid Input

Invalid web address.

Invalid Input

Invalid Input

Joomla Certified Administrator

Joomla Maintenance Plans

Get started with reliable monthly care provided by a Joomla certified company.

Invalid Input

Invalid web address.

Invalid Input

Invalid Input