Socialize Me: Oh, SSL, why are you so hard ?

Tuesday, October 14, 2008

Oh, SSL, why are you so hard ?

This is a summary email that was sent to me by a colleague after fixing a SSL issue he was having in his deployment. I'm capturing it here so I can refer to it later, and in case it helps you as well.

This is a different certificate problem than the one you folks encountered back in May of this year. That one produced an "Internal Server Error" in the browser when attempting a redirected login and had a corresponding "GSK_ERROR_BAD_CERT(gsk rc = 414)" message in the log. That was a case of a new certificate that didn't have a trusted signer. This particular problem resulted from a certificate that had a trusted signer, but recently expired and produced different symptoms.

Symptoms:

When passing from a logged-in user's Portal page to a Connections page (ie. Portal kicking off a new browser window to open, say, Activites using SSO), the notorious "Page can not be displayed" message appeared.

In the log, C:\IBM\HTTPServer\logs\error.log ,you see this error message:
"SSL0221E: SSL Handshake Failed, Invalid date."

In hindsight, this one's easy to figure out but it took an inexperienced person like me a while to track and confirm it. It means there is an expired certificate in use ... dates in a certificate are no longer valid.

Cause: Expired certificates

Using the key management utility (IKEYMAN), open the Key Database file C:\IBM\HTTPServer\bin\key.kdb and view the default Personal Certificate called "demoibm". It expired on June 26, 2008. Similarly when you look at Signer Certificates and open up "WebSphere Extracted Cert", you will find that also expired on June 25, 2008.

Solution

1.) Since the one and only certificate being used in the key database was now expired, it seemed to me that generating new certificates and continuing to manage an additional KDB was pointless, so I chose to use the already valid keys in this key database already in use by the web server:

C:\IBM\HTTPServer\Plugins\config\webserver1\plugin-key.kdb .

I redirected to that plugin-key.kdb by changing C:\IBM\HTTPServer\conf\httpd.conf as follows:

#Enable SSL for Lotus Connections
LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
LoadModule rewrite_module modules/mod_rewrite.so
#KeyFile "C:\IBM\HTTPServer\bin\key.kdb"<---- this was the old reference that I commented out
Keyfile "C:\IBM\HTTPServer\Plugins\config\webserver1\plugin-key.kdb"
SSLStashFile "C:\IBM\HTTPServer\Plugins\config\webserver1\plugin-key.sth"
Listen 0.0.0.0:443
<VirtualHost *:443>
ServerName connections.demoibm.com
SSLEnable
SSLProxyEngine on
RewriteEngine on
# Keep website (/html) on http for perf
RewriteRule ^/dogear/html(.*) http://connections.demoibm.com/dogear/html$1 [noescape,L,R]
#Put everything but login and api back to http
RewriteCond %{REQUEST_URI} !^/blogs/services/atom(.*)
RewriteCond %{REQUEST_URI} !^/blogs/roller-ui/login.do
RewriteCond %{REQUEST_URI} !^/blogs/j_security_check
RewriteRule ^/blogs/(.*) http://connections.demoibm.com/blogs/$1 [noescape,L,R]
</VirtualHost>
SSLDisable

2.) Along with changing the HTTPD.CONF file, I did the certificate insertions into the signer certificates of the CMSKeyStore as described in the technote from that other GSK problem: GSK_ERROR_BAD_CERT error configuring SSL between Plug-in and Application Server V6.1 I simply exported all three "default_x" personal certificates then imported (added) them to the signer certificates of the CMSKeyStore. (Before executing the instructions in that technote, see the note below about disabling security so you can use the WAS Integrated Solutions Console.)

2a.) Important note: Because authentication does not work due to the expired certificate, you can not successfully log into the WAS Integrated Solutions Console until you manually disable security. To do that, modify this file C:\IBM\WebSphere\AppServer\profiles\AppSrv01\config\cells\connectionsNode01Cell\security.xml. Look for the following text at about the fifth or so line down and change "enabled" to be "false" as shown in blue below.

useLocalSecurityServer="true" useDomainQualifiedUserNames="false" enabled="false" cacheTimeout="600"

After saving and closing security.xml, you will need to restart the Connections server. Upon return, start the Integrated Solutions Console with the URL "https://connections.demoibm.com:9043/ibm/console/logon.jsp" and just click the Log in button without entering user credentials.

3.) Upon completion of the work needed to be done in the Integrated Solutions Console, don't forget to go back and modify the security.xml and set "enabled=true" and restart the Connections server so security is properly invoked.

Final Note: Aside from the ISC admin console, directly logging into the Connections sever did not seem to work. Redirecting to it from a logged-in Portal page as described in the "Symptom" section above gets you to Connections authenticated via SSO. This behavior was observed even before the certificate expiration problem arose. Going through this exercise only restores it to it's pre-expiration condition which means you still can not directly login.

Lastly, I know much of this is in different pieces of the doc but I personally found the info in this posting in the Lotus Connections Forum to be useful as a newbie.