oracle.security.fed.event.EventException: Could not find the AuthnRequest associated to the Assertion

In this scenario, we have two OIF 11.1.1.7.0 servers set up on serverA acting as an IdP and serverB acting as an SP.

Start federation by going to the SP as follows:
http://10.10.107.100:7777/fed/user/testspsso

On the Initiate Federation SSO page, click Start SSO.

This directs the page to the IdP at:
http://serverA:7777/fed/idp/samlv20

On the login screen, enter a username and password that exists in the IdP:

Click Sign In.  The following error is shown on the web page:

Error 500--Internal Server Error

From RFC 2068 Hypertext Transfer Protocol -- HTTP/1.1: 10.5.1 500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request.

And on the server hosting the OIF SP services, the wls_oif1-diagnostic.log shows the following:

[2014-09-24T15:48:57.688-04:00] [wls_oif1] [ERROR] [FED-15011] [oracle.security.fed.eventhandler.profiles.sp.sso.assertion.Saml20AssertionProcessor] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 0050xJtWyGm5Yb^pxSt1iW0000jy0000E2,0:1] [APP: OIF#11.1.1.2.0] [URI: /fed/sp/authnResponse20] Cannot find the authentication request associated with the assertion.
[2014-09-24T15:48:57.692-04:00] [wls_oif1] [ERROR] [FED-12064] [oracle.security.fed.controller.ActionStateMachine] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 0050xJtWyGm5Yb^pxSt1iW0000jy0000E2,0:1] [APP: OIF#11.1.1.2.0] [URI: /fed/sp/authnResponse20] Exception: {0}[[
oracle.security.fed.event.EventException: Could not find the AuthnRequest associated to the Assertion
        at oracle.security.fed.eventhandler.profiles.sp.sso.assertion.Saml20AssertionProcessor.checkSubjectConfirmation(Saml20AssertionProcessor.java:1178)
        at oracle.security.fed.eventhandler.profiles.sp.sso.assertion.Saml20AssertionProcessor.processAssertion(Saml20AssertionProcessor.java:333)
        at oracle.security.fed.eventhandler.profiles.sp.sso.v20.ProcessResponseEventHandler.perform(ProcessResponseEventHandler.java:233)
        at oracle.security.fed.controller.ActionStateMachine.processEvent(ActionStateMachine.java:141)
        at oracle.security.fed.controller.EventControllerImpl.processEvent(EventControllerImpl.java:118)
        at oracle.security.fed.controller.ApplicationController.publishEvent(ApplicationController.java:425)
        at oracle.security.fed.controller.web.action.RequestHandlerContext.publishEvent(RequestHandlerContext.java:48)
        at oracle.security.fed.controller.web.action.RequestHandlerSupport.perform(RequestHandlerSupport.java:15)
        at oracle.security.fed.controller.ApplicationController.processServletRequest(ApplicationController.java:321)
        at oracle.security.fed.controller.web.servlet.FederationServlet.doGet(FederationServlet.java:151)
        at oracle.security.fed.controller.web.servlet.FederationServlet.doPost(FederationServlet.java:98)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
        at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
        at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:119)
        at java.security.AccessController.doPrivileged(Native Method)
        at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:324)
        at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:460)
        at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:103)
        at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:171)
        at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:71)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
        at oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:163)
        at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3730)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3696)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
        at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2273)
        at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2179)
        at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1490)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

]]

The main error being "oracle.security.fed.event.EventException: Could not find the AuthnRequest associated to the Assertion".  This has to with the how the JSESSIONID is set by the IdP and transferred to the SP.  According to Oracle Support Doc ID: 1067769.1 amongst other things, make sure that all access to the OIF use FQDN.

Once the request http://10.10.107.100:7777/fed/user/testspsso was changed to http://serverB:7777/fed/user/testpsso, everything worked as shown below:

Configuration settings are unavailable because OIF(11.1.1.2.0) is Down

From EM Fusion Middleware Control, select Identity and Access, then select OIF(11.1.1.2.0).

Left click on OIF(11.1.1.2.0) and select Administration and Server Properties.

The following error is displayed:
Configuration settings are unavailable because /Farm_IDMDomain_SP/IDMDomain_SP/wls_oif1/OIF(11.1.1.2.0) is down.

To fix this, click Farm on top left corner.  From the drop down select Monitoring Credentials.  Enter the weblogic username and password and click OK.

Stop the EMAGENT

cd $MW_HOME/asinst_1/bin

opmnctl stopproc ias-component=EMAGENT
opmnctl sartproc ias-component=EMAGENT

Restart the wls_oif1 server from Weblogic or use WLST.

Click Farm on top left corner.  From the drop down, select Monitoring Credentials.  The warning message will not be displayed and you should be able to access the resource.

MBean operation access denied. MBean: com.oracle.igf:type=Xml,name=IDSConfig Operation: listAllIdentityDirectoryService() Detail: Access denied. Required roles: Admin, Operator, Monitor, executing subject: principals=[weblogic, oimAdminGroup, OAMSystemAdminGroup]

Environment:
RHEL 5u6 x64
OAM 11gR2
OUD 11gR2
Weblogic 10.3.6.0

After migrating the default and system store to external LDAP, in this case OUD, the following error is reported from OAM console Launch Pad -> Configuration -> User Identity Stores: 

To resolve, add the Administrators to the cn=systemids and make weblogic a member of Administrators.  If using a GUI tool such as Apache Directory Studio do the following:

1. Expand the cn=systemids node
2. Select an entry under cn=systemids and right-click
3. Click New -> New Entry
4. On the Entry Creation Method screen, check "Use existing entry as template" and click Next
5. On the Object Classes screen, click Next without making any changes
6. On the Distinguished Name screen, change the RDN cn = <original_value> to RDN cn = Administrators and click Next (the DN preview box on the bottom should something like: cn=Administrators,cn=systemids,dc=acme,dc=com)
7. On the Attributes screen, click Finish
8. Log out and log back in to OAM console and access User Identity Stores.  The error will not appear.

The LDAP commands are as follows:
dn: cn=Administrators,cn=systemids,dc=acme,dc=com
changetype: add
objectClass: groupOfUniqueNames
objectClass: top
description: OIM administrator role
uniqueMember: cn=oimAdminUser,cn=systemids,dc=acme,dc=com
uniqueMember: cn=weblogic,cn=systemids,dc=acme,dc=com
uniqueMember: cn=xelsysadm,cn=systemids,dc=acme,dc=com
cn: Administrators

These can be saved to a file e.g. add_group.ldif and run as follows:

ldapadd -x -p 1389 -h localhost -D "cn=Directory Manager" -w password -f add_group1.ldif

X509 Authentication Using Oracle Access Manager (OAM) 11gR2PS2 and Oracle Unified Directory (OUD)

Assumptions

It is assumed the following products have been installed:
Oracle RDBMS 11gR2 - holding metadata 
Oracle Unified Directory 11.1.2.2.0 - LDAP for system and user store
Oracle Access Manager 11gR2PS2
Weblogic 10.3.6
Oracle OHS (which if I recall is installed with Weblogic)
Webgate 11g installed and configured


My environment consists of three RHEL 5.10 x64 VMs consisting of svrtoes01, svrtst02, and svrtst03 as follows:
Oracle RDBMS 11gR2 on server svrtoes01

Components installed for supporting the FMW (fusion middleware) stack are:

COMP_ID              OWNER                VERSION           MODIFIED   U
-------------------- -------------------- ----------------- ---------- -
APM                  SYSMAN_APM           11.1.1.3.0        2014-03-05 N
MDS                  SYSMAN_MDS           11.1.1.6.1        2014-03-05 N
OPSS                 SYSMAN_OPSS          11.1.1.6.0        2014-03-05 N
IAU                  DEV_IAU              11.1.1.7.0        2014-04-30 N
MDS                  DEV_MDS              11.1.1.7.0        2014-04-30 N
OAM                  DEV_OAM              11.1.2.2.0        2014-04-30 N
OID                  ODS                  11.1.1.7.0        2014-04-30 N
OPSS                 DEV_OPSS             11.1.1.7.2        2014-04-30 Y
OIM                  DEV_OIM              11.1.2.2.0        2014-05-08 N
ORASDPM              DEV_ORASDPM          11.1.1.7.0        2014-05-08 N
SOAINFRA             DEV_SOAINFRA         11.1.1.7.0        2014-05-08 N

See this link on what is installed on svrtst02 and svrtst03:
http://anotherdatabaseblog.blogspot.com/2014/05/oracle-access-manager-oam-111220-and.html

There are many resources on the web that show detailed instructions on how to install various components of the Oracle FMW stack.  I have found these to be very useful:
http://www.iamidm.com
http://fusionsecurity.blogspot.com
http://www.ateam-oracle.com
http://onlineappsdba.com

An 11g webgate was installed:
http://anotherdatabaseblog.blogspot.com/2014/05/create-and-deploy-11g-webgate.html

I used this link http://www.iamidm.com/2012/10/oam-11g-r2-lab4-protecting-secure-url.html to help me in setting up and testing access to a secure web page.  This link shows you how to setup a webpage hosted on OHS and protected by OAM.  I just extend the example to include X509 authentication.

However, in order to protect the resource and only allow access via X509 certificates, some changes are required in OAM.  To be clear, this exercise shows how to do X509 authentication (AUTHN) to allow access to a resource, in this case a simple web page.

It is assumed that Weblogic and OAM have been configured for SSL and that you have access to certificates issued by a CA (http://fusionsecurity.blogspot.com/2011/02/certificate-x509-authentication-in-oam.html).

Add Root and Intermediate Certificates to .oamkeystore

The oamkeystore is Access Managers keystore and is located in $MW_HOME/user_projects/domains/WLSDomain/config/fmwconfig/.

The root and intermediate (if any) certificates need to be installed for X509 authN to work.

First, get the password for the keystore as follows:

cd $MW_HOME/Oracle_IDM1/common/bin

./wlst.sh
connect()

domainRuntime()

listCred(map="OAM_STORE",key="jks")

Make a note of the password and then exit scripting tool

exit()

Install the root and intermediate certificates.  In my setup, I have my certs located in the /certs directory.  The root certificate is in a file called ISEDlabRoot.crt and the intermediate certificate is in a file called CADCA1.crt.  I need to change to the location where my certs are installed:

cd /certs

Then run the command to import the certs into the keystore:

keytool -importcert -alias ISEDlabRoot -file ISEDlabRoot.crt \
-keystore $MW_HOME/user_projects/domains/WLSDomain/config/fmwconfig/.oamkeystore \
-storepass oa6fgome4lsnf9c6ntoio1qc5p -storetype jceks

Answer 'yes' when prompted whether to trust this certificate

Successful import will respond with: Certificate was added to keystore

Import the intermediate cert into the keystore:

keytool -importcert -alias CADCA1 -file CADCA1.crt \
-keystore /opt/oracle/middleware/user_projects/domains/WLSDomain/config/fmwconfig/.oamkeystore \
-storepass oa6fgome4lsnf9c6ntoio1qc5p -storetype jceks

Successful import will respond with: Certificate was added to keystore

Configure Web Pages

I have two web pages:
secure.html
<i><html></html></i>
<i><body></body></i>
<i><a href="http://10.10.107.89:7777/logout2.html?end_url=">Logout</a></i>
<i></i>
<center>
<i> <h1>
 This is Secure Page, Only logged in users can view </h1></i></center><i> </i>
<i></i></div><i></i>
<i>
</i> Name it as <b>sample.html</b>
<b>
</b>

logout2.html
<i><html></html></i>
<i><body></body></i>
<i><b><span style="color: red;"><a href="http://10.10.107.89:7777/logout2.html?end_url=">Logout</a></span></b></i>
<i></i>
<center>
<i> <h1>
 This is Secure Page, Only logged in users can view </h1></i></center><i> </i>
<i></i>
<i></i>


These are installed $MW_HOME/Oracle_WT1/instances/instance1/config/OHS/ohs1/htdocs.  Note that end_url= is left blank when configured for X509 authN.

Configure Oracle Access Management

Login to the Oracle Access Management console
From Launch Pad -> Access Manager -> Application Domains
Click Search

Select RREG_OAM11G then click the edit icon

Click on Resources tab, then click Create

Select HTTP from the Type drop down

Host Identifier can be searched for using the search icon.  Enter RREG_HostId11G in the Host Identifier field

Enter /secure.html in Resource URL.  This is the resource that is going to be protected

Select Protected for Protection Level

Click Apply to continue

Close the RREG_OAM11G: RREG_HostID... tab

Click on the Autentication Policies tab then click on the Create Authentication Policy button

Enter a name for the policy, e.g X509_Test

Select X509Scheme for Authentication Scheme

The new policy will be displayed

Click on the Resources tab and the newly added resource that needs to be protected (secure.html) will be displayed.  Note that the new resource is not attached to any policy yet

Next, attach the resource to the newly created authentication policy.  Select the resource (secure.html) in the grid and click the Edit button.

From the Authentication Policy drop down, select X509_Test and click Apply

Close the RREG_OAM11G:RREG_HostId... tab

In the RREG_OAM11G tab, click the Search button which will show that secure.html is protected by the new X509_Test authentication scheme

Restart the OAM server for the changes to take effect

Now try and access the protected resource, secure.html in this case which hosted on http://10.10.107.89:7777/secure.html.  The HTTP request will be intercepted by the 11g Webgate and routed to OAM which will prompt you to present a client certificate:

The protected web page will be displayed:

Click the Logout link and you will be logged out and all session cookies will be removed:

You can view the Oracle Unified Directory (OUD) access_log located $OUD_HOME/logs and see the connection to the LDAP being made the the user CN being verified:

[21/May/2014:15:50:04 -0400] SEARCH REQ conn=29429 op=15 msgID=16 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=inetOrgPerson)(cn=weblogic))" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[21/May/2014:15:50:04 -0400] SEARCH RES conn=29429 op=15 msgID=16 result=0 nentries=1 etime=2
[21/May/2014:15:50:04 -0400] SEARCH REQ conn=29430 op=4 msgID=5 base="cn=weblogic,cn=systemids,dc=acme,dc=com" scope=base filter="(objectclass=inetOrgPerson)" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[21/May/2014:15:50:04 -0400] SEARCH RES conn=29430 op=4 msgID=5 result=0 nentries=1 etime=2

Also, in the oam_server1.out log located $MW_HOME/user_projects/domains/WLSDomain//servers/oam_server1/logs, you can view the SSL key negotiation and exchange in progress and verification of the certificates.  This is only true if SSL debugging is turned on.

And in $MW_HOME/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1 the access_log will show the initial connection being made as the user requests the resource.

Messing Around with C and Pointer Stuff

/*
An exercise in getting to grips with pointers and dynamic memory allocation. 

This program checks to see the largest chunk of memory that can be allocated on a RHEL system with 4GB memory.

Used some made up algorithm which is probably not very efficient but the purpose of this exercise was not to write efficient algorithms but just for my edification.
*/

#include <stdio.h>
#include <stdlib.h>

long lb = 0;
long ub = 4140949504;
const int O_SUCCESS = 0;
const int O_FAIL = 1;

void fx(long *, long *);
void range_calc(long *, long *, long *, int);

int main(void)
{
   char text[500];
   long sz, arr=10;
   fx(&lb, &ub);
   return 0;
}

void fx(long *p_lb, long *p_ub)
{
   long *array;
   long ctr = 0;
   int status = O_SUCCESS;
   long crnt_lb, crnt_ub, prev_lb, prev_ub;
   prev_lb = *p_lb; prev_ub = *p_ub;
   long mem_sz_to_test = prev_ub;

   while (1)
   {
      array = (long *)malloc(mem_sz_to_test * sizeof(long));

      if (array == 0)
      {
         printf("ERROR: While trying to allocate %lu bytes\n", mem_sz_to_test);
         // Split the range in half, test the lower limit
         crnt_ub = mem_sz_to_test;
         range_calc(&crnt_ub, &prev_lb, &crnt_lb, O_FAIL);
         mem_sz_to_test = crnt_lb;
         printf("1.\tlb = %lu, ub = %lu\n", crnt_lb, crnt_ub);
         printf("mem_sz_to_test = %lu\n", mem_sz_to_test);
      }
      else
      {
         printf("Allocated = %lu bytes\n", mem_sz_to_test * sizeof(long));
         free(array);
         prev_lb = crnt_lb;
         // Split the range in half, test the lower limit
         range_calc(&crnt_ub, &prev_lb, &crnt_lb, O_SUCCESS);
         printf("2.\tlb = %lu, ub = %lu\n", crnt_lb, crnt_ub);
         mem_sz_to_test = crnt_lb;
      }
      printf("ctr = %d\n\n", ctr++);
      if (ctr == 2500 || crnt_lb == crnt_ub) break;
   }
}


// Do the range split in half routine here
void range_calc(long *p_crnt_ub, long *p_prev_lb, long *p_crnt_lb, int STATUS)
{

   if (O_FAIL)
   {
      *p_crnt_lb = (*p_crnt_ub - *p_prev_lb)/2 + *p_prev_lb;
      if (*p_crnt_lb % 2 != 0)
         *p_crnt_lb = *p_crnt_lb + 1;
   }

   if (O_SUCCESS)
   {
      *p_crnt_lb = (*p_crnt_ub - *p_crnt_lb)/2 + *p_crnt_lb;
      if (*p_crnt_lb % 2 != 0)
         *p_crnt_lb = *p_crnt_lb + 1;
   }
}

Create and Deploy an 11g Webgate

Deploy Webgate

Note that this is an 11gR2PS2 environment running Weblogic 10.3.6 and Oracle Access Manager 11gR2PS2.

cd $MW_HOME/Oracle_OAMWebGate1/webgate/ohs/tools/deployWebGate

Run the following command (required for copying agent bits from the Webgate_Home directory to Webgate Instance location):
./deployWebGateInstance.sh -w $MW_HOME/Oracle_WT1/instances/instance1/config/OHS/ohs1 -oh $MW_HOME/Oracle_OAMWebGate1

Set LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MW_HOME/Oracle_WT1/lib

cd ../setup/InstallTools/

and run this command...
./EditHttpConf -w <Webgate_Instance_Directory> [-oh <Webgate_Oracle_Home>] [-o <output_file>]

where Webgate_Instance_Directory is the instance directory for ohs1
and Webgate_Oracle_Home is the home directory for the webgate

./EditHttpConf -w $MW_HOME/Oracle_WT1/instances/instance1/config/OHS/ohs1 -oh $MW_HOME/Oracle_OAMWebGate1 -o Edithttpconf.log

Sample output is shown below:
The web server configuration file was successfully updated
/opt/oracle/middleware/Oracle_WT1/instances/instance1/config/OHS/ohs1/httpd.conf has been backed up as
/opt/oracle/middleware/Oracle_WT1/instances/instance1/config/OHS/ohs1/httpd.conf.ORIG

Create Webgate

Oracle Access Management -> Launch Pad -> SSO Agents -> Create 11g Webgate
Choose a name, for example RREG_OAM11G and click Apply

In Logout Target URL, type end_url
In Logout URL, type
/logout1.html
/logout2.html

Click Apply again

Artifacts will be created in the following directory:
/opt/oracle/middleware/user_projects/domains/WLSDomain/output/OAM11gWebGate

Backup the OAM11GRequest.xml file
cd $MW_HOME/Oracle_IDM1/oam/server/rreg/input

cp OAM11GRequest.xml NewOAM11GRequest.xml

Edit NewOAM11GRequest.xml and add the correct values for serverAddress and agentBaseUrl
<serverAddress>http://localhost:7001</serverAddress>
<agentBaseUrl>http://localhost:7001</agentBaseUrl>

Start the process to complete the agent registration
cd ..
You should now be in the $MW_HOME/Oracle_IDM1/oam/server/rreg/bin directory.  Run the following command (with output shown):
./bin/oamreg.sh inband input/NewOAM11GRequest.xml

----------------------------------------
Request summary:                                                               
OAM11G Agent Name:RREG_OAM11G                                                  
Base URL:http://localhost:7001                                                 
URL String:RREG_HostId11G                                                      
Registering in Mode:inband                                                     
Your registration request is being sent to the Admin server at: http://localhost:7001
----------------------------------------  

Now copy the artifacts as follows:
cp $MW_HOME/Oracle_IDM1/oam/server/rreg/output/RREG_OAM11G/cwallet.sso $MW_HOME/Oracle_WT1/instances/instance1/config/OHS/ohs1/webgate/config

cp $MW_HOME/Oracle_IDM1/oam/server/rreg/output/RREG_OAM11G/ObAccessClient.xml $MW_HOME/Oracle_WT1/instances/instance1/config/OHS/ohs1/webgate/config

Restart OHS
cd $MW_HOME/Oracle_WT1/instances/instance1/bin
./opmnctl stopall
./opmnctl startall

...and check if domain got created in OAM as follows:
Launch Pad -> Access Manager -> Applications Domains
Click Search

Enable SSL Debugging for Oracle Access Manager 11gR2

For debugging SSL connections terminating on the Weblogic Server, from Weblogic Administration Console, click on Servers, select oam_server1:

Click on the Server Start tab and add -Dssl.debug=true -Dweblogic.StdoutDebugEnabled=true in the Arguments section:

Restart oam_server1.  SSL-debug information will be written to the oam_server1.log (located in $MW_HOME/domains/WLSDomain/servers/oam_server1/logs):

...
...
...
####<May 16, 2014 2:30:25 PM EDT> <Debug> <SecuritySSL> <svrtst02.isedlab.org> <oam_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <114848da72d7dcfc:-3a0c05c9:14605a3f20e:-8000-0000000000001794> <1400265025295> <BEA-000000> <Validating certificate 1 in the chain: Serial number: 207182277780947434404477757924094648847
Issuer:C=US, O=U.S. Government, OU=DoD, OU=NRO, CN=ISED lab Root
Subject:C=US, O=U.S. Government, OU=DoD, OU=NRO, OU=CA, CN=CAD CA 1
Not Valid Before:Thu Jun 14 10:00:16 EDT 2012
Not Valid After:Sun Jun 14 10:00:16 EDT 2015
Signature Algorithm:SHA1withRSA
>
####<May 16, 2014 2:30:25 PM EDT> <Debug> <SecuritySSL> <svrtst02.isedlab.org> <oam_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <114848da72d7dcfc:-3a0c05c9:14605a3f20e:-8000-0000000000001794> <1400265025295> <BEA-000000> <validationCallback: validateErr = 0>
####<May 16, 2014 2:30:25 PM EDT> <Debug> <SecuritySSL> <svrtst02.isedlab.org> <oam_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <114848da72d7dcfc:-3a0c05c9:14605a3f20e:-8000-0000000000001794> <1400265025296> <BEA-000000> <  cert[0] = Serial number: 85355980927748066409252166003794705697
Issuer:C=US, O=U.S. Government, OU=DoD, OU=NRO, OU=CA, CN=CAD CA 1
Subject:C=US, O=U.S. Government, OU=DoD, OU=NRO, CN=weblogic
Not Valid Before:Fri May 09 16:24:56 EDT 2014
Not Valid After:Sun Jun 14 09:59:16 EDT 2015
Signature Algorithm:SHA1withRSA
>
####<May 16, 2014 2:30:25 PM EDT> <Debug> <SecuritySSL> <svrtst02.isedlab.org> <oam_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <114848da72d7dcfc:-3a0c05c9:14605a3f20e:-8000-0000000000001794> <1400265025296> <BEA-000000> <  cert[1] = Serial number: 207182277780947434404477757924094648847
Issuer:C=US, O=U.S. Government, OU=DoD, OU=NRO, CN=ISED lab Root
Subject:C=US, O=U.S. Government, OU=DoD, OU=NRO, OU=CA, CN=CAD CA 1
Not Valid Before:Thu Jun 14 10:00:16 EDT 2012
Not Valid After:Sun Jun 14 10:00:16 EDT 2015
Signature Algorithm:SHA1withRSA
>
...
...
...

When using Firefox and passing your SSL certificate to the server, if the "Remember this decision" is checked, Firefox will not prompt you again for the cert:

While testing, it's good to reset this behavior so that you will be always prompted to select a certificate to present to the server.  To do this, from Firefox Tools -> Options -> Privacy -> clear your recent history:

Check Active Logins and un-check everything else and click Clear Now.  This will prompt for a client cert to be selected.

Oracle Access Manager (OAM) 11.1.2.2.0 and Oracle Unified Directory (OUD) 11.1.2.2.0 Integration

Oracle Access Manager (OAM) 11.1.2.2.0 and Oracle Unified Directory (OUD) 11.1.2.2.0 and WebLogic 10.3.6 Integration

OAM and WebLogic installed on svrtst02
OUD and WebLogic installed on svrtst03

Assumptions:  WebLogic 10.3.6 domain has been created on both svrtst02 and svrtst03 and extended for OAM.

Good idea to backup the configuration or server home before proceeding.

Install OUD on svrtst03 in $MW_HOME.

So, svrtst02 has the following installed:
oracle_common
Oracle_IDM1
Oracle_OAMWebGate1
Oracle_WT1
coherence_3.7
user_projects
wlserver_10.3

And, svrtst03 has the following installed:
oracle_common
Oracle_OUD1
Oracle_WT
user_projects
wlserver_10.3

Run oud-setup on svrtst03 either in GUI mode or CLI.  End result being that the following command is executed to create an LDAP directory store:

./oud-setup \
          --cli \
          --baseDN dc=acme,dc=com \
          --addBaseEntry \
          --ldapPort 1389 \
          --adminConnectorPort 4444 \
          --rootUserDN cn=Directory\ Manager \
          --rootUserPasswordFile ****** \
          --doNotStart \
          --ldapsPort 1636 \
          --useJavaKeystore /certs/svrtst03.isedlab.org.jks \
          --keyStorePasswordFile ****** \
          --certNickname svrtst03.isedlab.org\ u.s.\ government\ id \
          --serverTuning autotune \
          --importTuning autotune \
          --no-prompt \
          --noPropertiesFile

The LDAP directory instance is created in $MW_HOME/asinst_1/OUD.
export OUD_HOME=$MW_HOME/asinst_1/OUD

Start the instance as follows:
cd $OUD_HOME/bin
./start-ds

The default listening port is 1389, the SSL port is on 1636 and the management port is on 4444.

The instance can be managed from here:
http://svrtst03:7001/odsm

Create a scripts directory in $OUD_HOME/scripts

Quick check to make sure that everything is up and running, from the CLI:
cd $OUD_HOME/bin
./ldapsearch -h localhost -p 1389 -D "cn=directory manager" -w passw0rd -b "dc=acme,dc=com" "(objectclass=*)"

dn: dc=acme,dc=com
dc: acme
objectClass: domain
objectClass: top


Configuring OUD for OAM

The following links are useful:
http://docs.oracle.com/cd/E27559_01/install.1112/e27301/preconfigoud.htm

http://uberether.com/2012/configuring-oracle-unified-directory-as-an-identity-store-for-access-manager-11gr2-11-1-2/

From the Oracle documentation:
"Before you can use your LDAP directory as an Identity store, you must preconfigure it. The procedure in this section enables you to preconfigure Oracle Unified Directory (OUD) for using Oracle Unified Directory (OUD) as your LDAP Identity store."

Create the following file in $MW_HOME/asinst_1/OUD/scripts:
OUDContainers.ldif

dn:cn=oracleAccounts,dc=acme,dc=com
cn:oracleAccounts
objectClass:top
objectClass:orclContainer

dn:cn=Users,cn=oracleAccounts,dc=acme,dc=com
cn:Users
objectClass:top
objectClass:orclContainer

dn:cn=Groups,cn=oracleAccounts,dc=acme,dc=com
cn:Groups
objectClass:top
objectClass:orclContainer

dn:cn=Reserve,cn=oracleAccounts,dc=acme,dc=com
cn:Reserve
objectClass:top
objectClass:orclContainer

Next, import the OUD server:
cd $OUD_HOME/bin
./stop-ds
./import-ldif --backendID userRoot --append --ldifFile $OUD_HOME/scripts/OUDContainers.ldif
./start-ds 

Configure OIM proxy users and acis to communicate with OUD after installing OUD. Create the OIM Admin User, Group and the ACIs.

vi $OUD_HOME/scripts/oudadmin.ldif

dn: cn=systemids,dc=acme,dc=com
changetype: add
objectclass: orclContainer
objectclass: top
cn: systemids

dn: cn=oimAdminUser,cn=systemids,dc=acme,dc=com
changetype: add
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetorgperson
mail: oimAdminUser
givenname: oimAdminUser
sn: oimAdminUser
cn: oimAdminUser
uid: oimAdminUser
userPassword: passw0rd

dn: cn=oimAdminGroup,cn=systemids,dc=acme,dc=com
changetype: add
objectclass: groupOfUniqueNames
objectclass: top
cn: oimAdminGroup
description: OIM administrator role
uniquemember: cn=oimAdminUser,cn=systemids,dc=acme,dc=com

dn: cn=oracleAccounts,dc=acme,dc=com
changetype: modify
add: aci
aci: (target = "ldap:///cn=oracleAccounts,dc=acme,dc=com")(targetattr =
 "*")(version 3.0; acl "Allow OIMAdminGroup add, read and write access to
 all attributes"; allow (add, read, search, compare,write, delete, import,export)
 (groupdn = "ldap:///cn=oimAdminGroup,cn=systemids,dc=acme,dc=com");)

dn: cn=oimAdminUser,cn=systemids,dc=acme,dc=com
changetype: modify
add: ds-privilege-name
ds-privilege-name: password-reset

Run the following command to load the above LDIF file:

./ldapmodify --hostname localhost --port 1389 --bindDN "cn=Directory Manager" \
--bindPassword passw0rd --defaultAdd --filename $OUD_HOME/scripts/oudadmin.ldif

Create a weblogic account

vi $OUD_HOME/scripts/weblogic.ldif

dn: cn=weblogic,cn=systemids,dc=acme,dc=com
changetype: add
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetorgperson
mail: weblogic
givenname: weblogic
sn: weblogic
cn: weblogic
uid: weblogic
userPassword: passw0rd

And add it to the LDAP:
./ldapmodify --hostname localhost --port 1389 --bindDN "cn=Directory Manager" \
--bindPassword passw0rd --defaultAdd --filename $OUD_HOME/scripts/weblogic.ldif

Add weblogic account to the oimAdminGroup

vi $OUD_HOME/scripts/weblogicGroup.ldif

dn: cn=oimAdminGroup,cn=systemids,dc=acme,dc=com
changetype: modify
add: uniquemember
uniquemember: cn=weblogic,cn=systemids,dc=acme,dc=com

And add it to the LDAP:

./ldapmodify --hostname localhost --port 1389 --bindDN "cn=Directory Manager" \
--bindPassword passw0rd --defaultAdd --filename ../scripts/weblogicGroup.ldif


Add the global-aci to changelog node in OUD (I think this is only necessary if you setup replication.  See the documentation listed above for more information).

cd $OUD_HOME/bin

./dsconfig ->
2. Authentication and Authorization ->
2. Access Control Handler ->
1. View and edit the Access Control Handler ->
2. global-aci ->
2. Add one or more values ->

(target="ldap:///cn=changelog")(targetattr="*")(version 3.0; acl "External changelog access"; allow(read,search,compare,add,write,delete,export) groupdn="ldap:///cn=oimAdminGroup,cn=systemids,dc=acme,dc=com";)

Delete this one:
(target="ldap:///cn=changelog")(targetattr="*")(version 3.0; acl "External changelog access"; deny (all) userdn="ldap:///anyone";)

q from dsconfig
Start OAM...
Configuration -> User Identity Stores

From OAM ID Stores, click Create

Store Name: OUD
Store Type: OUD: Oracle Unified Directory
Location: 10.10.107.49:1389
Login ID Attribute: uid
User Password Attribute: userPassword
User Search Base: dc=acme,dc=com
User Filter Object Class: inetOrgPerson
Group Search Base: dc=acme,dc=com

Test the connection and if successful, click Apply:

Change the Default Store to OUD:

And click Apply

Change the System Store to OUD click the green plus sign to add users:

Click Search and add the selected users:

Then click Apply:

Click OK and enter a valid administrator username and password to validate the system administrator account:

Click Validate.  This can error "Group oimAdminGroup is already a member" can be ignored.

Now configure the IDMDomainAgent to use the new OUD store:

Launch Pad -> Access Manager -> Authentication Modules

Click Search, then select LDAP.  Change User Identity Store to OUD and click Apply:

Sign out and then sign back in.  The new new credential store is in use now.

One way to confirm is to check the OUD access logs on svrtst03.

cd $OUD_HOME/logs

Do a tail -f access and logon to Oracle Access Manager.  Typical output is shown below from the access log:

[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11517 op=23 msgID=24 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=inetOrgPerson)(uid=weblogic))" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11517 op=23 msgID=24 result=0 nentries=1 etime=3
[06/May/2014:10:33:56 -0400] BIND REQ conn=11526 op=3 msgID=4 type=SIMPLE dn="cn=weblogic,cn=systemids,dc=acme,dc=com"
[06/May/2014:10:33:56 -0400] BIND RES conn=11526 op=3 msgID=4 result=0 authDN="cn=weblogic,cn=systemids,dc=acme,dc=com" etime=2
[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11517 op=24 msgID=25 base="cn=weblogic,cn=systemids,dc=acme,dc=com" scope=base filter="(objectclass=inetOrgPerson)" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11517 op=24 msgID=25 result=0 nentries=1 etime=2
[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11529 op=35 msgID=36 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=inetOrgPerson)(uid=weblogic))" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11529 op=35 msgID=36 result=0 nentries=1 etime=3
[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11529 op=36 msgID=37 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=groupofuniquenames)(uniquemember=cn=weblogic,cn=systemids,dc=acme,dc=com))" attrs="orgunit,mail,cn,description,name,orclguid,rolecategory,org,objectclass,displayname"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11529 op=36 msgID=37 result=0 nentries=1 etime=2
[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11529 op=37 msgID=38 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=groupofuniquenames)(uniquemember=cn=oimAdminGroup,cn=systemids,dc=acme,dc=com))" attrs="orgunit,mail,cn,description,name,orclguid,rolecategory,org,objectclass,displayname"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11529 op=37 msgID=38 result=0 nentries=0 etime=1
[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11529 op=38 msgID=39 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=inetOrgPerson)(uid=weblogic))" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11529 op=38 msgID=39 result=0 nentries=1 etime=3
[06/May/2014:10:33:56 -0400] SEARCH REQ conn=11529 op=39 msgID=40 base="cn=weblogic,cn=systemids,dc=acme,dc=com" scope=base filter="(objectclass=inetOrgPerson)" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[06/May/2014:10:33:56 -0400] SEARCH RES conn=11529 op=39 msgID=40 result=0 nentries=1 etime=1
[06/May/2014:10:33:57 -0400] SEARCH REQ conn=11529 op=40 msgID=41 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=inetOrgPerson)(uid=weblogic))" attrs="uid,wirelessacctnumber,postalcode,manager,street,orclguid,obresponsetries,dateofbirth,uniquename,defaultgroup,telephonenumber,obresponsetimeout,orgunit,timezone,employeenumber,obYetToBeAnsweredChallenge,initials,activestartdate,description,maidenname,localityname,gender,objectclass,sn,oblastloginattemptdate,fax,middlename,homeaddress,country,obpasswordhistory,cn,oblastsuccessfullogin,oblastfailedlogin,preferredlanguage,pobox,mobile,hiredate,uiaccessmode,oblastresponseattemptdate,department,state,givenname,lastname,org,employeetype,title,obfirstlogin,name,obpasswordcreationdate,homephone,pager,mail,activeenddate,oblockouttime,obAnsweredChallenges,loginid,firstname,obpasswordexpmail,obpasswordchangeflag,postaladdress,obuseraccountcontrol,telephone,displayname,oblogintrycount"
[06/May/2014:10:33:57 -0400] SEARCH RES conn=11529 op=40 msgID=41 result=0 nentries=1 etime=3
[06/May/2014:10:33:57 -0400] SEARCH REQ conn=11529 op=41 msgID=42 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=groupofuniquenames)(uniquemember=cn=weblogic,cn=systemids,dc=acme,dc=com))" attrs="orgunit,mail,cn,description,name,orclguid,rolecategory,org,objectclass,displayname"
[06/May/2014:10:33:57 -0400] SEARCH RES conn=11529 op=41 msgID=42 result=0 nentries=1 etime=1
[06/May/2014:10:33:57 -0400] SEARCH REQ conn=11529 op=42 msgID=43 base="dc=acme,dc=com" scope=sub filter="(&(objectclass=groupofuniquenames)(uniquemember=cn=oimAdminGroup,cn=systemids,dc=acme,dc=com))" attrs="orgunit,mail,cn,description,name,orclguid,rolecategory,org,objectclass,displayname"
[06/May/2014:10:33:57 -0400] SEARCH RES conn=11529 op=42 msgID=43 result=0 nentries=0 etime=2
[06/May/2014:10:34:02 -0400] CONNECT conn=11544 from=10.10.107.89:44839 to=10.10.107.49:1389 protocol=LDAP
[06/May/2014:10:34:02 -0400] DISCONNECT conn=11544 reason="Client Disconnect"

WebLogic Integration

On svrtst02, login to WebLogic.

Security Realms -> my realm -> Providers

From Authentication Providers, click New

Select LDAP Authenticator for Type and click OK.

From Authenticator Providers, click on OUD_LDAP:

 Under the Common tab, change Control Flag to SUFFICIENT:

 Click on the Provider Specific tab and enter the relevant information for the new provider:

Click Save to complete.

 

Unable to Start Oracle Unified Directory

 

On RHEL 5.6, installed Oracle Unified Directory 11.1.2.2.0 from ofm_oud_generic_11.1.2.2.0_disk1_1of1.zip.

After installation into $MW_HOME/Oracle_OUD1 and configuration don't start the directory server as the directories $MW_HOME/Oracle_OUD1/logs and $MW_HOME/Oracle_OUD1/locks are not created by the oud-setup program.  Create these manually before starting the directory server.  If the directory server is started by oud-setup, then you are unable to stop using the stop-ds command because there is no server.pid file (stop-ds looks for this in the locks directory).  I ended up de-installing/re-installing when this first happened.

When trying to start using start-ds, the following error message is thrown:

severity=SEVERE_ERROR msgID=2359728 msg=The LDAP connection handler defined in configuration entry cn=LDAP Connection Handler,cn=Connection Handlers,cn=config was unable to bind to 0.0.0.0:389:  IOException(Address already in use)

Solution

cd $MW_HOME/Oracle_OUD1/config
cp -p config.ldif config.ldif.bak
vi config.ldif
Change this:
ds-cfg-listen-port: 389
to
ds-cfg-listen-port: 1389

Then issue start command.

Thanks to a anonymous reader for sharing the correct location as to where the true stop/start commands are:

cd $MW_HOME/asinst_1/OUD/bin

and issue the start command from there.

Installing OpenLDAP 2.4.39 on RHEL 5.6

Note: Very high level instructions.  Should work with OpenLDAP 2.4.38 (has been tested) as well and RHEL 5.10.  Possibly other versions also.

My directory structure is as follows:
software install location - /opt
schema file location - /usr/local/etc/
slapd.conf file location - /usr/local/etc/openldap/
database file location - /var/openldap/openldap-data/
certs location -  /usr/local/etc/openldap

Install Berkely DB

Download db-6.0.30.tar.gz and untar in /opt
cd /opt
tar xvf db-6.0.30.tar.gz
cd db-6.0.30/build_unix
../dist/configure --enable-cxx --prefix=/usr/local/BerkeleyDB.6.0
make
make install

Do this step so that libs can be found without having to go through creating messy links:
cd /etc/ld.so.conf.d
vi berkely_db.conf
/usr/lib

/sbin/ldconfig

Install OpenSSL

Download openssl-1.0.1g.tar.gz and untar in /opt
cd /opt
tar xvf openssl-1.0.1g.tar.gz
export CPPFLAGS="-I/usr/local/BerkeleyDB.6.0/include"
export LDFLAGS="-L/usr/local/BerkeleyDB.6.0/lib"
cd  openssl-1.0.1g
./config shared
make
make test
make install

Install Cyrus-SASL

Download cyrus-sasl-2.1.26.tar.gz and untar in /opt
cd /opt
tar xvf cyrus-sasl-2.1.26.tar.gz
cd db-6.0.20/
export CPPFLAGS="-I/usr/local/BerkeleyDB.6.0/include"
export LDFLAGS="-L/usr/local/BerkeleyDB.6.0/lib -L/usr/local/ssl/lib"
./configure --with-openssl=/usr/local/ssl --libdir=/usr/local/lib64
make
make install

After the install, note the following message:
********************************************************
* WARNING:
* Plugins are being installed into /usr/local/lib/sasl2,
* but the library will look for them in /usr/lib/sasl2.
* You need to make sure that the plugins will eventually
* be in /usr/lib/sasl2 -- the easiest way is to make a
* symbolic link from /usr/lib/sasl2 to /usr/local/lib/sasl2,
* but this may not be appropriate for your site, so this
* installation procedure won't do it for you.
*
* If you don't want to do this for some reason, you can
* set the location where the library will look for plugins
* by setting the environment variable SASL_PATH to the path
* the library should use.
********************************************************
I set the SASL_PATH environment variable in the .bash_profile file as follows:
SASL_PATH=/usr/local/lib/sasl2
export SASL_PATH

Install OpenLDAP

Download OpenLDAP openldap-2.4.39.tar.gz
cd /opt
tar xvf openldap-2.4.39.tar.gz
cd openldap-2.4.39
export CPPFLAGS="-I/usr/local/BerkeleyDB.6.0/include -I/usr/local/ssl/include"
export LDFLAGS="-L/usr/local/BerkeleyDB.6.0/lib -L/usr/local/ssl/lib"
./configure --with-tls --with-cyrus-sasl --sysconfdir=/usr/local --bindir=/usr/local --libdir=/usr/local/lib64
make depend
make
make test
make install

Copy Schema Files

mkdir /usr/local/etc
cp -rfp /opt/openldap-2.4.39/servers/slapd/schema /usr/local/etc/.

Copy slapd.conf

mkdir /usr/local/etc/openldap/
cp -rfp /opt/openldap-2.4.39/servers/slapd/slapd.conf  /usr/local/etc/openldap/

Generate  the Encrypted Admintrator LDAP Password

This can be entered into the slapd.conf file so that OpenLDAP can be started without user interaction:
/opt/openldap-2.4.39/servers/slapd/slappasswd -s password
{SSHA}O4iHZ1yg+f4ynVg1rrUjYMki2F6jTp7O

Copy this value and paste it into the /usr/local/etc/openldap/slapd.conf file:
rootpw          {SSHA}O4iHZ1yg+f4ynVg1rrUjYMki2F6jTp7O

The slapd.conf file may end up looking like this:
include         /usr/local/etc/schema/core.schema
include         /usr/local/etc/schema/cosine.schema
include         /usr/local/etc/schema/inetorgperson.schema

pidfile         /var/openldap/run/slapd.pid
argsfile        /var/openldap/run/slapd.args

#######################################################################
# BDB database definitions
#######################################################################

database        bdb
suffix          "dc=my-domain,dc=com"
rootdn          "cn=Manager,dc=my-domain,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw          {SSHA}O4iHZ1yg+f4ynVg1rrUjYMki2F6jTp7O
# The database directory MUST exist prior to running slapd AND
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory       /var/openldap/openldap-data/my-domain
# Indices to maintain
index   objectClass     eq
index   cn              eq
index   uid             eq

Starting OpenLDAP Services

cd /usr/local/etc/openldap/slapd.conf
nohup /opt/openldap-2.4.39/servers/slapd/slapd -f /usr/local/etc/openldap/slapd.conf -d 1 &

View status of the OpenLDAP server:
tail -f nohup.out

Stoping OpenLDAP Services

kill -INT `cat /var/openldap/run/slapd.pid`

Enabling SSL

Request a server certificate from your RA or CA.

Place the issuing CA certificate file here:
TLSCACertificateFile    /usr/local/etc/openldap/cacert.pem

The signed server certificate file here:
TLSCertificateFile      /usr/local/etc/openldap/servercrt.pem

And the private key file (only readable by the process that starts the OpenLDAP server) here:
TLSCertificateKeyFile   /usr/local/etc/openldap/serverkey.pem