Monday, April 26, 2010

Tips & Tricks To Make Apache Work With Jserv [ID 330250.1]

Tips & Tricks To Make Apache Work With Jserv [ID 330250.1]
Modified 21-APR-2010 Type BULLETIN Status ARCHIVED


PURPOSE
-------
This is a comprehensive guide aimed to explain the interlinked working
architecture of different files under $IAS_ORACLE_HOME/Apache directory and also
suggestions to follow when things are not working as they should.

Pre-Amble
---------
For easy navigation and reference let us first set the following variables in
the environment of the user that owns the Apache directory structure:

Variable Physical Path
COMMON_TOP ..../[sid]comn
APACHE_TOP $IAS_TOP/Apache
SCRIPT_TOP $COMMON_TOP/admin/scripts

Note: The environment variables will not be available to the Apache listener
runtime. In addition, you should not add these variables to the Applications
environment file. Keep your environment for administering; starting and stopping
Apache free of the Applications environment variables to avoid potential problems.

Startup
=======
The core of the Technology Stack is built around Apache Web Server.
So let that be our starting point.
First thing to be determined is which file Apache is using httpd.conf or httpds.conf? to
get its startup information.
This can be determined by examining the script $APACHE_TOP/Apache/bin/apachectl
which is ultimately called by the script $SCRIPT_TOP/adapcctl.sh when you start
and stop the web server.

adapcctl.sh => apachectl

In the script apachectl, there is a variable HTTPD defined points to either the
httpd or the httpds executable.
Example: $ grep HTTPD= apachectl
HTTPD=/apps/oracle/visora/iAS/Apache/Apache/bin/httpd

If the HTTPD variable in the apachectl script calls httpd, Apache configuration
depends on httpd.conf. If the apachectl script calls httpds, Apache configuration
depends on httpds.conf.

For Release 11i Applications, the Apache Web Server depends on the full
JDK 1.3 to be installed under (or symbolically linked from) $APACHE_TOP/jdk.
The latest rapidWiz (11.5.10) installs JDK 1.4 on its own.

Note: The following type of error will occur if you have pointed $APACHE_TOP/jdk
to a directory containing only the JRE, and try to execute a jsp through browser:

JSP Error:
Request URI:/html/jtflogin.jsp
Exception: java.lang.NoClassDefFoundError:sun/tools/javac/Main

If the installation process for your platform installs the JDK to a predefined
directory, symbolically link $APACHE_TOP/jdk to the actual location.
Here is an example (assuming that the JDK installed to /usr/local/jdk118_v1):
$ln -s /usr/local/jdk118_v1 $APACHE_TOP/jdk

To determine the version of the JDK that is in use by Apache,
use the following script as noted in Note:309769.1 (not AIX specific):

sh -c "`awk -F= '$1 ~ /^JSERVJAVA.*$/ {print $2}' \
$ORACLE_HOME/../iAS/Apache/Apache/bin/java.sh` -version;"

Files of Importance
===================
The files that are involved in making Apache work with JServ are the following:
I $APACHE_TOP/Apache/conf/httpd.conf
(or httpds.conf, depending on your platform)
II $APACHE_TOP/Jserv/etc/jserv.conf
III $APACHE_TOP/Jserv/etc/jserv.properties
IV $APACHE_TOP/Jserv/etc/zone.properties
V $APACHE_TOP/Apache/conf/oracle_apache.conf
VI $APACHE_TOP/Ojsp/conf/ojsp.conf

IMPORTANT NOTE
-.-.-.-.-.-.-.
All these files are controlled by AutoConfig and it is not supported to make
manual changes to these files. If there are any changes that you require for
your system to function that cannot be made using AutoConfig then raise a
Service Request with Oracle Support

I httpd(s).conf ( "master" file for Apache )
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Observe the include lines for the files jserv.conf and oracle_apache.conf.

# Include the configuration files needed for jserv
include "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/Jserv/etc/jserv.conf"

# Include the Oracle configuration file for custom settings
include "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/Apache/conf/oracle_apache.conf"

The file ojsp.conf is indirectly referenced for Apache's configuration using the
following line in oracle_apache.conf:
include "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/Ojsp/conf/ojsp.conf"

A line that begins with the word Alias maps a directory name viewable on the
browser URL to a physical directory on the file system. Alias directives must
have a trailing slash for both virtual and physical paths.
Check to make sure the trailing slashes are there. The aliases will not work
without them. The following aliases must be defined.
Some are redundant, but must be defined nonetheless:

Alias Physical Path
/OA_HTML/ $OA_HTML
/OA_MEDIA/ $OA_MEDIA
/OA_JAVA/ $OA_JAVA
/html/ $OA_HTML
/media/ $OA_MEDIA
The DocumentRoot directive defines the directory where the Apache listener will
initially take users when no virtual path is specified in the URL. This should
be defined as your $COMMON_TOP/portal directory.
Just below the DocumentRoot directive, the default directory permissions are
defined. Make sure to also check that the line that allows users to view the
DocumentRoot directory is set to $COMMON_TOP/portal.

DocumentRoot "[$COMMON_TOP]/portal"

Options FollowSymLinks
AllowOverride None


DocumentRoot "/apps/oracle/viscomn/portal/VIS_incq093r"

httpd.conf ----> oracle_apache.conf => ojsp.conf
----> Jserv.conf => jserv.properties => zone.properties

Make sure ServerName is the fully qualified host name.
$ grep ServerName httpd.conf
ServerName myHost.oracle.com

The following portion shows that Apache is listening through port 3333 for
oprocmgr .(From FND.H there is a separate oprocmgr.conf file).
Here it denies the access from anywhere except if you are in the same host.
So, comment out the line: Deny from all to make it accessible from anywhere.


Listen 3333
ProcNode myHost.oracle.com 3333

Port 3333

Order Deny,Allow
#Deny from all
Allow from localhost
Allow from sweetnsour.idc.oracle.com



Check the way the /etc/hosts file is set up.
It is required that you have your machine.domain (Fully Qualified Domain Name)
name before the entry with just the hostname.
i.e. If it is:
127.0.0.1 localhost
168.142.16.5 myHost myHost.oracle.com loghost

Then change it to:
127.0.0.1 localhost
168.142.16.5 myHost.oracle.com myHost loghost

Trouble Shooting Checks
=======================
1. Are their any existing Apache processes?
If so, shut them down or use a kill command to remove them.

2. Test for existing processes: Unix: ps ef | grep Apache
3. Check for Port Conflicts: $IAS_TOP/Apache/Apache/conf/
In Files httpd.conf httpd_pls.conf (Port)
mod_oprocmgr (Listen / Port )
$IAS_TOP/Jserv/etc/jserv.properties

4. In case you are using the same Port Number for Jserv as the one you are
using for Apache then this will cause a Port Conflict on the Web Server.
The Jserv must use a different Port that is not used on the Server.
(The port for Jserv is the one that Jserv will use to comunicate with the
Apache Server, so it must be different than the one that Apache uses for the Web Pages)
For example Change the line :

# Set the port Apache JServ listens to.
# Syntax: port=[1024,65535] (int)
# Default: 8007
port=2030

Using AutoConfig Change the port=2030 for a Port NOT used on the Server

Save jserv.properties -> Stop All the applications processes

Start ONLY Apache using : adapcctl.sh start

5. In case Apache is not starting up Disable GSM /OAM in the following way
mv $FND_TOP/bin/FNDSVCRG $FND_TOP/bin/FNDSVCRG_ori

6. Try to Restart Apache using adapcctl.sh in $OAD_TOP/admin/scripts/
Check if it starts without using the GSM? If yes that makes sure this is not a WebServer issue. Pass the tar to 99/GSM.

7. Apply Patch 3351007 or comment out the rotatelogs (transferlog) directives.
Also uncomment the following line as well
#CustomLog /usr/oracle/visora/iAS/Apache/Apache/logs/access_log common

8. Check whether you are no on iAS 1.0.2.2.2? If yes have you applied iAS rollup 4?
(How to check, “httpd –v”)

A rare error on error_log is :
script not found or unable to stat:
/opt/oracle/product/OAD2/806/tools/web60/cgi/jinit

This indicates that the Apache web server has not even gotten to the part of
actually calling any Jinitiator. So troubleshoot Apache not Jinitiator.
Also check that the cgi directory has only f60cgi and rwcgi60.

Performance Tuning
===================
Now that you have taken care to startup Apache tune it up so that processes are
not started up or shutdown heavily. The load on the web server governs this
tuning. one commonly practised rule of thumb is set :
MaxRequestsPerChild = high number [This is set to 0 (unlimited) by default]
MinSpareServers = low number (e.g. 0 or 1)
This configuration ensures that Apache processes are very rarely shutdown,
actually only when they reach the MaxRequestsPerChild value.
MaxClients can be set to a maximum of 1024 in iAS 1.0.2

Followings are some detail possibilities for tuning

1. Limit the number of processes (HTTPD on Unix; THREAD on NT) to avoid spawning
too many HTTPD processes (which is memory consuming).

2. Set the following directive in the Oracle HTTP Listener configuration file
httpd.conf: KeepAlive Off

If you must use KeepAlive On (for example, for another application), make sure
that KeepAliveTimeout is set to a low number (for example, 15 seconds, which
is the default).

3. Set the maxClient directive to a high value.
The best is to let the HTTP Listener control to create more HTTPD daemons.
Therefore set the maxClient directive to a high value in the configuration
file (httpd.conf). However, you need to consider the memory available on the
system when setting this parameter.

MaxClient=512(by default) means that the listener can create up to 512 HTTPD
processes to handle concurrent requests.

4. MinSpareServers / MaxSpareServers
If your HTTP requests come in bursts, and you want to reduce the time to start
the necessary HTTPD processes, you can set MinSpareServers and MaxSpareServers
within httpd.conf) to have an appropriate number of processes ready.
However, the default values of 5 and 10 respectively are sufficient for most sites.

5. The MaxRequestsPerChild directive sets the limit on the number of requests that
an individual child server process will handle. After MaxRequestsPerChild
requests, the child process will die.
If MaxRequestsPerChild is 0, then the process will never expire.
This directive sets the maximum configured value for ThreadsPerChild for the
lifetime of the Apache process. Any attempts to change this directive during a
restart will be ignored, but ThreadsPerChild can be modified during a
restart up to the value of this directive.

6. ThreadsPerChild
This directive sets the number of threads created by each child process.
The child creates these threads at startup and never creates more.

II $APACHE_TOP/Jserv/etc/jserv.conf
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The file jserv.conf references jserv.properties using the ApJServProperties
directive. The file should exist in the same directory ($APACHE_TOP/Jserv/etc)
as jserv.conf. Make sure the path is defined correctly and the file exists.

ApJServProperties /u08/app/oracle/viscomn/util/apache/1.3.9/Apache/Jserv/etc/jserv.properties

jserv.conf => jserv.properties

Trouble Shooting Checks
========================
Check if you are hitting the following issue documented in Oracle 9iAS release
notes Release 1 (v1.0.2.2.2) for Sun SPARC Solaris Part Number A95823-01

1.5.3.2.20 HTTP Server Infrastructure Watcher May Suspend JVM
The infrastructure contains a watcher component that monitors the automatically
started JServ processes. If a JServ process stops responding, the watcher terminates
it and starts another process to replace it.
If the JVM is heavily loaded or performing garbage collection, the JVM will also
be suspended.
To prevent the JVM from terminating, increase its timeout value (ApJServVMTimeout)
in the jserv.conf file, or decrease the JVM heap size to reduce garbage collection time.
Default is 90 seconds timeout and 3 retries. If that is not enough,
you have got some other problem that needs to be sorted.

To address this issue you need to first determine if Apache or Jserv is hanging
since in some cases Apache maybe the problem and not the JVM.
The easiest way to do that is to get thread dumps for the JVM when the hang happens.
This will show if the JVM is the problem and why.
Another test you can do during the hang is to access a static html page
(not cached yet in browser, you can cleanup your cache to make sure of that)
and a non OA based jsp page (such as isItWorking servlet).

III $APACHE_TOP/Jserv/etc/jserv.properties
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This file is organized in following 3 sections:
1: Execution Parameters 2: Servlet Zones parameters
3: CZ Mem & URL Parameters

Find the line that starts with wrapper.bin.
This should point to the full path to the java executable from the JDK 1.1.8
which you ensured was properly installed earlier. This should point to
$APACHE_TOP/jdk/bin/java.sh

wrapper.bin=/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/jdk/bin/java

Go to the "CZ Mem & URL Parameters" section.
There are 3 that need to be added at the end of this section:

wrapper.bin.parameters=-DJTFDBCFILE=/u08/app/oracle/visappl/fnd/11.5.0/secure/otclnx1_vis.dbc
wrapper.bin.parameters=-Dservice.Logging.common.filename=/tmp/ibe.log
wrapper.bin.parameters=-Dframework.Logging.system.filename=/tmp/fwsys.log

Go to the "Servlet Zones parameters" section.
jserv.properties => zone.properties
The file jserv.properties references zone.properties using the root.properties directive.

Check the entry for bindaddress:
# Syntax: bindaddress=[ipaddress] or [localhost] or [*]
# Default: localhost
bindaddress=myHost.oracle.com

If you notice that the bindaddress is set to the IP address or the hostname.domain
of the host machine, try replacing it with localhost:
For example: bindaddress=localhost

In order for this to work, there must be an entry in the /etc/hosts file for
that machine:
127.0.0.1 localhost loghost

Trouble Shooting Checks
=======================
Where to look when the going gets Tough
-- Apache is quite verbose in logging errors when they occur.
While Apache is down, rename or delete the following files so that you can
troubleshoot with clean copies to check the log files for the cause of the
problem.

$APACHE_TOP/Apache/logs/error_log
$APACHE_TOP/Apache/logs/access_log (not necessary to rename if rotatelogs is being used)
$APACHE_TOP/Jserv/logs/mod_jserv.log
$APACHE_TOP/Jserv/logs/jserv.log

After that Edit the httpd.conf/httpds.conf file to enable debug for apache.
Set the "LogLevel" parameter to debug
Edit the $IAS_TOP/Apache/Jserv/etc/jserv.conf to enable debug for the jserv module.
Set the "ApJServLogLevel" parameter to debug
Ensure that entry log=true is present in jserv.properties.

Check that the necessary libraries of class files exist on your server:
$APACHE_TOP/Jserv/libexec/ApacheJServ.jar
$APACHE_TOP/Jserv/libexec/ApacheJServ.jar
$APACHE_TOP/Jsdk/lib/jsdk.jar
$APACHE_TOP/Ojsp/lib/ojsp.jar
$APACHE_TOP/Ojsp/lib/servlet.jar
$APACHE_TOP/Ojsp/lib/xmlparserv2.jar
$APACHE_TOP/Ojsp/lib/translator.zip
$JAVA_TOP/apps.zip
$JAVA_TOP/jdbc111.zip

Check that the .dbc file in $FND_TOP/secure specifies that the THIN JDBC driver
should be used by Oracle Applications. The GUEST_USER_PWD should be a valid
Applications username and password.
Open the file in a text editor and verify that the following lines appear in it:

#DB Settings
#Sun Aug 06 10:31:21 EDT 2000
APPS_JDBC_DRIVER_TYPE=THIN
APPL_SERVER_ID=6E2ECF457C39DE9FE030028AE99E70823172354937397097845740928337FA42
TWO_TASK=VIS
GUEST_USER_PWD=GUEST/GUEST
DB_HOST=myHost.oracle.com
FNDNAM=apps
GWYUID=APPLSYSPUB/PUB
DB_PORT=11523

Keep in mind that JVMs often could not start in the first place due to the
failure to bind to socket. In such a scenario the OPM (Oracle Procedure Manager)
cannot find any of them. Since they are all dead, Apache also cannot mount the
oa_servlets directory to the JServ module and therefore it tries to interpret
/oa_servlets/ as a virtual directory (which it is not).
As a result client browser request cannot find Jsp pages when actually there is
nothing wrong with any of them.

If you observe in log file Jserv is not being able to create a socket on any of
the ports selected so far be sure every other problem in the logs is based upon
this initial one. Most of the time this may be a problem with the clone,
but is more likely a problem at the operating system level.

Always check does "netstat -a" show that ports range specified in
jserv.properties available? You can also try to expand this range of ports and
let the system pick through them one by one until you find an open port yourself.

Verify the JDK Settings for SUN following Note: 130091.1

Set jserv.properties parameter security.maxConnections higher than 256

** If you still have a issue at hand suggest customers to apply the following
patches.

a> Apply AOL/J Function Security Patch 3119955
Confirm by doing:
SQL> select * from ad_bugs where bug_number='3119955';

b> Apply TXK D for JDBC Drivers.
Confirm by doing:
SQL> select * from ad_bugs where bug_number='2902755';

IV $APACHE_TOP/Jserv/etc/zone.properties
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here the repositories directive defines a comma-delimited list of the locations of
class files Jserv considers as "root". By default, the directive should point to
the $APACHE_TOP/jserv/servlets directory (which contains classes to test Jserv).

repositories=/apps/oracle/visora/iAS/Apache/Jserv/servlets,/apps/oracle/viscomn/java/apps.zip,/apps/oracle/viscomn/java
servlet.oracle.jsp.JspServlet.initArgs=translate_params=true,developer_mode=false,page_repository_root=/apps/oracle/viscomn/_pages
zone.properties <-- Jserv/servlets + apps.zip + _pages Points to Remember ================== + The page_repository_root is an alternative root directory for the Oracle JSP container to use in loading and generating JSP pages as long as it is not set as one of the wrapper.classpath entry in jserv.properties. + The System Class Path is shared by all servlets zones and is defined using the "wrapper.classpath" directive in the jserv.properties file when JServ startup mode is set to either automatic or auto. + Due to performance issues, class reloading is non-functional. Nevertheless, that's one of the reasons why Apache Jserv has been deprecated by the Apache foundation in favour of tomcat. There is no way Oracle should recommend class reloading in a production environment since the performance constraints that will introduce, might render the application unusable. + If you are working on a customization and really need class reloading (which is not good and I will explain why later on), it would be a good idea to use either Tomcat as a standalone development environment, or better yet, use Jdeveloper with the OA extensions that will mimic most of the functionality found in java layer on Oracle Applications. + Classes defined in the System Class Path are not automatically reloaded by JServ when they are modified and JServ needs to be restarted in order for new classes to be loaded. On the other hand, classes defined in the Servlet Zone Class Path are automatically loaded by the Zone Class Loader when they are modified. + The repositories setting is for the classes you want to be reloaded upon modification. + Classes must be in the JSP classpath, not the system classpath, to be dynamically reloaded. If they are in the system classpath as well, the system class loader may take precedence in some circumstances, possibly interfering with JSP automatic-reloading functionality" + If there are classes you want to be loaded by the Oracle JSP class loader instead of the system class loader, then use the classpath configuration parameter, or place the classes in the Oracle JSP default classpath. From Note 211918.1 that says the following: Usually in normal servlets you can take advantage of class/es autoreloading by specifying those classes in the repositories section (in zone.properties) and setting autoreload.classes property to true. But Configurator servlet can not take advantage of this autoreload mechanism because of the Configurator engine library. Because of this dependency, servlets does not know how to autoreload the modified classes. So, the only option is to bounce the Apache to see the changes in Functional companion. And currently, there is no way to bounce the Apache from the web browser. Add "unsafe_reload=true" to the initArgs, like below servlet.oracle.jsp.JspServlet.initArgs=translate_params=true,developer_mode=true, unsafe_reload=true, page_repository_root=/apps/testora/iAS/testcache with the above settings actually JSP pages will get refreshed without relogin after modifying them. But still for class modification, you will have to bounce the Apache Web Server + Class reloading is not a good idea on apache jserv because there is no "concept" of a Web Application. Class reloading occurs in any J2EE java servlet/jsp container where the classes pertaining to a particular web application are reloaded via the container where the application as a whole is reloaded. In our deployment for Oracle Applications, we didn't have any other choice (OC4J was a promise back then) than going with apache jserv, so everything is deployed as a big application. + Individual class reloading might cause several issues (heap corruption, memory leaks) in a massive java application as Oracle Apps. Every time a class is reloaded, all the instantiated classes need to be reloaded as well (jdbc connection pool,session, cache engine) and should not be recommended under any circumstance. From an official Oracle's standpoint, extending Oracle Applications should be done using Framework methodologies (see www-apps.us.oracle.com/fwk/), which implies using jdeveloper as a development environment. Anything else should be considered a hack. V $APACHE_TOP/Apache/conf/oracle_apache.conf ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Note: On some platforms the oracle_apache.conf file may be missing. The oracle_apache.conf file is included in httpd.conf or httpds.conf for the sole purpose of referencing additional configuration files that are Oracle-specific. In addition to the configuration file for Oracle jsp's it will be used for configuring Oracle's implementation of xml and plsql. Check/define include line for the files ojsp.conf include "[$APACHE_TOP]/Ojsp/conf/ojsp.conf" include "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/modplsql/cfg/plsql.conf" include "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/Ojsp/conf/ojsp.conf" include "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/xdk/admin/xml.conf" oracle_apache.conf --> plsql.conf
--> ojsp.conf
--> xml.conf

VI $APACHE_TOP/Ojsp/conf/ojsp.conf
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note: On some platforms the conf subdirectory of $APACHE_TOP/Ojsp and the
ojsp.conf file may be missing.

The 2 types of directives in this file are Alias directives and ApJServAction
directives. The ApJservAction directives map a file extension to a class file
that defines how Jserv will execute it.

Check/define the Alias for jspsamples and jspdocs.
Make sure that the trailing slash appears for both the alias and the physical
directory.
# Set aliases for Oracle JSP
Alias /jspdocs/ "/u08/app/oracle/viscomn/util/apache/1.3.9/Apache/Ojsp/"
Alias /jspsample08/app/oracle/viscomn/util/apache/1.3.9/Apache/Ojsp/webroot/"

Check/define the ApJServAction directives for .jsp and .sqljsp files
# Associate OJSP extension with OJSP class to be called
ApJServAction .jsp /servlets/oracle.jsp.JspServlet
ApJServAction .sqljsp /servlets/oracle.jsp.JspServlet

Comprehensive Performance Tuning for the Entite Middle tier
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Whenever you are into some performance tuning issue with respect to Applications
find out the following information :

+ What version of Apache are you running?
Determine the current Apache version and patch level as follows:
%IAS_ORACLE_HOME%\Apache\Apache\Apache.exe -version
+ Check whether the database can still be accessed. (SQL*Plus)
( Confirms that number of connections has been exceeded. )
+ Implement Dead Connection Detection (DCD) in the following manner.
On the server side add in SQLNET.ORA:
SQLNET.EXPIRE_TIME= <# of minutes>
# of minutes is the time between successive probe packets sent from server to
the client to check if the connection is still alive.
( A time of 10 minutes is probably optimum for most applications. )

+ Check if stopping the concurrent manager makes a difference.
+ Following statements will give you a fair idea about the sessions in use
set linesize 132
set pagesize 100
col machine format a15
col OSuser format a12
col program format a30
SQL> select count(*) from v$session;
SQL> select count(*) from v$session where status='INACTIVE';
SQL> select count(*) from v$session where status='ACTIVE';
SQL> select machine, osuser, program, count(*) from v$session
group by machine, osuser, program order by 4 desc;
SQL> select count(status) Count, status, machine, program from v$session
where program like '%JDBC%' group by status, machine, program;
SQL> select count(status) Count, status, machine, module from v$session
where program = 'JDBC Thin Client' group by status, machine, module;
SQL> SELECT serial#, substr(program,1,20) program, status, to_char(logon_time,'DD-MON-YY HH24:SS') Login_Time,
to_char(sysdate-last_call_et/86400,'DD-MON-YY HH24:SS') Last_Activity FROM v$session
WHERE program like 'JDBC%' order by 4;

+ If you are particularly concerned about the inactive sessions, keep in mind there
will always be a certain number of Inactive sessions. They will get recreated
once Apache is bounced. Now the forms processes are a different matter, and
should be getting cleaned up, especially if they are abnormally terminated
forms sessions. If this is not working, this could go back to the current
forms version you are on.

+ Apache version 1.3.12 is notorious for not cleaning up after itself.
The recommendation is always to upgrade Apache to 1.3.19 (9iAS 1.0.2.2.2) per
Note.146468.1 at the earliest opportunity.

Between upgrading Apache and upgrading the JDBC driver, the latter would obviously
be of least impact to the system, but there is no guarantee that the latest JDBC
driver alone will resolve any issue with inactive sessions.

+ How many JVM's have you defined?
+ Have your appied iAS rollup patch 4 { 3072811}?
Development strongly recommends applying 3072811 as it is the latest iAS rollup patch.
+ What modules are you using in applications?
+ If you are not using ICSM, please check that parameter s_icsmstatus = 'disabled' in
your Applications Context (XML) file for Autoconfig.

+ What the profile option "Self Service Personal Home Page mode" is set to when
you get performance issues. In a general sense, the performance of the framework
style interface is much better than the PHP (plsql based) version. Most of all
development going forward is tied to framework and development had considered
desupporting PHP until customer demand insisted on keeping it.
The old sytle is desupported from 11.5.10 CU2 and has been strongly
recommended since FND.G

+ Have you applied the latest TXK patch at the time of writing is
4104924 TXK (FND) AUTOCONFIG TEMPLATE ROLLUP PATCH K (July 2005) for Autoconfig
and roll up patch 5 for 9iAS 1.0.2.2.2.
+ Have you applied Patch 3902820 - New CONSOLIDATED FIXES - POST AOLJ PATCH 3313291
+ When you bounce the Apache server, does the memory consumption continue to increase
from that point?
+ Development has stated if performance problems are shown with Jserv processes
in 11i, JDK 1.4 should be implement. The JDK is the engine for the Jserv.
The steps also move the JDBC driver to the current version. It would recommend
to test the JDK 1.4. (246105.1)
Secondly, if you run the following command, it will give you the number of
JDBC connections you have on the instance.
select program, count(*) from v$session group by program;

+ When memory usage is high, what is the result of the following query?:
select count(*) from v$session where program like 'JDBC%';
The above output may show the memory is being consumed by large counts of JDBC
connections. The JDBC Pool in e-Business Suite is implemented through the core
java technology framework for the entire suite (named AOL/J). AOL/J uses the
dbc file in order to obtain the parameters needed to create a database connection.
AOL/J technology uses JDBC, mostly and more intensively, in web applications.
When starting iAS HTTPD it launches the ApacheJserv engine.
During the startup of ApacheJerv is when the pool is built, based on the
parameters specified by the DBC file.

When using the default settings in a heavily used system, it is possible to
encounter situations where the size of the pool grows, leaving the database
running out of processes or out of connections.

+ If this condition surfaces, the best way to deal with this situation is to
re-configure the AOL/J database connection pool parameters.
The values in the DBC file will override any values set in SSP_INIT.txt,
but you should not put these values in ssp_init.txt anymore. The file
'ssp_init.txt' is now only used for iProcurement specific settings
I would like to expound on the explanation above to clear the doubt on how the
JDBC pool monitor will behalve with the following parameters set:

FND_MAX_JDBC_CONNECTIONS=500
FND_JDBC_BUFFER_MIN=5
FND_JDBC_BUFFER_MAX=25%
FND_JDBC_BUFFER_DECAY_INTERVAL=60
FND_JDBC_BUFFER_DECAY_SIZE=1
FND_JDBC_USABLE_CHECK=true
FND_JDBC_CONTEXT_CHECK=true
FND_JDBC_PLSQL_RESET=false

Note We are changing the default values for the parameter FND_JDBC_MAX_CONNECTIONS
to 100 in Bug#3186367.

When apache starts, it will create a buffer of 5 JDBC connections (FND_JDBC_BUFFER_MIN=5)
plus a few more as java dependant products initialize. As users login, those 5
connections will get used and more connections will be created to replenish the
buffer. These connections will increase as needed up to the limit of 500
(FND_MAX_JDBC_CONNECTIONS), which is probably too high for user load to hit.
If the buffer decay interval is too high then the pool cleanup thread will not
kick in often enough. The default value is 60 seconds. If the interval is too
high and the decay size is too small then insufficient number of connections
will be cleaned each time the cleanup thread sweeps the pool.
The default for decay size is 1.
You may want to up the decay size and lower the interval to make the pool cleanup
thread more aggressive. I suggest you the following:

FND_JDBC_BUFFER_DECAY_INTERVAL = 60
FND_JDBC_BUFFER_DECAY_SIZE = 1

As users log off, the JDBC connections will die off at a rate of 1 every 60 seconds
(FND_JDBC_BUFFER_DECAY_SIZE=1, FND_JDBC_BUFFER_DECAY_INTERVAL=60), but this
decay won't start until you have reached 125 JDBC connections (25% of 500 where
FND_JDBC_BUFFER_MAX=25% and FND_MAX_JDBC_CONNECTIONS=500).
The decay will slowly reduce the number of inactive JDBC connections down to
FND_JDBC_BUFFER_MIN=5, with the least recently used ones being removed first.

Note that each of these parameters are PER JVM.
If your jserv.conf has 3 groups of 1 JVM each (OACoreGroup, DiscoGroup,
XmlSvcsGrp) for a total of 3 JVM's, each with a FND_JDBC_BUFFER_MAX of 125 for
a total of 375.

Make a back up of the DBC file(s) in $FND_TOP/secure.
You may have two dbc files instance_sid.domain and instance_sid. You can tune
these as you see fit, such as increasing the decay rate and decreasing the
FND_JDBC_BUFFER_MAX, but whatever you do they MUST BE SPECIFIED in the DBC file.
In order to see the results it would be best to do this at a time when you can
bounce the database as well as the web server.
Look at the AOL/J pool status monitor to verify the changes Over a period of
time take snapshots of the pool monitor and then your connections at the database
level.

Point to Remember
=================
If multiple JVMs are implemented (ie: Jserv load balancing -multiple JVMs in a
single box-, HTTP Load balancing -Multiple Web Servers connecting to the same
instance- or a combination of both), it is important to note that each JVM will
create its own JDBC Pool.
Under these circumstances,it is very difficult to troubleshoot what the problem
is, since some users might be getting errors when trying to obtain connections
from the pool, while when trying to use the diagnostics tools provided to
troubleshoot the problem, a different JVMs could be accessed, obtaining the
wrong results for the tests.
+ Currently there are 8 user-configurable pool parameters.
4 govern pool size, 2 control the thread used to maintain the pool, and
3 control the types of safety checks the pool performs on connections.
All connection pool parameters are configured by setting the corresponding
environment variable in the dbc file.

+ The fundamental purpose of a web server is to deliver files, and when the
JInitiator needs jar files to build the client side of a forms connection the
Apache web server is the logical choice for sending them. The forms process does
not have a mechanism for this.

+ All of the Apache web server controlled JVMs use the DBC and ssp_init.txt files
on the forms/web tier and they are the bulk of the JDBC connections.
One exception that I know of is the concurrent request "OAM Dashboard Collector"
that runs as a java program and makes up to 2 JDBC connections everytime it runs.
It is typically scheduled to run every 10 minutes and on slower systems these
requests tend to back up and stack up the JDBC connections.

The "OAM Dashboad Collector" is the concurrent request that updates the statuses
of everything for the Oracle Applications Manager. If you cancel this request,
then the Oracle Applications Manager console will have stale data.
Ref: Item 11 in Note:226826.1 under "Troubleshooting".
If you don't use the Oracle Applications Manager this probably will not matter
to you, but if you do find more than 1 of these requests running or in pending
status you may want to cancel them temporarily and see if that alleviates your
JDBC session problem.

How to Disable the "OAM Application Dashboard Collection"
1. Bring up applications.
2. To disable the "OAM Application Dashboard Collection" agent, we are
going to do this from the concurrent managers:
2a. Login as someone who has the System Administrator Responsibility
(Do not confuse with System Administration - OSSWA)
2b. Pick System Adminstrator
2c. Navigation Steps:
Concurrent >> Program >> Define
Under Short Name Query up 'FNDOAMCOL'
This will bring up "OAM Application Dashboard Collection"
2d. Uncheck the "Enabled" check box.
2e. Save the record.
3. Bounce applications. (Specifically the Concurrent Managers)
Note: The reason why we have to bounce apps is that the OAM Dashboard Collection is scheduled via OAM.
Prior to "OAM Rollup Patch A (3119911)" this was defaulted to be turned on.
We want to disable this, shutdown the concurrent managers, and then kill any scheduled requests from
trying to launch again.
4. Log back into System Administrator Responsiblity
Go to Concurrent >> Requests
Choose "All My Requests" >> Find
If you see any "OAM Application Dashboard Collection" Pending Scheduled.
Cancel these requests.

Still if you see a CPU spike by JAVA process, then run
$ps -aef |grep java --followed by this SQL statement:

SELECT s.username||' '||s.osuser||' ('|| DECODE(s.terminal,'unknown','?',
'UNKNOWN','?',s.terminal) ||')' userinfo, s.sid||','|| s.serial# ||'
('|| s.audsid ||')' "sid (audsid)", p.spid, w.event , w.p1
FROM v$session s, v$process p ,v$session_wait w
WHERE p.addr = s.paddr AND s.sid = w.sid AND s.process = '&java_process_id'
ORDER BY s.osuser, s.terminal ;

Try to associate the corresponding database processes:
SELECT sid, username, module, program, status FROM v$session WHERE status='INACTIVE' ;

In case you see a corresponding database process, then gather more information
about that session. Use : Note 169630.1: bde_session.sql
- Expensive SQL and resources utilization for given Session ID.

+ Did you make the change to the heap size because you suspected that GC
activity is causing the hang?
If yes, how does the activity compare before and after the change.
With a smaller heap you will be spending more time in GC since it will
happen more frequently.

Another thing to mention here is that when the JVM stops responding to the ping
coming from OPMN, it will be bounced rather than suspended.
To take a thread dump on Unix/Linux, you need to issue the following:
kill -3 jvmpid
where jvmpid is the jvm process id on the OS.

The output will go to $iAS/Apache/Jserv/logs/jvm/OACoreGroup.*.stdout file.
Run the command for all JVMs during the hang.
You should run it 3 times per JVM spaced out by 10 seconds each to establish a
pattern.

The thread dump will show all threads within a particular JVM along with state
and stack for each one of those threads. So for example, if the reason for the
hang is a lock that all threads is waiting for then we will be able to see that.
In other words, it will show what the JVM is doing when it is not responding to
requests. When you take multiple thread dumps for every JVM we will be able to
establish a pattern of what every thread is doing and approximately for how long.
Please also capture the GC activity with timestamps (not natively available if
you are on 1.3.1 Unix/Linux or Windows) and the actual clock time when the JVM
was started and when it went into the hanging phase.
This will be needed to find out if the JVM at the same time of the hang was busy
doing long or excessive GC activity.

Run the following script to determine your patchset level

REM =====================
REM "BEGIN CHECK PATCHES"
REM =====================

set pagesize 1000;
set linesize 141;
spool /tmp/supp_patch.txt;

select bug_number "BUG",
decode(bug_number,
3594604, 'TXK Rollup Patch I',
4068109, 'POST ADX.E[.1] CONSOLIDATED UPDATE',
3072811, 'MLR FOR IAS1.0.2.2 ROLLUP4',
4090504, 'REHOST ORACLE 9iR2 JDBC DRIVERS (v9.2.0.5) WITH ORACLE APPS 11i',
3214685, 'APPS NAV PORTLET: NEED TO RELEASE THE AM FROM THE NAVIGATE PAGE',
3543254, 'Rollup for JTT Session Management',
3005778, 'INACTIVE JDBC CONNECTIONS PERSIST AFTER IPROCURMENT TERMINATED',
3075636, 'INACTIVE JDBC CONNECTIONS PERSIST AFTER IPROCURMENT TERMINATED 2',
3551500, 'UNRELEASED JDBC CONNECTIONS IN IPROCUREMENT 11.5.9',
3137283, 'JVM memory leak',
3298118, 'Extra connections being used under load and connection not released',
3308185, 'Navigates from receiving and connection is not released',
3315942, 'JDBC Connection Leak when navigate to iProcurement Home Page',
3334973, 'APPSPERF: FND: Server side profile cache misses',
3308185, 'Navigates from receiving and connection is not released',
3549568, 'Fixes for connection leaks',
3819096, 'FND_PROFILE.VALUE_SPECIFIC returns Site Value if no context passed',
3968216, 'Set ORACLE.JDBC.FREEMEMORYONENTERIMPLICITCACHE to TRUE') "Patches Applied",
b.LAST_UPDATE_DATE "APPLIED"
from AD_BUGS b where b.BUG_NUMBER in
('4068109','3072811','4090504','3214685','3543254','3005778','3075636','3551500','3137283','3298118','3308185','3315942',
'3334973','3549568','3819096','3308185' ,'3968216');
spool off;

REM ===================
REM "END CHECK PATCHES"
REM ===================

Follow these steps to clean up fnd_nodes table :

1. Make a backup copy of fnd_nodes
2. EXEC FND_CONC_CLONE.SETUP_CLEAN;
COMMIT;
EXIT;
3. Source the iAS env.
4. Then run adlnkiAS.sh from $iAS_Oracle_home/bin.
5. Clear OS memory ( Contact your O/S administrator )
6. Stop all services.
7. Retest the issue using the following login URL:
http://:/OA_HTML/AppsLocalLogin.jsp
<

A Guide to Understanding and Implementing SSL with Oracle Applications 11i

11i: A Guide to Understanding and Implementing SSL for Oracle Applications [ID 123718.1]
Modified 29-OCT-2009 Type WHITE PAPER Status PUBLISHED
A Guide to Understanding and Implementing SSL with Oracle Applications 11i

This document contains information for implementing SSL with Oracle E-Business Suite Release 11i. It provides guidelines for a successful implementation and covers specific steps for configuring SSL with Oracle HTTP Server, Oracle Forms 6i server, and Oracle Database server. The most current version of these notes is 123718.1 on Oracle MetaLink. There is a change log at the end of this document.

Chapter 1. Overview

Concepts and Terminology
SSL Implementations with the Applications Technology Stack

Chapter 2. Certificate Provisioning

Common Set-up Steps
Certificate Provisioning for Oracle HTTP Server
Certificate Provisioning for Oracle Forms 6i Server
Certificate Provisioning for XML Publisher or Business Intelligence Publisher
Certificate Provisioning for Oracle Database Server
Working with Unrecognized CA Root Certificates

Chapter 3. Configuring SSL for AutoConfig-enabled System

Common Configuration Steps
Configuring SSL with Oracle HTTP Server
Configuring SSL with Oracle Forms 6i Server
Configuring SSL with Oracle Database Server

Chapter 4. Configuring SSL for non AutoConfig-enabled System

Common Configuration Steps
Configuring SSL with Oracle HTTP Server
Configuring SSL with Oracle Forms 6i Server
Configuring SSL with Oracle Database Server
Additional Configuration Steps

Chapter 5. Verifying SSL Configuration

Verifying SSL Set-up for Oracle HTTP Server
Verifying SSL Set-up for Oracle Forms 6i Server
Verifying SSL Set-up for Oracle Database Server

Chapter 6. Troubleshooting

Client Connections
Startup/Connection to Oracle HTTP Server
Connection to Oracle Forms Server
Use of SSL with Oracle Portal

Appendices

A. Current Limitations and Considerations
B. Firewalls
C. Use of the Forms Listener Servlet
D. Converting OpenSSL Certificates to Wallet Format for use with Web Cache

Related Documentation

Chapter 1. Overview

This document describes procedures for configuring SSL with Oracle E-Business Suite Release 11i. It provides guidelines for a successful implementation and covers specific steps for configuring SSL with Oracle HTTP Server, Oracle Forms 6i Server, and Oracle Database Server. You MUST configure SSL for both the Oracle HTTP Server and Oracle Forms (either the the Forms 6i Server or the Forms Listener Servlet), these cannot be configured independently.

Oracle XML Gateway's Transport Agent and Oracle Workflow's Web services are the other two components that support both Client Authentication and Server Authentication and require manual steps for SSL configurations. For detailed SSL configuration steps, see the following Oracle MetaLink notes:

Oracle Transport Agent

* For 11.5.10, see the Oracle XML Gateway's User Guide Release 11i in the Virtual Applications Documentation Library.
* For 11.5.9 or earlier, see Oracle MetaLink Note 152775.1 Installing Oracle XML Gateway and Oracle Workflow with Oracle Applications 11i.

Oracle Web Services

* Oracle XML Gateway's User Guide Release 11i, in the Virtual Applications Documentation Library.


Concepts and Terminology

Secure Sockets Layer (SSL)

SSL is a technology that defines the essential functions of mutual authentication, data encryption, and data integrity for secure transactions. Exchange of data between the client and server in such secure transactions is said to use the Secure Sockets Layer (SSL).

Transport Layer Security (TLS)

Transport Layer Security is a cryptographic protocol that ensures privacy between communicating applications and their users on the Internet. While SSL is supported with all versions of the Oracle Application Server, TLS requires a minimum of Application Server 10.1.2.0.
Oracle Applications Release 11i supports the use of SSL.
Oracle Applications Release 12 supports the use of both SSL and TLS.

Certificate Authority (CA)

A Certificate Authority is a trusted third party responsible for issuing, revoking, and renewing digital certificates. All digital certificates are signed with the Certificate Authority's private key to ensure authenticity. The Certificate Authority's Public Key is widely distributed.

Private (Server) Key

The private key file is a digital file that you generate and for use to decrypt messages sent to you. The certificate request (CSR) that you send to your Certificate Authority (CA) is derived from this private key. Therefore, the resulting digital certificate (containing the your public key) which is issued by your CA, is bound to this private key.

Certificate Signing Request (CSR)

A Certificate Signing Request (CSR) is a digital file which contains your public key and your name. You send the CSR to a Certifying Authority (CA) to be converted into a real Certificate.

Digital Certificate (Public Key)

A digital certificate is an electronic document that binds an identity to a pair of electronic keys that can be used to encrypt and sign digital information. Certificates are issued by a trusted third party, called a Certification Authority (CA). The document is usually in a standard X509 format and contains three elements:

* Entity attributes (information about your organization)
* Public key (which is bound to your organization)
* Digital signature of the trusted CA private key

Secure Server Certificates

Secure Server Certificates are 128 bit certificates which provide 128 bit SSL encryption. If a browser has 128 bit support, then encryption is negotiated to 128 bits. However, if the browser only supports 40 bit encryption, the level of encryption, regardless of a 128 bit certificate, will be negotiated down to 40 bits.

Global Server Certificates

Global Server Certificates, also referred to as Server Gated Cryptography, are 128 bit certificates that enable all browsers to use 128 bit encryption, even if the browser only supports 40 bit encryption. A global server certificate usually has 2 parts: the certificate itself and an extra intermediate certificate which is used to provide the step-up. The marketing names of these certificates vary depending on the company that issues the certificate, for example, Thawte call them 128 bit SuperCerts.

Note that it is not possible to get trial versions of global server certificates, therefore it is not possible to test unless one is purchased.


SSL Implementations with the Applications Technology Stack

Oracle E-Business Suite Release 11i has three interfaces for transactions between client and server:

* Oracle HTTP Server for all communications
* Oracle HTTP Server for initial connection, and Oracle Forms 6i Server for subsequent connections.
* Oracle Database Server for communication with Oracle HTTP Server using UTL_HTTP package

This chapter illustrates the SSL implementations - based on the three interfaces listed above - with the Applications Technology stack.

* Section 1.1 . SSL Implementation with Oracle HTTP Server
* Section 1.2 . SSL Implementation with Oracle Forms 6i Server
* Section 1.3 . SSL Implementation with Oracle Database Server


Section 1.1. SSL Implementation with Oracle HTTP Server

The implementation of SSL for the Oracle HTTP server, is based on mod_ssl (http://www.modssl.org) and openssl (http://www.openssl. org) technology. The Oracle HTTP server installation under Oracle E-Business Suite Release 11i includes the mod_ssl and openssl components. Once openssl has been used to generate your digital certificate and the proper directives for Oracle HTTP server have been defined, the Oracle HTTP server is capable of starting and running in SSL mode.
How SSL works with Oracle HTTP Server

1. The client sends a request to the server using HTTPS connection mode.
2. The server presents its certificate to the client. This certificate contains the server's identifying information.
3. The client checks its list of Trust points and compares the information in the certificate with the server's public key. If it matches, the server is authenticated as a trusted server.
4. The client sends the server a list of the encryption levels, or ciphers, that it can use.
5. The server receives the list and selects the strongest level of encryption that they have in common.
6. The client creates a session key which is used to encrypt the data and sends this session key to the server which can decrypt the data with its private key.

Section 1.2. Oracle 6i Forms 6i Server

Oracle Form 6i Server can operate in three separate modes: socket, HTTP and HTTPS. HTTP or HTTPS connection mode can be used for internet transactions (outside the firewall), whereas socket mode can only be used for an organization's intranet (inside the firewall). HTTPS corresponds with SSL and is strongly recommended if a company is to allow transactions over the internet.

The implementation of SSL for the Oracle Forms 6i Server makes use of Oracle Wallet Manager, which is included in the Developer 6i Server install under the application server Oracle home from version 8.0.6 onwards. The tool is Java-based.
How SSL works with Oracle Forms 6i Server

1. The client sends a request to the server using the HTTPS protocol.
2. The applet is downloaded and connects to the Forms Server Listener through HTTPS using the connectMode parameter of HTTPS.
3. The server sends its certificate to the client and the client searches its list of Trust points. The list of Trust points is stored in the certdb.txt file.
4. The server is identified and authenticated.
5. The client sends the server its list of encryption levels and the server chooses the strongest common level of encryption for communication.
6. Communication can now take place between the client and server.

Section 1.3. Oracle Database Server

Oracle products such as Oracle Configurator, Order Management, Order Capture, Quoting, iPayment, iRecruitment, iStore, and Pricing access data over the Internet in HTTP or HTTPS connection mode. The implementation of SSL for the Oracle database server which acts as a client sending requests to the Web server makes use of the Oracle Wallet Manager for setting up an Oracle wallet.
How SSL works with Oracle Database Server

1. The UTL_HTTP package is used for making HTTP callouts from SQL and PL/SQL to a Web node (Oracle HTTP server).
2. When the package fetches data from a Web site using HTTPS, it needs to specify the location to the Oracle wallet that resides on the database server. This wallet contains the certificate for the Certifying Authority (CA) who signed the Web node's server certificate.

Chapter 2. Certificate Provisioning

The SSL configuration includes two major steps: the acquisition of the necessary certificates, and the modifications to the various configuration files maintained by the E-Business Suite Release 11i system. This chapter contains detailed instructions for provisioning certificates that are required by the E-Business Suite Technology stack. Oracle MetaLink Note 119873.1 explains the implementation of self-signed certificates. This document expands on that information and provides guidance for third-party CA certificates, and non-default naming port conventions.

In April 2006 Verisign began issuing Extended Validation (EV) certificates. Please see Note 436312.1 - Information on using new 3 tier certificates issued by
VeriSign with Applications 11i and be sure you understand the implications before using this type of certificate.

* Common Set-up Steps
* Option 2.1 . Certificate Provisioning for Oracle HTTP Server
* Option 2.2 . Certificate Provisioning for Oracle Forms 6i Server
* Option 2.3 . Certificate Provisioning for XML Publisher or Business Intelligence Publisher
* Option 2.4 . Certificate Provisioning for Oracle Database Server
* Working with Unrecognized CA Root Certificate

Common Set-up Steps

Before proceeding with any set-up steps, you must review the information in Appendix A (Current Limitations and Considerations). If you have read this, you understand the following:

* You will be using two different certificates, one for the Oracle HTTP Server and one for the Oracle Forms 6i Server.
* If you are not on Oracle Forms version 6.0.8.10.3, at least one of your certificates (for Oracle Forms) will be limited to a size of 512 bits.
* If you have not upgraded the version of JInitiator 1.1.8.3, the initial download of jar files for Forms (Core) Applications files will be over the non-SSL Web server port.

For testing purposes, Verisign will allow your organization to apply for multiple free trial certificates. Each time you generate a certificate request (CSR), however, you must supply an Organizational Unit Name that is different. For example, to trial two certificates for your organization, one for the Oracle HTTP server and one for Oracle Forms server, you could call your Organizational Unit "My Unit - Apache" for the first CSR and "My Unit - Forms" for the second CSR.

The following instructions frequently reference the objects described below. Please set appropriate values for these variables in the environment that you are working in. You MUST use the directory structure and naming conventions contained herein to avoid problems within the SSL infrastructure.
Variable

Value
COMMON_TOP
refers to the common top directory containing the administration scripts with a default name in the form of [SID]comn

SCRIPT_TOP


refers to the directory in the common top directory tree that contains the server administration scripts with a default name in the form of $COMMON_TOP/admin/scripts or $COMMON_TOP/admin/scripts/[SID] in later releases

APACHE_TOP
refers to the installation location for the Oracle HTTP Server (powered by Apache). Before 11i.2, the default for this was $COMMON_TOP/util/apache/1.3.9/Apache, but from 11i.2 onwards this is held within the iAS Oracle Home with a default name in the form of ../[SID]ora/iAS/Apache
OPENSSL_TOP
refers to the open_ssl directory which will be found in the $APACHE_TOP directory
OPENSSL_CONF
refers to the openssl.cnf file which will be found in the $OPENSSL_TOP/bin directory


Option 2.1. Certificate Provisioning for Oracle HTTP Server

Follow the steps listed below to provision the necessary private key and certificate files for the SSL implementation for the Oracle HTTP Server.
Create the certificate using openssl

1. Create a working directory

1. Create a working directory that will eventually hold the private key file and certificate files. Listed below are commands for Unix.

% mkdir $COMMON_TOP/admin/certs

% mkdir $COMMON_TOP/admin/certs/apache

In an AutoConfig-enabled environment, the above directory is the value pointed to by the context variable 's_web_ssl_directory'.
2. Create two other subdirectories under the above directory.

% mkdir $COMMON_TOP/admin/certs/apache/ssl.crt

% mkdir $COMMON_TOP/admin/certs/apache/ssl.key

The above directories hold the certificate file and private key, respectively.

2. Create a random number file

The value of the RANDFILE parameter in the $OPENSSL_CONF file is the file that is used as a source of random characters when generating the private key. On UNIX the default value is:

RANDFILE = $ENV::HOME/.rnd

If you want to use this file to assist with the encryption algorithm, you should ensure that this file exists in the specified location.

If this file does not already exist, follow the steps listed in the example below to create it:

1. Change to a directory that contains some binary files such as ORACLE_HOME/bin of your Oracle 8.0.6 installation:

% cd $ORACLE_HOME/bin
2. Execute the following command to generate the random character file:

% $OPENSSL_TOP/bin/openssl sha1 or* > $HOME/.rnd

3. Create the Server PEM-encoded Private Key file (apache_1024.key)

1. Change to the working directory:

% cd $COMMON_TOP/admin/certs/apache
2. Type the following command to generate your RSA private key (apache_1024.key):

% $OPENSSL_TOP/bin/openssl genrsa -des3 -out apache_1024.key 1024

You will be prompted to enter the PEM pass phrase, which must be at least 4 characters in length. Remember this pass phrase; unless you decide to unencrypt your server key using step 4 below, you will be prompted to enter in the next step and each time you start up Apache in SSL mode.

4. Optionally, unencrypt the value of the Server PEM-encoded Private Key file

If this is done, you will not be prompted for the PEM pass phrase every time you start up the Oracle HTTP server (Note that if anyone gets this key they will be able to impersonate you on the net, so consider leaving it encrypted and entering the PEM pass phrase each time you start the Oracle HTTP server).

Copy the private key (apache_1024.key) that you created in step 3 above to a backup file (apache_1024.key.bak) then type the following command to invoke the RSA key processing tool:

% $OPENSSL_TOP/bin/openssl rsa -in apache_1024.key.bak -out apache_1024.key

Note on Windows NT/2000 you must perform this step. This limitation will be fixed in Apache 2.0.

5. Edit the default openssl configuration file

Edit the default openssl configuration file, $OPENSSL_CONF, so that it can be used to generate the certificate signing request.

1. Locate the line that reads dir = ./demoCA and modify this line to point to the full directory path where the directory of your key and request file are located (the full path to $COMMON_TOP/admin/certs/apache as created in step 1 above).
2. On NT, comment out the lines that set the RANDFILE, oid_file and oid_section variables:

#RANDFILE =$ENV::HOME/.rnd
#oid_file=$ENV::HOME/.oid
#oid_section=new_oids

6. Generate the certificate signing request (CSR)

From the working directory, $COMMON_TOP/admin/certs/apache, type the following command to generate your CSR (apache_1024.csr) derived from the private key (apache_1024.key).

% $OPENSSL_TOP/bin/openssl req -config $OPENSSL_CONF -new -key apache_1024.key -out apache_1024.csr

You will be prompted for the following:

1. Enter PEM pass phrase: Enter the pass phrase chosen in step 4.
2. Country Name (2 letter code) [AU]:

State or Province Name (full name) [Some-State]:
Locality Name (e.g., city) []:
Organization Name (e.g., company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (e.g., section) []:

3. Common Name (e.g., YOUR name) []: Enter the fully qualified name of your server.
For example: orl-sun.oracle.com
4. E-mail Address []: Enter your e-mail address. This is where the certificate will be sent.
5. Please enter the following 'extra' attributes: These questions are optional.

Now the working directory contains a private key (apache_1024.key) and a certificate request (apache_1024.csr). Both are text files containing encrypted information and can be viewed. The contents of certificate request be used in the next step.

7. Submit your certificate request (CSR) to your certificate authority (CA)

The request process is generally explained in detail by your CA of choice. Go to the web site of your CA to follow their specific process for requesting a certificate. This document was written based on experience requesting a trial certificate from Verisign.

Go to www.verisign.com, to request a trial certificate. From Verisign's home page, there should be a link for getting a trial certificate (if this is not obvious, search on "trial certificate").

The Verisign web site will lead you through the steps of requesting your certificate. When asked to submit your CSR, paste the contents of your certificate request (apache_1024.csr) beginning with and including

-----BEGIN CERTIFICATE REQUEST-----

and ending with and including

-----END CERTIFICATE REQUEST-----

Click Submit, then enter the additional requested information, including your e-mail address again for the certificate. You should receive the certificate via e-mail within a couple of hours.

8. Create the Server PEM-encoded Certificate file

Once you have received your certificate, use your e-mail application to save the entire e-mail message to a text file under $COMMON_TOP/admin/certs/apache.

Name the file apache_1024.crt and ensure that it is located under $COMMON_TOP/admin/certs/apache.

It is very important to do this so that you do not corrupt your certificate by cutting and pasting from your e-mail application. In addition, you must use a text editor, rather than a word processing application for your editions to the resulting text file.

If you have saved your e-mail message to a machine that is not your server, transfer the file over to the server and into the working directory ($COMMON_TOP/admin/certs/apache), for example, using ftp in binary mode.

In the working directory, open the file in a text editor and delete all the lines except for the certificate. The certificate consists of all lines beginning with and including

-----BEGIN CERTIFICATE-----

and ending with and including

-----END CERTIFICATE-----

Once your changes are made, save the file with the name apache_1024.crt.

At this stage, if you are using a global server certificate, you will also have to retrieve the intermediate certificate which should be accessible via a URL in the e-mail that you receive containing your certificate. Transfer the file over to the server and into the working directory ($COMMON_TOP/admin/certs/apache), for example, using ftp in binary mode.

Save this intermediate certificate in a file named intermediate.crt.

9. Create the file of PEM-encoded Server Certificates (ca.crt)

1. Change directory to the working directory:

% cd $COMMON_TOP/admin/certs/apache
2. Backup the existing file, if any, under $COMMON_TOP/admin/certs/apache/ssl.crt:

% cp $COMMON_TOP/admin/certs/apache/ssl.crt/ca.crt $COMMON_TOP/admin/certs/apache/ssl.crt/ca.crt.bak
3. Use FTP (in binary mode) to transfer apache_1024.crt to your PC and open it with Internet Explorer.
4. On the Certification Path tab click on the first line and then View Certificate. This will open a new window showing the certificate for the root Certifying Authority (CA).
5. Select the Details tab in the new window. Click Copy to File to start the export wizard.
6. Click Next to continue.
7. Select Base-64 encoded X.509 (.CER) and click next.
8. Enter ca1 as the name and click ok to export the certificate. After the certificate is exported, you can close this window.
9. Repeat steps d through h for each line on the Certification Path tab. Incrementing the file name each time by 1, i.e. ca2, ca3.

NOTE:

It is important that you do the lines in order. You will not be able to view the certificate for the last line. In most cases there will be only 1 certificate for you to view.

10. Close the wizard and IE.
11. Use FTP (in binary mode) to transfer the files you created back to the server under $COMMON_TOP/admin/certs/apache.
12. On the server under $COMMON_TOP/admin/certs/apache, concatenate the files in reverse order (ca2.cer, ca1.cer) and save as ca.crt

NOTE:

The first certificate in the file will be for the CA signing the server certificate. The next certificate is for the CA who issued the first certificate, and so on down to the last certificate which is of the root CA. When there is only 1 certificate created in step h, it means that the root CA was the one who signed the server certificate.

10. Copy server key and certificate file to final destination

1. Change directory to the working directory:

% cd %COMMON_TOP/admin/certs/apache
2. Copy the certificate file to $COMMON_TOP/admin/certs/ssl.crt with a different file name:

% cp apache_1024.crt %COMMON_TOP/admin/certs/apache/ssl.crt/server.crt
3. Copy the private key file to $COMMON_TOP/admin/certs/ssl.key with a different name:

% cp apache_1024.key %COMMON_TOP/admin/certs/apache/ssl.key/server.key
4. Copy the concatenated PEM-encoded CA Certificate file to $COMMON_TOP/admin/certs/apache/ssl.crt:

% cp $APACHE_TOP/Apache/conf/ssl.crt/ca-bundle.crt $COMMON_TOP/admin/certs/apache/ssl.crt/ca-bundle.crt

NOTE: The ca-bundle.crt contains the Root Certificates for Certifying Authorities. You should download and apply patch 5946797 which will update your ca-bundle.crt to the latest version. This will help to avoid issues caused by expiring certificates.

5. Copy the PEM-encoded Server Certificate Chain file to $COMMON_TOP/admin/certs/apache/ssl.crt

% cp ca.crt $COMMON_TOP/admin/certs/apache/ssl.crt/ca.crt

1. If you received the file of global server certificates, copy this file to $COMMON_TOP/admin/certs/apache/ssl.crt

% cp intermediate.crt $COMMON_TOP/admin/certs/apache/ssl.crt/intermediate.crt

Multiple Web Server Nodes

If you have multiple Web Server nodes where each node contains an Oracle HTTP Server, you must request a unique certificate for each machine. If you try to use the same certificate on all machines, you will see the following type of error in the ssl_engine_log:

[11/Dec/2000 22:25:48 01283] [warn] Init: (sundial:443) RSA server certificate
CommonName (CN) `otclnx1.us.oracle.com' does NOT match server name! ?

To use a unique certificate on each machine, perform all of the implementation steps in this chapter for each Web Server node running in HTTPS mode.

Web Server Wallet (Optional)

If you will be connecting to a LDAP server (e.g. OID) from the iAS 1.0.2.2.2 Oracle Home and that LDAP server is SSL enabled you will need to create a webserver wallet to successfully connect to the LDAP server. Oracle Wallet Manger includes the certificates for the most common CA's every time a new wallet is created. If you are not able to connect to your ldap server after creating this wallet refer to the section below titled "Working with Unrecognized CA Root Certificate".

1. Navigate the to iAS Oracle Home (cd $IAS_ORACLE_HOME)
2. Source the env file.
3. Be sure your DISPLAY environment variable is pointing to a valid display.
4. cd $IAS_ORACLE_HOME/Apache/Apache/conf
5. For LINUX only - set the environment variable THREADS_FLAG=native. (export THREADS_FLAG=native)
6. Launch Oracle Wallet Manager: owm&
1. Select Wallet > New
2. Select No when prompted with the message: "Your default directory doesn't exist. Do you wish to create it now?"
3. In the New Wallet dialogue box, create a Wallet Password and click OK. (Make sure you remember this password. You will be prompted for the password each time you open this wallet with Oracle Wallet Manager).
4. Select No when prompted with the message: "A new empty wallet has been created. Do you wish to create a certificate request at this time?"
5. Select Wallet and ensure that the Auto Login checkbox is checked.
6. Select Wallet > Save and save the new wallet in $IAS_ORACLE_HOME/Apache/Apache/conf.
7. Exit Wallet Manager
You should now see 2 new files: ewallet.der and cwallet.sso


Option 2.2. Certificate Provisioning for Oracle Forms 6i Server

Starting with Forms / Reports 6i patchset 4 the Forms Listener Servlet is introduced as an alternative architecture to the original Forms Listener Server. Running the Forms Listener Servlet, you still have the f60webmx processes, but they are now managed by a servlet rather than a dedicated listener process. All communication between the client PC and the forms server is routed via the apache port and forms servlet eliminating the need for the additional certificate required by the Forms Listener Server architecture. HTTPS Users wishing to use J2SE 1.4.2.x must use the Forms Listener Servlet. See Appendix C for more information on the Forms Listener Servlet.

The following steps are necessary only if you are using the Forms Listener Server

Step 2.2.1. Use Wallet Manager to create certificate

In order for the forms server to run in https mode, you must use the Oracle Wallet Manager utility delivered with Developer 6i. Developer 6i is installed in the Oracle 8.0.6 Oracle Home on your application server. All references to ORACLE_HOME in this Part refer to the home for Oracle 8.0.6 on the application server.

1. Create a working directory

Create a working directory to hold the Oracle Forms 6i Wallet:

% mkdir $COMMON_TOP/admin/certs/forms

2. Start Wallet Manager

On UNIX, as the oracle user,

1. Change to the directory of ORACLE_HOME of your Oracle 8.0.6 installation

% cd $ORACLE_HOME
2. Source the environment file [sid].env.
3. Set the DISPLAY variable to display to the IP address from where you are working. For example:

% export DISPLAY=138. 22.155.16:0.0

where 138.22.155.16 is the IP address of your workstation.
4. For LINUX only - set the environment variable THREADS_FLAG=native. For example:

% export THREADS_FLAG=native
5. Run owm, which is located in $ORACLE_HOME/bin directory of your Oracle 8.0.6 installation.

On Windows NT/2000, run Oracle Wallet Manager using Start > Programs > Oracle for Windows NT > Oracle Wallet Manager

3. Create a new wallet and certificate request (CSR)

NOTE: When you create a new wallet in Wallet Manager, this step of creating a CSR is the only way to enable the function of importing a User Certificate. (Operations > Import User Certificate). If a CSR has not been generated within a specific wallet, the option to import a User Certificate will remain disabled. (See comments in Appendix A, Incompatibility of Apache / Oracle Forms 6i Certificate formats).

Follow the steps listed below to create a new wallet and certificate request (CSR):

1. Select Wallet > New
2. Select No when prompted with the message:

"Your default directory doesn't exist. Do you wish to create it now?"
3. In the New Wallet dialogue box, create a Wallet Password and click OK.

Make sure you remember this password. You will be prompted for the password each time you open this wallet with Oracle Wallet Manager.
4. Select Yes when prompted with the message:

"A new empty wallet has been created. Do you wish to create a certificate request at this time?"
5. Fill out the fields in the Create Certificate Request dialogue box, for example:

Common Name: Fully qualified host name of your server
Organizational Unit:
Organization:
Locality/City:
State/Province:
Country:
Key Size: 512

Note that as discussed in Appendix A (Current Limitations & Considerations), if you are not on Oracle Forms version 6.0.8.10.3 or higher, the list of values for Key Size will only contain 512 bits.

6. Select OK when prompted with the message:

"A certificate was created. Please submit this to a Certificate Authority or you can use "Export Certificate Request" to export it into a file."

4. Select Wallet and ensure that the Auto Login checkbox is checked

This ensures that the wallet is open to connect via SSL.

5. Save the new wallet

Select Wallet > Save to save the new wallet.

Save the wallet files to the working directory chosen for the wallet, for example:

% $COMMON_TOP/admin/certs/forms

(Use the value of $COMMON_TOP, rather than the variable.)

Two new files will be created in your wallet directory:

cwallet.sso -- Auto Login is checked; wallet is open
ewallet.der -- stores your certificate once imported

6. Submit your certificate request (CSR) to your certificate authority(CA)

Within the Wallet Manager navigator window, highlight the Certificate: [Requested] branch of the Wallet. On the right hand side you will see your CSR in the Certificate Request window. This is the window you will cut and paste from when prompted by your CA.

The request process is generally explained in detail by your CA of choice. Go to the web site of your CA to follow their specific process for requesting a certificate. This document was written based on experience requesting a trial certificate from Verisign. The process for requesting a production certificate is almost identical.

To request a trial certificate go to www.verisign.com. There should be a link for getting a trial certificate (if this is not obvious, search on "trial certificate").

The Verisign web site will lead you through the steps for requesting your certificate. When asked to submit your CSR, paste the contents of your CSR from Wallet Manager beginning with and including

-----BEGIN CERTIFICATE REQUEST-----

and ending with and including

-----END CERTIFICATE REQUEST-----

Click Submit.

If you have created the certificate request with a 512 bit key size, Verisign will warn you that it has detected a 512 bit key size. As explained in Appendix A (Current Limitations & Considerations), the 512 key size is the only option in Release 11i for Forms 6i version below 6.0.8.10.3.

Click Continue, then enter the additional requested information, including your e-mail address again for the certificate. You should receive the certificate via e-mail within a couple of hours.

7. Create the certificate file

Once you have received your certificate, use your e-mail application to save the entire e-mail message to a text file. Based on the convention in this document, name the file forms_[key_size].crt, where [key_size] is the number of bits chosen in Wallet Manager when creating the certificate request. As an example, forms_1024.crt, if 1024-bit key size was chosen. It is very important to save the message as a text file so that you do not corrupt your certificate by cutting and pasting from your e-mail application. In addition, you must use a text editor, rather than a word processing application for your editions to the resulting text file.

If you have saved your e-mail message to a machine that is not your server, transfer the file over to the server and into the working directory ($COMMON_TOP/admin/certs/forms) , for example using binary ftp.

In the working directory, open the file in a text editor (for example, vi) and delete all the lines except for the certificate. The certificate consists of all lines beginning with and including

-----BEGIN CERTIFICATE-----

and ending with and including

-----END CERTIFICATE-----

Once your editions are made, save the file, again with the name forms_[key_size].crt, for example, forms_1024.crt.

8. Import your User Certificate into Wallet Manager

Back in Wallet Manager,

1. Operations > Import User Certificate
2. Select the radio button for

Select a file that contains the certificate and click OK
3. Enter the full path of your working directory where you have stored your certificate file (forms_[key_size].crt), for example:

/u08/app/oracle/viscomn/admin/certs/forms

(Assuming that COMMON_TOP is /u08/app/oracle/viscomn).
4.

Highlight the certificate file (forms_[key_size].crt) and click OK.
5. The bottom status bar of Wallet Manager should display the message

Your certificate has been successfully imported.
6. In the navigator window under Wallet, you can highlight the Certificate: [Ready]

to see the unencrypted values of your certificate.

NOTE: The following error message when attempting to import the User Certificate indicates that you are working with an unrecognized CA root certificate:

User certificate import has failed because the CA certificate doesn't exist.
Do you want to import a CA certificate now?

This error occurs when Wallet Manager attempts to import your User Certificate, but does not recognize the digital signature of the CA who issued this certificate to you. Wallet Manager then offers you the option to import the CA certificate under the "Trusted Certificates" branch of this Wallet, so that it can finish the proper import of your User Certificate. This message will be received when using Trial Certificates and if you have chosen a Certifying Authority whose root certificate wasn't included as one of the Oracle Wallet Manager default root certificates.

At this point, it is best to leave the error message above open on the terminal and continue with the installation of the CA root certificate that is a pre-requisite of importing the user certificate in Oracle Wallet Manager by following the certificate authority?s instructions to obtain an X.509 format *.cer file (as documented in Section Working with Unrecognized CA Root Certificate / Phase 1 - Web browser below) then using it to import the digital certificate of your CA as trusted to successfully finish the import of your User Certificate (as documented in Section Working with Unrecognized CA Root Certificate / Phase 2- Oracle Forms, steps 1c onwards)

The bottom status bar of Wallet Manager should display the message:

Your certificate has been successfully imported.

You should now be able to highlight Certificate: [Ready] in the navigator window under Wallet to see the unencrypted values of your certificate.
2.2.2 Multiple Forms Server nodes

If you have multiple Forms Server nodes, you can request a unique certificate for each node, or you can use the same certificate on all the nodes.
Unique Certificate for each Forms Server node

To use a unique certificate on each machine, perform all of the implementation tasks in Step 2.2.1 for each Forms Server node running in HTTPS mode.
Identical Certificate for all Forms Server nodes

To use the same certificate on all the Forms server nodes, perform all of the procedures in Step 2.2.1 on one of the Forms Server machines to create a wallet that contains a certificate. Then, copy the wallet file, ewallet.der, to the other Forms Server machines running in HTTPS mode. Copy the file to the directory specified in the FORMS60_WALLET environment variable. Use this wallet to complete Chapter Two on all additional machines. Be sure that Auto Login is set to ON for all machines.


Option 2.3. Certificate Provisioning for XML Publisher or Business Intelligence Publisher

If you are using XML or BI Publisher you will need to install your server certificate into the cacerts file on the Concurrent Manager (CCM) node. This step is necessary whether you are using demo, self-signed, or production certificates.

1. FTP the HTTP server certificate server.crt and ca.crt ( from Option 2.1 step 10) to the CCM node.
2. Source your envirorment file.
3. Copy server.crt to the $OA_JRE_TOP directory.
4. Change to the $OA_JRE_TOP directory and issue the following commands:
./jre/bin/keytool -import -alias ApacheCA -file ./ca.crt -trustcacerts -v -keystore $OA_JRE_TOP/jre/lib/security/cacerts
./jre/bin/keytool -import -alias ApacheServer -file ./server.crt -trustcacerts -v -keystore $OA_JRE_TOP/jre/lib/security/cacerts

For the password enter -> changeit
Trust this certificate? -> Yes
5. Bounce the CCM node for the change to the cacerts file to take effect.

Option 2.4. Certificate Provisioning for Oracle Database Server

Oracle products such as Oracle Configurator, Order Management, iStore, Order Capture, Quoting, iPayment, iStore, and Pricing rely on the Oracle Wallet to establish a successful connection in SSL mode from the Database tier. This section contains instructions for the SSL Set-up for Oracle Database Server using the Oracle Wallet Manager.
Use Wallet Manager to create the wallet on the database server

E-Business Suite 11i10 customers using the Oracle Configurator batch validation feature (Order Management, iStore, Order Capture and Quoting) as well as all E-Business Suite 11i customers using iPayment (since 11i9) and Pricing need to create a wallet on the database tier containing the certificate for the Certifying Authority (CA) who signed the Middle Tier's server certificate and have auto login enabled.

This section contains instructions to modify configuration files and profiles that may be maintained by the AutoConfig infrastructure.

(Note: MetaLink Note 165195.1 explains how to determine if you are AutoConfig enabled).

1. Apply any pre-requisite patches

If your E-Business Suite system is 11.5.9 or below and you are using AutoConfig to manage your system, apply the Techstack Advanced Utilities patch (bug ref 2864765) and any pre-requisite patches.

If your E-Business Suite system is 11.5.9 or below and you are not using AutoConfig, you will need to apply the patch for bug ref 3797160 and any pre-requisite patches.

These patches are shipped with 11.5.10.

Patch No Description Comments
3797160 Provides new profile definition for FND_DB_WALLET_DIR for non AutoConfig-enabled
2864765 Advanced Utilities Patch for AutoConfig-enabled

2. Create a wallet directory on the database tier

Log on to the database tier as the user that owns the oracle files

Source the environment on the database tier.

Create a directory under $ORACLE_HOME/appsutil to hold the new wallet using the following command:

% $ORACLE_HOME/appsutil/wallet

3. Start Wallet Manager

On UNIX, as the oracle user on the database-tier node,

1. Change to the wallet directory that you have just created.

% cd $ORACLE_HOME/appsutil/wallet
2. Ensure that your environment is correctly sourced.
3. Set the DISPLAY variable to display to the IP address from where you are working. For example:

% export DISPLAY=138. 22.155.16:0.0

where 138.22.155.16 is the IP address of your workstation.
4. For LINUX only - set the environment variable THREADS_FLAG=native. For example:

% export THREADS_FLAG=native
5. Run owm, which is located under the $ORACLE_HOME/bin directory of the Database tier. For example:

% $ORACLE_HOME/bin/owm &
6. On Windows NT/2000, run Oracle Wallet Manager using Start > Programs > Oracle for Windows NT > Oracle Wallet Manager

or Start > Programs > Oracle for Windows NT > Integrated Management Tools > Wallet Manager.

4. Create a new wallet

1. On the Oracle Wallet Manger Menu:

Navigate to Wallet -> New.

2. Select No when prompted with the message:

"Your default directory doesn't exist. Do you wish to create it now?"

3. In the New Wallet dialogue box, create a Wallet Password and click OK.

Make sure you remember this password. You will be prompted for the password each time you open this wallet with Oracle Wallet Manager.

4. Select No when prompted with the message:

"A new empty wallet has been created. Do you wish to create a certificate request at this time?"

Oracle Wallet Manger includes the certificates for the most common CA's every time a new wallet is created. If the CA who signed your server certificate is not in the default listing follow these instructions to include CA Certificates:

1. Access your web page.
2. Double-click on the padlock at the bottom of the page to view the Certificates.

NOTE:

if there is no padlock, then on the top toolbar select File->Properties->Certificates

3. Click on the Certification Path Tab.
4. For each certificate listed:

Click View Certificate

Click Details Tab

Click Copy to File - follow directions and export in the Base 64 encoded X.509 (.CER) format.

5. FTP these files in binary mode to the db tier and import into your wallet.

5. Be sure the AutoLogin feature is checked.

When the AutoLogin feature is checked the wallet can be accessed by OS processes (i.e. sqlplus) that are owned by the same owner who created the wallet. To open the wallet using Oracle Wallet Manager a password will still be required.

NOTE: On Windows NT/2000, the wallet files must be generated and owned by the same user that starts up both the Database and Listener services.

6. Save the wallet

Save the wallet in $ORACLE_HOME/appsutil/wallet and exit Wallet Manager.


Continue with the configuration

After completing the above steps successfully, proceed to one of the following chapters to complete the configuration:

* If you are using AutoConfig to manage your system, proceed to Chapter 3.3 - Configuring SSL for AutoConfig-enabled System / Configuring SSL with Oracle Database Server for AutoConfig-enabled environment.
* If you are not using AutoConfig to manage your system, proceed to Chapter 4.3 - Configuring SSL for non AutoConfig-enabled System / Configuring SSL with Oracle Database Server for non AutoConfig-enabled environment.


Working with Unrecognized CA Root Certificate

Your certificate is digitally signed by your CA. If your browser & JInitiator recognize the CA's signature, they will accept the secure communication. For practical purposes, you can think of the CA's digital signature as the CA root certificate. A particular CA can have more than one digital signature or CA root certificate. For example, Verisign's trial certificates are digitally signed differently than their permanent certificates. In other words the CA root for Verisign's trial certificates is different from the CA root for Verisign's permanent certificates.

Note: This Section is only necessary if you have created a Verisign Trial certificate or you are otherwise using a certificate authority root that is not already contained in the database of trusted CA roots in your browser or in JInitiator. For permanent certificates with the most recognized CA's, no client side steps are necessary. These steps will be necessary if you were prompted " Do you want to import a CA certificate now?" when importing your user certificate into the Oracle Wallet Manager.
Phase 1 - Web Browsers

Typically your CA will give you instructions for installing the necessary CA root in your browser, if it is not commonly contained in MS Internet Explorer's (MSIE's) or Netscape's seeded database of trusted CA root certificates. This document was written based on experience using a trial certificate from Verisign, which required that the trial CA root be installed in each browser and JInitiator.

In the e-mail that is received from Verisign, it will specify that you must install the Test CA Root in each browser you plan to use as part of your test of SSL Trial SSL Server ID. The URL from which to complete this step will be given in e-mail.

Currently you must use MS Internet Explorer (MSIE) in order to complete the task of installing a trial or non-recognized CA root certificate for JInitiator. This is because Netscape does not have the ability of exporting trusted CA Root certificates. Therefore, if your preferred browser is Netscape, you will need to install the CA root into Netscape as your preferred browser and also into MSIE for the purpose of exporting the CA root certificate in the format needed by JInitiator.

1. Install in the browser

Using your browser, go to the URL that your CA instructs for installing it's root certificate. For Verisign, the page describes how to install the Test CA Root for either browser Netscape or MSIE. Typically the CA will have a link that you click to install the CA root in your browser. Follow the instructions for the browser you are currently using. The install may go very smoothly but if not, see the step below for the specific browser you are currently working with.

a. Netscape

Currently Netscape only allows you to view, install or delete trusted CA root certificates.

When installing the CA root from the CA's web site, you should have no trouble in just clicking on the link instructed by your CA. Netscape's New Certificate Authority Wizard will spawn and lead you through the process. When asked for the purpose of accepting this CA, select the option for Certifying Network Sites.

- You can give it a friendly name, for example, "Verisign Test CA Root"

To view the trusted CA roots in the Netscape's database, select Tools > Security Info. In the navigator tree under Certificates, select Signers. The list of trusted CA root certificates appears in the window.

b. Microsoft Internet Explorer (MSIE)

MSIE allows you to view, import, export and delete trusted CA root certificates.

When installing the CA root from the CA's web site (eg http://www.verisign.com/server/trial/faq/index.html), you may see a File Download dialogue box when you click on the link instructed by your CA. Go ahead and save to a file with the following specifications:

File name: CA_root
Save as type: All Files (*.*)

To work with trusted CA roots in the MSIE database, select Tools > Internet Options. Click Content tab. In the Certificates block, click the Certificates button, then Click the Trusted Root Certification Authorities. This is the MSIE Certificate Manager. From here you can import the file you saved as CA_root into the database.

Click the import button to allow the Certificate Manager Import Wizard to lead you through selecting the file you saved from the CA web site (CA_root) for import.

This process, documented in full, is as follows:

From Microsoft Internet Explorer 5.0:

* Click on the Accept Button at the top of the page to accept the conditions of use
* A Dialog Box will appear that says "You have chosen to download a file from this location: "getcacert from digitalid.verisign.com" Click the option that says "Save this file to Disk." Click OK
* Select the location of the file, leave name and file type to default
* Prompt will appear: Download Complete, click Close.
* Open IE 5.0 Browser
* Go to Tools/Internet Options/ Content/ Certificates (trusted root authorities tab)/ Import
* Certificate Manager Import Wizard: click Next
* Browse to the location of the recently stored root (done in step 3), select ALL files for file type.
* Select "getcacert" and click open
* Click Next
* Select "place all certificates in the following store" option
* Highlight "Trusted Root Certification Authorities" and click OK
* Click Next
* Click Finish
* User will be prompted and asked if they wish to add the following certificate to the root store, click Yes.
* To Verify the installation of the "Intermediate Root CA" into your browser, go into:
* Go to Tools/Internet Options/ Content/ Certificates (trusted root authorities tab) It should read as "For VeriSign Authorized Testing Only".

2. Export the certificate for use with Oracle Forms components

Now that you have the CA root certificate installed in MSIE, you can export it in a format that is recognized by Wallet Manager & JInitiator.

In MSIE,

* Select Open the Certificate Manager, then Tools > Internet Options
* Click on the Content tab
* Click on the Certificates button in the Certificates block
* Click on the Trusted Root Certification Authorities tab
* Highlight the entry for your CA's root certificate
* Click on the export button to allow the Certificate Manager Export Wizard to lead you through the export.
* Select Base64 encoded X.509(.CER) as the format you want to export
* Use the browse tab to select an appropriate directory to save the file with an extension of .cer as its name. root for example, CA_root.cer
* Transfer the CA_root.cer file over to the server and into the working directory ($COMMON_TOP/admin/certs/forms), for example using ftp.

The Base64 encoded CA root certificate will be used in the following Part for Wallet Manager.

Phase 2 - Oracle Forms

The Wallet Manager error message:

User certificate import has failed because the CA certificate doesn't exist.
Do you want to import a CA certificate now?

received when importing your user certificate in Section 2.2.1 (Certificate Provisioning for Oracle Forms 6i Server - Use Wallet Manager to create certificate) above indicates that the user certificate is a trial certificate or that you are using a certificate for which the signature is not already contained in Wallet Manager's database of trusted CA root certificates. If you did not complete the steps in Section 2.2.1, you must do so now. Once these steps are complete, continue the steps below to successfully complete the import of your user certificate into Wallet Manager.

1. Import CA root certificate into Wallet Manager's database

If you have exited Wallet Manager since attempting Section 2.2.1 (Certificate Provisioning for Oracle Forms 6i Server- Use Wallet Manager to create certificate) above, you will need to start up Wallet Manager again and open the wallet which you saved earlier:

a. Start Wallet Manager.

On UNIX, as the oracle user, change to the directory of your Oracle 8.0.6 $ORACLE_HOME and source the environment file [sid].env. Set the UNIX environment variable DISPLAY for Wallet Manager to display to the IP address from where you are working. For example:

$ export DISPLAY=138.22.155.16:0.0

where 138.22.155.16 is the IP address of your workstation.

Run owm, which is located in the 8.0.6 $ORACLE_HOME/bin directory.

On Windows NT/2000, run Oracle Wallet Manager using Start > Programs > Oracle for Windows NT > Oracle Wallet Manager

b. Open the wallet you saved in Section 2.2.1, Step 4.

Select Wallet > Open

Select Yes when prompted with the message:

"Your default directory doesn't exist. Do you wish to continue?"

In the Select A Directory dialogue box, navigate to the directory name you chose for your wallet, for example:

$COMMON_TOP/admin/certs/forms

(Use the value of $COMMON_TOP, rather than the variable.)

In the Open Wallet dialogue box, enter your wallet password and click OK.

c. Attempt to import your user certificate into this wallet again.

Operations > Import User Certificate

Select the radio button for

Select a file that contains the certificate and click OK.

Enter the full path of your working directory where you have stored your certificate (forms_[key_size].crt), for example:

$COMMON_TOP/admin/certs/forms

(Use the value of $COMMON_TOP, rather than the variable.)

Highlight the certificate file (forms_[key_size].crt) and click OK.

Answer yes to Wallet Manager's question:

User certificate import has failed because the CA certificate doesn't exist.
Do you want to import a CA certificate now?

Select the option:

Select a file that contains the certificate

Enter the path and filename for the CA root certificate, for example:

$COMMON_TOP/admin/certs/forms/CA_root.cer

(Use the value of $COMMON_TOP, rather than the variable.)

Click OK to finish the import of the CA root certificate as well as your user certificate.

The bottom status bar of Wallet Manager should display the message:

Your certificate has been successfully imported.

In the navigator window under Wallet, you can highlight Certificate: [Ready] to see the unencrypted values of your certificate.

2. Update JInitiator's CA root certificate database

Now you must inform JInitiator on the client that there is a new CA root certificate that it should consider as trusted. Follow the steps below to complete this task:

1. Select Operations > Export All Trusted Certificates
2. Select your working directory on the server ($COMMON_TOP/admin/certs/forms)

Specify the file name JInit_certdb.txt

Click OK.

3. Exit Wallet Manager
4. Transfer the file from the server over to the client PC that has JInitiator installed, using binary mode ftp. Even though it is an ASCII file, it is very important that the file be transferred in binary mode for JInitiator to be able to read it properly.
5. On the client PC, backup certdb.txt file for the version of Jinitiator that is being invoked.

For example:

c:\Program Files\Oracle\JInitiator 1.1.8.16\lib\security\certdb.txt

6. Copy the new database file, JInit_certdb.txt into this directory with the file name certdb.txt.

Now JInitiator will recognize your CA root certificate as trusted. This must be done for every client PC. For more information see Metalink Note 147836.1.
3. Continue with the configuration

* If you are using AutoConfig to manage your system, proceed to Section 3.2. Configuring SSL with Oracle Forms 6i Server for AutoConfig-enabled environment
* If you are not using AutoConfig to manage your system, proceed to Section 4.2. Configuring SSL with Oracle Forms 6i Server for non AutoConfig-enabled environment.


Chapter 3. Configuring SSL for AutoConfig-enabled System

This section contains instructions to modify configuration files and profiles of systems that are managed by AutoConfig. If you are not using AutoConfig to manage your system, you must follow the instructions described in Chapter 4. Configuring SSL for non AutoConfig-enabled System.

* Common Configuration Steps
* Option 3.1 . Configuring SSL with Oracle HTTP Server
* Option 3.2 . Configuring SSL with Oracle Forms 6i Server
* Option 3.3 . Configuring SSL with Oracle Database Server


Common Configuration Steps

The following instructions frequently reference the objects described below. Please set appropriate values for these variables in the environment that you are working in.
Variable

Value
COMMON_TOP
refers to the common top directory containing the administration scripts with a default name in the form of [SID]comn

SCRIPT_TOP


refers to the directory in the common top directory tree that contains the server administration scripts with a default name in the form of $COMMON_TOP/admin/scripts or $COMMON_TOP/admin/scripts/[SID] in later releases

APACHE_TOP
refers to the installation location for the Oracle HTTP Server (powered by Apache). Before 11i.2, the default for this was $COMMON_TOP/util/apache/1.3.9/Apache, but from 11i.2 onwards this is held within the iAS Oracle Home with a default name in the form of ../[SID]ora/iAS/Apache
OPENSSL_TOP
refers to the open_ssl directory which will be found in the $APACHE_TOP directory
OPENSSL_CONF
refers to the openssl.cnf file which will be found in the $OPENSSL_TOP/bin directory


Option 3.1 Configuring SSL with Oracle HTTP Server
Prerequisite

A prerequisite of implementing SSL with Oracle HTTP server is to successfully set-up Apache as the single web listener for the Release 11i Applications instance. Although this single listener configuration is the default in 11i.2 and later Rapid installs, maintenance packs do not update the web server configuration. If installation of Oracle Applications Release 11i was originally done using version 11i.1 of Rapid install, then administrators will have to configure Apache as the single web listener for the implementation regardless of the current maintenance pack level. Manual configuration can be performed using the steps in sections I through IV of the MetaLink document: Note 119873.1 Apache Single Listener Configuration for Applications 11.5.1. (Section V of this note contains basic information on the Apache portion of SSL implementation).
Step 3.1.1. Configuring SSL with Oracle HTTP Server using Configuration Wizards

The configuration of SSL with the Oracle HTTP server can be performed by using the Configuration Wizards utility. This utility is available as a command-line interface and also from OAM. This utility is shipped with the E-Business Suite Release 11.5.10.

If you are using 11.5.9 or lower and would like to use this utility, apply the following prerequisite patches to all Web, Forms, and Concurrent Processing nodes:

Patch No Description Comments
3258830 OAM Patchset H About Oracle Applications Manager Mini-pack 11i.OAM.H - Note 258330.1
2864765 Advanced Utilities Patch Running Configuration Wizards from the Command Line in Oracle Applications 11i - Note 277574.1

The Configuration Wizards utility automates the configuration process described in this section. The utility is available as a command-line interface and also from the Oracle Applications Manager (OAM). For additional information on the usage of this utility, please refer to the following documents:

1. Oracle MetaLink Note 277574.1 - "Running Configuration Wizards from the Command Line in Oracle Applications 11i"

2. OAM On-line Help: Help -> Oracle Applications Manager -> System Configuration -> AutoConfig -> Configuration Wizards

After completing the configuration of SSL with Oracle HTTP server using the Configuration wizards utility, run AutoConfig as described in MetaLink Note 165195.1.

Step 3.1.2. Testing

You should now proceed to Chapter 5.1 - Verifying SSL Set-up for Oracle HTTP Server to test the Applications sign-on process.

No more steps are required in the current section 3.1.
Step 3.1.3. Configuring SSL with Oracle HTTP Server

You can skip this step if you already used the Configuration Wizards to configure SSL with Oracle HTTP server as described in step 3.1.1.

If your E-Business Suite 11i instance is managed by AutoConfig, run AutoConfig as described in MetaLink Note 165195.1 or in the AD utilities manual after ensuring that the latest AutoConfig Template Rollup Patch (TXK (FND) Patch O:5478710) has been applied and that the following parameters are correctly set in your context xml file:

1. set the %s_url_protocol variable to https
2. set the %s_local_url_protocol variable to https
3. set the %s_webentryurlprotocol variable to https
4. set the %s_frmConnectMode variable to https
5. set the %s_webssl_port variable to the Apache SSL port required
6. set the %s_active_webport variable to the same value as that for the %s_webssl_port variable
7. set the %s_webport variable to the same value as that for the %s_webssl_port variable
Note: prior to TXK (FND) AutoConfig Template Rollup Patch F (3104607 December 2003) this value was set to the non-ssl Apache Port.
8. set the %s_web_ssl_directory variable to point to the full directory path of the directory that is to contain the .crt and .key files that you are using for Apache eg <$COMMON_TOP>/admin/certs/apache
9. set %s_apps_portal_url variable to https
10. run AutoConfig as described in MetaLink Note 165195.1
11. If Single Sign-On is enabled:
1. Run $OAD_TOP/admin/install//ssodatan.sh apps
2. Re-register the partner application.

Using a SSL Accelerator (Optional)

If either of the following conditions are true:

* Your HTTP load-balancer also acts as a SSL Accelerator
* You use an SSL Accelerator as your Primary Web Entry point which is the initial target of interfacing client communication

Then you must complete the following steps:

1. Set the %s_webentryhost variable to the SSL Accelerator hostname.
2. Set the %s_webentrydomain variable to the SSL Accelerator domain name
3. Set the %s_webentryurlprotocol variable to "https".
4. Set the %s_active_webport variable to the value of the SSL Accelerator's external interfacing port.
5. Set the %s_webssl_port variable to the same value as the "Active Web Port".
6. Set the %s_login_page variable to "https://<'Web Host entry point'>.<'Web domain entry point'>:<'Active Web Port'>"

Step 3.1.4. Testing

You should now proceed to Chapter 5.1 - Verifying SSL Set-up for Oracle HTTP Server to test the Applications sign-on process.


Option 3.2 Configuring SSL with Oracle Forms 6i Server
Step 3.2.1. Configuring SSL with Oracle Forms 6i Server

This section contains instructions to modify configuration files and profiles that may be maintained by the AutoConfig infrastructure. Please note that in order to configure SSL with the Oracle Forms 6i Server you must have already configured SSL for the Oracle HTTP Server..

NOTE: This step is not necessary if you are using the Forms Listener Servlet architecture.

If you are AutoConfig enabled, run AutoConfig as described in MetaLink Note 165195.1 or in the 11i AD Utilities Manual after ensuring that the variables that enable forms to connect in https mode are correctly set in your context xml file.

Note that if you are using version 11.5.7 or earlier of the context xml files and autoconfig, you will require a patch to access these variables as described in bug ref 2340379
Step 3.2.2. Testing

You should now proceed to Option 5.2 - Verifying SSL Set-up for Oracle Forms 6i Server to test the Forms startup and Applications sign-on process.


Option 3.3 Configuring SSL with Oracle Database Server
Step 3.3.1. Required Patches and Prerequisite Steps

If your E-Business Suite system is 11.5.9 or below and you are using AutoConfig to manage your system, apply the following prerequisite patches to all Web, Forms, Admin, and Concurrent Processing nodes. The following patches are shipped with the E-Business Suite Release 11.5.10.

Patch No Description Comments
2864765 Advanced Utilities Patch for AutoConfig-enabled

Before proceeding with the configuration, you must also have completed Chapter 2, Option 2.3 - Certificate Provisioning for Oracle Database Server to create the necessary wallet files.
Step 3.3.2. Configuring SSL with Oracle Database Server
Update the Database Wallet Directory profile option using AutoConfig

Perform the steps listed below to update the Database Wallet Directory profile option value. Note that the Techstack Advanced Utilities Patch Rollup A (bug ref 2864765) and any pre-requisite patches must have been applied as required in step 3.3.1.

1. Follow Oracle MetaLink Note 165195.1 to copy AutoConfig to the RDBMS ORACLE_HOME under the section titled "Section 5: Maintaining System Configurations".
2. Run AutoConfig on the database-tier node as per Oracle MetaLink Note 165195.1.

Step 3.3.3. Testing

You should now proceed to Chapter 5.3 - Verifying SSL Set-up for Oracle Database Server to test the wallet set-up.


Chapter 4. Configuring SSL for non AutoConfig-enabled System

If you are not using AutoConfig to manage your system, please proceed with the manual steps described in this chapter. If you are using AutoConfig to manage your system, you must follow the instructions described in Chapter 3. Configuring SSL for AutoConfig-enabled System.

* Common Configuration Steps
* Option 4.1 . Configuring SSL with Oracle HTTP Server
* Option 4.2 . Configuring SSL with Oracle Forms 6i Server
* Option 4.3 . Configuring SSL with Oracle Database Server
* Additional Configuration Steps


Common Configuration Steps

The following instructions frequently reference the objects described below. Please set appropriate values for these variables in the environment that you are working in.
Variable

Value
COMMON_TOP
refers to the common top directory containing the administration scripts with a default name in the form of [SID]comn

SCRIPT_TOP


refers to the directory in the common top directory tree that contains the server administration scripts with a default name in the form of $COMMON_TOP/admin/scripts or $COMMON_TOP/admin/scripts/[SID] in later releases

APACHE_TOP
refers to the installation location for the Oracle HTTP Server (powered by Apache). Before 11i.2, the default for this was $COMMON_TOP/util/apache/1.3.9/Apache, but from 11i.2 onwards this is held within the iAS Oracle Home with a default name in the form of ../[SID]ora/iAS/Apache
OPENSSL_TOP
refers to the open_ssl directory which will be found in the $APACHE_TOP directory
OPENSSL_CONF
refers to the openssl.cnf file which will be found in the $OPENSSL_TOP/bin directory


Option 4.1 Configuring SSL with Oracle HTTP Server
Prerequisite

A prerequisite of implementing SSL with Oracle HTTP server is to successfully set-up Apache as the single web listener for the Release 11i Applications instance. Although this single listener configuration is the default in 11i.2 and later Rapid installs, maintenance packs do not update the web server configuration. If installation of Oracle Applications Release 11i was originally done using version 11i.1 of Rapid install, then administrators will have to configure Apache as the single web listener for the implementation regardless of the current maintenance pack level. Manual configuration can be performed using the steps in sections I through IV of the MetaLink document: Note 119873.1 Apache Single Listener Configuration for Applications 11.5.1. (Section V of this note contains basic information on the Apache portion of SSL implementation).
Step 4.1.1. Configuring SSL with Oracle HTTP Server

This section contains manual instructions for configuring SSL with Oracle HTTP Server for non AutoConfig-enabled systems.
Determine which configuration file Apache is currently using

By default, E-Business Suite installations that were created using Rapid Install 11i.1 through 11i.5 originally installed Oracle HTTP Server 1.3.9 from iAS 1.0.0. In these environments on unix, either the httpds or the httpd executable may be used. To determine which of these is in use, open the file $APACHE_TOP/Apache/bin/apachectl and check the parameter HTTPD. If HTTPD is pointing to the httpds executable, then httpds.conf is being used as the configuration file, if HTTPD is pointing to the httpd executable, then httpd.conf is being used.

E-Business Suite installations created with Rapid Install 11i.7 and higher versions use the Oracle HTTP Server 1.3.19 from iAS 1.0.2.2. This uses the httpd executable for both SSL and normal http which is based on the parameters that are set in the httpd.conf configuration file. For these versions, the script httpdsctl is the standard script for starting Apache in SSL mode.

Windows NT/2000 does not use the apachectl file. On these platforms, Apache is implemented through use of services. By default the apache service points to the httpd.conf file during startup. This is defined in an Oracle Applications environment through use of the adsvapc.cmd file in the admin\install directory in the Oracle Applications common top directory tree. If you are not sure which configuration file Apache is currently using, open the adsvapc.cmd file and check the value of the -f variable in the call to invoke Apache.exe.
Backup the current configuration file (either httpd.conf or httpds.conf) before making any changes

You should back up the current configuration file: httpd.conf or httpds.conf before making any changes:

$ cd $APACHE_TOP/Apache/conf
$ cp httpd.conf httpd.conf.BAK

or

$ cp httpds.conf httpds.conf.BAK

Edit the SSL directives for Apache in your environment's configuration file

Decide on the port that you want Apache to use for SSL communication.

The standard port for SSL communication is 443, but it need only be a free port on your server. Keep in mind that on UNIX, if you have chosen port 443 (or any port under 1024) for your SSL port, you will have to start Apache as root, otherwise you should connect as the oracle user.

Assuming that COMMON_TOP is /u08/app/oracle/viscomn.

Open httpds.conf in an editor.

a. Find the main section:

### Section 2: 'Main' server configuration

Find the subsection:

## SSL Support

Ensure the block reflects your original non-SSL port for the first Listen directive and the chosen SSL port for the second Listen directive.

For example:


Listen 8800
Listen 443


b. Find the main section:

### Section 3: Virtual Hosts

Ignore the first (commented out) reference to VirtualHost _default_:, but change the second reference to ensure your true choice for the SSL port is reflected in the line beneath the SSL Virtual Host Context Section:



Find the subsection:

## SSL Virtual Host Context

Find and edit the DocumentRoot directive to point to your portal. For example (note the trailing slash):

DocumentRoot "/u08/app/oracle/viscomn/portal/"

Find and edit the SSLCertificateFile directive to point to your server certificate (server.crt). For example:

SSLCertificateFile
/u08/app/oracle/viscomn/admin/certs/apache/ssl.crt/server.crt

Find and edit the SSLCertificateKeyFile directive to point to your private key (server.key). For example:

SSLCertificateKeyFile
/u08/app/oracle/viscomn/admin/certs/apache/ssl.key/server.key

If you are using global server certificates, find and edit the SSLCertificateChainFile directive to point to your intermediate certificate (intermediate.crt). For example:

SSLCertificateChainFile
/u08/app/oracle/viscomn/admin/certs/apache/ssl.crt/intermediate.crt

If you are not using global server certificates, and you previously created the file of PEM-encoded Server Certificates (ca.crt), find and edit the SSLCertificateChainFile directive to point to your file of PEM-encoded Server Certificates (ca.crt). For example:

SSLCertificateChainFile
/u08/app/oracle/viscomn/admin/certs/apache/ssl.crt/ca.crt

On NT, if you are using an earlier version of Oracle Applications than 11i.4, you will also need to change the line that defines SSLMutex file:\Apache\logs\ssl_mutex to read

SSLMutex sem

Save and close the file httpds.conf

Modify the Apache service to start up in SSL mode

On UNIX, to start Apache in SSL mode, the Apache control script (adapcctl.sh) in $COMMON_TOP/admin/scripts should be altered to invoke the httpdsctl script instead of the default apachectl script. Before this change can be made, it is necessary to customize and test the httpdsctl script as follows:

a. Backup the httpdsctl file before making any changes:

$ cd $APACHE_TOP/Apache/bin
$ cp httpdsctl httpdsctl.BAK

b. Transfer the Oracle-specific environment that is set-up in the apachectl script to the httpdsctl script.

The configuration section begins with the line:

# |||||||||||||||||||| START CONFIGURATION SECTION||||||||||||||||||||

and ends with the line:

# |||||||||||||||||||| END CONFIGURATION SECTION||||||||||||||||||||

Many of the lines in the configuration section can be copied from apachectl to httpdsctl, but there are also variables in this section that must be defined differently between starting in SSL vs. NonSSL mode, such as the variables PIDFILE (which should point to httpds.pid in the $APACHE_TOP/Apache/logs directory) and HTTPD (which should point to httpds in the $APACHE_TOP/Apache/bin directory, thus invoking the settings in the httpds.conf file).

(Note that if you are on 11i.1 and have manually converted to using Apache as the single listener, this will look a bit different to a fresh install of 11i.2 or higher).

In 11i.1, by following Note 119873.1 Apache Single Listener Configuration for Applications 11.5.1, you will have added lines to set and to export the following Oracle-specific variables:

ORACLE_HOME
LD_LIBRARY_PATH
SHLIB_PATH
LIBPATH
WV_GATEWAY_CFG
NLS_LANG

And also for 11i.1, by following the instructions in Note 119873.1, the value for ulimit will have been altered as follows:

# Self-Service Applications often need more file handles ulimit -n 1024

ulimit

In 11i.2, apachectl will typically have only the following Oracle-specific environment variables:

ORACLE_HOME
LD_LIBRARY_PATH or SHLIB_PATH or LIBPATH (platform dependent)
WV_GATEWAY_CFG

The important thing is to set and export the Oracle-specific variables in httpdsctl that you know to be working in apachectl.

c. Save the changes that you have made to the httpdsctl script.

Once you have transferred the necessary environment settings from apachectl to httpdsctl and checked your work carefully, save and close the file httpdsctl

d. Test the changes that you have made to the httpdsctl script file

(Note that if you are using a trial certificate or a non-recognized CA root certificate you will need to complete the steps in Section Working with Unrecognized CA Root Certificate / Phase 1 - Web browser before performing this step).

Use the httpdsctl script to start Apache in SSL mode as follows:

$APACHE_TOP/Apache/bin/httpdsctl startssl

If your server key is encrypted, you will be prompted for the PEM pass phrase that you chose when creating your server's private key. Also, if you have chosen the default port 443 (or any other port number less than 1024) for your SSL port, you will have to start Apache as root.

Make sure that Apache can startup successfully before continuing. Ensure that there are spawned processes for Apache through use of the command ps -ef | grep util and that you successfully get an SSL connection to the default Apache banner screen using https:/: where is the fully qualified name of the machine running Apache and is the SSL port number defined in httpds.conf.

e. Replace the call to apachectl with a call to httpdsctl in the control script that starts the Apache server.

First, make a backup of $SCRIPT_TOP/adapcctl.sh

$ cd $SCRIPT_TOP
$ cp adapcctl.sh adapcctl.sh.BAK

Then, edit the file adapcctl.sh in $COMMON_TOP/admin/scripts as follows:

Find the line :

[YOUR_COMMON_TOP]/util/apache/1.3. 9/Apache/Apache/bin/apachectl start

Change "apachectl start" to "httpdsctl startssl", for example:

[YOUR_COMMON_TOP]/util/apache/1.3. 9/Apache/Apache/bin/httpdsctl startssl

Find the line :

[YOUR_COMMON_TOP]/util/apache/1.3. 9/Apache/Apache/bin/apachectl stop

Change "apachectl stop" to "httpdsctl stop", for example:

[YOUR_COMMON_TOP]/util/apache/1.3. 9/Apache/Apache/bin/httpdsctl stop

Save and close the file adapcctl.sh

On Windows NT/2000, the Apache service should be configured to use SSL through use of the adsvapc.cmd file that is shipped with Oracle Applications in the admin\install directory in the Oracle Applications common top directory tree. Follow the instructions below to change the Apache service to run in SSL mode:

f. Make a backup of the adsvapc.cmd file in your \admin\install directory

g. Edit the adsvapc.cmd file as follows to use the httpds.conf file and the SSL definitions contained within it:

Change:

e:\oracle\prodcomn\util\OamkSvc.exe -si "Oracle Apache Server PROD" -e -a -c "e:\oracle\prodora\iAS\Apache\Apache\Apache.exe -f e:\oracle\prodora\iAS\Apache\Apache\conf\httpd.conf"

to

e:\oracle\prodcomn\util\OamkSvc.exe -si "Oracle Apache Server PROD" -e -a -c "e:\oracle\prodora\iAS\Apache\Apache\Apache.exe -D SSL -f e:\oracle\prodora\iAS\Apache\Apache\conf\httpds.conf"

h. Deinstall the existing Apache service.

From a non-sourced command window with the PATH variable unset (eg PATH set to null) run:

adsvapc.cmd -deinstall

(You may get an exiting with status 1 the first time this is run. Rerun it a second time and it should exit with status 0.)

i. Install the modified Apache service.

In the same command window run:

adsvapc.cmd

This should recreate the service with reference to httpds and will also use the SSL portions of that file.

(Note that if you are using a trial certificate or a non-recognized CA root certificate you will need to complete the steps in Section Working with Unrecognized CA Root Certificate / Phase 1 - Web browser before testing your work.

Step 4.1.1. Testing

You should now proceed to Chapter 5.1 - Verifying SSL Set-up for Oracle HTTP Server to test the Applications sign-on process. Once the test result was verified, you must also complete step 4.1.2 below.
Step 4.1.2. Additional Configuration Steps

Once you have determined that the Oracle HTTP server can run in SSL mode and connections to Core Forms and Self-Service Web Applications is successful, you should now proceed to Option 4.4 - Additional Configuration Steps to complete the manual configuration.


Option 4.2 Configuring SSL with Oracle Forms 6i Server

If you are not using AutoConfig to manage your system, please proceed with the manual steps described in this section to complete the configuration of SSL with Oracle Forms 6i Server.

Set the following SSL environment variables for Developer 6i Forms server by defining them somewhere below the setting of the ORACLE_HOME variable in the environment file for your 8.0.6 Oracle Home: $ORACLE_HOME/[sid].env

On Windows NT/2000 platforms, these variables should also be set in the registry under \HK_LOCAL_MACHINE\SOFTWARE\ORACLE using the regedit or regedt32 command.

Variable

Value
FORMS60_WALLET


specifies the directory created for your wallet that holds the certificate used on the server.

For example: $COMMON_TOP/admin/certs/forms

FORMS60_HTTPS_NEGOTIATE_DOWN


specifies whether to allow the Forms Server to negotiate encryption down to the highest level supported by the client (TRUE) or to ensure that all encryption occurs at the level at which it is running (FALSE). This should be set to TRUE.

Step 4.2.1 Configuring SSL with Oracle Forms 6i Server
Edit the FORMS60_WEB_CONFIG_FILE (appsweb.cfg)

The FORMS60_WEB_CONFIG_FILE variable is defined in your $APACHE_TOP/Apache/conf/apps.conf file for Apache to use during the initial connection in order to spawn a forms session. It is defined for Release 11i Apps to be $OA_HTML/bin/appsweb.cfg. This file must be edited to allow forms sessions in https mode.

1. Make a backup of the appsweb.cfg file in the Oracle Applications HTML bin ($OA_HTML/bin) directory tree
2. Edit appsweb.cfg
3. Find the connectMode parameter in the section:

; Forms Client-Server Communication Mode: socket, http, or https

4. Set the connectMode parameter to be https. For example:

connectMode=https

5. Save and close the file appsweb.cfg

Note that this file should always be kept in sync with the version at $FND_TOP/resource/appsweb.cfg. Be sure any changes you make are reflected in both locations.
Amend the forms server process to startup in https mode

In order for the forms server process to startup in https mode on UNIX, you must edit your forms server control script (adfractl.sh, adfroctl.sh or adfrmctl.sh) under $SCRIPT_TOP directory. If you originally did an install of 11i.1, you may be using either adfractl.sh or adfroctl.sh. If your original install was of 11i.2 or above, you are probably using adfrmctl.sh

1. Make a backup of the forms startup script file in use ($SCRIPT_TOP/adfractl.sh, adfroctl.sh or adfrmctl.sh)

For example, where is the name of the forms startup script in use:

$ cp $SCRIPT_TOP/ $SCRIPT_TOP/

2. Edit the and change the command that invokes f60ctl to use mode https, for example

$ vi $SCRIPT_TOP/

3. Alter the line that starts with f60ctl start port=$FORMS_PORT mode=http exe=f60webmx to read

f60ctl start port=$FORMS_PORT mode=https exe=f60webmx >> $FRMLOG 2>/dev/null

4. Save and close the forms startup script file

On Windows NT/2000 platforms the forms server process is implemented through use of services. The forms service should be configured to use https through use of the adsvfrm.cmd file that is shipped with Oracle Applications in the admin\install directory in the Oracle Applications common top directory tree. Follow the instructions below to change the forms service to run in https mode on Windows NT/2000:

1. Make a backup of the adsvfrm.cmd file in your \admin\install directory
2. Edit adsvfrm.cmd file as follows to change the connect mode from socket to https:

Change:

\util\OamkSvc.exe -si OracleFormsServer-Forms60 -e -a -t -c "<8.0.6 ORACLE_HOME>\bin\ifsrv60.exe -listen port=9000 mode=SOCKET log=\admin\log\forms_server.log"

to

\util\OamkSvc.exe -si OracleFormsServer-Forms60 -e -a -t -c "<8.0.6 ORACLE_HOME>\bin\ifsrv60.exe -listen port=9000 mode=HTTPS log=\admin\log\forms_server.log"

3. Deinstall the existing forms service. From a command window run:

adsvfrm.cmd -deinstall

(You may get an exiting with status 1 the first time this is run. Rerun it a second time and it should exit with status 0.)

4. Install the modified forms service. In the same command window run:

adsvfrm.cmd

This should recreate the service using https.

Note: Recreating the service in this way will reset the mode of the service in the registry. You can check the settings using the registry editor by running regedit or regedt32.

For 11i.1, go to the Registry key \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\OracleFormsServerSID, where SID is the name of your database instance. Double-click on Mode value and ensure that it is set to https.

For 11i.2 and later, go to the Registry key \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\OracleFormsServerSID\Application, where SID is the name of your database instance. Double-click on the Parameters value and ensure that the mode is set to https.

Note also that in 11i.1, Mode and Port are separate registry values, but in later releases, they are combined into a single Parameters registry key.
5. Change the ?Log on As? property of the forms service from ?System? to ?This Account? with a Windows NT Administrator user on the host machine using Control Panel > Services. Then restart the service.

Step 4.2.2 Testing

You should now proceed to Option 5.2 - Verifying SSL Set-up for Oracle Forms 6i Server to test the Forms startup and Applications sign-on process.


Option 4.3 Configuring SSL with Oracle Database Server
Step 4.3.1. Required Patches and Prerequisite Steps

If your E-Business Suite system is 11.5.9 or below and you are not using AutoConfig to manage your system, apply the following prerequisite patches to all Web, Forms, Admin, and Concurrent Processing nodes. The following patches are shipped with the E-Business Suite Release 11.5.10.

Patch No Description Comments
3797160 Provides new profile definition for FND_DB_WALLET_DIR for non AutoConfig-enabled

You must also have generated the necessary Wallet files as described in Chapter 2, Option 2.3 - Certificate Provisioning for Oracle Database Server to create the necessary wallet files.
Step 4.3.2. Configuring SSL with Oracle Database Server
Manually update the Database Wallet Directory profile option

You may omit this step if you are using AutoConfig to manage your configuration. If you are not using AutoConfig, you will need to manually update the Database Wallet Directory profile option value that was created by the patch for bug ref 3797160. Perform the steps listed below to update this profile option value:

1. Sign on to the E-Business Suite as the System Administrator responsibility.
2. Navigate to the System Profile Values screen.
3. Query the Database Wallet Directory profile option.
4. Enter the full path to the wallet directory that was created earlier on the database tier. Assuming that the value of ORACLE_HOME of your database server installation is /u08/app/oracle/visdb, the value profile option value must be set as follows:

/u08/app/oracle/visdb/appsutil/wallet
5. Save your changes.

Step 4.3.3. Testing

You can now go to Chapter 5.3 - Verifying SSL Set-up for Oracle Database Server to test the wallet set-up.


Option 4.4 Additional Configuration Steps

Follow the instructions in this section only if you are not using AutoConfig to manage your system and only after you have determined that the Oracle HTTP server and Forms server can run in SSL mode and connections to Core Forms and Self-Service Web Applications is successful. This section describes how you can change the profile options and html page links for proper user navigation for secure communication.
4.4.1 Profile Options

Sign on to Core Forms Applications as System Administrator and navigate to Profile > System

1. Query the profile option: Applications Web Authentication Server

This profile option deserves special note. It is used internally in Self-Service Web Applications code only for servers using SSL. It should be set as follows:

http:// rather than https://

the non-SSL port for Apache

the machine name, not qualified by domain

Example: http://orl-sun:8800

2. Changes to other profile options

Query each of the following profile options to change each to use https:// instead of http:// and the secure SSL port instead of the non-SSL port for Apache.

1. Application Framework Agent

Example:

https://orl-sun.us.oracle.com:443

2. Applications Help Web Agent

Example:

https://orl-sun.us.oracle.com:443/pls/prod

3. Applications Web Agent

Example:

https://orl-sun.us.oracle.com:443/pls/prod

4. ICX: Report Images

Example:

https://orl-sun.us.oracle.com:443/OA_MEDIA

5. ICX: Report Launcher

Example:

https://orl-sun.us.oracle.com:443/dev60cgi/rwcgi60

6. ICX: Report Link

Example:

https://orl-sun.us.oracle.com:443/pls/prod/

7. ICX: Report Cache

Example:

https://orl-sun.us.oracle.com:443/CACHE

8. ICX: Requisition Server

Example:

https://orl-sun.us.oracle.com:443

3. Profile option ICX: FORMS LAUNCHER using the non-SSL Apache port

Make sure to leave the profile option ICX: FORMS LAUNCHER as using the non-SSL Apache port.

This is because JInitiator 1.1.7.27 requires that the initial connection to the Web server must be via the non-SSL port to load the jar files.

This limitation is described in Appendix A (Current Limitations and Considerations).

For example:

http://:/dev60cgi/f60cgi

4.4.2 Self Service Starting Page URL

Sign on to Self-Service Web Applications as System Administrator and navigate to System Administration-> General Application Options.

Change the Starting Page URL to use https:// instead of http://and the secure SSL port instead of the non-SSL port for Apache.

For example:

https://:/OA_HTML/US/ICXINDEX.htm

4.4.3 HTML pages

There are four HTML files must be edited for proper user navigation for secure communication via the Applications Release 11i portal page. For each of files listed, make a backup before editing, then edit the line(s) indicated for that file such that the link uses https:// instead of http:// and the secure SSL port instead of the non-SSL port for Apache.

1. Make a backup of each file

$ cd $COMMON_TOP/portal

$ cp aplogon.html aplogon.html.BAK

$ cp applist.html applist.html.BAK

$ cp appos090.html appos090.html.BAK

$ cp appetmp.html appetmp.html.BAK

2. Edit $COMMON_TOP/portal/aplogon.html

Find and edit the line that provides the link for the Personal Home Page:



3. Edit $COMMON_TOP/portal/applist.html

Find and edit the line that provides the link for the Online Help to read (this is all one line):

11i Online help

4. Edit $COMMON_TOP/portal/appos090.html

Find and edit the lines that define the example link for the Personal Home Page to read (this is all one line):

https://:/OA_HTML/US/ICXINDEX.htm


5. Edit $COMMON_TOP/portal/appetmp.html

Find and edit the lines that define your Web Listener Port. Change the value of [port] to reflect the Apache SSL port, for example:

Web Listener Port
WEBPRT
SSL port

4.4.4 Re-Register the Oracle Applications Framework Web Provider with Oracle Portal (Optionally Required)

If you are using Oracle Portal the Oracle Applications Framework Web Provider for the E-Business Suite environment must be re-registered in the Oracle Portal Repository so that it uses the https protocol and the secure ssl port.

Sign on to Oracle Portal as a portal administrator and navigate to:
"Administer" tab -> "Portlets" region -> "Edit a Provider Registration" section
Select your Framework Web Provider from the list, and click "Edit"
On the "Connection" tab update the URL to use https and the same SSL port you configured for Apache.
Click "Apply" and then "OK".
Select your Framework Web Provider from the list again, and click "Refresh"

Note: the navigation above is for Portal10g. When navigating in prior releases the nomenclature may be slightly different.


Chapter 5. Verifying SSL Configuration

This chapter contains instructions for testing the various SSL configurations previously performed as documented in this document.

* Option 5.1 . Verifying SSL Set-up for Oracle HTTP Server
* Option 5.2 . Verifying SSL Set-up for Oracle Forms 6i Server
* Option 5.3 . Verifying SSL Set-up for Oracle Database Server

Option 5.1 Verifying SSL Set-up for Oracle HTTP Server
Test Startup of Apache and Apps Signon

Startup the Apache server in SSL mode on Unix by invoking apacctl.sh (which will invoke the httpdsctl script) or on Windows NT/2000 by starting the Apache service.

On UNIX, you can ensure that there are several spawned processes for Apache through use of the following command:

% ps -ef | grep util

On NT/2000, you should use the processes taskbar to check for Apache processes.

If your server key is encrypted, you will be prompted for the PEM pass phrase that you chose when creating your server's private key.

Ensure that Apache can startup successfully before continuing and that you successfully get an SSL connection to the default Apache banner screen using

https://:

where

is the fully qualified name of the machine running Apache

is the SSL port number defined in httpds.conf.


Test the Forms Listener Servlet (Conditional)

If you have chosen to use the alternative Forms Listener Servlet architecture introduced in Forms/Reports 6i patchset 4 you should test that you can connect successfully to forms. Logon to your E-Business Suite Home Page and choose the System Administrator responsibility. Selecting any of the functions should launch forms in a new window.
Option 5.2 Verifying SSL Set-up for Oracle Forms 6i Server
Test Startup of Oracle Forms 6i Server and Apps Signon

If you are using a trial certificate or a non-recognized CA root certificate, you will need to complete the steps in Section IV, Part B (Working with an unrecognized CA Root certificate - Phase 2 Oracle Forms) before testing your work.

1. Start up the forms servers

On UNIX, startup the forms metrics server (adfmsctl.sh) and forms metrics client (adfmcctl.sh) as usual. Use your edited $SCRIPT_TOP/adfractl.sh (or adfroctl.sh, or adfrmctl.sh) to startup the forms server in https mode:

$SCRIPT_TOP/adfractl.sh start

On NT start the forms service using the control panel.

2. Make sure that the forms server has started successfully before continuing:

On UNIX, check for a process for f60srvm and a process similar to f60webmx webfile= HTTPS9,4,PID20 which indicates that forms is running in HTTPS mode using the command:

$ ps -ef | grep f60

On Windows NT/2000, check the services panel and task bar for forms server processes

3. Test that you can connect to Forms (Core) Applications

Remember that using JInitiator 1.1.7.27 requires that the initial connection to the Web server must be via the non-SSL port to load the jar files. This limitation is described in Appendix A (Current Limitations and Considerations).

4. Complete the steps in Chapter 6.1 (Troubleshooting - Client Connections)

Complete the steps in Chapter 6.1 (Troubleshooting - Client Connections) to prepare the client then test signon to Forms using the following URL for UNIX:

http://[host.domain]:[port] /dev60cgi/f60cgi

or, for NT:

http://[host.domain]:[port] /dev60cgi/if60cgi

where [host.domain] is the fully qualified hostname of the machine running Apache

and [port] is the non-SSL web listener port.

NOTE:

If the forms server is not starting successfully, or you cannot connect to Forms go over the above steps, checking for mistakes. The answer is most likely somewhere in the set-up instructions above. If you continue to have problems, check Chapter 6.3 (Troubleshooting - Connection to Oracle Forms Server) for more help.


Option 5.3 Verifying SSL Set-up for Oracle Database Server
Test wallet setup

To test that the wallet is properly set up and accessible, login to SQLPLUS as the apps user and execute the following:

utl_http.request('[address to access]', '[proxy address]', 'file:[full path to wallet directory]', null)

where:

'[address to access]' = the url for your Oracle Applications Rapid Install Portal, for example: 'https://www.oracle.com:4443'

[proxy address]' = the url of your proxy server, or NULL if you are not using a proxy server, for example: 'http://proxy.com:80'

'file:[full path to wallet directory]' = the location of your wallet directory, for example: 'file:/d1/dg02db/9.2.0/appsutil/wallet'

The final parameter is the wallet password, which is set to null by default.

NOTE:

You must use the prefix 'file:' and only the directory is specified, not the actual wallet files.

Examples:

utl_http.request('https://www.oracle.com:4443','http://proxy.com:80', 'file:/d1/dg02db/9.2.0/appsutil/wallet', null)

utl_http.request('https://www.oracle.com:4443',null, 'file:/d1/dg02db/9.2.0/appsutil/wallet', null)



If the wallet has been properly set up, you will be returned the first 2,000 characters of the html page.

If you receive any errors check the following:

For AutoConfig enabled instances:

Be sure you have followed the instructions in Oracle MetaLink Note 165195.1 under the section titled "Section 5: Maintaining System Configurations" to copy AutoConfig to the RDBMS ORACLE_HOME and Run AutoConfig on the database tier node.

For non AutoConfig enabled instances:

Be sure the Database Wallet Directory profile option exactly matches the location of the wallet directory.



Chapter 6. Troubleshooting
6.1. Client Connections
6.1.1 Refresh the Client Caches

The browser and JInitiator both have a mechanism for remembering activities that are performed repeatedly. In a production environment these mechanisms dramatically increase the performance of the client. However, when testing any changes made to the servers, we often want to start with a clean slate. The following steps allow you to test your changes with confidence that client caching is not obscuring the result.

1. Clean out your browser cache:

1. Netscape

Edit > Preferences > Advanced > Cache

Click the button Clear Memory Cache & Clear Disk Cache

2. MSIE

Tools > Internet Options > General Tab

In the Temporary Internet Files block, click Delete Files...

In the History block, click Clear History

2. Clean out the JInitiator cache

Delete all files contained in the directory

c:\Program Files\Oracle\JInitiator 1.1.7.27 Export\jcache

3. Exit your browser completely and start a new session.

6.1.2 Disable Browser Proxy Settings

If your organization uses a firewall and the network administrator has not yet opened the necessary port in the firewall you will need to disable the browser proxy settings:

1. Netscape

Edit > Preferences > Advanced > Proxies

Click radio button for Direct Connection to the Internet

2. MSIE

Tools > Internet Options > Connections Tab

In the Local Area Network (LAN) block, click LAN Settings.

Uncheck the checkbox for Use a Proxy Server

6.1.3 Enable JInitiator's Java Console

Enabling JIniatitor's Java console allows you to view all activity and errors for JInitiator for forms connections.

1. Start > Programs > Jinitiator
2. Check "Show Java Console"


6.2. Startup/Connection to Oracle HTTP Server
6.2.1 Startup of Apache Web Server

If the Apache Server does not start, use the following checklist to investigate any problem areas

1. Check the following files for typing mistakes or missing environment variables:

* httpds.conf
* httpdsctl
* adapcctl.sh

Note, from Rdbms 8.1.7 onwards, the following commands will run a syntax check on the httpd.conf or httpds.conf configuration files.

on Windows :

\Apache\Apache>apache -t

on Unix :

/Apache/Apache/bin>httpdsctl -configtest

Note that this only checks that the Apache syntax is correct, not that directories or files referenced actually exist.

2. Check the messages in the Apache log files which are located in the Apache/logs directory of the Apache directory tree.

* error_log
* ssl_engine_log

The messages they contain are often quite helpful and informative. Use them as search strings in the MetaLink technical database to help diagnose any problems.
6.2.2 Connection to Apache Web Server

1. Contact your local firewall administrator

Contact your local firewall administrator if you see the following message when connecting to the SSL port:

Forbidden

You are not permitted to access the remote system.

This error indicates that the firewall has not been opened for the Apache SSL port. If you are inside the firewall, unset the proxy settings in your browser as described in section 6.1.2. Disable Browser Proxy Settings above.

2. Possible problem with browser proxy settings

There may be a problem with the browser proxy settings if everything works perfectly with Netscape, but attempts to use Internet Explorer fail with the following error message:

You are not Authorized to view the page,

HTTP Error 403 Forbidden.

If this error message occurs, try unchecking the proxy server in browser settings as described in section 6.1.2. Disable Browser Proxy Settings above. IE only works with proxy server settings for port 443, any SSL port that is not 443 gets routed through proxy even if bypass proxies is set to on.

3. Connected to non-SSL port but not SSL port

If you can connect to the non-SSL port and not to the SSL port, you should check whether the Apache server has started on the SSL ports.

On Unix, use the following command to check:

ps -ef

On NT, use the following command to check:

netstat -a

If the SSL ports are not being used, you should check the service definitions in httpds.conf and, on NT in the adsvapc.cmd file to ensure that the syntax to start and run the service using SSL is correct.

4. Connected to non-SSL port but connection to SSL port is refused

If you can connect to the non-SSL port, but your connection is refused when connecting to the SSL port, check the contents of the ssl_engine_log file in the Apache/logs directory in the Apache directory tree. If you see a message in the following format:

[11/Dec/2000 22:25:48 01283] [warn] Init: (sundial:443) RSA server certificate
CommonName (CN) `otclnx1.us.oracle.com' does NOT match server name! ?

this indicates that you have either specified your machine name (Common Name) incorrectly when creating the certificate request for Apache, or you have tried to copy the certificate and server key over from another web server machine and not made the correct amendments.

6.3. Connection to Oracle Forms Server
6.3.1 JInitiator

If JInitiator never spawns when you attempt to connect to Oracle Forms, check what mode is being used for the initial connection. For the initial download of jar files, Jinitiator 1.1.7.27 needs to connect to the non-SSL port, from version 1.1.8.3 jar files can be downloaded over the SSL connection.
6.3.2 Problems with Certificates

1. On NT, if you have successfully created the forms service, then attempt to make a connection, but the Java Console hangs after the lines

connectMode=HTTPS

Cipher capability:128 bit

and the htts.log file shows the error ?open wallet failed?. You should ensure that the forms service has been started with the service property ?Log on AS? set to This Account with a Windows NT Administrator user on the forms server host machine. If this is correct and the error persists, you should also ensure that the cwallet.sso file that is created through use of the Oracle Wallet Manager is in the FORMS60_WALLET directory that is nominated in the 8.0.6 ORACLE_HOME environment file and that this value also matches the registry setting for FORMS60_WALLET

2. There may be a problem with the certificate in use if the Java Console downloads the jar files when you attempt to connect to Oracle Forms but then displays messages like the following:

Opening http://otclnx1.us.oracle.com:11082/OA_JAVA/oracle/jinitiator/ProxySettings.class no proxy
connectMode=HTTPS
Cipher capability:40 bit.
javax.net.ssl.SSLException: SSL handshake failed: X509CertChainInvalidErr
at oracle.security.ssl.OracleSSLSocketImpl.
startHandshake(OracleSSLSocketImpl.java)
at oracle.forms.net.HTTPSStream.connect(Unknown Source)
at oracle.forms.net.HTTPConnection.connect(Unknown Source)
at oracle.forms.engine.Runform.initConnection(Unknown Source)
at oracle.forms.engine.Runform.startRunform(Unknown Source)
at oracle.forms.engine.Main.createRunform(Unknown Source)
at oracle.forms.engine.Main.start(Unknown Source)
at sun.applet.JinitAppletPanel.run(Compiled Code)
at java.lang.Thread.run(Thread.java:466)

This message indicates that you have imported an unrecognized certificate into JInitiator's database of trusted CA's.

3. The certificate in use may have expired if you are able to connect successfully using Socket or HTTP mode, but when you attempt to connect in HTTPS mode using the Forms listener, the JAVA CONSOLE displays an SSL handshake failure and then gives an error showing that it is unable to connect to the machine name in the following format:

no proxy
@ proxyHost=d2kqa-apps4.us.oracle.com
proxyPort=9096
connectMode=HTTPS
Cipher capability:128 bit.
javax.net.ssl.SSLException: SSL handshake failed:X509CertExpiredErr
at oracle.security.ssl.OracleSSLSocketImpl.startHandshake(OracleSSLSocketImpl.java)
at oracle.forms.net.HTTPSStream.connect(Unknown Source)
at oracle.forms.net.HTTPConnection.connect(Unknown Source)
at oracle.forms.engine.Runform.initConnection(Unknown Source)
at oracle.forms.engine.Runform.startRunform(Unknown Source)
at oracle.forms.engine.Main.createRunform(Unknown Source)
at oracle.forms.engine.Main.start(Unknown Source)
at sun.applet.JinitAppletPanel.run(Compiled Code)
at java.lang.Thread.run(Thread.java:466)

Error messages in this format indicate that the certificate of your Forms server has expired and should be replaced with a valid certificate.

4. There may be a problem with the certdb.txt file if signon to Forms is successful when you attempt to connect to Oracle Forms, but the Java Console shows messages similar to the following:

connectMode=HTTPS
IO Exception caught: java.io.IOException: h:\PROGRA~1\oracle\JINITI~1.27E\lib\security\certdb.txt
java.lang.NullPointerException:
IO Exception caught: java.io.IOException: h:\PROGRA~1\oracle\JINITI~1.27E\lib\security\certdb.txt
java.lang.NullPointerException:
Cipher capability:40 bit.
Negotiated Cipher Suite:40 bit.
Forms Applet version is : 4

This message suggests that the certdb.txt file either has not been transferred from the server correctly or that the file is missing from the Jinitiator\lib\security directory eg:

c:\Program Files\Oracle\JInitiator \lib\security

Follow the instructions in Section 2. Update JInitiator's CA root certificate database to resolve this issue.



6.3.3 Reported Problems with Multi-User Installs

For multi-user installs (i.e. an oracle user and an applmgr user), one customer has reported problems starting the Forms server as applmgr. The symptoms reported were that when attempting to connect to forms, the Java console indicated problems accessing the jar files. Further information was acquired by starting the forms server with the log= parameters and examining the log file, which showed an error reading the wallet (as specified by the environment variable FORMS60_WALLET). The resolution in this customer's case was to start the forms server as oracle, rather than applmgr.

If this seems to be an issue, before starting the forms server as oracle, check the following:

1. You are not attempting to connect to Forms and download jar files over the SSL port of Apache.
2. For this wallet, ensure that the Auto Login checkbox is checked.
3. Ensure that on UNIX machines, the oracle and applmgr users are in the same UNIX group and that the group has full permissions on the directory and files in FORMS60_WALLET.


6.4 Use of SSL with Oracle Portal
6.4.1 Known Issues

1. Customers wishing to run the Oracle Portal Parallel Page Engine in https mode in environments that have been installed using versions of Rapid Install earlier than version 11.5.10 need to apply patch 1824205 to the iAS ORACLE_HOME. Without this patch, the jserv.log will show the following error after enabling SSL for the HTTP listener:

[01/04/2003 03:51:14:785 PST] page/ContentFetcher Unexpected Exception Request

Failed: java.lang.NullPointerException: SSL context null in setting cipher list name=content-fetcher16 label=page
url=https://.../!PORTAL30.wwpob_page.show?_pageid=51 time=866ms timeout=60000ms process=Reading
[01/04/2003 03:51:14:977 PST] page/Content Fetcher Caught:
java.lang.NullPointerException: SSL context null in setting cipher list
Source)
at HTTPClient.ExtBufferedInputStream.available(ExtBufferedInputStream.java:320)
at HTTPClient.StreamDemultiplexor.available(StreamDemultiplexor.java:383)
at HTTPClient.RespInputStream.available(RespInputStream.java:191)
at java.io.InputStreamReader.inReady(InputStreamReader.java:194)
at java.io.InputStreamReader.fill(InputStreamReader.java:166)
at java.io.InputStreamReader.read(InputStreamReader.java:244)
at oracle.webdb.page.ContentFetcher.run(ContentFetcher.java:524)

2. Customers using OAS10g with Internet Explorer cannot log into OracleAS Portal when the SSO HTTP Server and the middle tier WebCache (or HTTP Server if the WebCache is not used) are running on a single machine and using the same ServerName. After submitting the login information, the browser is redirected to the Portal home page and the following error is raised:

Error: Unexpected error encountered in wwsec_app_priv.process_signon (ORA-06502: PL/SQL: numeric or value error: character string buffer too small) (WWC-41417)

or

Error: The decryption of the authentication information was unsuccessful.
This may be caused by data inconsistency, an incorrect encryption key in this application's configuration, or an illegal access attempt. Please notify your administrator. (WWC-41454)

To resolve this MSIE error do one of the following:

1. Use Netscape Navigator to login to OracleAS Portal.
2. Install the Infrastructure tier and the Application Server middle-tier on two separate machines.
3. Follow the instructions in Metalink Note 285986.1 to add an additional virtual hostname.

3. After upgrading to ATG-H and HR FP-J you receive this warning when logging into Portal:

"There are secure and non-secure items on this page. Do you want to proceed"

This is fixed in 11.5.10 CU2 for ATG Product family (4125550) and one-off patch 4734630 ONE-OFF REQUEST FOR SCHEME (HTTP/HTTPS) FIXES CONTAINED IN CU2 FOR PORTAL NAV which can be applied on top of 11.5.10 Consolidated Update (CU1) for ATG Product Family (4017300).

Appendix A. Current Limitations & Considerations

This section describes the current issues your organization must consider in implementing SSL for Oracle Applications. Many of the limitations described are being addressed by Oracle Development at the time of writing this document and may be resolved by the time of your reading. Please check MetaLink for the status of referenced bugs or contact Oracle Support for current information.
Incompatibility of Apache / Oracle Forms 6i Certificate formats

The process of generating a certificate request (CSR) and the resulting certificate from openssl (used for Apache SSL implementation) cannot be used by Oracle Wallet Manager in an Oracle Forms 6i SSL implementation. This is because:

* If the original certificate was generated with Oracle Wallet Manager, the private key is not in a suitable format for Apache to use for decryption of messages.
* If the original certificate was generated using openssl, Wallet Manager does not have the capability of recognizing the existence of the private key.

The implication is that two certificates must be used by one organization wishing to implement SSL for both the Oracle Application's Web and Forms servers.

Bug 1483135 was originally logged for this incompatibility between the technology stack components.

Bug 1533603 has been raised because Oracle Wallet Manager does not use its private key in a standard format that can be interpreted by the Apache Server.

Bug 1491508 has been logged for the inability of Oracle Wallet Manager to import a User Certificate if the CSR is generated elsewhere.

As of 9i implementations of Oracle Applications, both iAS and Database stacks will be united so that the Apache server can understand Oracle Wallets. The Oracle Wallet format will also be amended allowing it to inter operate with browsers and openssl. If you have 9i Oracle Wallet Manager and an earlier version of apache, you will be able to use the openssl pkcs12 utility to convert the apache format to a standard format and open it in Wallet Manager. These changes are incorporated into iAS version 2.0.
Key Size Limitation of 512 bits for Oracle Forms 6i Server

Due to restrictions imposed by the United States during the initial development of Oracle Applications Release 11i, rapid installs up to 11i.4 include the Export Edition of Developer 6i. The result of this being that Oracle Wallet Manager and Forms 6i Server supported a private/public key size no longer than 512 bits and corresponding 40-bit client-server message encryption.

Since the original release of Oracle Applications Release 11i, laws concerning encryption strength have relaxed, Developer 6i has gained an ?approved product? status effectively removing the distinction between domestic and export encryption and a key size of 1024 bits and corresponding 128-bit client-server message encryption has become the recommended standard.

Bug 1365794 has been logged for this limitation of the Forms Server included with Release 11i technology stack.

Bug 1713616 provides details of a one off patch that can be applied to Developer 6i to upgrade from the export to the domestic edition to gain the benefit of the improved encryption levels that it supplies. This patch is required for any Release 11i implementations that have been installed with RapidInstall versions 11i.3 or earlier that would like to use the higher encryption levels supported in the domestic version of the Developer 6I base. Any new implementations that have been installed with version 11i.4 or later of RapidInstall will already incorporate the domestic version of Developer 6i. (On NT 4.0 / Win 2000, any patch for 6I will upgrade the export version to domestic). From Forms 6i patch 5 (6.0.8.14) onwards, the domestic edition will be applied to all implementations.

If you are not sure which version of forms your install of Release 11i Applications is using, you can check this by setting up the environment for the 8.0.6 ORACLE_HOME and typing f60run. This will display a page of information to the terminal, the first line of which shows the Forms Runtime version.
JInitiator 1.1.7.27 cannot download jar files via SSL

When connecting to core Oracle Applications (Forms), Oracle JInitiator 1.1.7.27 must download jar files to the client using a non-SSL Web connection. After jar files are downloaded, subsequent communication between the Oracle Forms 6i Server and JInitiator will be in https mode (as shown by the Java console for JInitiator).

Since non-SSL web connections are not configured by the latest versions of AutoConfig patches (TXK AutoConfig Template Rollup F (November 2003), bug 3104607 onwards), customers that are using AutoConfig must use JInitiator 1.1.8.3 or above which have the capability to download jar files over an SSL Web connection.

Please check certify for certification of your specific product configuration and be sure to follow any special instructions which may include applying specific patches or changes to the FORMS60_WEB_CONFIG_FILE (appsweb.cfg). More information on this topic may be found in the MetaLink document: " Upgrading the JInitiator version used with Oracle Applications 11i"
Forms 6i Wallet Manager does not work with new Verisign Extended Validation Certificates.

In April 2006 Versign began issuing Extended Validation (EV) Certificates. While Apache/iAS will function just fine with these new certificates, the Forms 6i Wallet Manager will not. You will need to use the Forms Listener Servlet if you wish to enable SSL for Forms. Please see Metalink Note 436312.1 for further information.


Appendix B. Firewalls

In order for users outside your organization's intranet to make SSL connections, the network administrator must open the following ports in the firewall for network traffic:

* Apache SSL port
* Apache non-SSL port (necessary for Forms connections until JInitiator is capable of initial download of jar files over SSL connection)
* Forms Listener port
* TCF Server port (if used in your environment, TCF SSL not covered in this document)

Appendix C. Use of the Forms Listener Servlet

Forms 6i Patchset 4 onwards will include a new architecture for the Forms Server to improve the deployment of Forms applications through the internet. The current Forms Listener will continue to be supported in socket mode for an intranet usage. The new Forms Server architecture is servlet based and is called the Forms Listener Servlet. Using this new feature, all the communication between the Forms client and the Forms Server goes through the web server (instead of a direct connection between the Forms client and the Forms Server). This type of connection is totally standard and the requirements in terms of proxy and firewall settings are the same as any web based application (like a standard servlet for instance).

Here are some of the benefits of the architecture:

* Support for a broader range of firewalls and proxies
* No protocol restriction (HTTP/1.1 or HTTP1.0)
* No extra process to manage (No Forms listener process)
* No specific certificate to purchase/manage for SSL deployment (Done at the web server level)
* Standard Load balancing techniques support
* Internet Explorer 5.0 native support in intranet and internet deployment

The Forms Listener Servlet has been designed to work with iAS. It has been built using standard technology so it should work with any Servlet engine 2.0 or above. Further documentation on its implementation is available in white papers that have been written as well as Oracle Metalink Note 247136.1 - Forms 6i Listener Servlet For Web Deployment of Forms in Oracle 9iAS Rel 1.
Implementing Forms Listener Servlet

AutoConfig-enabled system

To implement Forms Listener Servlet, you can use the Configuration Wizards utility. This utility is available as a command-line interface and also from Oracle Applications Manager (OAM). This utility is shipped with the E-Business Suite Release 11.5.10.

If you are using 11.5.9 or lower and would like to use this utility, apply the following prerequisite patches to all Web, Forms, and Concurrent Processing nodes:
Patch No Description Comments
3258830 OAM Patchset H About Oracle Applications Manager Mini-pack 11i.OAM.H - Note 258330.1
2864765 Advanced Utilities Patch Running Configuration Wizards from the Command Line in Oracle Applications 11i - Note 277574.1

The Configuration Wizards utility automates the configuration of Forms Listener Servlet. For additional information on the usage of this utility, please refer to the following documents:

1. Oracle MetaLink Note 277574.1 - "Running Configuration Wizards from the Command Line in Oracle Applications 11i"

2. OAM On-line Help: Help -> Oracle Applications Manager -> System Configuration -> AutoConfig -> Configuration Wizards

After completing the configuration of SSL with Oracle HTTP server using the Configuration wizards utility, run AutoConfig as described in MetaLink Note 165195.1.

Non AutoConfig-enabled system

If you are not using AutoConfig to manage your system, see the Oracle MetaLink Note 201340.1 Using Forms Listener Servlet with Oracle Applications 11i.


Appendix D. Converting OpenSSL Certificates to Wallet Format for use with Web Cache

Using Web Cache with SSL requires an Oracle Wallet. Your exisitng Apache server.key, server.crt, and ca.crt files on the Applications Middle Tier can be converted into an Oracle Wallet format using the following instructions and openssl version 0.9.7 or higher.

* Copy the existing server.crt, ca.crt, and server.key files from $COMMON_TOP/admin/certs/apache/ssl.crt and $COMMON_TOP/admin/certs/apache/ssl.key directories to a temporary directory.
* Run the following command which will combine the certificate's public key, private key and root certficate into one file which can be be imported into Oracle Wallet Manager.

/usr/bin/openssl pkcs12 -export -descert -in server.crt -inkey server.key -certfile ca.crt -name -out ewallet.p12
where is a name you give for the certificate and private key.

You will be prompted to Enter Export Password:
This password will be required to open the wallet.

* Copy ewallet.p12 to Webcache wallet directory as the user who owns the Web Cache file system.
If you FTP the file be sure to do so in binary mode.
* Ensure that your Web Cache environment is correctly sourced.
* Enable AutoLogin for the wallet

Set the DISPLAY variable to display to the IP address from where you are working.
For example: % export DISPLAY=138. 22.155.16:0.0
where 138.22.155.16 is the IP address of your workstation.

For LINUX only - set the environment variable THREADS_FLAG=native.
For example: export THREADS_FLAG=native

Run owm, which is located under the $ORACLE_HOME/bin directory.
For example: $ORACLE_HOME/bin/owm &

On Windows NT/2000, run Oracle Wallet Manager using Start > Programs > Oracle for Windows NT > Oracle Wallet Manager or Start > Programs > Oracle for Windows NT > Integrated Management Tools > Wallet Manager.

On the Oracle Wallet Manger Menu:
Navigate to Wallet -> Open
Select Yes when prompted with the message:
" Your default directory doesn't exist. Do you wish to continue?"
Use the Select Directory window to navigate to your wallet directory.
Click ok and enter the wallet password when prompted to open the wallet.

On the Oracle Wallet Manger Menu:
Be sure the AutoLogin feature is checked.

When the AutoLogin feature is checked the wallet can be accessed by OS processes (i.e. sqlplus) that are owned by the same owner who created the wallet. To open the wallet using Oracle Wallet Manager a password will still be required.

Save the wallet and exit.

You should now have the following wallet files in your wallet directory.
ewallet.p12
cwallet.sso


Related Documentation

Apache SSL Configuration

* Apache Single Listener Configuration for Applications 11.5.1 (Note 119873.1)
* Configuring Oracle HTTP Server with SSL and Global Server Certificates (Note 137136.1)
* How to be a Certificate Authority (CA) with Oracle HTTP Server on Unix (Note 117022.1)
* How to Request and Configure Test SSL Certificate with for Oracle9iASv1 on Unix (Note 114444.1)
* Advanced Configurations and Topologies for Enterprise Deployments of E-Business Suite 11i (Note 217368.1)

Forms SSL Configuration

* How to Deploy Forms via HTTPS with Forms Server 6i (Note 106350.1)
* Forms 6i Listener Servlet For Web Deployment of Forms in Oracle 9iAS Rel 1 (Note 247136.1)
* Deploying Forms Applications to the Web with Forms Server Release 6i, Part no. A73071_01 (PDF, 1.8Kb)
* How to make Jinitiator work with Apache with SSL (HTTPS) with a Verisign Trial Certificate and 1.0.2.2.2 (Note 147836.1)

Using SSL with Portal

* Configuring SSL for Portal on UNIX (Note 161099.1)
* Configuring SSL for Portal on NT (Note 136153.1)
* How to Configure Portal with SSL and Global Server Certificates (Note 137145.1)