Scope of this Article:
First of all identify the patches required to be applied for your environment and download and keep zip files in specific location on the server. for example I have downloaded and kept patches zip files at below location on my all nodes of the RAC cluster.
/backup/PATCH/Jan2020
Few Important points:
1. This document can be used to apply PSU patch for Unix/Linux and some of the portion can be used for Windows as well.
2. This method is used to apply patch for 12cR2, which is quite different than application of patch in 11g and prior.
3. Binary Patching activity will be performed as a root user.
4. Only GI patch needs to be downloaded for RAC(which includes GI+RDBMS components), same patch needs to be applied on both Grid Infra home and RDBMS Home.
5. datapatch, which is post SQL Load/execution needs to performed only once on any single node , when binary patching completed on all/both the nodes.
6. It's recommended that patch related readme needs to be followed , there may be small changes here and there considering the patch version and OS version.
7. For RAC environment, Rolling forward method is the best as there will NOT be complete application downtime, however in flight transaction may impact while performing activity.
###1.1 Patch Information
The Grid Infrastructure patches are cumulative and include the Database Update Content and CPU program security content.
###1.2 Patch Installation and Deinstallation
==>1.2.1 Patch Installation Prerequisites
Note: Make sure that crontab jobs or any other scheduled jobs should be commented during patching.
-->1.2.1.1 OPatch Utility Information
You must use the OPatch utility version 12.2.0.1.17 or later to apply this patch for all platforms. Oracle recommends that you use the latest released OPatch version for 12.1 releases,
which is available for download from My Oracle Support patch 6880880 by selecting ARU link for the 12.2.0.1.0 release. It is recommended that you download the Opatch utility and
the patch in a shared location to be able to access them from any node in the cluster for the patch application on each node.
When patching the GI Home, a shared location on ACFS only needs to be unmounted on the node where the GI Home is being patched.
The new opatch utility should be updated in all the Oracle RAC database homes and the GI home that are being patched.
-->1.2.1.2 Validation of Oracle Inventory
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note:Perform this on both the nodes:
For Grid Home:
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_B4.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_B4.lst
For RDBMS Home as oracle user:
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_B4.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_B4.lst
~~>1.2.1.2 Validation of Oracle Inventory
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
~~>1.2.1.3 Download and Unzip the Patch : I assume this step has been done
~~>1.2.1.4 Run OPatch Conflict Check
Mentioned steps to apply patch for GI and RDBMS homes and database using rolling method with minimum to no downtime can be used for any quarterly patches, however this particular article was prepared while applying Jan'20 PSU patch -Oracle Database Patch 30501932 - GI Jan 2020 Release Update 12.2.0.1.200114 on IBM AIX 64 bit Power systems. Below mentioned steps will give you fair idea to apply PSU Patch, however I would highly encourage you to go through the patch readme thoroughly before proceeding with the patching. It's always better to apply patch on lower environment first before applying it on production environment.
First of all identify the patches required to be applied for your environment and download and keep zip files in specific location on the server. for example I have downloaded and kept patches zip files at below location on my all nodes of the RAC cluster.
/backup/PATCH/Jan2020
Few Important points:
1. This document can be used to apply PSU patch for Unix/Linux and some of the portion can be used for Windows as well.
2. This method is used to apply patch for 12cR2, which is quite different than application of patch in 11g and prior.
3. Binary Patching activity will be performed as a root user.
4. Only GI patch needs to be downloaded for RAC(which includes GI+RDBMS components), same patch needs to be applied on both Grid Infra home and RDBMS Home.
5. datapatch, which is post SQL Load/execution needs to performed only once on any single node , when binary patching completed on all/both the nodes.
6. It's recommended that patch related readme needs to be followed , there may be small changes here and there considering the patch version and OS version.
7. For RAC environment, Rolling forward method is the best as there will NOT be complete application downtime, however in flight transaction may impact while performing activity.
###1.1 Patch Information
The Grid Infrastructure patches are cumulative and include the Database Update Content and CPU program security content.
Configuration GI Version Database Versions Patch OPatch Command(1) Comments
----------------------------------------------------------------- ------------- ----------------- -------- ----------------- ---------------------------------------------------
GI Home in conjunction with RAC, RACOne, or Single Instance home 12.2.0.1 12.2.0.1 GI RU opatchauto GI Home and all the Database Homes will be patched
Table 1-2 Patch Numbers Getting Installed as Part of this Bundle Patch
Patch Number Description Applicable Homes
================== =================== ============================
30593149 Database Jan 2020 Release Update 12.2.0.1.200114 Only DB Home for non-Oracle RAC setup.
Both DB Homes and Grid Home for Oracle RAC setup.
30585969 OCW Jan 2020 Release Update 12.2.0.1.200114 Both DB Homes and Grid Home
30586063 ACFS Jan 2020 Release Update 12.2.0.1.200114Footnote 2 Only Grid Home
26839277 DBWLM Release Update 12.2.0.1.170913Footnote 2 Only Grid Home
30591794 Tomcat Release Update 12.2.0.1.0Footnote 2 Only Grid Home
###1.2 Patch Installation and Deinstallation
==>1.2.1 Patch Installation Prerequisites
Note: Make sure that crontab jobs or any other scheduled jobs should be commented during patching.
-->1.2.1.1 OPatch Utility Information
You must use the OPatch utility version 12.2.0.1.17 or later to apply this patch for all platforms. Oracle recommends that you use the latest released OPatch version for 12.1 releases,
which is available for download from My Oracle Support patch 6880880 by selecting ARU link for the 12.2.0.1.0 release. It is recommended that you download the Opatch utility and
the patch in a shared location to be able to access them from any node in the cluster for the patch application on each node.
When patching the GI Home, a shared location on ACFS only needs to be unmounted on the node where the GI Home is being patched.
The new opatch utility should be updated in all the Oracle RAC database homes and the GI home that are being patched.
oracle@PRODHOSTDB01:[/home/oracle]$cd $ORACLE_HOME/OPatch
oracle@PRODHOSTDB01:[/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch]$opatch version
OPatch Version: 12.2.0.1.6 <--Which is less than 12.2.0.1.17 so we need to download latest 6880880 support patch.
OPatch succeeded.
oracle@PRODHOSTDB01:[/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch]$
Download the OPatch utility to a temporary directory.
For each Oracle RAC database home and the GI home that are being patched, run the following commands as the home owner to extract the OPatch utility.
Unzip the patch as grid home owner except for installations that do not have any grid homes
PATCH Location: /backup/PATCH/Jan2020
$ unzip <OPATCH-ZIP> -d <ORACLE_HOME>
$ <ORACLE_HOME>/OPatch/opatch version
The version output of the previous command should be 12.2.0.1.17 or later.
~~~>Make sure that you change below permission for directory from where you are applying patches otherwise you may face issue while patching due to permission issue.
oracle@PRODHOSTDB01:[/backup/PATCH/Jan2020]$chmod -R 775 /backup/PATCH/Jan2020
UnZip the 6880880:
###Backing of the Binaries homes: make sure to backup binaries on both the nodes
~~~>Make sure that you change below permission otherwise you may face issue while patching due to permission issue.
oracle@PRODHOSTDB01:[/backup/PATCH/Jan2020]$chmod -R 775 /backup/PATCH/Jan2020
~~~>Make sure that you change below permission otherwise you may face issue while patching due to permission issue.
oracle@PRODHOSTDB01:[/backup/PATCH/Jan2020]$chmod -R 775 /backup/PATCH/Jan2020
For GI Home: Run it as GRID user
-->To run backup of home in background
cd /backup/PATCH/Jan2020
nohup tar cvf - /u01/app/12.2.0/grid | gzip > /backup/PATCH/Jan2020/GRID_HOME_bkp1_30012020.tar.gz &
-->to run backup of home in foreground
tar cvf - /u01/app/12.2.0/grid | gzip > /backup/PATCH/Jan2020/Grid_home_bkp1_30012020.tar.gz
OH home: RUN it as Oracle user
-->To run backup of home in background
cd /backup/PATCH/Jan2020
nohup tar cvf - /u01/app/oracle/product/12.2.0.1/dbhome_1 | gzip > ORACLE_HOME_bkp1_30012020.tar.gz &
OR
-->to run backup of home in foreground
cd /backup/PATCH/Jan2020
tar cvf - /u01/app/oracle/product/12.2.0.1/dbhome_1 | gzip > /backup/PATCH/Jan2020/ORACLE_HOME_bkp1_08102019.tar.gz
-->1.2.1.2 Validation of Oracle Inventory
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note:Perform this on both the nodes:
For Grid Home:
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_B4.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_B4.lst
For RDBMS Home as oracle user:
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_B4.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_B4.lst
~~>>Execute below SQL to check the status of the patches from DB:
sqlplus / as sysdba
spool /backup/PATCH/Jan2020/datapatch_status_B4.lst
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a70
col bundle_series for a20
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series from dba_registry_sqlpatch;
spool off
sqlplus / as sysdba
spool /backup/PATCH/Jan2020/datapatch_status_B4.lst
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a70
col bundle_series for a20
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series from dba_registry_sqlpatch;
spool off
~~>1.2.1.2 Validation of Oracle Inventory
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
~~>1.2.1.3 Download and Unzip the Patch : I assume this step has been done
~~>1.2.1.4 Run OPatch Conflict Check
•For Grid Infrastructure Home, as home user: Change appropriate patch location
Login as grid
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30593149 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30593149.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30585969 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30585969.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30586063 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30586063.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/26839277 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_26839277.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30591794 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30591794.txt
•For Database home, as home user:Change appropriate patch location
Login as oracle
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30593149 |tee -a /backup/PATCH/Jan2020/oracle_ConflictCheck_30501932_30593149.txt
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30585969 |tee -a /backup/PATCH/Jan2020/oracle_ConflictCheck_30501932_30585969.txt
The report will indicate the interim patches that conflict with the patch 30501932 and the interim patches for which patch 30501932 is a superset.
Note:
When OPatch starts, it validates the patch and ensures that there are no conflicts with the software already installed in the ORACLE_HOME.
OPatch categorizes conflicts into the following types:
Conflicts with a patch already applied to the ORACLE_HOME.
In this case, stop the patch installation and contact Oracle Support Services.
Conflicts with subset patch already applied to the ORACLE_HOME.
In this case, continue with the patch installation because as the new patch contains all the fixes from the existing patch in the ORACLE_HOME.
And, in any case, the subset patch will automatically be rolled back prior to the installation of the new patch.
-->1.2.1.5 Run OPatch SystemSpace Check
Check if enough free space is available on the ORACLE_HOME filesystem for the patches to be applied as given below:
⇒For Grid Infrastructure Home, as home user:grid
===============================
1.Create file /tmp/patch_list_gihome.txt with the following content:
su - grid
cd /backup/PATCH/Jan2020
echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30586063" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/26839277" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30591794" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
cat /backup/PATCH/Jan2020/patch_list_gihome.txt
2.Run the opatch command to check if enough free space is available in the Grid Infrastructure Home:
$GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_gihome.txt
⇨For Database home, as home user:
===============================
su - oracle
1.Create file /backup/PATCH/Jan2020/patch_list_dbhome.txt with the following content:
echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
cat /backup/PATCH/Jan2020/patch_list_dbhome.txt
2. Run opatch command to check if enough free space is available in the Database Home:
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_dbhome.txt
The command output reports pass and fail messages as per the system space availability:
If OPatch reports Prereq "checkSystemSpace" failed., then cleanup the system space as the required amount of space is not available.
If OPatch reports Prereq "checkSystemSpace" passed., then no action is needed. Proceed with patch installation.
~~~~~~~~~~~~~~~~~~~~~~output of checkSystemSpace execution~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
===>Node-1: checkSystemSpace for GRID Home
==================================
Login as grid
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30593149 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30593149.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30585969 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30585969.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30586063 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30586063.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/26839277 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_26839277.txt
$GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30591794 |tee -a /backup/PATCH/Jan2020/grid_ConflictCheck_30501932_30591794.txt
•For Database home, as home user:Change appropriate patch location
Login as oracle
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30593149 |tee -a /backup/PATCH/Jan2020/oracle_ConflictCheck_30501932_30593149.txt
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /backup/PATCH/Jan2020/30501932/30585969 |tee -a /backup/PATCH/Jan2020/oracle_ConflictCheck_30501932_30585969.txt
The report will indicate the interim patches that conflict with the patch 30501932 and the interim patches for which patch 30501932 is a superset.
Note:
When OPatch starts, it validates the patch and ensures that there are no conflicts with the software already installed in the ORACLE_HOME.
OPatch categorizes conflicts into the following types:
Conflicts with a patch already applied to the ORACLE_HOME.
In this case, stop the patch installation and contact Oracle Support Services.
Conflicts with subset patch already applied to the ORACLE_HOME.
In this case, continue with the patch installation because as the new patch contains all the fixes from the existing patch in the ORACLE_HOME.
And, in any case, the subset patch will automatically be rolled back prior to the installation of the new patch.
-->1.2.1.5 Run OPatch SystemSpace Check
Check if enough free space is available on the ORACLE_HOME filesystem for the patches to be applied as given below:
⇒For Grid Infrastructure Home, as home user:grid
===============================
1.Create file /tmp/patch_list_gihome.txt with the following content:
su - grid
cd /backup/PATCH/Jan2020
echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30586063" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/26839277" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30591794" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
cat /backup/PATCH/Jan2020/patch_list_gihome.txt
2.Run the opatch command to check if enough free space is available in the Grid Infrastructure Home:
$GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_gihome.txt
⇨For Database home, as home user:
===============================
su - oracle
1.Create file /backup/PATCH/Jan2020/patch_list_dbhome.txt with the following content:
echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
cat /backup/PATCH/Jan2020/patch_list_dbhome.txt
2. Run opatch command to check if enough free space is available in the Database Home:
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_dbhome.txt
The command output reports pass and fail messages as per the system space availability:
If OPatch reports Prereq "checkSystemSpace" failed., then cleanup the system space as the required amount of space is not available.
If OPatch reports Prereq "checkSystemSpace" passed., then no action is needed. Proceed with patch installation.
~~~~~~~~~~~~~~~~~~~~~~output of checkSystemSpace execution~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
===>Node-1: checkSystemSpace for GRID Home
==================================
"/backup/PATCH/Jan2020/patch_list_gihome.txt" [New file] 7 lines, 202 characters
grid@PRODHOSTDB01:[/backup/PATCH/Jan2020]$cat "/backup/PATCH/Jan2020/patch_list_gihome.txt"
/backup/PATCH/Jan2020/30501932/30593149
/backup/PATCH/Jan2020/30501932/30585969
/backup/PATCH/Jan2020/30501932/30586063
/backup/PATCH/Jan2020/30501932/26839277
/backup/PATCH/Jan2020/30501932/30591794
grid@PRODHOSTDB01:[/backup/PATCH/Jan2020]$GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_gihome.txt
ksh: GRID_HOME/OPatch/opatch: not found.
grid@PRODHOSTDB01:[/backup/PATCH/Jan2020]$$GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_gihome.txt
Oracle Interim Patch Installer version 12.2.0.1.19
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/12.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/12.2.0/grid/oraInst.loc
OPatch version : 12.2.0.1.19
OUI version : 12.2.0.1.4
Log file location : /u01/app/12.2.0/grid/cfgtoollogs/opatch/opatch2020-01-30_11-23-25AM_1.log
Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
~~>>CheckSystemSpace for Oracle_home:RDBMS on node-1
<!–– output text begin ––>
oracle@PRODHOSTDB01:[/home/oracle]$
oracle@PRODHOSTDB01:[/home/oracle]$echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
oracle@PRODHOSTDB01:[/home/oracle]$echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
oracle@PRODHOSTDB01:[/home/oracle]$cat /backup/PATCH/Jan2020/patch_list_dbhome.txt
/backup/PATCH/Jan2020/30501932/30593149
/backup/PATCH/Jan2020/30501932/30585969
oracle@PRODHOSTDB01:[/home/oracle]$$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_dbhome.txt
Oracle Interim Patch Installer version 12.2.0.1.19
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/12.2.0.1/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/12.2.0.1/dbhome_1/oraInst.loc
OPatch version : 12.2.0.1.19
OUI version : 12.2.0.1.4
Log file location : /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatch/opatch2020-01-30_11-25-59AM_1.log
Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
oracle@PRODHOSTDB01:[/home/oracle]$
===>Node-2: checkSystemSpace for GRID Home
==================================
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#su - grid
grid@PRODHOSTDB02:[/home/grid]$
grid@PRODHOSTDB02:[/home/grid]$cd /backup/PATCH/Jan2020
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$
echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$echo "/backup/PATCH/Jan2020/30501932/30586063" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$echo "/backup/PATCH/Jan2020/30501932/26839277" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$echo "/backup/PATCH/Jan2020/30501932/30591794" >> /backup/PATCH/Jan2020/patch_list_gihome.txt
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$cat /backup/PATCH/Jan2020/patch_list_gihome.txt
/backup/PATCH/Jan2020/30501932/30593149
/backup/PATCH/Jan2020/30501932/30585969
/backup/PATCH/Jan2020/30501932/30586063
/backup/PATCH/Jan2020/30501932/26839277
/backup/PATCH/Jan2020/30501932/30591794
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$$GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_gihome.txt
Oracle Interim Patch Installer version 12.2.0.1.19
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/12.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/12.2.0/grid/oraInst.loc
OPatch version : 12.2.0.1.19
OUI version : 12.2.0.1.4
Log file location : /u01/app/12.2.0/grid/cfgtoollogs/opatch/opatch2020-01-30_11-44-12AM_1.log
Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$exit
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#su - oracle
oracle@PRODHOSTDB02:[/home/oracle]$
oracle@PRODHOSTDB02:[/home/oracle]$echo "/backup/PATCH/Jan2020/30501932/30593149" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
oracle@PRODHOSTDB02:[/home/oracle]$echo "/backup/PATCH/Jan2020/30501932/30585969" >> /backup/PATCH/Jan2020/patch_list_dbhome.txt
oracle@PRODHOSTDB02:[/home/oracle]$cat /backup/PATCH/Jan2020/patch_list_dbhome.txt
/backup/PATCH/Jan2020/30501932/30593149
/backup/PATCH/Jan2020/30501932/30585969
oracle@PRODHOSTDB02:[/home/oracle]$
oracle@PRODHOSTDB02:[/home/oracle]$
oracle@PRODHOSTDB02:[/home/oracle]$$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /backup/PATCH/Jan2020/patch_list_dbhome.txt
Oracle Interim Patch Installer version 12.2.0.1.19
Copyright (c) 2020, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/12.2.0.1/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/12.2.0.1/dbhome_1/oraInst.loc
OPatch version : 12.2.0.1.19
OUI version : 12.2.0.1.4
Log file location : /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatch/opatch2020-01-30_11-45-00AM_1.log
Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
oracle@PRODHOSTDB02:[/home/oracle]$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~End of Ouput ChecySystemSpace ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~==>1.2.2 One-off Patch Conflict Detection and Resolution
Run as ROOT user:
export GRID_HOME=/u01/app/12.2.0/grid
$GRID_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $GRID_HOME
export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1
$ORACLE_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $ORACLE_HOME
===You can use below command to analyze both the home togather.
export GRID_HOME=/u01/app/12.2.0/grid
$GRID_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze
===
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~output of execution of analyze~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
===>Node -1 for GRID home:
=============================
<!–– output text begin ––>
root@PRODHOSTDB01:[/]#export GRID_HOME=/u01/app/12.2.0/grid
root@PRODHOSTDB01:[/]#$GRID_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $GRID_HOME
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Thu Jan 30 11:34:02 2020
System initialization log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-01-30_11-34-11AM.log.
Session log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/opatchauto2020-01-30_11-34-53AM.log
The id for this session is ZE7B
Executing OPatch prereq operations to verify patch applicability on home /u01/app/12.2.0/grid
Patch applicability verified successfully on home /u01/app/12.2.0/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Analysis for applying patches has completed successfully:
Host:PRODHOSTdb01
CRS Home:/u01/app/12.2.0/grid
Version:12.2.0.1.0
==Following patches were SUCCESSFULLY analyzed to be applied:
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-35-18AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30586063
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-35-18AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/26839277
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-35-18AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30591794
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-35-18AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-35-18AM_1.log
OPatchauto session completed at Thu Jan 30 11:35:30 2020
Time taken to complete the session 1 minute, 28 seconds
root@PRODHOSTDB01:[/]#
===>Node -1 for RDBMS home:
=============================
root@PRODHOSTDB01:[/]#export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1
root@PRODHOSTDB01:[/]#$ORACLE_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $ORACLE_HOME
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Thu Jan 30 11:30:21 2020
System initialization log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2020-01-30_11-30-30AM.log.
Session log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-01-30_11-31-23AM.log
The id for this session is YCBP
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Analysis for applying patches has completed successfully:
Host:PRODHOSTdb01
RAC Home:/u01/app/oracle/product/12.2.0.1/dbhome_1
Version:12.2.0.1.0
==Following patches were SKIPPED:
Patch: /backup/PATCH/Jan2020/30501932/30586063
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/26839277
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/30591794
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY analyzed to be applied:
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-31-45AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-31-45AM_1.log
OPatchauto session completed at Thu Jan 30 11:32:46 2020
Time taken to complete the session 2 minutes, 25 seconds
root@PRODHOSTDB01:[/]#
===>Node -2 for GRID home:
=============================
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#export GRID_HOME=/u01/app/12.2.0/grid
$GRID_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $GRID_HOME
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1
$GRID_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $GRID_HOME
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Thu Jan 30 11:47:24 2020
System initialization log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-01-30_11-47-33AM.log.
Session log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/opatchauto2020-01-30_11-48-10AM.log
The id for this session is W7XV
Executing OPatch prereq operations to verify patch applicability on home /u01/app/12.2.0/grid
Patch applicability verified successfully on home /u01/app/12.2.0/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Analysis for applying patches has completed successfully:
Host:PRODHOSTdb02
CRS Home:/u01/app/12.2.0/grid
Version:12.2.0.1.0
==Following patches were SUCCESSFULLY analyzed to be applied:
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-48-36AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30586063
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-48-36AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/26839277
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-48-36AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30591794
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-48-36AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-48-36AM_1.log
OPatchauto session completed at Thu Jan 30 11:48:46 2020
Time taken to complete the session 1 minute, 23 seconds
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#
===>Node -2 for RDBMS home:
=============================
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#$ORACLE_HOME/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -analyze -oh $ORACLE_HOME
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Thu Jan 30 11:48:47 2020
System initialization log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2020-01-30_11-48-55AM.log.
Session log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-01-30_11-49-45AM.log
The id for this session is KAXM
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Analysis for applying patches has completed successfully:
Host:PRODHOSTdb02
RAC Home:/u01/app/oracle/product/12.2.0.1/dbhome_1
Version:12.2.0.1.0
==Following patches were SKIPPED:
Patch: /backup/PATCH/Jan2020/30501932/30586063
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/26839277
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/30591794
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY analyzed to be applied:
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-50-07AM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_11-50-07AM_1.log
OPatchauto session completed at Thu Jan 30 11:51:07 2020
Time taken to complete the session 2 minutes, 21 seconds
root@PRODHOSTDB02:[/backup/PATCH/Jan2020]#
==> Check all the CRS services are listed...
ps -ef|grep d.bin
$crsctl status res -t
Node-1:
====================
grid@PRODHOSTDB01:[/home/grid]$ps -ef|grep d.bin
root 14156106 1 1 Jan 27 - 1:08 /u01/app/12.2.0/grid/bin/cssdagent
grid 14549326 1 0 Jan 27 - 5:54 /u01/app/12.2.0/grid/bin/oraagent.bin
root 14614814 1 0 Jan 27 - 2:35 /u01/app/12.2.0/grid/bin/octssd.bin reboot
grid 14942694 16777666 0 Jan 27 - 0:25 /u01/app/12.2.0/grid/bin/evmlogger.bin -o /u01/app/12.2.0/grid/log/[HOSTNAME]/evmd/evmlogger.info -l /u01/app/12.2.0/grid/log/[HOSTNAME]/evmd/evmlogger.log
root 15073760 1 0 Jan 27 - 3:43 /u01/app/12.2.0/grid/bin/ohasd.bin reboot _ORA_BLOCKING_STACK_LOCALE=AMERICAN_AMERICA.WE8ISO8859P1
root 15466990 1 0 Jan 27 - 0:00 /bin/sh /u01/app/12.2.0/grid/bin/ocssd
root 15860074 1 0 Jan 27 - 1:07 /u01/app/12.2.0/grid/bin/cssdmonitor
grid 16122176 1 0 Jan 27 - 0:31 /u01/app/12.2.0/grid/bin/mdnsd.bin
root 16318772 1 0 Jan 27 - 4:11 /u01/app/12.2.0/grid/bin/orarootagent.bin
grid 16515512 1 0 Jan 27 - 2:32 /u01/app/12.2.0/grid/bin/gipcd.bin
grid 16646534 15466990 1 Jan 27 - 5:59 /u01/app/12.2.0/grid/bin/ocssd.bin
grid 16777666 1 0 Jan 27 - 2:36 /u01/app/12.2.0/grid/bin/evmd.bin
grid 17236460 1 1 Jan 27 - 0:54 /u01/app/12.2.0/grid/bin/scriptagent.bin
grid 18219414 1 0 Jan 27 - 0:01 /u01/app/12.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -no_crs_notify -inherit
grid 18481628 1 0 Jan 27 - 0:01 /u01/app/12.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -no_crs_notify -inherit
root 18547174 1 0 Jan 27 - 8:59 /u01/app/12.2.0/grid/bin/ologgerd -M
root 18809210 1 0 Jan 27 - 7:54 /u01/app/12.2.0/grid/bin/osysmond.bin
grid 19595702 1 0 Jan 27 - 0:37 /u01/app/12.2.0/grid/bin/gpnpd.bin
root 19726724 1 0 Jan 27 - 75:05 /u01/app/12.2.0/grid/bin/orarootagent.bin
grid 19792224 1 0 Jan 27 - 0:28 /u01/app/12.2.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit
grid 19988910 1 0 Jan 27 - 0:02 /u01/app/12.2.0/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit
grid 23331312 1 0 Jan 27 - 54:21 /u01/app/12.2.0/grid/bin/oraagent.bin
grid 24773044 1 0 Jan 27 - 0:02 /u01/app/12.2.0/grid/bin/tnslsnr MGMTLSNR -no_crs_notify -inherit
root 26935722 1 0 Jan 27 - 4:23 /u01/app/12.2.0/grid/bin/crsd.bin reboot
grid 29688294 42008834 0 11:58:57 pts/2 0:00 grep d.bin
oracle 30474534 1 0 Jan 28 - 0:47 /u01/app/12.2.0/grid/bin/oraagent.bin
grid@PRODHOSTDB01:[/home/grid]$crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.DATA.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.FRA.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
OFFLINE OFFLINE PRODHOSTdb02 STABLE
ora.GIMR.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.INDX.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
OFFLINE OFFLINE PRODHOSTdb02 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.OCR.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.REDO.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
OFFLINE OFFLINE PRODHOSTdb02 STABLE
ora.net1.network
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.ons
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.proxy_advm
OFFLINE OFFLINE PRODHOSTdb01 STABLE
OFFLINE OFFLINE PRODHOSTdb02 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE PRODHOSTdb01 169.254.113.95 10.80
.0.2,STABLE
ora.asm
1 ONLINE ONLINE PRODHOSTdb01 Started,STABLE
2 ONLINE ONLINE PRODHOSTdb02 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.PRODHOSTdb01.vip
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.PRODHOSTdb02.vip
1 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.testdb.db
1 ONLINE ONLINE PRODHOSTdb01 Open,HOME=/u01/app/o
racle/product/12.2.0
.1/dbhome_1,STABLE
2 ONLINE ONLINE PRODHOSTdb02 Open,HOME=/u01/app/o
racle/product/12.2.0
.1/dbhome_1,STABLE
ora.testdb.testdb_svc.svc
1 ONLINE ONLINE PRODHOSTdb01 STABLE
2 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.mgmtdb
1 ONLINE ONLINE PRODHOSTdb01 Open,STABLE
ora.qosmserver
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.scan1.vip
1 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.scan2.vip
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.scan3.vip
1 ONLINE ONLINE PRODHOSTdb01 STABLE
--------------------------------------------------------------------------------
grid@PRODHOSTDB01:[/home/grid]$
Node-2:
=====================
<!–– output text begin ––>
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$ps -ef|grep d.bin
root 13631752 1 2 Jan 27 - 48:45 /u01/app/12.2.0/grid/bin/orarootagent.bin
root 13828582 1 0 Jan 27 - 0:00 /bin/sh /u01/app/12.2.0/grid/bin/ocssd
root 13959532 1 0 Jan 27 - 5:09 /u01/app/12.2.0/grid/bin/crsd.bin reboot
grid 14287182 16318832 0 Jan 27 - 0:25 /u01/app/12.2.0/grid/bin/evmlogger.bin -o /u01/app/12.2.0/grid/log/[HOSTNAME]/evmd/evmlogger.info -l /u01/app/12.2.0/grid/log/[HOSTNAME]/evmd/evmlogger.log
grid 14352682 13828582 0 Jan 27 - 5:51 /u01/app/12.2.0/grid/bin/ocssd.bin
grid 14811500 1 0 Jan 27 - 0:29 /u01/app/12.2.0/grid/bin/mdnsd.bin
root 14942564 1 0 Jan 27 - 6:51 /u01/app/12.2.0/grid/bin/osysmond.bin
root 15008054 1 0 Jan 27 - 1:07 /u01/app/12.2.0/grid/bin/cssdagent
root 15139242 1 1 Jan 27 - 2:32 /u01/app/12.2.0/grid/bin/octssd.bin reboot
root 15663392 1 0 Jan 27 - 1:05 /u01/app/12.2.0/grid/bin/cssdmonitor
grid 16187658 1 0 Jan 27 - 8:20 /u01/app/12.2.0/grid/bin/oraagent.bin
grid 16253186 1 0 Jan 27 - 2:30 /u01/app/12.2.0/grid/bin/gipcd.bin
grid 16318832 1 0 Jan 27 - 2:33 /u01/app/12.2.0/grid/bin/evmd.bin
root 16384338 1 0 Jan 27 - 3:31 /u01/app/12.2.0/grid/bin/ohasd.bin reboot _ORA_BLOCKING_STACK_LOCALE=AMERICAN_AMERICA.WE8ISO8859P1
grid 16515334 1 0 Jan 27 - 0:02 /u01/app/12.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -no_crs_notify -inherit
root 16581012 1 0 Jan 27 - 2:48 /u01/app/12.2.0/grid/bin/orarootagent.bin
grid 16712134 1 1 Jan 27 - 23:35 /u01/app/12.2.0/grid/bin/oraagent.bin
grid 16843084 1 0 Jan 27 - 0:33 /u01/app/12.2.0/grid/bin/tnslsnr ASMNET1LSNR_ASM -no_crs_notify -inherit
grid 16908772 1 0 Jan 27 - 0:36 /u01/app/12.2.0/grid/bin/gpnpd.bin
grid 18547146 1 0 Jan 27 - 0:01 /u01/app/12.2.0/grid/bin/tnslsnr LISTENER -no_crs_notify -inherit
oracle 19530086 1 0 Jan 28 - 0:44 /u01/app/12.2.0/grid/bin/oraagent.bin
grid 35389790 19595636 0 11:59:26 pts/0 0:00 grep d.bin
grid@PRODHOSTDB02:[/backup/PATCH/Jan2020]$
grid@PRODHOSTDB02:[/home/grid]$crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.DATA.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.FRA.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.GIMR.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.INDX.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.OCR.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.REDO.dg
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.net1.network
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.ons
ONLINE ONLINE PRODHOSTdb01 STABLE
ONLINE ONLINE PRODHOSTdb02 STABLE
ora.proxy_advm
OFFLINE OFFLINE PRODHOSTdb01 STABLE
OFFLINE OFFLINE PRODHOSTdb02 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE PRODHOSTdb01 169.254.113.95 10.80
.0.2,STABLE
ora.asm
1 ONLINE ONLINE PRODHOSTdb01 Started,STABLE
2 ONLINE ONLINE PRODHOSTdb02 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.PRODHOSTdb01.vip
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.PRODHOSTdb02.vip
1 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.testdb.db
1 ONLINE ONLINE PRODHOSTdb01 Open,HOME=/u01/app/o
racle/product/12.2.0
.1/dbhome_1,STABLE
2 ONLINE ONLINE PRODHOSTdb02 Open,HOME=/u01/app/o
racle/product/12.2.0
.1/dbhome_1,STABLE
ora.testdb_svc.svc
1 ONLINE ONLINE PRODHOSTdb01 STABLE
2 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.mgmtdb
1 ONLINE ONLINE PRODHOSTdb01 Open,STABLE
ora.qosmserver
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.scan1.vip
1 ONLINE ONLINE PRODHOSTdb02 STABLE
ora.scan2.vip
1 ONLINE ONLINE PRODHOSTdb01 STABLE
ora.scan3.vip
1 ONLINE ONLINE PRODHOSTdb01 STABLE
--------------------------------------------------------------------------------
grid@PRODHOSTDB02:[/home/grid]$
==>1.2.4 Patch Installation
###########################################################################################
==>check status of database connection and distribution before starting patch or any other information like invalid objects etc , you can capture it at this point
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
run your scripts if any to gather any details before proceeding with the patch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now shutdown the instance on the node-1, either you can use srvctl stop instance command or you can manually shutdown like below. I always prefere to manually shutdown instance like below:
oracle@PRODHOSTDB01:[/home/oracle]$sqlplus '/ as sysdba'
SQL*Plus: Release 12.2.0.1.0 Production on Thu Jan 30 12:11:54 2020
Copyright (c) 1982, 2017, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
SQL> shut immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>
==>Patching Oracle RAC Database Homes and GI Separately
•Case 1: Oracle RAC, where the GI Home and the Database Homes are not shared and ACFS file system is not configured.
root@PRODHOSTDB01:[/]#ps -ef|grep sql
root 29360580 43123118 0 12:14:52 pts/2 0:00 grep sql
root@PRODHOSTDB01:[/]#
==>For Grid Home:
================
---------------------------------------------------------------------------
su - grid
export GRID_HOME=/u01/app/12.2.0/grid <--Not Mandatory
which opatch
opatch version
==> Please make sure to change the full permission otherwise patching may fail, due to file permission issue.
root@IPYDRDB01:[/backup/Jan2020]#chmod 777 -r /backup/Jan2020/29699173
echo $PATH
export PATH=/u01/app/12.2.0/grid/OPatch:$PATH
opatch version
root@# /u01/app/12.2.0/grid/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/12.2.0/grid
==>Oracle - DB:
================
-------------------------------------------------------------------------
su - oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1
export PATH=/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:$PATH
echo $PATH
which opatch
opatch version
root@# /u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/oracle/product/12.2.0.1/dbhome_1
You can either patch GRID and RDBMS Homes separately or in single shot using below command which will patch both the homes with single command on the node.
====>To patch both the nodes to gather run below command.
As root user, execute the following command on each node of the cluster:
# <GI_HOME>/OPatch/opatchauto apply <UNZIPPED_PATCH_LOCATION>/29708720
~~~~~~~~~~~~~~~~~~~~~~Output patch apply~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
==>Node-1 : Grid Patching:
=============================
root@PRODHOSTDB01:[/]#export GRID_HOME=/u01/app/12.2.0/grid
root@PRODHOSTDB01:[/]#echo $PATH
/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB01:[/]#export PATH=/u01/app/12.2.0/grid/OPatch:$PATH
root@PRODHOSTDB01:[/]#echo $PATH
/u01/app/12.2.0/grid/OPatch:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB01:[/]#opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
root@PRODHOSTDB01:[/]#/u01/app/12.2.0/grid/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/12.2.0/grid
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /u01/app/12.2.0/grid/OPatch /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin/usr/java8_64/bin.
OPatchauto session is initiated at Thu Jan 30 12:17:43 2020
System initialization log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-01-30_12-17-51PM.log.
Session log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/opatchauto2020-01-30_12-18-19PM.log
The id for this session is JS77
Executing OPatch prereq operations to verify patch applicability on home /u01/app/12.2.0/grid
Patch applicability verified successfully on home /u01/app/12.2.0/grid
Bringing down CRS service on home /u01/app/12.2.0/grid
Prepatch operation log file location: /u01/app/grid/crsdata/PRODHOSTdb01/crsconfig/crspatch_PRODHOSTdb01_2020-01-30_12-18-58AM.log
CRS service brought down successfully on home /u01/app/12.2.0/grid
Start applying binary patch on home /u01/app/12.2.0/grid
Successfully executed command: /usr/sbin/slibclean
Binary patch applied successfully on home /u01/app/12.2.0/grid
Starting CRS service on home /u01/app/12.2.0/grid
Postpatch operation log file location: /u01/app/grid/crsdata/PRODHOSTdb01/crsconfig/crspatch_PRODHOSTdb01_2020-01-30_12-28-34AM.log
CRS service started successfully on home /u01/app/12.2.0/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:PRODHOSTdb01
CRS Home:/u01/app/12.2.0/grid
Version:12.2.0.1.0
Summary:
==Following patches were SUCCESSFULLY applied:
Patch: /backup/PATCH/Jan2020/30501932/26839277
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-21-35PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-21-35PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30586063
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-21-35PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30591794
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-21-35PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-21-35PM_1.log
OPatchauto session completed at Thu Jan 30 12:40:54 2020
Time taken to complete the session 23 minutes, 12 seconds
root@PRODHOSTDB01:[/]#
~~~~~~~~~~RDBMS/ORACLE_HOME patching output~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
root@PRODHOSTDB01:[/]#
root@PRODHOSTDB01:[/]#
root@PRODHOSTDB01:[/]#export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1
root@PRODHOSTDB01:[/]#export PATH=/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:$PATH
root@PRODHOSTDB01:[/]#echo $PATH
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:/u01/app/12.2.0/grid/OPatch:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB01:[/]#which opatch
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch
root@PRODHOSTDB01:[/]#opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
root@PRODHOSTDB01:[/]#/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/oracle/product/12.2.0.1/dbhome_1
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch /u01/app/12.2.0/grid/OPatch /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Thu Jan 30 12:48:26 2020
System initialization log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2020-01-30_12-48-35PM.log.
Session log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-01-30_12-49-28PM.log
The id for this session is 1172
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Preparing to bring down database service on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Successfully prepared home /u01/app/oracle/product/12.2.0.1/dbhome_1 to bring down database service
Bringing down database service on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Database service successfully brought down on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Performing prepatch operation on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Perpatch operation completed successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Start applying binary patch on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Successfully executed command: /usr/sbin/slibclean
Binary patch applied successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Performing postpatch operation on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Postpatch operation completed successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Starting database service on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Database service successfully started on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Preparing home /u01/app/oracle/product/12.2.0.1/dbhome_1 after database service restarted
No step execution required.........
Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0.1/dbhome_1
SQL patch applied successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:PRODHOSTdb01
RAC Home:/u01/app/oracle/product/12.2.0.1/dbhome_1
Version:12.2.0.1.0
Summary:
==Following patches were SKIPPED:
Patch: /backup/PATCH/Jan2020/30501932/30586063
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/26839277
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/30591794
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY applied:
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-50-06PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-01-30_12-50-06PM_1.log
Patching session reported following warning(s):
_________________________________________________
[WARNING] The database instance 'TESTDB1' from '/u01/app/oracle/product/12.2.0.1/dbhome_1', in host'PRODHOSTdb01' is not running. SQL changes, if any, will not be applied.
To apply. the SQL changes, bring up the database instance and run the command manually from any one node (run as oracle).
Refer to the readme to get the correct steps for applying the sql changes.
OPatchauto session completed at Thu Jan 30 12:54:26 2020
Time taken to complete the session 6 minutes, 0 second
root@PRODHOSTDB01:[/]#
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
####>Start the Cluster service if not started automatically:
>>As ROOT user and make sure GRID_HOME is set
crsctl start crs
root@TESTDB02:[/backup/PATCH]#echo $GRID_HOME
/u01/app/12.2.0.1/grid
root@TESTDB02:[/backup/PATCH]#$GRID_HOME/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
root@TESTDB02:[/backup/PATCH]#crsctl status res -t
>>To check status of Cluster services (all the CRS services are listed).
$crsctl status res -t
ps -ef|grep d.bin
sqlplus / as sysdba
>>>Take the lsinvntory details for DB and GRID homes.
opatch lsinventory -detail> grid_OPatch_lsinventory_detail_After_DBGrid_patch.lst
/u01/app/oracle/product/12.2.0/dbhome_1/OPatch/opatch lsinventory -detail >rdbms_OPatch_lsinventory_detail_After_DB_Patch.lst
==>start the instance:
grid@PRODHOSTDB01:[/home/grid]$srvctl status database -d TESTDB
Instance TESTDB1 is not running on node PRODHOSTdb01
Instance TESTDB2 is running on node PRODHOSTdb02
srvctl start instance -d TESTDB -i TESTDB1
srvctl status database -d TESTDB
grid@PRODHOSTDB01:[/home/grid]$srvctl status database -d TESTDB
Instance TESTDB1 is running on node PRODHOSTdb01
Instance TESTDB2 is running on node PRODHOSTdb02
grid@PRODHOSTDB01:[/home/grid]$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
grid@TESTDB01:[/home/grid]$crs_stat -t
grid@PRODHOSTDB01:[/home/grid]$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM.lsnr ora....er.type ONLINE ONLINE PRODHOSTdb01
ora.DATA.dg ora....up.type ONLINE ONLINE PRODHOSTdb01
ora.FRA.dg ora....up.type ONLINE ONLINE PRODHOSTdb01
ora.GIMR.dg ora....up.type ONLINE ONLINE PRODHOSTdb01
ora.INDX.dg ora....up.type ONLINE ONLINE PRODHOSTdb01
ora....ER.lsnr ora....er.type ONLINE ONLINE PRODHOSTdb01
ora....AF.lsnr ora....er.type OFFLINE OFFLINE
ora....N1.lsnr ora....er.type ONLINE ONLINE PRODHOSTdb01
ora....N2.lsnr ora....er.type ONLINE ONLINE PRODHOSTdb02
ora....N3.lsnr ora....er.type ONLINE ONLINE PRODHOSTdb02
ora.MGMTLSNR ora....nr.type ONLINE ONLINE PRODHOSTdb01
ora.OCR.dg ora....up.type ONLINE ONLINE PRODHOSTdb01
ora.REDO.dg ora....up.type ONLINE ONLINE PRODHOSTdb01
ora.asm ora.asm.type ONLINE ONLINE PRODHOSTdb01
ora.cvu ora.cvu.type ONLINE ONLINE PRODHOSTdb02
ora....01.lsnr application ONLINE ONLINE PRODHOSTdb01
ora....b01.ons application ONLINE ONLINE PRODHOSTdb01
ora....b01.vip ora....t1.type ONLINE ONLINE PRODHOSTdb01
ora....02.lsnr application ONLINE ONLINE PRODHOSTdb02
ora....b02.ons application ONLINE ONLINE PRODHOSTdb02
ora....b02.vip ora....t1.type ONLINE ONLINE PRODHOSTdb02
ora.TESTDB.db ora....se.type ONLINE ONLINE PRODHOSTdb01
ora....svc.svc ora....ce.type ONLINE ONLINE PRODHOSTdb01
ora.mgmtdb ora....db.type ONLINE ONLINE PRODHOSTdb01
ora....network ora....rk.type ONLINE ONLINE PRODHOSTdb01
ora.ons ora.ons.type ONLINE ONLINE PRODHOSTdb01
ora.proxy_advm ora....vm.type OFFLINE OFFLINE
ora.qosmserver ora....er.type ONLINE ONLINE PRODHOSTdb02
ora.scan1.vip ora....ip.type ONLINE ONLINE PRODHOSTdb01
ora.scan2.vip ora....ip.type ONLINE ONLINE PRODHOSTdb02
ora.scan3.vip ora....ip.type ONLINE ONLINE PRODHOSTdb02
grid@PRODHOSTDB01:[/home/grid]$
grid@TESTDB01:[/home/grid]$crsctl status res -t
grid@TESTDB01:[/home/grid]$
###>>Get the post Patch evidence from Node-1 After patch application:
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following command as respective Oracle home owner to check the consistency.
Note:Perform this on both the nodes:
For Grid:
============
$ /u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_After.lst
/u01/app/12.2.0/grid/OPatch/opatch lsinventory -detail -oh /u01/app/12.2.0/grid |tee -a /backup/PATCH/Jan2020/grid_OPatch_lsinventory_detail_After.lst
For RDBMS as oracle user:
=============
If this command succeeds, it lists the Oracle components that are installed in the home. Save the output so you have the status prior to the patch apply.
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_After.lst
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch lsinventory -detail -oh /u01/app/oracle/product/12.2.0.1/dbhome_1 |tee -a /backup/PATCH/Jan2020/rdbms_OPatch_lsinventory_detail_After.lst
~~~~~~~~~~~~~~~~~~~~~completion of node-1 patching activity~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
###====>Perform the similar steps on Node-2:
==>1.2.4 Patch Installation ON NODE-2
########################################################################################
sqlplus / as sysdba
01:28:49 SQL> shut immediate;
Note: Post shutdown of the instance one, observed that connections/txns where moved slowly to node-2, after 10-15 min all connection(total 3593) were moved to node-2. Below is the output from instance -2:
Patching Oracle RAC Database Homes and GI Separately
•Case 1: Oracle RAC, where the GI Home and the Database Homes are not shared and ACFS file system is not configured.
oracle@:[/home/oracle]$ps -ef| grep sqlplus
oracle 9307060 7275390 0 03:13:07 pts/3 0:00 grep sqlplus
oracle@:[/home/oracle]$
==>Grid HOME:
==============
---------------------------------------------------------------------------
==> Please make sure to change the full permission otherwise patching may fail, due to file permission issue.
root@TESTDB02:[/backup/PATCH]#chmod -R 775 /backup/PATCH/Jan2020
-->Make sure that instance is down, This is optional but it's better to shutdown instance manually to make sure it is cleanly shutting down.
Login as oracle and shutdown instance. "Shutdown immediate;"
-->Now as a root User:
export GRID_HOME=/u01/app/12.2.0/grid <--Not Mandatory
echo $PATH
export PATH=/u01/app/12.2.0/grid/OPatch:$PATH
which opatch
opatch version
root@# /u01/app/12.2.0/grid/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/12.2.0.1/grid
==>Oracle - DB:
==============
-------------------------------------------------------------------------
-->As a root user:
export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1 <-- Not mandatory
export PATH=/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:$PATH
echo $PATH
which opatch
opatch version
root@# /u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/oracle/product/12.2.0.1/dbhome_1
==>start the instance as an oracle user:
su - oracle
srvctl start instance -d TESTDB -i TESTDB2
srvctl status database -d TESTDB
~~~~~~~~~~~~output of patch application on node-2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
oracle@PRODHOSTDB02:[/home/oracle]$sqlplus / as sysdba
SQL*Plus: Release 12.2.0.1.0 Production on Sun Feb 2 14:19:07 2020
Copyright (c) 1982, 2017, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL>
-----------------
root@PRODHOSTDB02:[/]#echo $PATH
/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB02:[/]#export PATH=/u01/app/12.2.0/grid/OPatch:$PATH
root@PRODHOSTDB02:[/]#echo $PATH
/u01/app/12.2.0/grid/OPatch:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB02:[/]#opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
root@PRODHOSTDB02:[/]#which opatch
/u01/app/12.2.0/grid/OPatch/opatch
root@PRODHOSTDB02:[/]#/u01/app/12.2.0/grid/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/12.2.0/grid
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /u01/app/12.2.0/grid/OPatch /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Sun Feb 2 14:39:53 2020
System initialization log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-02-02_02-40-02PM.log.
Session log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/opatchauto2020-02-02_02-40-41PM.log
The id for this session is XEQZ
Executing OPatch prereq operations to verify patch applicability on home /u01/app/12.2.0/grid
Patch applicability verified successfully on home /u01/app/12.2.0/grid
Bringing down CRS service on home /u01/app/12.2.0/grid
Prepatch operation log file location: /u01/app/grid/crsdata/PRODHOSTdb02/crsconfig/crspatch_PRODHOSTdb02_2020-02-02_02-41-20PM.log
CRS service brought down successfully on home /u01/app/12.2.0/grid
Start applying binary patch on home /u01/app/12.2.0/grid
Successfully executed command: /usr/sbin/slibclean
Binary patch applied successfully on home /u01/app/12.2.0/grid
Starting CRS service on home /u01/app/12.2.0/grid
Postpatch operation log file location: /u01/app/grid/crsdata/PRODHOSTdb02/crsconfig/crspatch_PRODHOSTdb02_2020-02-02_02-49-56PM.log
CRS service started successfully on home /u01/app/12.2.0/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:PRODHOSTdb02
CRS Home:/u01/app/12.2.0/grid
Version:12.2.0.1.0
Summary:
==Following patches were SUCCESSFULLY applied:
Patch: /backup/PATCH/Jan2020/30501932/26839277
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_14-42-55PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_14-42-55PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30586063
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_14-42-55PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30591794
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_14-42-55PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_14-42-55PM_1.log
OPatchauto session completed at Sun Feb 2 14:55:05 2020
Time taken to complete the session 15 minutes, 12 seconds
root@PRODHOSTDB02:[/]#
==>Output of patch apply for RDBMS Home on Node-2:
===================================================
root@PRODHOSTDB02:[/]#echo $PATH
/u01/app/12.2.0/grid/OPatch:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB02:[/]#export PATH=/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:$PATH
root@PRODHOSTDB02:[/]#echo $PATH
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch:/u01/app/12.2.0/grid/OPatch:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin
root@PRODHOSTDB02:[/]#which opatch
/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatch
root@PRODHOSTDB02:[/]#opatch version
OPatch Version: 12.2.0.1.19
OPatch succeeded.
root@PRODHOSTDB02:[/]#
root@PRODHOSTDB02:[/]#/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch/opatchauto apply /backup/PATCH/Jan2020/30501932 -oh /u01/app/oracle/product/12.2.0.1/dbhome_1
which: 0652-141 There is no sudo in /bin /usr/bin /usr/local/bin /u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch /u01/app/12.2.0/grid/OPatch /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/java8_64/jre/bin /usr/java8_64/bin.
OPatchauto session is initiated at Sun Feb 2 15:03:59 2020
System initialization log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchautodb/systemconfig2020-02-02_03-04-07PM.log.
Session log file is /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/opatchauto2020-02-02_03-04-47PM.log
The id for this session is F3V4
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0.1/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Preparing to bring down database service on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Successfully prepared home /u01/app/oracle/product/12.2.0.1/dbhome_1 to bring down database service
Bringing down database service on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Database service successfully brought down on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Performing prepatch operation on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Perpatch operation completed successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Start applying binary patch on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Successfully executed command: /usr/sbin/slibclean
Binary patch applied successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Performing postpatch operation on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Postpatch operation completed successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Starting database service on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Database service successfully started on home /u01/app/oracle/product/12.2.0.1/dbhome_1
Preparing home /u01/app/oracle/product/12.2.0.1/dbhome_1 after database service restarted
No step execution required.........
Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0.1/dbhome_1
SQL patch applied successfully on home /u01/app/oracle/product/12.2.0.1/dbhome_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:PRODHOSTdb02
RAC Home:/u01/app/oracle/product/12.2.0.1/dbhome_1
Version:12.2.0.1.0
Summary:
==Following patches were SKIPPED:
Patch: /backup/PATCH/Jan2020/30501932/30586063
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/26839277
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /backup/PATCH/Jan2020/30501932/30591794
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY applied:
Patch: /backup/PATCH/Jan2020/30501932/30585969
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_15-05-26PM_1.log
Patch: /backup/PATCH/Jan2020/30501932/30593149
Log: /u01/app/oracle/product/12.2.0.1/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-02-02_15-05-26PM_1.log
Patching session reported following warning(s):
_________________________________________________
[WARNING] The database instance 'TESTDB2' from '/u01/app/oracle/product/12.2.0.1/dbhome_1', in host'PRODHOSTdb02' is not running. SQL changes, if any, will not be applied.
To apply. the SQL changes, bring up the database instance and run the command manually from any one node (run as oracle).
Refer to the readme to get the correct steps for applying the sql changes.
OPatchauto session completed at Sun Feb 2 15:09:45 2020
Time taken to complete the session 5 minutes, 47 seconds
root@PRODHOSTDB02:[/]#
root@PRODHOSTDB02:[/]#
root@PRODHOSTDB02:[/]#su - oracle
oracle@PRODHOSTDB02:[/home/oracle]$
oracle@PRODHOSTDB02:[/home/oracle]$
oracle@PRODHOSTDB02:[/home/oracle]$srvctl start instance -d TESTDB -i TESTDB2
oracle@PRODHOSTDB02:[/home/oracle]$
oracle@PRODHOSTDB02:[/home/oracle]$srvctl status database -d TESTDB
Instance TESTDB1 is running on node PRODHOSTdb01
Instance TESTDB2 is running on node PRODHOSTdb02
oracle@PRODHOSTDB02:[/home/oracle]$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~END of output of patch application on binary on node-2~~~~~~~~~~~~~~~~~~~~~~~~~~
####==>Datapatch application: only should be run from one node only.
==>Run as an oracle user
cd $ORACLE_HOME/OPatch
./datapatch -verbose
~~~~~~~~~~~~~~~~~~~~~~ ==>o/p of :Datapatch application: only should be run from one node only.~~~~~~~~~~~~
oracle@PRODHOSTDB02:[/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch]$./datapatch -verbose
SQL Patching tool version 12.2.0.1.0 Production on Sun Feb 2 15:21:24 2020
Copyright (c) 2012, 2020, Oracle. All rights reserved.
Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_34800046_2020_02_02_15_21_24/sqlpatch_invocation.log
Connecting to database...OK
Bootstrapping registry and package to current versions...done
Determining current state...done
Current state of SQL patches:
Bundle series DBRU:
ID 200114 in the binary registry and not installed in the SQL registry
Adding patches to installation queue and performing prereq checks...
Installation queue:
Nothing to roll back
The following patches will be applied:
30593149 (DATABASE JAN 2020 RELEASE UPDATE 12.2.0.1.200114)
Installing patches...
Patch installation complete. Total patches installed: 1
Validating logfiles...
Patch 30593149 apply: WITH ERRORS (RETRYABLE)
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30593149/23339232/30593149_apply_TESTDB_2020Feb02_15_22_57.log (errors)
Error at line 10007: ORA-04021: timeout occurred while waiting to lock object SYS.DBMS_AQ
Error at line 10051: ORA-04021: timeout occurred while waiting to lock object SYS.DBMS_AQADM_VAR
Error at line 10057: ORA-04021: timeout occurred while waiting to lock object SYS.DBMS_AQADM_SYS
Adding patches to retry installation queue...
Retry installation queue:
Nothing to roll back
The following patches will be applied:
30593149 (DATABASE JAN 2020 RELEASE UPDATE 12.2.0.1.200114)
Installing patches...
Patch installation complete. Total patches installed: 1
Validating logfiles...
Patch 30593149 apply: SUCCESS
logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/30593149/23339232/30593149_apply_TESTDB_2020Feb02_16_11_06.log (no errors)
SQL Patching tool complete on Sun Feb 2 16:21:59 2020
oracle@PRODHOSTDB02:[/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch]$
oracle@PRODHOSTDB02:[/u01/app/oracle/product/12.2.0.1/dbhome_1/OPatch]$
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~End of datapatch application~~~~~~~~~~~~~~~~~~~~~~~~~
-----While datapatch faced issue of below as AQ job were running: If you also encounter similar issue while apply datapatch and $AQ- Advanced queue jobs are running then refer this article for the solution. :datapatch execution takes time and fails with ORA-04021 as AQ jobs are running while PSU Patching on 12c
~~>Check the status of datapatch/modified SQL
set line 500
col patch_id for 9999999999
col patch_uid for 9999999999
col version for a10
col action for a10
col install_id for 99
col action_time for a30
col description for a60
col bundle_series for a20
col status for a30
select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series,status from dba_registry_sqlpatch;
SQL> set line 500
SQL> col patch_id for 9999999999
SQL> col patch_uid for 9999999999
SQL> col version for a10
SQL> col action for a10
SQL> col install_id for 99
SQL> col action_time for a30
SQL> col description for a60
SQL> col bundle_series for a20
SQL> col status for a30
SQL>
SQL> select patch_id, patch_uid, version, action, install_id, action_time, description, bundle_series,status from dba_registry_sqlpatch;
PATCH_ID PATCH_UID VERSION ACTION INSTALL_ID ACTION_TIME DESCRIPTION BUNDLE_SERIES STATUS
----------- ----------- ---------- ---------- ---------- ------------------------------ ------------------------------------------------------------ -------------------- ------------------------------
30593149 23339232 12.2.0.1 APPLY 1 02-FEB-20 04.11.06.311937 PM DATABASE JAN 2020 RELEASE UPDATE 12.2.0.1.200114 DBRU WITH ERRORS (RETRYABLE)
30593149 23339232 12.2.0.1 APPLY 1 02-FEB-20 04.21.59.239175 PM DATABASE JAN 2020 RELEASE UPDATE 12.2.0.1.200114 DBRU SUCCESS
SQL>
Hope this helps.
