ORA-00205: error in identifying control file after GI PSU 12.2.0.1.200114 upon instance startup

Problem Statement:

Today, I encountered interesting but weird problem while successfully applying Patch 30501932 - GI Jan 2020 Release Update 12.2.0.1.200114 on node-1  when I tried to shutdown immediate for instance -2 for applying patch, it did not shutdown and session and was keep waiting for shutdown so I had to abnormally stop shutdown and issued shutdown abort, but post shutdown when I tried to start up instance-2, it throw below set of errors:

SQL> startup

ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file '/u01/app/oracle/product/12.2.0.1/dbhome_1/dbs/inittestdb2.ora'
SQL> exit


Created pfile from SPFILE on node-1 and copied to node-2 and tried starting instance 2 using newly created pfile:

SQL> startup
ORA-01078: failure in processing system parameters
LRM-00109: could not open parameter file '/u01/app/oracle/product/12.2.0.1/dbhome_1/dbs/inittestdb2.ora'
SQL>
SQL>  startup pfile='/tmp/inittestdb.ora';
ORACLE instance started.

Total System Global Area 5.5298E+10 bytes
Fixed Size                 29964104 bytes
Variable Size            8724154552 bytes
Database Buffers         4.6439E+10 bytes
Redo Buffers              104251392 bytes

ORA-00205: error in identifying control file, check alert log for more info

Below is the list of Error reported in  alert.log file:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
WARNING: failed to register ASMB0 with ASM instance
2020-01-30T14:32:27.133429+03:00
Errors in file /u01/app/oracle/diag/rdbms/testdb/testdb2/trace/testdb2_asmb_31261082.trc:
ORA-12547: TNS:lost contact
ORA-12547: TNS:lost contact
WARNING: ASMB0 exiting with error
2020-01-30T14:32:27.137941+03:00
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA/testdb/CONTROLFILE/current.261.1030897861'
ORA-17503: ksfdopn:2 Failed to open file +DATA/testdb/CONTROLFILE/current.261.1030897861
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-12547: TNS:lost contact
ORA-12547: TNS:lost contact
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA/testdb/CONTROLFILE/current.262.1030897861'
ORA-17503: ksfdopn:2 Failed to open file +DATA/testdb/CONTROLFILE/current.262.1030897861
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-12547: TNS:lost contact
ORA-12547: TNS:lost contact
2020-01-30T14:32:27.145437+03:00
ORA-205 signalled during: ALTER DATABASE   MOUNT...
2020-01-30T14:44:12.824562+03:00
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Issue Observation:

I observed that even though +ASM2 was up and all disks were mounted on node-2 still Instance-2 was not able to access diskgroups and hence upon DB startup  it could not access SPFILE located on ASM diskgroup and hence oracle tried to start Instance with default PFILE but still it couldn't start as controlfile was not accessible due to diskgroup accessibility issue. 

Further more Observed that after successfully applying GI Jan 2020 PATCH on node-1, even without applying patch on node-2, it had changed setuid permission  for GI oracle binary file on node-2 and because of that Oracle was not able to access diskgroup upon instance startup, please find below evidence:


setuid permission for owner and group is changed from s to x and that's the issue.

On Node-2:

oracle@TESTHOST02:[/u01/app/12.2.0/grid/bin]$ls -rlta |grep oracle
-rw-------    1 grid     oinstall  579571814 Jan 27 13:45 oracleO
-rwxr-x--x    1 grid     oinstall  579571814 Jan 27 14:29 oracle
oracle@TESTHOST02:[/u01/app/12.2.0/grid/bin]$


Please observe setuid for GI Oracle binary on node-1, which is proper.

On Node-1:
===================
root@TESTHOST01:[/]#ls -lrta /u01/app/12.2.0/grid/bin/oracle
-rwsr-s--x    1 grid     oinstall  583710697 Jan 30 12:27 /u01/app/12.2.0/grid/bin/oracle
root@TESTHOST01:[/]#

Solution:

Perform below steps on node-2, (affected node), which should fix the issue and Instance on node-2 should come up just fine without any issue.

1.shutdown CRS as root

root@TESTHOST02:[/]#/u01/app/12.2.0/grid/bin/crsctl stop crs

2. Set setuid  for Grid oracle binary. 


root@TESTHOST02:[/u01/app/12.2.0/grid/bin]$ls -rlta |grep oracle
-rwxr-x--x    1 grid     oinstall  579571814 Jan 27 14:29 oracle


root@TESTHOST02:[/]#chmod 6751 /u01/app/12.2.0/grid/bin/oracle

root@TESTHOST02:[/]#ls -lrta /u01/app/12.2.0/grid/bin/oracle
-rwsr-s--x    1 grid     oinstall  579571814 Jan 27 14:29 /u01/app/12.2.0/grid/bin/oracle

3. Start the CRS

root@TESTHOST02:[/]#/u01/app/12.2.0/grid/bin/crsctl start crs