When trying to bring up the Oracle Service Group - listener-failing to stay online - TNS-01106: Listener using listener name listener has already been started



ERROR CODEMESSAGE
TNS-01106 Listener using listener name listener has already been started

Oracle Errors ; Oracle Error  TNS-01106 Listener using listener name string has already been started Cause Another listener is already listening on one of the IP addresses specified.
Action Shutdown the previous listener specified by the listener name before starting this one.



DIAGNOSTIC STEPS

Cleared listener resource fault(s)

# hares -clear listener

Brought the Service Group online to allow for the VCS engine_A.log log file contents to be reviewed

# hagrp -online DB_Group -sys alpimdb01

Froze the Service Group enabling the user to troubleshoot the possible timeout issue prior to the resource faulting.

# hagrp -freeze  DB_Group -sysalpimdb01

Db_OAM_VIP  10.162.13.125 is fine

Unable to ping DB_VIP  172.18.160.77


The following route modifications were not done earlier

route delete gateway 192.168.82.1
route delete default 192.168.82.1


netstat -rn  revealed that the netmask was wrong for the DB_VIP virtual IP.

# hastop -all-force

Edited the main.cf and changed the netmask to 255.255.255.0


IPMultiNIC DB_OAM_VIP (
Address =10.162.13.125
NetMask = 255.255.255.0
MultiNICResName =MNIC_OAM
IfconfigTwice = 1
)

IPMultiNIC DB_VIP (
Address= 172.18.160.77
NetMask =255.255.255.0          was255.255.254.0
MultiNICResName = MNIC_Res
IfconfigTwice =1
)

# hastart

Offlined the Service Group again

#  hagrp -offline DB_Group -sys alpimdb01

#hares -clear listener





PROBLEMDESCRIPTION

Unable to keep VCS listener process online

CAUSE

Once the netmask was amended in the main.cf, the VCS controlled resource was then able to ping the DB_VIPaddress

IPMultiNIC DB_VIP (
Address = 172.18.160.77
NetMask =255.255.255.0          was255.255.254.0
MultiNICResName = MNIC_Res
IfconfigTwice =1
)

# ifconfig -a
lo0flags=2001000849UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL mtu 8232 index1
       inet 127.0.0.1 netmask ff000000
ce0flags=1000843UP,BROADCAST,RUNNING,MULTICAST,IPv4 mtu 1500 index2
       inet 172.18.160.121 netmask ffffff00 broadcast172.18.160.255
       ether 0144f1ee4d5
ce01flags=1000843UP,BROADCAST,RUNNING,MULTICAST,IPv4 mtu 1500 index2
       inet 172.18.160.77 netmask ffffff00 broadcast172.18.160.255
ce1 flags=1000843UP,BROADCAST,RUNNING,MULTICAST,IPv4 mtu 1500index 3
       inet 10.162.13.102 netmask ffffff00broadcast 10.162.13.255
       ether 0144f1ee4d6
ce11 flags=1000843UP,BROADCAST,RUNNING,MULTICAST,IPv4 mtu 1500 index3
       inet 10.162.13.125 netmask ffffff00 broadcast10.162.13.255
ce2 flags=1000843UP,BROADCAST,RUNNING,MULTICAST,IPv4 mtu 1500index 4
       inet 192.168.82.189 netmask fffffe00broadcast 192.168.83.255
       ether 0144f1ee4bd

#  ping 172.18.160.77
172.18.160.77 is alive


SOLUTION

Steps

1.] ps -ef output revealed additional LISTENER process was running:

oracle 3322 1 0051414  000 applodigooraclebintnslsnr LISTENER -inherit
oracle 2599 1 0051300  002 ora_mmon_ODGSRC oracle 3703 1 0 051520  000apploracleproduct10.2.0bintnslsnr LISTENER -inherit
oracle 2952 1 0051309  000 ora_arc0_ODGSRC



2.] engine_A log stated the following details:


Copyright (c) 1991, 2006, Oracle. All rights reserved.

TNS-01106 Listener using listener name listener has already been started
+====================================================================+
20091031051324 VCS INFO V-16-1-50135 User root fired command hagrp -freeze DB_Group from localhost q


3.] Stopped the current listener process

killed the 3322 listener process

4.] Started the lsnrctl start

Which is using Oracle product version10.2.0