Error Message

Feb  6 03:23:44 system407 AgentFramework[1057]: [ID 702911 daemon.notice] VCS ERROR V-16-1-13067 Thread(7) Agent is calling clean for resource(Vol22-cfs) because the resource became OFFLINE unexpectedly, on its own.

Engine_A.log

2012/02/06 03:19:33 VCS INFO V-16-2-13075 (system407) Resource(Vol22-cfs) has reported unexpected OFFLINE 1 times, which is still within the ToleranceLimit(2).
2012/02/06 03:21:39 VCS INFO V-16-2-13075 (system407) Resource(Vol22-cfs) has reported unexpected OFFLINE 2 times, which is still within the ToleranceLimit(2).
2012/02/06 03:23:44 VCS ERROR V-16-2-13067 (system407) Agent is calling clean for resource(Vol22-cfs) because the resource became OFFLINE unexpectedly, on its own.
2012/02/06 03:23:44 VCS NOTICE V-16-20011-5511 (system407) CFSMount:Vol22-cfs:clean:clean unexpected offline : Mount Point : /pvcs6
2012/02/06 03:23:51 VCS INFO V-16-2-13068 (system407) Resource(Vol22-cfs) - clean completed successfully.
2012/02/06 03:23:51 VCS ERROR V-16-2-13073 (system407) Resource(Vol22-cfs) became OFFLINE unexpectedly on its own. Agent is restarting (attempt number 1 of 2) the resource.
 

Underlying scsi errors caused to invoke the file system disable. V-2-31: vx_disable file system disabled.

/etc/vx/dmpevents log:  (Scsi Errors on the same Disk)
Mon Feb  6 03:17:51.725: I/O analysis done as DMP_PATH_OKAY on Path c4t5005552323d3s2 belonging to Dmpnode emc1_1b48
Mon Feb  6 03:17:51.725: I/O retry(4) on Path c4t5005552323d3s2 belonging to Dmpnode emc1_1b48
Mon Feb  6 03:17:51.738: SCSI error occured on Path c4t5005552323d3s2: opcode=0x2a reported reservation conflict (status=0x18, key=0x0, asc=0x0, ascq=0x0)                                    -----------> Underlying scsi error on the same disk.
Mon Feb  6 03:17:51.738: I/O analysis done as DMP_DISK_FAILURE on Path c4t5005552323d3s2 belonging to Dmpnode emc1_1b48
Mon Feb  6 03:17:51.738: I/O error occured (errno=0x5) on Dmpnode emc1_1b48

dmesg output :  
Feb  6 03:10:01 system407 syslogd: configuration restart
Feb  6 03:17:51 system407 vxdmp: [ID 443116 kern.notice] NOTICE: VxVM vxdmp V-5-0-0 i/o error occured (errno=0x5) on dmpnode 327/0xbd2
Feb  6 03:17:51 system407 vxio: [ID 917197 kern.warning] WARNING: VxVM vxio V-5-0-1266 Subdisk liv_globalpvcspdg52-03 block 123952: Uncorrectable write error
Feb  6 03:17:51 system407 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 43 mesg 037: V-2-37: vx_metaioerr - vx_inode_iodone - /dev/vx/dsk/pvcspdg/Vol_pvcs6 file system meta data write error in dev/block 0/167641648
Feb  6 03:17:51 system407 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 44 mesg 037: V-2-37: vx_metaioerr - vx_inode_iodone - /dev/vx/dsk/pvcspdg/Vol_pvcs6 file system meta data write error in dev/block 0/167641648
Feb  6 03:17:59 system407 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 45 mesg 037: V-2-37: vx_metaioerr - vx_cbuf_revoke_blk - /dev/vx/dsk/pvcspdg/Vol_pvcs6 file system meta data write error in dev/block 0/167641648
Feb  6 03:17:59 system407 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 46 mesg 031: V-2-31: vx_disable - /dev/vx/dsk/liv_globalpvcspdg/Vol22_cfs file system disabled                                                                                ----------------------> file system disabled due to block errors.

Feb  6 03:17:59 system407 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 48 mesg 037: V-2-37: vx_metaioerr - vx_inode_iodone - /dev/vx/dsk/liv_globalpvcspdg/Vol_pvcs6 file system meta data write error in dev/block 0/197696
Feb  6 03:17:59 system407 vxfs: [ID 702911 kern.warning] WARNING: msgcnt 47 mesg 037: V-2-37: vx_metaioerr - vx_inode_iodone - /dev/vx/dsk/pvcspdg/Vol_pvcs6 file system meta data write error in dev/block 0/201136

Unmount the file system and use fsck to run a full structural check. If the problem is a disk failure, replace the disk before the file system is mounted for write access.

Applies To

Solaris 10. SFHACFS :  5.1SP1