0

We have a redhat virtual machine (vmware vsphere) that is hosting the database for our application. The application runs normally, until at some point it crashes with the error below seen on db2diag:

2015-03-05-14.35.13.380939+000 E1406160203E1391      LEVEL: Error (OS)
PID     : 19969                TID : 47269155956480  PROC : db2sysc 0
INSTANCE: db2inst1             NODE : 000            DB   : SIMTVAIS
APPHDL  : 0-15775              APPID: ::ffff:10.86.22.164.51178.150305082202
AUTHID  : ESERVIS              HOSTNAME: DbSimpl
EDUID   : 120                  EDUNAME: db2agent (SIMTVAIS) 0
FUNCTION: DB2 UDB, oper system services, sqloseekwrite64, probe:40
MESSAGE : ZRC=0x840F0001=-2079391743=SQLO_ACCD "Access Denied"
DIA8701C Access denied for resource "", operating system return code was "".
CALLED  : OS, -, pwrite                           OSERR: EROFS (30)
DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
  File Handle              = 751
  File System Block Size   = 4096 bytes
  File System Type         = ext4
  File Handle Flags :
  Require Sector Align   = No
  DIO/CIO Mode           = No
Raw Block Device       = No
Reserved Handle        = No
Flush On Close         = No
Thread-Level Lock      = No
Write-through Mode     = Yes
File Not Tracked       = Yes
DATA #2 : unsigned integer, 8 bytes
4096
DATA #3 : signed integer, 8 bytes
1172877312
DATA #4 : signed integer, 8 bytes
-1
DATA #5 : String, 105 bytes
Search for ossError*Analysis probe point after this log entry for
further self-diagnosis of this problem.

When we check the OS syslog we see :

Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: timing out command, waited 180s
Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: [sdb] Unhandled error code
Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: [sdb] CDB: Write(10): 2a00
Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: [sdb] Unhandled error code
Mar  5 14:34:53 DbSimpl kernel: 08 82 44 b9 00 00 08 00
Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
Mar  5 14:34:53 DbSimpl kernel: Buffer I/O error on device dm-4, logical block 17844119
Mar  5 14:34:53 DbSimpl kernel: sd 2:0:1:0: [sdb] CDB: Write(10): 2a 00 09 1c 4b 41 00 00 08 00
Mar  5 14:34:53 DbSimpl kernel: lost page write due to I/O error on dm-4

Can anyone help me figure out the root cause of the problem?

1
  • physical storage ok? you say vmware, I assume sdb is a virtual disk
    – Dan
    Mar 6, 2015 at 7:33

1 Answer 1

0

The I/O subsystem where your /dev/sdb is served from is not responding in a timely manner; your DB2 guest server tried to write something to that disk, but gave up after three minutes of waiting. Investigate that side of things.

6
  • thanks for your answer, can you help me on how to check the error on the subsystem ?
    – hamzahir
    Mar 6, 2015 at 7:06
  • I'm not a psychic so I need lots more data before I can help. I have no idea where that /dev/sdb is connected to and how loaded that target might possibly be. Is that some kind of iSCSI target? A SAN? Are there just a few other virtual machines or hundreds/thousands of them? Is your DB2 server the only one having issues or are other VMs having problems as well? How about the logs at the VMware side? And so on, and so on... Mar 6, 2015 at 7:19
  • thanks for your quick answer, i am going to check on the vmware logs and get back to you, so if i understand well the database server db2 isnt the issue here, right ?
    – hamzahir
    Mar 6, 2015 at 7:42
  • Not the server itself, probably. Hard to say with 100% confidence with such little background information -- imagine if we switched places and you should answer to me with only data being two short pastes and no information about other environment whatsoever. Mar 6, 2015 at 7:48
  • it is an iSCSI target and there are other virtual machines alongside but the problem is only visible on the db2 server; i am gonna get back to you when i have more data on the vmware logs
    – hamzahir
    Mar 6, 2015 at 8:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .