Having a problem with vCenter service stopping daily

Question

I have vCenter 4 running on a Windows 2008 VM and an Oracle 11g R2 database. I have identical configurations in an Operational Environment and a Non-Operational Environment. In my NOE, every morning I have to start the vCenter service in Windows--it fails overnight. The logs show a DB error, but nothing specific. Because Oracle 11.2 is not officially supported by VMware, they can't help me figure it out. Is there anyone with some Oracle & VMware experience who can help me figure this out? I've attached the logs from when it quits at night.

[2010-12-10 02:22:53.021 06560 error 'App'] [VdbStatement::PrepareHandle] Error on preparing statement INSERT INTO VPX_EVENT (EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME, USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME, COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME, DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME, NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME, CHANGE_TAG_ID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[2010-12-10 02:22:56.021 06560 error 'App'] [VdbStatement::PrepareHandle] Error on preparing statement INSERT INTO VPX_EVENT (EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME, USERNAME, CATEGORY, VM_ID, VM_NAME, HOST_ID, HOST_NAME, COMPUTERESOURCE_ID, COMPUTERESOURCE_TYPE, COMPUTERESOURCE_NAME, DATACENTER_ID, DATACENTER_NAME, DATASTORE_ID, DATASTORE_NAME, NETWORK_ID, NETWORK_NAME, NETWORK_TYPE, DVS_ID, DVS_NAME, CHANGE_TAG_ID) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[2010-12-10 02:22:56.021 06560 error 'App'] Error inserting events: ODBC error: (08S01) - [Oracle][ODBC][Ora]ORA-03113: end-of-file on communication channel
Process ID: 6553
Session ID: 147 Serial number: 2246

[2010-12-10 02:22:56.021 06560 warning 'VpxProfiler'] VpxdMoEventManager::ProcessEventChanges took 3015 ms
[2010-12-10 02:22:56.021 06560 error 'App'] [ProcessEventChanges] Unhandled exception.
[2010-12-10 02:22:56.021 06560 warning 'VpxProfiler'] [VpxdHostSync] ProcessChanges host:hostname (xx.xx.xx.xx) took 3109 ms
[2010-12-10 02:22:56.021 06560 warning 'VpxProfiler'] InvtHostSyncLRO::StartWork took 4359 ms
[2010-12-10 02:22:56.021 06560 error 'App'] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) ODBC error: (08S01) - [Oracle][ODBC][Ora]ORA-03113: end-of-file on communication channel
Process ID: 6553
Session ID: 147 Serial number: 2246

[2010-12-10 02:22:56.021 06560 info 'App'] Forcing shutdown of VMware VirtualCenter now

Answer 1

My experience with vCenter and Oracle is only anecdotal but I've never been impressed with vCenter running against an Oracle DB, there are always issues even with the supported versions. Oher Oracle\vCenter problems have cropped up here before and my experience doesn't seem to be unique ( e.g. VMWare VCenter Server 4.1 & Oracle ).

In your case my only advice is that you should never run vCenter against an unsupported DB - if your environment is not large (less than 5 hosts / 50 VMs) you are in a much better support position if you use SQL Express, despite its limitations. Given the licensing costs of any VMware infrastructure that is larger than that you really should push to get budget to install a supported DB server.

Answer 2

I'm not so good with Oracle, but I have some basic vCenter Questions:

• Is your vCenter host 64bit are you running 4.1?

• What build of vSphere are you running, have you upgraded to 4.1?

• Are you using 32bit or 64bit DSNs?

Answer 3

I think the problem is a odbc driver

This Error:

2010-12-10 02:22:56.021 06560 error 'App'] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) ODBC error: (08S01) - [Oracle][ODBC][Ora]ORA-03113: end-of-file on communication channel
Process ID: 6553
Session ID: 147 Serial number: 2246

shows that the connection (in my opinion) breaked up. The best way is to go the supported way. But here is a (german) Site about configuring an Oracle ODBC Connection (the screenshots should help)

http://www.virture.de/oracle-odbc-64bit-und-32bit-dsn-auf-einem-system-mit-dem-oracle-instant-client/

Answer 4

It is probably a firewall timing out the connection between the server and the database. See http://stackoverflow.com/questions/83068/ora-03113-end-of-file-on-communication-channel-after-long-inactivity-in-asp-net for more information.