0

I'm an administrator IT and also repair desktop computers & servers for many years.

We have an DL580 G7 whose warranty expired 12/31/2015. Originally supplied with Xeon E7530 and standard memory cartridges, we recently upgraded it to E7-4870 and E7 memory cartridges. Since then, we are in big trouble to ensure that it works. In order to locate the issue, I proceeded with a sequential test:

1) PROC1=A, PROC2=B, PROC3=X, PROC4=X, POST=TRUE
2) PROC1=C, PROC2=D, PROC3=X, PROC4=X, POST=TRUE
3) PROC1=A, PROC2=X, PROC3=B, PROC4=X, POST=FALSE
4) PROC1=A, PROC2=X, PROC3=X, PROC4=B, POST=TRUE
5) PROC1=A, PROC2=C, PROC3=X, PROC4=B, POST=TRUE
6) PROC1=A, PROC2=C, PROC3=D, PROC4=B, POST=FALSE
7) PROC1=E, PROC2=F, PROC3=X, PROC4=X, POST=TRUE
8) PROC1=E, PROC2=X, PROC3=F, PROC4=X, POST=TRUE
9) PROC1=E, PROC2=G, PROC3=F, PROC4=H, POST=TRUE

A,B,C,D are E7-4870 (SLC3T) 130W TDP
E,F,G,H are E7530 (SLBRJ) 105W TDP
X is empty socket

TRUE = Server post and run OS (WS2012 R2 DataCenter)
FALSE = Server hang before post (display last post code 4048 on ilo3, then post code disappear and no video, same result connected to VGA port)

Server parts:
DL580 G7 (584087-421)
Motherboard (512843-001/591196-001)
CPU Board (583367-001/591197-001)
I/O Board (512844-001/591199-001)
E7 ram cartridge (650761-001/647058-001)
Standard ram cartridge (617524-001/591198-001)
Micron 4GB ram dimm (500203-061)
4xPSU 1200W (438203-001/498152-001)

System ROM P65 10/01/2013 (reset to defaults)
iLO Firmware Version 1.85

According to Hp QuickSpecs for this particular ProLiant server, E7-4800 series are only supported in DL580 G7 CTO (643086-B2x) & Preconfigured models (6430xx-xx1/6967xx-xx1). From what I know, the main difference with a entry-level are E7 cartridges. Motherboard, CPU board and I/O board are the same part numbers.

To resume, it seems that server doesn't post when socket PROC3 is mounted with a E7-4870 (test all four 4870 on it, same result). I suspect a electrical/power issue on this socket because TDP is higher than E7530.

I plan to replace CPU board first, any advices are welcome.

Do any of you have already met such a problem ?

3

We have just upgraded three DL580 G7's that are model 584084-001 to Xeon E7-4870 processors. We found that what mattered in our environment was the revision of the main system board (not the CPU board/tray or SPI riser). After reading that last response, I checked iLO to verify all of our PAL versions. We do have one server that is running 4 E7-4870 processors with 1TB of memory on a CPU board with PAL version 0x10.

We found that as long as the system MainIO board (hp part 512843-001) is revision 0B or later, it works. Two of the three servers had revision 0A of the MainIO board and they would not post with the new processors. On those systems, all we replaced was the mainboard and the result was a working server.

Everything just worked on the server with the newer MainIO board revision, so we initially tried moving its CPU/Memory tray to the servers that were not working. When they still failed to work we decided that the issue lied outside of the CPU/Memory tray. The revision on the MainIO board was the only difference between the non-working servers and the working one.

We ordered the mainboards from our usual supplier, but had no way to specify revision. We just ordered and then verified the revision before installing.

EDIT: Adding the diagnostic info on our working CPLD / PAL versions:

Server 1:
 - CPLD PAL0:   ProLiant DL580 G7 SPI Board PAL version 0x0F
 - CPLD PAL1:   ProLiant DL580 G7 MainIO Board PAL version 0x0F
 - CPLD PAL2:  ProLiant DL580 G7 CPU Board PAL version 0x0E

Server 2:
 - CPLD PAL0:   ProLiant DL580 G7 SPI Board PAL version 0x0F
 - CPLD PAL1:   ProLiant DL580 G7 MainIO Board PAL version 0x0F
 - CPLD PAL2:   ProLiant DL580 G7 CPU Board PAL version 0x0F

Server 3:
 - CPLD PAL0:   ProLiant DL580 G7 SPI Board PAL version 0x10
 - CPLD PAL1:   ProLiant DL580 G7 MainIO Board PAL version 0x11 bootleg
   0x10
 - CPLD PAL2:   ProLiant DL580 G7 CPU Board PAL version 0x10
  • Great response @Clint ! Now, we know more precisely that the main incompatibility issue is the MainIO board version (see my edited answer with MainIO pics). About yours servers 1&2, new MainIO PAL1 is 0x0F. On non-working boards, what was PAL1 number ? Also, like me, you have to spend several hundreds of bucks, if not thousands, to get workings expensive servers which could be updated by a free firmware update from hp (ex: a DL980 G7 is basically a DL580 G7 extended to 8 sockets). I really don't like this kind of practice like last bios & SPP download locked if you have no valid warranty. – Edgar1 Apr 16 '16 at 18:48
  • I think the PAL1 number was 0x0E, am but not certain. We had to spend about $500 on each of those two servers to get them upgraded to the E7 series Xeons. (refurbished parts, $300 for the MainIO board and $200 on the E7 compatible memory cartridges). My main gripe was that not all DL580 G7's were the same... and not even all 584084-001 model number servers were the same. – Clint Apr 18 '16 at 12:37
  • And good news, you can even get refurbished boards in V.B03 for $82 (without shipping) at pcserverandparts.us/. Too late for me... – Edgar1 Apr 18 '16 at 20:41
2

This is a "call HP support" situation.

That is all.

Regardless of your warranty status, you're trying to make something work that requires specialized knowledge. I'll assume you've upgraded all of the firmware, but it also seems like you've recognized, but ignored a serious compatibility issue.

2

Like ewwhite notified it, I just ignored a serious compatibility issue.

I finally got my hands on a 643086-B21 CTO model, fitted with two E7-8837 on PROC1 & PROC2.

Sequential test done (I,J are E7-8837, SLC3N, 130W TDP):

1) PROC1=I, PROC2=J, PROC3=X, PROC4=X, POST=TRUE (E7 dimm cartridges)
2) PROC1=A, PROC2=B, PROC3=C, PROC4=D, POST=TRUE (E7 dimm cartridges)
3) PROC1=E, PROC2=F, PROC3=G, PROC4=H, POST=TRUE (Standard dimm cartridges)

System ROM is P65 10/01/2013

As you can see, everything works fine (4xE7-4870 & 4xE7530 cpus).

CTO Server parts:
Motherboard (512843-001/591196-001)
CPU Board (583367-001/591197-001)
I/O Board (512844-001/591199-001)

Comparing to 584057-421 model, hardware, p/n boards are exactly the same.

So where is the difference ?

In this kind of server, each board (SPI, MainIO, CPU) have is own chip ROM (Lattice EEPROM/Flash), named CPLD for Complex Programmable Logic Device and PAL for Programmable Array Logic. It can explain why some Hp ProLiant servers take so much time to boot.

You can see number version in iLO3 menu under Information/Diagnostics/iLO Self-Test Results:

For DL580 G7 CTO p/n 643086-B21:

CPLD - PAL0 = ProLiant DL580 G7 SPI Board PAL version 0x10 (p/n 591199-001, Lattice chip sticker = SPI PAL, 512844v4, 6470)
CPLD - PAL1 = ProLiant DL580 G7 MainIO Board PAL version 0x11 bootleg 0x10 (p/n 591196-001, Lattice chip sticker = Sys PAL, 20110215, 5290)
CPLD - PAL2 = ProLiant DL580 G7 CPU Board PAL version 0x10 (p/n 591197-001, Lattice chip sticker = CPLD, 0x1010, EB96)

enter image description here

For DL580 G7 p/n 584087-421:

CPLD - PAL0 = ProLiant DL580 G7 SPI Board PAL version 0x0F (p/n 591199-001, Lattice chip sticker = SPI PAL, R18, 3484)
CPLD - PAL1 = ProLiant DL580 G7 MainIO Board PAL version 0x0E (p/n 591196-001, Lattice chip sticker = Sys PAL, R18, 6D87)
CPLD - PAL2 = ProLiant DL580 G7 CPU Board PAL version 0x0E (p/n 591197-001, Lattice chip sticker = CPLD, R18, B5B6)

enter image description here

For two DL580 G7s (boot on two servers, no CPLD update required):

CPLD - PAL3 = ProLiant DL580 G7 Combo PCIe & PCI-X Board PAL version 0x06 (p/n 591204-001, Lattice chip sticker = CPLD, R18, 80B6)
CPLD - PAL3 = ProLiant DL580 G7 Standard PCIe Board PAL version 0x06 (p/n 591205-001, Lattice chip sticker = CPLD, R18, 80B6)

Note: 4xE7-4870 with CPU Board PAL v0x10 won't boot on a DL580 G7 p/n 584087-421. So replace/update CPU Board is not sufficient, you have to replace/update MainIO/SPI boards too (not separatly tested).

As tests have shown before, A DL580 G7 CTO supports E7500 processors, so updating CPLD firmwares on a standard server wouldn't induce a compatibility issue. So if you plan to upgrade your standard DL580 G7 server with E7 processors, you have to update your CPLD chips. Unfortunately, I haven't found anywhere a CPLD firmware update for DL580 G7 servers. It seems that Hp doesn't offer it to download and I doubt they ever will. However, you can find it for many others ProLiant servers:
For example, hp documentation p/n AM426-9011 B, describe E7 family processor upgrade on DL980 G7 server (page 6, Flash the FPGA using the FPGA Smart Component). You can get complete iso CPLD update firmware at hp ftp server (/pub/softlib2/software1/pubsw-linux/p2139015018/v70029/dl980g7-offline-cpld-fpga-fmp-flash-1.0-8.zip)
And, for those who might be interested, I also found a CPLD update for DL585 G7 (/pub/softlib2/software1/sc-linux-fw/p2039955643/v85801/cp019851.scexe)

As has been said before, E7-4800 series are only supported in DL580 G7 CTO (643086-B2x) & Preconfigured models (6430xx-xx1/6967xx-xx1). But if you get failed board, especially for second market, you have to replace it with the right part number and take care to get the right compatible CPLD chip version !

Also, for people who still want to upgrade a standard DL580 G7 with E7 Westmere processors without CPLD update, keep in mind that:
- Only in PROC1 & PROC2 sockets (to get a symmetrical configuration) and so, maximum memory amount is split by two,
- A 2xE7-4870 configuration is faster than a 4xE7530 configuration and less power hungry (2xE7-4870, 20 cores=208W, 4xE7530, 24 cores=337W, under WS2012 R2 idle mode),
- A Expansion board can't be added as you have to populate PROC3.

Issue solved.

EDIT: In response to @Clint, detailed MainIO boards versions:

584087-421 enter image description here

643086-B21 enter image description here

0

Thanks to all of you for the superb information. You really saved me on this. For those of you wandering in later, here's the bottom line:

You have main IO board 591196-001 rev 0A. That's the board with the PCI slots. The label over near the connectors to the CPU board says rev 0A. You need rev 0B.

The server will not post and boot if you have an E7 processor in slot 3 and the rev 0A main board installed. As soon as you remove processor 3 or install a rev 0B board, everything will work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.