Like ewwhite notified it, I just ignored a serious compatibility issue.
I finally got my hands on a 643086-B21 CTO model, fitted with two E7-8837 on PROC1 & PROC2.
Sequential test done (I,J are E7-8837, SLC3N, 130W TDP):
1) PROC1=I, PROC2=J, PROC3=X, PROC4=X, POST=TRUE (E7 dimm cartridges)
2) PROC1=A, PROC2=B, PROC3=C, PROC4=D, POST=TRUE (E7 dimm cartridges)
3) PROC1=E, PROC2=F, PROC3=G, PROC4=H, POST=TRUE (Standard dimm cartridges)
System ROM is P65 10/01/2013
As you can see, everything works fine (4xE7-4870 & 4xE7530 cpus).
CTO Server parts:
CPU Board (583367-001/591197-001)
I/O Board (512844-001/591199-001)
Comparing to 584057-421 model, hardware, p/n boards are exactly the same.
So where is the difference ?
In this kind of server, each board (SPI, MainIO, CPU) have is own chip ROM (Lattice EEPROM/Flash), named CPLD for Complex Programmable Logic Device and PAL for Programmable Array Logic. It can explain why some Hp ProLiant servers take so much time to boot.
You can see number version in iLO3 menu under Information/Diagnostics/iLO Self-Test Results:
For DL580 G7 CTO p/n 643086-B21:
CPLD - PAL0 = ProLiant DL580 G7 SPI Board PAL version 0x10 (p/n 591199-001, Lattice chip sticker = SPI PAL, 512844v4, 6470)
CPLD - PAL1 = ProLiant DL580 G7 MainIO Board PAL version 0x11 bootleg 0x10 (p/n 591196-001, Lattice chip sticker = Sys PAL, 20110215, 5290)
CPLD - PAL2 = ProLiant DL580 G7 CPU Board PAL version 0x10 (p/n 591197-001, Lattice chip sticker = CPLD, 0x1010, EB96)
For DL580 G7 p/n 584087-421:
CPLD - PAL0 = ProLiant DL580 G7 SPI Board PAL version 0x0F (p/n 591199-001, Lattice chip sticker = SPI PAL, R18, 3484)
CPLD - PAL1 = ProLiant DL580 G7 MainIO Board PAL version 0x0E (p/n 591196-001, Lattice chip sticker = Sys PAL, R18, 6D87)
CPLD - PAL2 = ProLiant DL580 G7 CPU Board PAL version 0x0E (p/n 591197-001, Lattice chip sticker = CPLD, R18, B5B6)
For two DL580 G7s (boot on two servers, no CPLD update required):
CPLD - PAL3 = ProLiant DL580 G7 Combo PCIe & PCI-X Board PAL version 0x06 (p/n 591204-001, Lattice chip sticker = CPLD, R18, 80B6)
CPLD - PAL3 = ProLiant DL580 G7 Standard PCIe Board PAL version 0x06 (p/n 591205-001, Lattice chip sticker = CPLD, R18, 80B6)
Note: 4xE7-4870 with CPU Board PAL v0x10 won't boot on a DL580 G7 p/n 584087-421. So replace/update CPU Board is not sufficient, you have to replace/update MainIO/SPI boards too (not separatly tested).
As tests have shown before, A DL580 G7 CTO supports E7500 processors, so updating CPLD firmwares on a standard server wouldn't induce a compatibility issue. So if you plan to upgrade your standard DL580 G7 server with E7 processors, you have to update your CPLD chips. Unfortunately, I haven't found anywhere a CPLD firmware update for DL580 G7 servers. It seems that Hp doesn't offer it to download and I doubt they ever will. However, you can find it for many others ProLiant servers:
For example, hp documentation p/n AM426-9011 B, describe E7 family processor upgrade on DL980 G7 server (page 6, Flash the FPGA using the FPGA Smart Component). You can get complete iso CPLD update firmware at hp ftp server (/pub/softlib2/software1/pubsw-linux/p2139015018/v70029/dl980g7-offline-cpld-fpga-fmp-flash-1.0-8.zip)
And, for those who might be interested, I also found a CPLD update for DL585 G7 (/pub/softlib2/software1/sc-linux-fw/p2039955643/v85801/cp019851.scexe)
As has been said before, E7-4800 series are only supported in DL580 G7 CTO (643086-B2x) & Preconfigured models (6430xx-xx1/6967xx-xx1).
But if you get failed board, especially for second market, you have to replace it with the right part number and take care to get the right compatible CPLD chip version !
Also, for people who still want to upgrade a standard DL580 G7 with E7 Westmere processors without CPLD update, keep in mind that:
- Only in PROC1 & PROC2 sockets (to get a symmetrical configuration) and so, maximum memory amount is split by two,
- A 2xE7-4870 configuration is faster than a 4xE7530 configuration and less power hungry (2xE7-4870, 20 cores=208W, 4xE7530, 24 cores=337W, under WS2012 R2 idle mode),
- A Expansion board can't be added as you have to populate PROC3.
EDIT: In response to @Clint, detailed MainIO boards versions: