3

I have a t2.medium instance running in AWS on the CentOS 7 AMI and I noticed in top and free it's basically reporting 3.45GiB of total memory. If I launch an instance with the AWS linux (I think I tried the RedHat one too) it reports about 4GiB of memory. Looking around the web I can't see anyone else saying why this would be. Does anyone know why they report differently or if it's a problem that can be fixed?

CentOS: KiB Mem : 3619008 total, 1096408 free, 1218216 used, 1304384 buff/cache

AWS Linux: Mem: 4049884k total, 459488k used, 3590396k free, 11440k buffers

EDIT

I've also launched a RHEL with 1GiB of RAM:

KiB Mem : 1015472 total, 620588 free, 73452 used, 321432 buff/cache

So now with close to the same OS but reporting different details, I'd assume there is some bug in this version of the AMI?

EDIT 2 cat /proc/meminfo output with kdump turned off: MemTotal: 3783872 kB MemFree: 2682052 kB MemAvailable: 2799084 kB Buffers: 1328 kB Cached: 301760 kB SwapCached: 0 kB Active: 822816 kB Inactive: 153564 kB Active(anon): 676708 kB Inactive(anon): 25668 kB Active(file): 146108 kB Inactive(file): 127896 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 4194300 kB SwapFree: 4194300 kB Dirty: 292 kB Writeback: 0 kB AnonPages: 673292 kB Mapped: 47560 kB Shmem: 29084 kB Slab: 46348 kB SReclaimable: 23912 kB SUnreclaim: 22436 kB KernelStack: 3648 kB PageTables: 25784 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 6086236 kB Committed_AS: 1746036 kB VmallocTotal: 34359738367 kB VmallocUsed: 16640 kB VmallocChunk: 34359717628 kB HardwareCorrupted: 0 kB AnonHugePages: 147456 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 67584 kB DirectMap2M: 4126720 kB

EDIT 3

I've turned off Kdump and have reclaimed a little bit of space but still missing ~200MB.

dmesg output:

[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-327.10.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Tue Feb 16 17:03:50 UTC 2016 [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.10.0-327.10.1.el7.x86_64 root=UUID=dd8f0afe-5a00-40ff-b538-5fa0a351cdba ro console=tty0 console=ttyS0,115200 [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable ... [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.4 present. [ 0.000000] DMI: Xen HVM domU, BIOS 4.2.amazon 12/07/2015 [ 0.000000] Hypervisor detected: Xen HVM [ 0.000000] Xen version 4.2. [ 0.000000] Xen Platform PCI: I/O protocol version 1 [ 0.000000] Netfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated NICs. [ 0.000000] Blkfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated disks. You might have to change the root device from /dev/hd[a-d] to /dev/xvd[a-d] in your root= kernel command line option [ 0.000000] HVMOP_pagetable_dying not supported [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] AGP: No AGP bridge found [ 0.000000] e820: last_pfn = 0x110000 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF write-combining [ 0.000000] C0000-FFFFF write-back [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000F0000000 mask 3FFFF8000000 uncatchable ... [ 0.000000] 2 disabled ... [ 0.000000] 7 disabled [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 [ 0.000000] e820: last_pfn = 0xf0000 max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000fbba0-0x000fbbaf] mapped at [ffff8800000fbba0] [ 0.000000] Base memory trampoline at [ffff880000098000] 98000 size 24576 [ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff] [ 0.000000] [mem 0x00000000-0x000fffff] page 4k [ 0.000000] BRK [0x01f33000, 0x01f33fff] PGTABLE ... [ 0.000000] init_memory_mapping: [mem 0x10fe00000-0x10fffffff] [ 0.000000] [mem 0x10fe00000-0x10fffffff] page 2M [ 0.000000] BRK [0x01f36000, 0x01f36fff] PGTABLE [ 0.000000] init_memory_mapping: [mem 0x10c000000-0x10fdfffff] [ 0.000000] [mem 0x10c000000-0x10fdfffff] page 2M [ 0.000000] init_memory_mapping: [mem 0x100000000-0x10bffffff] [ 0.000000] [mem 0x100000000-0x10bffffff] page 2M [ 0.000000] init_memory_mapping: [mem 0x00100000-0xefffffff] [ 0.000000] [mem 0x00100000-0x001fffff] page 4k [ 0.000000] [mem 0x00200000-0xefffffff] page 2M [ 0.000000] RAMDISK: [mem 0x33a7c000-0x35d35fff] [ 0.000000] ACPI: RSDP 00000000000ea020 00024 (v02 Xen) [ 0.000000] ACPI: XSDT 00000000fc00f5a0 00054 (v01 Xen HVM 00000000 HVML 00000000) ... [ 0.000000] ACPI: SSDT 00000000fc00f560 00031 (v02 Xen HVM 00000000 INTL 20090123) [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000010fffffff] [ 0.000000] Initmem setup node 0 [mem 0x00000000-0x10fffffff] [ 0.000000] NODE_DATA [mem 0x10ffd6000-0x10fffcfff] [ 0.000000] [ffffea0000000000-ffffea00043fffff] PMD -> [ffff88010b600000-ffff88010f5fffff] on node 0 [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal [mem 0x100000000-0x10fffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009dfff] ... [ 0.000000] On node 0 totalpages: 1048477 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3997 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 15296 pages used for memmap [ 0.000000] DMA32 zone: 978944 pages, LIFO batch:31 [ 0.000000] Normal zone: 1024 pages used for memmap [ 0.000000] Normal zone: 65536 pages, LIFO batch:15 [ 0.000000] ACPI: PM-Timer IO Port: 0xb008 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ... [ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x00] disabled) [ 0.000000] ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) ... [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level) [ 0.000000] ACPI: IRQ0 used by override. ... [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] smpboot: Allowing 15 CPUs, 13 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x0009e000-0x0009ffff] ... [ 0.000000] PM: Registered nosave memory: [mem 0xfc000000-0xffffffff] [ 0.000000] e820: [mem 0xf0000000-0xfbffffff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on Xen HVM [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:15 nr_cpu_ids:15 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 31 pages/cpu @ffff88010fc00000 s87168 r8192 d31616 u131072 [ 0.000000] pcpu-alloc: s87168 r8192 d31616 u131072 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 -- [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 1032072 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.10.0-327.10.1.el7.x86_64 root=UUID=dd8f0afe-5a00-40ff-b538-5fa0a351cdba ro console=tty0 console=ttyS0,115200 [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 [ 0.000000] AGP: Checking aperture... [ 0.000000] AGP: No AGP bridge found [ 0.000000] Memory: 4008808k/4456448k available (6440k kernel code, 262540k absent, 185100k reserved, 4266k data, 1620k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=15, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=15. [ 0.000000] Offload RCU callbacks from all CPUs [ 0.000000] Offload RCU callbacks from CPUs: 0-14. [ 0.000000] NR_IRQS:327936 nr_irqs:952 0 [ 0.000000] xen:events: Xen HVM callback vector for event delivery is enabled [ 0.000000] Console: colour VGA+ 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] Cannot get hvm parameter CONSOLE_EVTCHN (18): -22! [ 0.000000] console [ttyS0] enabled [ 0.000000] allocated 16777216 bytes of page_cgroup [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups [ 0.000000] hpet clockevent registered [ 0.000000] tsc: Detected 2500.056 MHz processor [ 0.005000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5000.11 BogoMIPS (lpj=2500056) [ 0.013004] pid_max: default: 32768 minimum: 301 [ 0.017040] Security Framework initialized [ 0.020007] SELinux: Initializing. [ 0.023011] SELinux: Starting in permissive mode [ 0.024124] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.030768] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.035392] Mount-cache hash table entries: 4096 [ 0.039203] Initializing cgroup subsys memory ... [ 0.060004] Initializing cgroup subsys hugetlb [ 0.064066] CPU: Physical Processor ID: 0 [ 0.067002] CPU: Processor Core ID: 0 [ 0.071304] mce: CPU supports 2 MCE banks [ 0.074023] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0 Last level dTLB entries: 4KB 512, 2MB 0, 4MB 0 tlb_flushall_shift: 6 [ 0.084252] Freeing SMP alternatives: 28k freed [ 0.091062] ACPI: Core revision 20130517 [ 0.097913] ACPI: All ACPI Tables successfully acquired [ 0.102004] ftrace: allocating 24584 entries in 97 pages [ 0.133003] Switched APIC routing to physical flat. [ 0.139406] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0 [ 0.154095] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz (fam: 06, model: 3e, stepping: 04) [ 0.162007] Xen: using vcpuop timer interface [ 0.162014] installing Xen timer for CPU 0 [ 0.166087] cpu 0 spinlock event irq 53 [ 0.167041] Performance Events: unsupported p6 CPU model 62 no PMU driver, software events only. [ 0.171416] NMI watchdog: disabled (cpu0): hardware events not enabled [ 0.172074] installing Xen timer for CPU 1 [ 0.173068] smpboot: Booting Node 0, Processors #1cpu 1 spinlock event irq 59 [ 0.188029] Brought up 2 CPUs [ 0.189005] smpboot: Total of 2 processors activated (10000.22 BogoMIPS) [ 0.190596] devtmpfs: initialized [ 0.194990] EVM: security.selinux [ 0.199005] EVM: security.ima [ 0.201006] EVM: security.capability [ 0.205725] atomic64 test passed for x86-64 platform with CX8 and with SSE [ 0.210134] NET: Registered protocol family 16 [ 0.214116] ACPI: bus type PCI registered [ 0.218006] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.222332] PCI: Using configuration type 1 for base access [ 0.227085] ACPI: Added _OSI(Module Device) [ 0.231006] ACPI: Added _OSI(Processor Device) [ 0.234005] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.238007] ACPI: Added _OSI(Processor Aggregator Device) [ 0.243367] ACPI: EC: Look up EC in DSDT [ 0.246343] ACPI: Interpreter enabled [ 0.250016] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20130517/hwxface-571) [ 0.257007] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20130517/hwxface-571) [ 0.265014] ACPI: (supports S0 S3 S4 S5) [ 0.268005] ACPI: Using IOAPIC for interrupt routing [ 0.272041] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.345050] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.346011] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI] [ 0.347015] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM [ 0.348081] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.349808] acpiphp: Slot [0] registered ... [ 0.378310] acpiphp: Slot [31] registered [ 0.379309] PCI host bridge to bus 0000:00 [ 0.380023] pci_bus 0000:00: root bus resource [bus 00-ff] ... [ 0.390382] pci 0000:00:01.1: reg 0x20: [io 0xc100-0xc10f] [ 0.390920] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] ... [ 0.394697] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.394733] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug, * this clock source is slow. Consider trying other clock sources [ 0.396352] pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4 ACPI [ 0.397918] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000 [ 0.398172] pci 0000:00:02.0: reg 0x10: [mem 0xf0000000-0xf1ffffff pref] [ 0.398431] pci 0000:00:02.0: reg 0x14: [mem 0xf3000000-0xf3000fff] [ 0.400291] pci 0000:00:03.0: [5853:0001] type 00 class 0xff8000 [ 0.400632] pci 0000:00:03.0: reg 0x10: [io 0xc000-0xc0ff] [ 0.400898] pci 0000:00:03.0: reg 0x14: [mem 0xf2000000-0xf2ffffff pref] [ 0.403567] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11) ... [ 0.414208] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11) [ 0.435706] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.436083] xen:balloon: Initialising balloon driver [ 0.441026] xen_balloon: Initialising balloon driver [ 0.445070] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none [ 0.452007] vgaarb: loaded [ 0.454004] vgaarb: bridge control possible 0000:00:02.0 [ 0.465074] SCSI subsystem initialized [ 0.469047] ACPI: bus type USB registered [ 0.480707] usbcore: registered new interface driver usbfs [ 0.485015] usbcore: registered new interface driver hub [ 0.496039] usbcore: registered new device driver usb [ 0.508075] PCI: Using ACPI for IRQ routing [ 0.512007] PCI: pci_cache_line_size set to 64 bytes [ 0.512427] e820: reserve RAM buffer [mem 0x0009e000-0x0009ffff] [ 0.512543] NetLabel: Initializing [ 0.516005] NetLabel: domain hash size = 128 [ 0.520004] NetLabel: protocols = UNLABELED CIPSOv4 [ 0.534049] NetLabel: unlabeled traffic allowed by default [ 0.535076] HPET: 3 timers in total, 0 timers will be used for per-cpu timer [ 0.536042] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.539006] hpet0: 3 comparators, 64-bit 62.500000 MHz counter [ 0.775037] Switching to clocksource xen [ 0.780573] pnp: PnP ACPI init [ 0.783587] ACPI: bus type PNP registered [ 0.787272] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved [ 0.792532] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active) [ 0.792586] pnp 00:01: Plug and Play ACPI device, IDs PNP0103 (active) [ 0.792620] system 00:02: [io 0x08a0-0x08a3] has been reserved ... [ 0.802087] system 00:02: [io 0x04d0-0x04d1] has been reserved [ 0.806860] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active) [ 0.806884] pnp 00:03: [dma 4] [ 0.806920] pnp 00:03: Plug and Play ACPI device, IDs PNP0200 (active) [ 0.806942] xen: --> pirq=16 -> irq=8 (gsi=8) [ 0.806955] pnp 00:04: Plug and Play ACPI device, IDs PNP0b00 (active) ... [ 0.806993] xen: --> pirq=17 -> irq=12 (gsi=12) [ 0.807002] pnp 00:06: Plug and Play ACPI device, IDs PNP0f13 (active) [ 0.807019] xen: --> pirq=18 -> irq=1 (gsi=1) [ 0.807032] pnp 00:07: Plug and Play ACPI device, IDs PNP0303 PNP030b (active) [ 0.807048] xen: --> pirq=19 -> irq=6 (gsi=6) [ 0.807051] pnp 00:08: [dma 2] [ 0.807069] pnp 00:08: Plug and Play ACPI device, IDs PNP0700 (active) [ 0.807095] xen: --> pirq=20 -> irq=4 (gsi=4) [ 0.807108] pnp 00:09: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.807142] system 00:0a: [io 0x10c0-0x1141] has been reserved [ 0.811864] system 00:0a: [io 0xb044-0xb047] has been reserved [ 0.816579] system 00:0a: Plug and Play ACPI device, IDs PNP0c02 (active) [ 0.835070] pnp: PnP ACPI: found 11 devices [ 0.838578] ACPI: bus type PNP unregistered [ 0.849503] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7] ... [ 0.849507] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff] ... [ 0.849541] NET: Registered protocol family 2 [ 0.854104] TCP established hash table entries: 32768 (order: 6, 262144 bytes) [ 0.860212] TCP bind hash table entries: 32768 (order: 7, 524288 bytes) [ 0.865152] TCP: Hash tables configured (established 32768 bind 32768) [ 0.870016] TCP: reno registered [ 0.873214] UDP hash table entries: 2048 (order: 4, 65536 bytes) [ 0.877886] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes) [ 0.882952] NET: Registered protocol family 1 [ 0.886554] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.891169] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.895949] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.900811] pci 0000:00:02.0: Boot video device [ 0.900878] PCI: CLS 0 bytes, default 64 [ 0.900930] Unpacking initramfs... [ 1.370039] Freeing initrd memory: 35560k freed [ 1.382776] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.387812] software IO TLB [mem 0xec000000-0xf0000000] (64MB) mapped at [ffff8800ec000000-ffff8800efffffff] [ 1.395560] RAPL PMU detected, API unit is 2^-32 Joules, 3 fixed counters 163840 ms ovfl timer [ 1.402348] hw unit of domain pp0-core 2^-16 Joules [ 1.406346] hw unit of domain package 2^-16 Joules [ 1.410208] hw unit of domain dram 2^-16 Joules [ 1.414438] microcode: CPU0 sig=0x306e4, pf=0x1, revision=0x416 ... [ 1.423672] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba [ 1.430934] sha1_ssse3: Using AVX optimized SHA-1 implementation [ 1.435656] sha256_ssse3: Using AVX optimized SHA-256 implementation [ 1.440884] futex hash table entries: 4096 (order: 6, 262144 bytes) [ 1.445953] Initialise system trusted keyring [ 1.449714] audit: initializing netlink socket (disabled) [ 1.454053] type=2000 audit(1460748590.383:1): initialized [ 1.479381] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 1.485298] zpool: loaded [ 1.488040] zbud: loaded [ 1.490923] VFS: Disk quotas dquot_6.5.2 [ 1.494507] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.499951] msgmni has been set to 7387 [ 1.503432] Key type big_key registered [ 1.506711] SELinux: Registering netfilter hooks [ 1.507730] NET: Registered protocol family 38 [ 1.511538] Key type asymmetric registered [ 1.515054] Asymmetric key parser 'x509' registered [ 1.519027] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251) [ 1.525577] io scheduler noop registered [ 1.529021] io scheduler deadline registered (default) [ 1.533344] io scheduler cfq registered [ 1.537151] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 1.541386] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 [ 1.546422] intel_idle: does not run on family 6 model 62 [ 1.546499] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 1.552505] ACPI: Power Button [PWRF] [ 1.556208] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1 [ 1.562654] ACPI: Sleep Button [SLPF] [ 1.566048] GHES: HEST is not enabled! [ 1.569688] xen: --> pirq=21 -> irq=28 (gsi=28) [ 1.569765] xen:grant_table: Grant tables using version 1 layout [ 1.574524] Grant table initialized [ 1.577755] Cannot get hvm parameter CONSOLE_EVTCHN (18): -22! [ 1.582372] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 1.614794] 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 1.620038] Non-volatile memory driver v1.3 [ 1.623599] Linux agpgart interface v0.103 [ 1.627302] crash memory driver: version 1.1 [ 1.630956] rdac: device handler registered [ 1.634477] hp_sw: device handler registered [ 1.637974] emc: device handler registered [ 1.641456] alua: device handler registered [ 1.645089] libphy: Fixed MDIO Bus: probed [ 1.648698] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 1.653541] ehci-pci: EHCI PCI platform driver [ 1.657751] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 1.662445] ohci-pci: OHCI PCI platform driver [ 1.666172] uhci_hcd: USB Universal Host Controller Interface driver [ 1.671042] usbcore: registered new interface driver usbserial [ 1.675477] usbcore: registered new interface driver usbserial_generic [ 1.680211] usbserial: USB Serial support registered for generic [ 1.684936] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12 [ 1.693451] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 1.697654] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 1.701942] mousedev: PS/2 mouse device common for all mice [ 1.707169] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2 [ 1.714459] rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0 [ 1.720970] rtc_cmos 00:04: alarms up to one day, 114 bytes nvram, hpet irqs [ 1.726774] cpuidle: using governor menu [ 1.730197] hidraw: raw HID events driver (C) Jiri Kosina [ 1.734504] usbcore: registered new interface driver usbhid [ 1.738851] usbhid: USB HID core driver [ 1.742264] drop_monitor: Initializing network drop monitor service [ 1.747070] TCP: cubic registered [ 1.750203] Initializing XFRM netlink socket [ 1.753891] NET: Registered protocol family 10 [ 1.757968] NET: Registered protocol family 17 [ 1.762078] Loading compiled-in X.509 certificates [ 1.765952] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717' [ 1.773397] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b' [ 1.781437] Loaded X.509 cert 'CentOS Linux kernel signing key: 756354566b6c2e3c931c0a3bf44c8294fccbce3a' [ 1.806084] registered taskstats version 1 [ 1.811489] Key type trusted registered [ 1.816618] Key type encrypted registered [ 1.821587] IMA: No TPM chip found, activating TPM-bypass! [ 1.826295] xenbus_probe_frontend: Device with no driver: device/vfb/0 ... [ 1.852051] xenbus_probe_frontend: Device with no driver: device/vif/0 [ 1.857158] rtc_cmos 00:04: setting system clock to 2016-04-15 19:29:51 UTC (1460748591) [ 1.866113] Freeing unused kernel memory: 1620k freed [ 1.874542] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) [ 1.887720] systemd[1]: Detected virtualization xen. [ 1.891810] systemd[1]: Detected architecture x86-64. [ 1.896050] systemd[1]: Running in initial RAM disk. [ 1.904088] systemd[1]: No hostname configured. [ 1.907882] systemd[1]: Set hostname to <localhost>. [ 1.911832] systemd[1]: Initializing machine ID from random generator. [ 1.956032] systemd[1]: Reached target Local File Systems. [ 1.960872] systemd[1]: Starting Local File Systems. [ 1.967877] systemd[1]: Reached target Swap. [ 1.971511] systemd[1]: Starting Swap. [ 1.977704] systemd[1]: Reached target Timers. [ 1.981363] systemd[1]: Starting Timers. [ 1.987805] systemd[1]: Created slice -.slice. [ 1.991574] systemd[1]: Starting -.slice. [ 1.998610] systemd[1]: Listening on udev Control Socket. [ 2.002939] systemd[1]: Starting udev Control Socket. [ 2.010145] systemd[1]: Listening on udev Kernel Socket. [ 2.014478] systemd[1]: Starting udev Kernel Socket. [ 2.021584] systemd[1]: Listening on Journal Socket. [ 2.025579] systemd[1]: Starting Journal Socket. [ 2.029510] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ 2.035921] systemd[1]: Starting Dispatch Password Requests to Console Directory Watch. [ 2.045313] systemd[1]: Reached target Paths. ... [ 2.091990] systemd[1]: Starting Journal Service... [ 2.099117] systemd[1]: Starting Create list of required static device nodes for the current kernel... [ 2.111135] systemd[1]: Starting dracut cmdline hook... [ 2.120901] systemd[1]: Reached target Sockets. [ 2.124667] systemd[1]: Starting Sockets. [ 2.131983] systemd[1]: Started Journal Service. [ 2.214968] RPC: Registered named UNIX socket transport module. [ 2.219794] RPC: Registered udp transport module. [ 2.223605] RPC: Registered tcp transport module. [ 2.227558] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 2.396079] tsc: Refined TSC clocksource calibration: 2500.000 MHz [ 2.404145] FDC 0 is a S82078B [ 2.414902] libata version 3.00 loaded. [ 2.415783] ata_piix 0000:00:01.1: version 2.13 [ 2.416790] scsi host0: ata_piix [ 2.420209] scsi host1: ata_piix [ 2.424025] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14 [ 2.429172] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15 [ 2.441339] xen_netfront: Initialising Xen virtual ethernet driver [ 2.503105] blkfront: xvda: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled; [ 2.526726] xvda: xvda1 [ 2.538259] blkfront: xvdb: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled; [ 2.553835] xvdb: unknown partition table [ 2.561480] blkfront: xvdm: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled; [ 2.564125] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3 [ 2.580077] xvdm: unknown partition table [ 2.596005] blkfront: xvdl: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled; [ 2.606573] xvdl: unknown partition table [ 2.610467] Setting capacity to 8388608 [ 2.614016] xvdm: detected capacity change from 0 to 4294967296 [ 2.735355] SGI XFS with ACLs, security attributes, no debug enabled [ 2.742377] XFS (xvda1): Mounting V4 Filesystem [ 3.881703] XFS (xvda1): Ending clean mount [ 4.275374] systemd-journald[125]: Received SIGTERM from PID 1 (systemd). [ 4.380362] SELinux: Disabled at runtime. [ 4.384006] SELinux: Unregistering netfilter hooks [ 4.403113] type=1404 audit(1460748594.045:2): selinux=0 auid=4294967295 ses=4294967295 [ 4.429738] ip_tables: (C) 2000-2006 Netfilter Core Team [ 4.434250] systemd[1]: Inserted module 'ip_tables' [ 4.643894] Installing knfsd (copyright (C) 1996 okir@monad.swb.de). [ 4.852747] systemd-journald[389]: Received request to flush runtime journal from PID 1 [ 4.980173] type=1305 audit(1460748594.622:3): audit_pid=432 old=0 auid=4294967295 ses=4294967295 res=1 [ 5.025952] piix4_smbus 0000:00:01.3: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr [ 5.044837] [drm] Initialized drm 1.1.0 20060810 [ 5.071499] input: PC Speaker as /devices/platform/pcspkr/input/input4 [ 5.090615] [TTM] Zone kernel: Available graphics memory: 1891936 kiB [ 5.095585] [TTM] Initializing pool allocator [ 5.101566] [TTM] Initializing DMA pool allocator [ 5.106805] [drm] fb mappable at 0xF0000000 [ 5.110834] [drm] vram aper at 0xF0000000 [ 5.110835] [drm] size 33554432 [ 5.110835] [drm] fb depth is 24 [ 5.110836] [drm] pitch is 3072 [ 5.123760] fbcon: cirrusdrmfb (fb0) is primary device [ 5.138883] ppdev: user-space parallel port driver [ 5.160861] Error: Driver 'pcspkr' is already registered, aborting... [ 5.175471] AES CTR mode by8 optimization enabled [ 5.178614] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) [ 5.184269] alg: No test for crc32 (crc32-pclmul) [ 5.186337] Console: switching to colour frame buffer device 128x48 [ 5.256497] cirrus 0000:00:02.0: fb0: cirrusdrmfb frame buffer device [ 5.256498] cirrus 0000:00:02.0: registered panic notifier [ 5.256502] [drm] Initialized cirrus 1.0.0 20110418 for 0000:00:02.0 on minor 0 [ 5.279587] Adding 4194300k swap on /dev/xvdm. Priority:-1 extents:1 across:4194300k SSFS [ 6.678040] ISOFS: Unable to identify CD-ROM format.

22
  • Exactly what are you seeing? Apr 8, 2016 at 22:49
  • Should I post a screenshot of the two tops?
    – Rudiger
    Apr 8, 2016 at 22:50
  • 1
    Copying and pasting is better. Apr 8, 2016 at 22:53
  • 1
    sorry, added. I'm guessing the AWS Linux is reporting it as KB not KiB? The t2 instance is sold as 4GiB, thought it would be around the 4000000KiB
    – Rudiger
    Apr 8, 2016 at 22:54
  • vmstat -S K -s | fgrep 'total mem' and/or fgrep MemTotal /proc/meminfo
    – poige
    Apr 11, 2016 at 8:33

2 Answers 2

2
+50

Maybe you have crashkernel enabled on your CentOS instances.

It basically reserves a portion of your memory to boot a, well, "crash kernel" in the case the main kernel crashed/deadlocked.

Please show the output of cat /proc/cmdline to verify.

UPDATE: Regarding the other 200 MB "lost", it seems from /proc/meminfo that they are reserved for use by the PCI bus/devices. The line:

f0000000-fbffffff : PCI Bus 0000:00

encompasses 201326591 KB, or about 200 MB.

Why that memory is reserved for a CentOS7 AMI but not for the AWS linux?

  • First, the two AMI can expose different virtualized hardware platform. So, the reserved PCI space can be different
  • Second, AWS provides two AMI types: HVM and PV. The latter can avoid some PCI remap, resulting in more memory available.

UPDATE: it seems that dmesg confirms some memory is reserved for device mappings. The line Memory: 4008808k shows 185496 KB less than full 4 GB (4194304 KB)

19
  • BOOT_IMAGE=/boot/vmlinuz-3.10.0-327.10.1.el7.x86_64 root=UUID=dd8f0afe-5a00-40ff-b538-5fa0a351cdba ro crashkernel=auto console=tty0 console=ttyS0,115200n8 console=ttyS0,115200 LANG=en_US.UTF-8
    – Rudiger
    Apr 11, 2016 at 10:03
  • Reading into it, looks like as I have 4GB it could be causing the issue but probably only 128MB worth which is short of the 512MB or so currently missing.
    – Rudiger
    Apr 11, 2016 at 10:12
  • I did find running dimes showed Reserving 161MB of memory at 656MB for crashkernel (System RAM: 4095MB). 161MB recovered would help, would just disabling the service and restarting recover it?
    – Rudiger
    Apr 11, 2016 at 11:00
  • To recover the memory, you not only have to disable the kdump service, but also to remove the crashkernel=auto option from the boot command line. A suggestion: as messing with boot parameters can bring the system in a non-bootable state, always create an AWS snapshot of the affected volume.
    – shodanshok
    Apr 11, 2016 at 13:23
  • Thanks, I might avoid this but remember for the next VM I build. It doesn't account for the other 300MB though.
    – Rudiger
    Apr 11, 2016 at 19:51
0

It looks like a 32bit kernel with centos.

2
  • 3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
    – Rudiger
    Apr 10, 2016 at 1:45
  • That and the AMI link is 64 bit.
    – Rudiger
    Apr 10, 2016 at 1:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.