Hot answers tagged

23

There is no problem. Even if there is still available RAM, the Linux Kernel will move memory pages which are hardly ever used into swap space. It’s good to swap out memory pages that have been inactive for a while, keeping often-used data in cache; this is the desired situation of the Kernel. You can have more control on this by using vfs_cache_pressure (...


21

Because: Very, very bad things will happen if someone gains unauthorized access to your storage network Very, very bad things will happen if you have your iSCSI traffic not separated, and someone finds it a great idea to muck around with the STP topology at an edge switch, with the result of your entire network AND your storage subsystem going down at once ...


20

Short answer: This is the results of network latency and a serial workload (as you imposed by using direct=1, sync=1 and iodepth=1). Long answer: using direct=1, sync=1 and iodepth=1 you created a serial workload, as new writes can not be queued before the previous write was committed and confirmed. In other word, writes submission rate strictly depend on ...


17

To have multiple initiators share a single target, whether over ISCSI, Fibre Channel or other SAN solution, you need a cluster-aware filesystem. VMWare ESXi does this with VMFS. Veritas offers one in Veritas Cluster Suite. Sun offered one back in the day that would cause no end of troubles. Oracle did this with RAC, until they got wise and changed to ...


14

You need to have some sort of either a clustered file system or distributed lock manager to avoid metadata damage done to your file system with concurrent writes. There's a good summary of this issue on StarWind forum. https://forums.starwindsoftware.com/viewtopic.php?f=5&t=1392


13

Use LACP for NFS. Use MPIO for iSCSI. If your hypervisor hosts don't have storage interface redundancy, that's where you should focus your attention; no hacks, no bullshit. Add an additional NIC to your hosts and configure MPIO.


12

If each host only has one interface for iSCSI, then you won't be able to use MPIO with the setup you've described here. However, you should be able to configure the FreeNAS system to use Link Aggregation (LACP), so that you can service two hosts simultaneously each at 1Gb (for a total of 2Gb from the FreeNAS). Instead of MPIO, look into LACP (or, get a ...


11

Can you expand a bit on your iSCSI architecture? How many initiator/target addresses are you working with, how many physical switches, all one subnet or multiple? The basic answer is: because MPIO manages end-to-end connectivity paths, and is better at storage connectivity load balancing and connection resilience than generic network redundancy and load ...


11

To squeeze maximum performance out of iSCSI connected storage you should use Jumbo Frames and MPIO (not LACP). RDMA/iSER is recommended if you can do that. AOE (ATA over Ethernet) is old and is shit. We’ve got rid of Coraid years ago, already. We are using StarWind https://www.starwindsoftware.com/ as iSCSI target quite a while already and StarWind asked ...


9

It's not a direct answer to your question, but a more traditional architecture for this sort of thing would be to use HAST and CARP to take care of the storage redundancy. A basic outline (see the linked documentation for better details): Machine A ("Master") Configure the HAST daemon & create an appropriate resource for each pool-member ...


8

After some more investigation, it turns out that the issue was someone (cough) enabling flow control on the SAN switches on the advice of an internet article talking about the very same hardware and explicitly encouraging Flow Control to be used. After disabling flow control, Load Average and I/O Wait immediately reduced, and after 24 hours it is completely ...


8

Any online writable disk can be corrupted by a malicious software or user. Underestimating them by assuming they cannot find a file share is a mistake. Last line of defense for important data is always tested, cold offline backups. And in this case, the important data may include backups themselves! Think about the possible ways to make backup archives ...


7

You don't need any additional mezzanine cards for the HP ProLiant BL460c Gen8 system to accommodate iSCSI use. This may depend slightly on which interconnects you're using in the C3000/C7000 blade chassis and what's already in use, but the system has onboard NICs available.


7

Clustered file systems (such as OCFS2) require block-based sharing (such as iSCSI, FC, FCoE etc.) so that they can manage locking on a distributed basis, this can't be done with NAS protocols which have a smaller functional capability list.


7

This question depends heavily upon your VMware vSphere licensing tier..., the applications and VMs you intend to run and the amount of storage space and performance profile you need. Answer that first, as an organization that can afford to properly license ten ESXi hosts in a vSphere cluster should be prepared for a higher-end storage solution and ...


7

You need to create another vSwitch to contain your iSCSI ports. This is separate from the vSwitch you created that holds your management and virtual machine traffic: Something like: Once you get past this point, the process is straightforward.


7

The rationale behind this in RFC 3720 is that above all, IQNs should be unique. The date prepended is a reasonable guarantee that the entity that controlled the domain name represented (in the naming auth field) at that time a "naming authority" who could ensure uniqueness - domain names change hands all the time and since the only other unique stuff going ...


7

If you do have just one Synology box shared between your Hyper-V machines, it can still be considered as a SPoF. It dies, you lose all the LUNs sitting there. For true HA you would need a second NAS box and some replication mechanism. Well, one thing that comes to mind is HA cluster with two Synology NAS devices. But AFAIK it doesnt rate well in the world of ...


7

I see similar issue posted on technet. Is there any chance to see what does Event Log show? For the case, from my experience, Microsoft iSCSI target is too slow. Currently, I use StarWind Virtual SAN Free that delivers really fast iSCSI server with the additional functionality of cache and log-structured file system (snapshots, dedupe, etc.). I suggest you ...


7

Using one drive for multiple connections without clustered file system is a suicide. Data corruption is the first that you will see after a few data iterations. You must run clustered file system to avoid such issues. There are a lot of topics answering such scenario, but this one is my favorite. https://forums.starwindsoftware.com/viewtopic.php?f=5&t=...


7

First off, get rid of NIC teams for your storage and synchronization networks. ISCSI networks leverage MPIO, there is no need for trunking, teaming, bonding, etc. Also, don't forget to enable MPIO in Add Roles and Features wizard. Assuming you want to use the 10.0.0.x networks for iSCSI traffic, your ISCSI Initiator configuration should look like this: ...


7

Good question. I think your sparse zvol block size should be 128k. Your ZIO scheduler settings should all be higher, like minimum 10 and max 64. zfs_txg_timeout should be much longer. I do 15 or 30s on my systems. I think the multiple RAIDZ3's (or was that a typo) are overkill and play a big part in the performance. Can you benchmark with RAIDZ2? Edit: ...


7

iSCSI as a protocol is not limiting performance or latency on spindle (HDD) drives. It may limit on SSD and will limit on NVMe. The performance in this case depends on: 1) Physical storage. You can increase amount of disk in RAID and use RAID10 for the best one. Video streaming uses large blocks, so properly aligned stripe size may also help. 2) Network ...


7

That is normal behavior for a non-clustered file system. To use iSCSI SAN with Ubuntu compute servers, a clustered file system should be used. You should probably learn more about GPFS, GFS2, Lustre, GlusterFS, and OCFS2 and use one of them on top of iSCSI SAN. Edit: Good description of what’s going on can be found here: https://forums.starwindsoftware.com/...


7

Leaving the question of iSCSI vulnerability when using it without a clustered file system but with multiple initiators aside, I can hardly find any clear reason why file sharing protocol would be more secure in terms of ransomware comparing to iSCSI. You get CHAP to strengthen authentication and IPSec to secure data transfer over the network. Here is a good ...


6

Don't do this unless you know exactly what you're doing. Really only do it on your dedicated iSCSI NICs and connected switch ports and SAN NICs. There's really not many reasons to have non-storage ports set for Jumbo Frames with modern equipment.


6

You didn't provide the version of CentOS installed as your VMWare guest... but let's assume it's CentOS 6.x. The default CFQ I/O scheduler is a poor choice for guest virtual machines (and most systems). You'll want to modify it. I used to just recommend setting the scheduler to deadline, but there's an easier method now for RHEL/CentOS systems... The best ...


6

(My initial answer was premature. As promised, I've rewritten it after having gotten everything working.) First of all, I've found that in general iSCSI-boot-enabling software is half-baked, and the disparate systems involved interoperate very poorly. For this reason I recommend instead going with a hardware-based solution such as iSCSI HBAs if possible. ...


6

You're compressing (gzip, even) 62 GB of data. At 40 MB/s, that transfer would take ~30 minutes on its own. However, gzip involves the CPU greatly. If you're pinning both cores at 100%, then there's your answer.


6

You have enough interfaces to build a workgroup switch with. As this configuration is not employed as often and thus not tested as thoroughly, expect oddities coming from that alone. Also, as your setup is quite complex, you should try isolating the issue by simplifying it. This is what I would do: rule out the simple cases, e.g. by checking the link stats ...


Only top voted, non community-wiki answers of a minimum length are eligible