Fibre Channel Storage

Use this page to understand how Karios connects hosts to SAN storage over Fibre Channel (FC), which kind of FC pool fits your case, and what both kinds share: arrays, ports, paths, host readiness, and health. The step-by-step tasks are on the two pool pages below.

For iSCSI pools, which share the same model over the storage network, see iSCSI Storage.

UI path: Control Center -> Storage -> Instance Storage

What the SAN Fabric Is

The SAN fabric is the part of Karios that puts VM disks on block storage a host reaches through a storage network: a Fibre Channel fabric or an iSCSI network. Every host in the pool’s scope sees each disk through several paths at once (multipath). If one cable, switch port, or array port fails, the disk keeps working on the other paths.

Three words come up on every page:

Term

Meaning

Protocol

How the host reaches the storage: fc (Fibre Channel) or iscsi. It is a property of the pool. One array can serve pools of both protocols.

Driver

How Karios talks to the storage. comstar manages a COMSTAR array (illumos or a compatible ZFS appliance) over SSH and carves a volume for every disk. generic manages nothing: it attaches LUNs that already exist on any vendor’s array.

Pool

One instance storage pool in the platform. It names its protocol, its driver, the array (for comstar) or the array ports to watch (for generic), its scope, and its path floor.

Two Kinds of FC Pool

Both are added from Add Instance Storage with Protocol set to fc.

Question

Managed array (COMSTAR)

Generic FC (existing LUNs)

Who creates the LUNs

Karios. It carves one volume on the array for every VM disk.

You. Karios only attaches LUNs your array already presents to the hosts.

Best for

New VM disks, root disks, and templates on SAN storage.

Bringing existing data on your own FC array into a VM as a data disk.

What Karios changes on the storage

FC target group, host groups, volumes, and views on the array.

Nothing. It never creates, resizes, wipes, or deletes a LUN, and never changes the array’s masking or the switch’s zoning.

Where masking is done

Karios adds each host’s initiator WWPN to a host group on the array and shows each volume to the host that runs the VM.

You mask each LUN to the hosts on your array. Karios reads what the hosts see.

Scope

Zone or Cluster.

Zone only.

Who can set it up

A platform administrator registers the array. Users with storage manage rights create pools on it.

Platform administrator only.

Pick the page for your case:

Before You Start

  • [ ] Every host that should use the pool has an FC host bus adapter (HBA) with its link up. A host without an HBA cannot use an FC pool; see Hosts Without an HBA.

  • [ ] You know each host’s initiator WWPN and the array’s target port WWPNs. Karios shows both on the Hosts rows of the create form once a host has reported; the switch’s name server and the array’s port list show them too.

  • [ ] The switch zoning lets each host’s HBA reach the array’s target ports. One zone per host that holds the host’s WWPN and the array’s target ports is the usual layout. Putting every host and every array port in one zone also works. Only ports in the same zone can talk; the array’s masking then decides which LUNs each host gets.

  • [ ] The array has at least two target ports in the zone, ideally on separate switches, so each disk gets two or more paths.

  • [ ] You know which storage tag (if any) should steer VM disks to this pool.

Tip

The create form’s Hosts card shows Show zoning commands for a host that is not zoned yet. It prints the switch commands for that host and the array’s ports, with the active zoneset and VSAN you enter. Paste them on the switch in a change window.

Ports, Paths, and Minimum Paths

A path is one route from a host HBA port to one array target port. A host with one HBA port zoned to two array ports has two paths to every LUN.

  • Ports to watch for health lists the array’s target ports. Every online port is ticked unless you untick it, and at least one stays ticked. Unticking a port only stops health from counting it. On a managed array the pool is served through every port the array has set up either way; on a generic pool it narrows which paths the LUN list counts.

  • Minimum paths is how many working paths a host must have to a disk. Karios proposes a value from what the hosts report, capped at the number of ports you ticked. The default is 2; it is 1 when a single port is ticked or when no host has reported yet.

  • If Minimum paths is 1 and a host has measured one path, the form shows Single path: no redundancy. To continue you must tick the acknowledgement. On such a pool one failed cable, HBA port, or array port stalls the VMs on that host until the path returns.

Tip

Keep Minimum paths at 2 or more in production. Use 1 only for a lab array that really has a single port.

Host Readiness

Before a pool exists, Karios already knows what each host sees on the fabric. The create form’s Hosts card shows one row per host in scope:

Readiness

Meaning

Ready

The host’s HBA is online and it sees the array’s target ports. The row shows how many ports it sees, for example sees 2 of 2.

Not zoned

The HBA is online but sees none of the array’s ports. Zone the host’s WWPN to the array’s ports on the switch, then click Re-check.

No FC HBA

No FC adapter was found on this host. It cannot use an FC pool.

Unknown

The host has not reported yet, or has not reported for about three minutes. A host reports once when its services start and then every minute while it holds at least one SAN pool. On a fresh host, restarting the Karios host service makes it report at once.

The create is held only while no host in scope reaches the array. If some hosts are ready and others are not, the pool is created and its health reads Degraded until the rest are zoned.

Pool Health

Each pool’s detail page has a Fabric card. Its Health row rolls up what the hosts reported last.

Health

Meaning

Up

Every host in scope reaches the pool on at least Minimum paths paths.

Degraded

Some hosts cannot reach the pool, a host that can reach it is short of ports or of the minimum paths, or a disk on the pool reads Critical on a host.

Pending hosts

At least one host has not answered yet, or has not reported for about three minutes, and none has refused.

Down

No host can reach the pool.

No hosts

The pool’s scope has no hosts in it.

The Registration row shows whether the platform knows the pool (up, Waiting for the platform, or error), as on an iSCSI pool.

If a pool stays Down for about five minutes, Karios stops placing new disks on it. The card then shows Placement: Disabled by the fabric. Disks already on the pool are left alone. Placement comes back by itself a few minutes after a host can reach the pool again.

Hosts Without an HBA

A pool can be scoped to a cluster or a zone whose hosts do not all have an FC adapter.

  • A host with no HBA is listed as No FC HBA and never holds the pool. VMs with disks on the pool are placed on the hosts that do reach it. If every host reaches it, the pool is Up; otherwise it reads Degraded, which is a label, not a block.

  • A host added to the cluster later starts as Unknown, then Not zoned or Ready once it reports. Zone it, and the pool’s health picks it up on the host’s next report. No change to the pool is needed.

  • To keep the pool Up on a mixed cluster, scope the pool to a cluster whose hosts all have HBAs, or accept Degraded as the normal state of a zone-wide pool.

What Happens on the Host

You do not need this to use the feature; it helps when you read a host’s logs.

  1. Karios tells the host that a disk is coming. The host asks its FC adapters to look for new LUNs (a scoped rescan; the server’s internal disk controller is never rescanned).

  2. The kernel finds one SCSI disk per path. The host’s multipath service groups the paths that carry the same identity into one device, /dev/mapper/3<identity>.

  3. Karios waits for that device, checks that it has at least Minimum paths paths, and hands it to the hypervisor as the VM’s disk.

  4. When a disk grows on the array, the host re-reads each path’s size and resizes the multipath device, and the running VM sees the new size.

  5. On detach, the hypervisor releases the disk, the multipath device is flushed, and the paths are dropped from the host.

Storage Tags

  • One storage tag must point to one kind of SAN pool. Karios refuses a tag that another SAN pool with a different protocol or QoS setting already uses.

  • The tag karios-mapped-lun is reserved for pools of existing LUNs (generic iSCSI and generic FC). The UI shows it as the locked chip Reserved for existing LUNs. You cannot add it to any other pool.

Limits on Both Pool Types

  • Volume snapshots are not available. The UI turns snapshot creation off for these disks and says why. A snapshot request sent through the API is refused with KF-4016. Keep disks that need snapshots on RBD storage.

  • Storage QoS (Min IOPS, Max IOPS, or custom IOPS) is refused with KF-4016, because these pools never enforce it. Hypervisor QoS, which the host applies, stays available.

  • FC has no CHAP. Access is decided by the switch’s zoning and the array’s masking. Keep the fabric dedicated to storage.

Common Errors

Karios errors start with a KF- code. The pool pages list the codes specific to each pool type.

Code

What it means

What to do

KF-4013

A host sees fewer paths than Minimum paths. The message names the host, its initiator WWPN, and how many of the pool’s ports it is logged in on.

Bring the missing array port back, zone the host to it, or lower Minimum paths.

KF-4014

A host cannot reach the array at all.

Check the HBA link, the zoning, and the array’s port state.

KF-4016

A snapshot, or Storage QoS, asked on a pool that does not support it.

Keep such disks on RBD storage. Leave IOPS limits off the volume profile.

KF-4098

The platform refused the call.

Read the details shown with the error.

KF-5022

Karios could not read what it needed to check. Nothing was sent.

Retry.

Unable to create a deployment or Insufficient capacity when a VM starts

The platform could not place the VM. When the VM has a disk on an FC pool, the usual cause is that Karios refused the attach on its host: too few paths (KF-4013) or a LUN in conflict (KF-4018).

Open the host’s rows on the pool’s Fabric card, or the LUN’s State reason. The platform’s management log carries the full KF- message.

Done Criteria

  • [ ] You know which pool type fits your workload.

  • [ ] Every host in scope shows Ready on the create form’s Hosts card, or you have decided which hosts stay outside the pool.

  • [ ] You finished the tasks on the matching pool page.