iSCSI Pools on a Managed Array (COMSTAR)

Use this page when Karios should create and manage the disks on your SAN array. Karios logs in to the array, carves one volume for every VM disk, and shows it to the right hosts.

If you want to attach LUNs that already exist on your own target, use Generic iSCSI Pools (Existing LUNs) instead. For shared concepts (portals, paths, health), see iSCSI Storage.

UI path: Control Center -> Storage -> Instance Storage

When to Use

  • You have a COMSTAR array (illumos or a compatible ZFS appliance) on the storage network.

  • You want VM root disks, data disks, and templates to live on that array.

  • You want Karios to create, grow, and remove the array volumes for you.

What Karios Does on the Array

Once a pool is in use, Karios manages these objects on the array:

  • One target group named karios-iscsi. The pool’s targets join it.

  • One host group per Karios host, named kh-<host name>. Each host is admitted by its own initiator IQN.

  • For each VM disk: one ZFS volume under the dataset you name, one logical unit, and one view that shows it to the host group of the host running the VM.

Karios does not create the iSCSI target itself. You create it on the array before you add the pool.

Before You Start

  • [ ] You are a platform administrator. Only a platform administrator can register an array. Creating a pool on a registered array needs the storage manage permission.

  • [ ] The array’s STMF service is online.

  • [ ] The ZFS pool that will hold the volumes exists, for example sanpool. Pick a dataset path under it, for example sanpool/karios. The dataset itself can be new.

  • [ ] An iSCSI target exists on the array. Bind it to a named target portal group so the pool form can list its portals for you to tick. A target bound only to the default portal group lists none, and you then type each portal’s address, port, and target IQN (see Task 2).

  • [ ] Use a target that is dedicated to Karios. A target must join karios-iscsi, and COMSTAR allows that only while the target is offline. Karios refuses to take a target offline while it carries sessions.

  • [ ] You have a management address, a username, and a password for an SSH account that can run the array’s COMSTAR and ZFS commands.

  • [ ] You have the array’s SSH host key. From a machine you trust, run ssh-keyscan -t ecdsa <management address>. On the array console, check the fingerprint with ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub.

  • [ ] The storage network and portals meet the checklist in iSCSI Storage.

Important

This pool type has no CHAP. Hosts are admitted by per-host IQN masking on the array. Keep the storage network isolated from guest and public traffic.

Task 1: Register the Array

When to Use: Do this once per array, before you create the first pool on it.

Purpose: Store the array’s address, credentials, and host key so Karios can manage it.

Steps:

  1. Open Control Center -> Storage -> Instance Storage.

  2. Click Add Instance Storage. The Add New Instance Storage panel opens.

  3. Set Protocol to iscsi. If a Driver field appears, pick comstar (carves volumes on an array).

  4. If no array is registered yet, the Register array form is already open. Otherwise, click Register array next to Array.

  5. Fill in the fields:

    Field

    What to enter

    Array name

    1 to 128 characters: lowercase letters, numbers, and hyphens. For example comstar-lab.

    Driver

    comstar.

    Dataset

    The ZFS dataset Karios carves volumes under, for example sanpool/karios. Required.

    Management address

    IP address or hostname of the array’s management interface.

    Username and Password

    The SSH account Karios uses on the array.

    Verify the array's SSH host key

    On by default. Leave it on.

    Array SSH host key

    Paste the key line from ssh-keyscan. Required while verification is on.

  6. Click Register. Karios probes the array before it stores it.

Expected Outcome:

  • The array appears in the Array list and is selected.

  • Under the selected array, the panel shows SSH host key verified: and the key.

If this fails:

  1. KF-5010: the array did not answer. Check the management address, the network path from the management node, and the credentials.

  2. KF-5011: the array answered but the probe failed. Check the username and password, and that the array really runs COMSTAR. The probe also fails if STMF is not online or if the ZFS pool named in Dataset does not exist.

  3. A wrong host key is refused. Read the key again with ssh-keyscan and compare the fingerprint on the array console.

Warning

If you turn off Verify the array's SSH host key, the form shows SSH host key not verified under the selected array. Karios then accepts any key that answers at that address, so its login could go to an impostor. Use this only in a lab.

Task 2: Create the Pool

When to Use: Do this after the array is registered, once per pool you need.

Purpose: Create an iSCSI pool that places VM disks on the array.

Steps:

  1. In Add New Instance Storage, set Scope:

    • Zone: every cluster in the zone can use the pool.

    • Cluster: one cluster only. Pick the Pod and Cluster.

  2. Pick the Zone and enter a Name for the pool.

  3. Set Protocol to iscsi. If a Driver field appears, pick comstar (carves volumes on an array). Then pick the registered array in Array.

  4. Under Portals, tick the portals this pool uses. The list shows the portals the array lists, grouped by target. None is ticked for you, because arrays also list their management portals. Pick two or more on the storage network.

  5. If the array lists no portal, typed rows appear instead. For each row enter the portal address, the port (default 3260), and the target IQN. The IQN must be lowercase, for example iqn.2010-08.org.illumos:02:target. Click Add portal for more rows.

  6. Pick Minimum paths. The list stops at the number of portals you picked, because a host gets at most one path per portal. The default is 2, or 1 if you picked a single portal.

  7. Optional: add Storage tags to steer disks of certain volume profiles to this pool.

  8. Click Add Instance Storage.

Expected Outcome:

  • The pool appears in the list with protocol ISCSI COMSTAR.

  • On the pool’s detail page, the Fabric card shows Health as Up once every host has logged in.

If this fails:

  1. KF-4014: a host cannot reach the portals or log in. Read the line for each host, then check the portal address, port, and storage network.

  2. KF-4013: a host sees fewer paths than Minimum paths. Check that every picked portal belongs to the target and is up on the array, or lower Minimum paths.

  3. A message that the target “must join target group karios-iscsi” means the target carries sessions. The message lists the commands to run in a maintenance window. Or use a target dedicated to Karios.

  4. A portal the array does not serve is refused. Refresh the portal list and pick again.

Fabric Card Reference

Open the pool from the list to see its Fabric card.

Row

What it shows

Health

Pool health from the hosts. See iSCSI Storage.

Registration

Whether the platform has accepted the pool.

Array and Dataset

The registered array and the dataset volumes are carved under.

Protocol

ISCSI COMSTAR.

Minimum paths

The pool’s path floor. A Single path chip appears when the pool or a host has only one path.

Redundancy

How many portals the array serves and how many this pool uses.

CHAP

None for this pool type.

Admin switch

The pool’s admin state, for example Up, Disabled, or Maintenance.

Portals and Hosts

Each portal, and each host’s reachability.

Last error

The most recent error, if any.

Task 3: Work With Disks on the Pool

When to Use: Use this when you create, grow, or remove VM disks that land on the pool.

Purpose: Know what Karios does on the array for each disk action.

Steps:

  1. Create VMs and disks as usual. See Block Storage. Disks land on the pool when its scope and storage tags match the volume profile.

  2. When a disk is first attached to a VM, Karios creates its volume on the array. Each attach shows the volume to that VM’s host. Each detach removes the view again.

  3. To grow a disk, use Update Disk Size in Block Storage. Disks can only grow. A smaller size is refused, because shrinking would destroy data.

  4. When you deploy VMs from a template, Karios keeps one copy of the template on the pool and clones each root disk from it on the array. This saves space and time.

Expected Outcome:

  • On the pool’s Fabric card, the Hosts rows name no disk as unhealthy.

  • In Block Storage, the Fabric paths panel reads Healthy for a disk held by the node that answers the request. For a disk held by another host it reads Not connected with a note, as described in iSCSI Storage.

If this fails:

  1. KF-4013 on attach: the host sees fewer paths than the minimum. Check the portals and the host’s path count under Hosts on the Fabric card.

  2. KF-4092: the template copy on the pool still has disks cloned from it. Delete those disks first.

  3. KF-5012: the array refused the operation. The network is fine. Check space and state on the array.

Task 4: Disable, Maintain, Change Scope, or Delete the Pool

When to Use: Use this to stop new disks, to service the array, or to retire the pool.

Purpose: Run pool lifecycle actions safely.

Steps:

  1. In the pool row, open the Actions menu.

  2. Use Disable pool to stop new disks while existing VMs keep running.

  3. Use Enable maintenance for an approved maintenance window.

  4. To change scope, first disable the pool. Then choose Change scope and pick Zone or a Cluster.

  5. To delete the pool:

    1. Move or delete every disk on it. Delete is refused while guest disks exist, with pool "<name>" still has N volume(s); delete them before deleting the pool. Cached template copies are removed with the pool. If one is still being written to a host, wait and retry.

    2. If the pool reads Disabled, choose Enable pool first. Maintenance can only start from Up. Then choose Enable maintenance and wait for the pool to read Maintenance.

    3. Choose Delete, type the pool name, and confirm.

Warning

The delete dialog warns that every VM disk stored on the pool will be lost and cannot be recovered. Check that no disk you need is still on the pool.

Expected Outcome:

  • Disabled and maintenance states show in the list and on the Admin switch row.

  • A deleted pool is gone from the list.

Limits

  • No CHAP. Use an isolated storage network.

  • No volume snapshots (KF-4016).

  • No Storage QoS. IOPS limits on a volume profile that can land here, or on a disk of this pool, are refused with KF-4016.

  • Disks can grow but not shrink.

Troubleshooting

Code or message

What it means

What to do

KF-5010

The array did not answer.

Check the management address, the network path, and the credentials.

KF-5011

The array answered, but the probe failed.

Check the credentials, that the driver matches the array, that STMF is online, and that the dataset’s ZFS pool exists.

KF-5012

The array refused the operation.

Check space and state on the array.

KF-4013

A host sees fewer paths than the minimum.

Check every portal is up and belongs to the target, or lower Minimum paths.

KF-4014

A host cannot reach the portals or log in.

Read each host’s line for the failed portals and the cause. Then check the portal address, port, and storage network.

KF-4016

A snapshot, or Storage QoS (Min or Max IOPS), asked on this pool.

Keep disks that need snapshots on RBD storage. Leave Min and Max IOPS off the volume profile.

KF-4092

The template copy still has clones.

Delete the disks cloned from it first.

Placement: Disabled by the fabric

No host could reach the pool for about five minutes.

Fix the storage network or array. Placement comes back by itself a few minutes after a host can reach the pool again.

SSH host key not verified

The array was registered without a host key.

In Add New Instance Storage, pick the array and click Verify host key.

Done Criteria

  • [ ] The array is registered with a verified SSH host key.

  • [ ] The pool reads Up in Health and lists two or more portals.

  • [ ] A test disk on a running VM leaves the Hosts rows on the Fabric card without an unhealthy-disk line, and Fabric paths never reads Degraded or Critical.

  • [ ] Your team knows that snapshots and CHAP are not available on this pool.