iSCSI Pools on a Managed Array (COMSTAR)
Use this page when Karios should create and manage the disks on your SAN array. Karios logs in to the array, carves one volume for every VM disk, and shows it to the right hosts.
If you want to attach LUNs that already exist on your own target, use Generic iSCSI Pools (Existing LUNs) instead. For shared concepts (portals, paths, health), see iSCSI Storage.
UI path: Control Center -> Storage -> Instance Storage
When to Use
You have a COMSTAR array (illumos or a compatible ZFS appliance) on the storage network.
You want VM root disks, data disks, and templates to live on that array.
You want Karios to create, grow, and remove the array volumes for you.
What Karios Does on the Array
Once a pool is in use, Karios manages these objects on the array:
One target group named
karios-iscsi. The pool’s targets join it.One host group per Karios host, named
kh-<host name>. Each host is admitted by its own initiator IQN.For each VM disk: one ZFS volume under the dataset you name, one logical unit, and one view that shows it to the host group of the host running the VM.
Karios does not create the iSCSI target itself. You create it on the array before you add the pool.
Before You Start
[ ] You are a platform administrator. Only a platform administrator can register an array. Creating a pool on a registered array needs the storage manage permission.
[ ] The array’s STMF service is online.
[ ] The ZFS pool that will hold the volumes exists, for example
sanpool. Pick a dataset path under it, for examplesanpool/karios. The dataset itself can be new.[ ] An iSCSI target exists on the array. Bind it to a named target portal group so the pool form can list its portals for you to tick. A target bound only to the default portal group lists none, and you then type each portal’s address, port, and target IQN (see Task 2).
[ ] Use a target that is dedicated to Karios. A target must join
karios-iscsi, and COMSTAR allows that only while the target is offline. Karios refuses to take a target offline while it carries sessions.[ ] You have a management address, a username, and a password for an SSH account that can run the array’s COMSTAR and ZFS commands.
[ ] You have the array’s SSH host key. From a machine you trust, run
ssh-keyscan -t ecdsa <management address>. On the array console, check the fingerprint withssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub.[ ] The storage network and portals meet the checklist in iSCSI Storage.
Important
This pool type has no CHAP. Hosts are admitted by per-host IQN masking on the array. Keep the storage network isolated from guest and public traffic.
Task 1: Register the Array
When to Use: Do this once per array, before you create the first pool on it.
Purpose: Store the array’s address, credentials, and host key so Karios can manage it.
Steps:
Open
Control Center -> Storage -> Instance Storage.Click
Add Instance Storage. TheAdd New Instance Storagepanel opens.Set
Protocoltoiscsi. If aDriverfield appears, pickcomstar (carves volumes on an array).If no array is registered yet, the
Register arrayform is already open. Otherwise, clickRegister arraynext toArray.Fill in the fields:
Field
What to enter
Array name1 to 128 characters: lowercase letters, numbers, and hyphens. For example
comstar-lab.Drivercomstar.DatasetThe ZFS dataset Karios carves volumes under, for example
sanpool/karios. Required.Management addressIP address or hostname of the array’s management interface.
UsernameandPasswordThe SSH account Karios uses on the array.
Verify the array's SSH host keyOn by default. Leave it on.
Array SSH host keyPaste the key line from
ssh-keyscan. Required while verification is on.Click
Register. Karios probes the array before it stores it.
Expected Outcome:
The array appears in the
Arraylist and is selected.Under the selected array, the panel shows
SSH host key verified:and the key.
If this fails:
KF-5010: the array did not answer. Check the management address, the network path from the management node, and the credentials.KF-5011: the array answered but the probe failed. Check the username and password, and that the array really runs COMSTAR. The probe also fails if STMF is not online or if the ZFS pool named inDatasetdoes not exist.A wrong host key is refused. Read the key again with
ssh-keyscanand compare the fingerprint on the array console.
Warning
If you turn off Verify the array's SSH host key, the form shows SSH host key not
verified under the selected array. Karios then accepts any key that answers at that
address, so its login could go to an impostor. Use this only in a lab.
Task 2: Create the Pool
When to Use: Do this after the array is registered, once per pool you need.
Purpose: Create an iSCSI pool that places VM disks on the array.
Steps:
In
Add New Instance Storage, setScope:Zone: every cluster in the zone can use the pool.Cluster: one cluster only. Pick thePodandCluster.
Pick the
Zoneand enter aNamefor the pool.Set
Protocoltoiscsi. If aDriverfield appears, pickcomstar (carves volumes on an array). Then pick the registered array inArray.Under
Portals, tick the portals this pool uses. The list shows the portals the array lists, grouped by target. None is ticked for you, because arrays also list their management portals. Pick two or more on the storage network.If the array lists no portal, typed rows appear instead. For each row enter the portal address, the port (default
3260), and the target IQN. The IQN must be lowercase, for exampleiqn.2010-08.org.illumos:02:target. ClickAdd portalfor more rows.Pick
Minimum paths. The list stops at the number of portals you picked, because a host gets at most one path per portal. The default is2, or1if you picked a single portal.Optional: add
Storage tagsto steer disks of certain volume profiles to this pool.Click
Add Instance Storage.
Expected Outcome:
The pool appears in the list with protocol
ISCSI COMSTAR.On the pool’s detail page, the
Fabriccard showsHealthasUponce every host has logged in.
If this fails:
KF-4014: a host cannot reach the portals or log in. Read the line for each host, then check the portal address, port, and storage network.KF-4013: a host sees fewer paths thanMinimum paths. Check that every picked portal belongs to the target and is up on the array, or lowerMinimum paths.A message that the target “must join target group karios-iscsi” means the target carries sessions. The message lists the commands to run in a maintenance window. Or use a target dedicated to Karios.
A portal the array does not serve is refused. Refresh the portal list and pick again.
Fabric Card Reference
Open the pool from the list to see its Fabric card.
Row |
What it shows |
|---|---|
|
Pool health from the hosts. See iSCSI Storage. |
|
Whether the platform has accepted the pool. |
|
The registered array and the dataset volumes are carved under. |
|
|
|
The pool’s path floor. A |
|
How many portals the array serves and how many this pool uses. |
|
|
|
The pool’s admin state, for example |
|
Each portal, and each host’s reachability. |
|
The most recent error, if any. |
Task 3: Work With Disks on the Pool
When to Use: Use this when you create, grow, or remove VM disks that land on the pool.
Purpose: Know what Karios does on the array for each disk action.
Steps:
Create VMs and disks as usual. See Block Storage. Disks land on the pool when its scope and storage tags match the volume profile.
When a disk is first attached to a VM, Karios creates its volume on the array. Each attach shows the volume to that VM’s host. Each detach removes the view again.
To grow a disk, use
Update Disk Sizein Block Storage. Disks can only grow. A smaller size is refused, because shrinking would destroy data.When you deploy VMs from a template, Karios keeps one copy of the template on the pool and clones each root disk from it on the array. This saves space and time.
Expected Outcome:
On the pool’s
Fabriccard, theHostsrows name no disk as unhealthy.In Block Storage, the
Fabric pathspanel readsHealthyfor a disk held by the node that answers the request. For a disk held by another host it readsNot connectedwith a note, as described in iSCSI Storage.
If this fails:
KF-4013on attach: the host sees fewer paths than the minimum. Check the portals and the host’s path count underHostson theFabriccard.KF-4092: the template copy on the pool still has disks cloned from it. Delete those disks first.KF-5012: the array refused the operation. The network is fine. Check space and state on the array.
Task 4: Disable, Maintain, Change Scope, or Delete the Pool
When to Use: Use this to stop new disks, to service the array, or to retire the pool.
Purpose: Run pool lifecycle actions safely.
Steps:
In the pool row, open the
Actionsmenu.Use
Disable poolto stop new disks while existing VMs keep running.Use
Enable maintenancefor an approved maintenance window.To change scope, first disable the pool. Then choose
Change scopeand pick Zone or a Cluster.To delete the pool:
Move or delete every disk on it. Delete is refused while guest disks exist, with
pool "<name>" still has N volume(s); delete them before deleting the pool. Cached template copies are removed with the pool. If one is still being written to a host, wait and retry.If the pool reads
Disabled, chooseEnable poolfirst. Maintenance can only start fromUp. Then chooseEnable maintenanceand wait for the pool to readMaintenance.Choose
Delete, type the pool name, and confirm.
Warning
The delete dialog warns that every VM disk stored on the pool will be lost and cannot be recovered. Check that no disk you need is still on the pool.
Expected Outcome:
Disabled and maintenance states show in the list and on the
Admin switchrow.A deleted pool is gone from the list.
Limits
No CHAP. Use an isolated storage network.
No volume snapshots (
KF-4016).No Storage QoS. IOPS limits on a volume profile that can land here, or on a disk of this pool, are refused with
KF-4016.Disks can grow but not shrink.
Troubleshooting
Code or message |
What it means |
What to do |
|---|---|---|
|
The array did not answer. |
Check the management address, the network path, and the credentials. |
|
The array answered, but the probe failed. |
Check the credentials, that the driver matches the array, that STMF is online, and that the dataset’s ZFS pool exists. |
|
The array refused the operation. |
Check space and state on the array. |
|
A host sees fewer paths than the minimum. |
Check every portal is up and belongs to the target, or lower |
|
A host cannot reach the portals or log in. |
Read each host’s line for the failed portals and the cause. Then check the portal address, port, and storage network. |
|
A snapshot, or Storage QoS (Min or Max IOPS), asked on this pool. |
Keep disks that need snapshots on RBD storage. Leave Min and Max IOPS off the volume profile. |
|
The template copy still has clones. |
Delete the disks cloned from it first. |
|
No host could reach the pool for about five minutes. |
Fix the storage network or array. Placement comes back by itself a few minutes after a host can reach the pool again. |
|
The array was registered without a host key. |
In |
Done Criteria
[ ] The array is registered with a verified SSH host key.
[ ] The pool reads
UpinHealthand lists two or more portals.[ ] A test disk on a running VM leaves the
Hostsrows on theFabriccard without an unhealthy-disk line, andFabric pathsnever readsDegradedorCritical.[ ] Your team knows that snapshots and CHAP are not available on this pool.