This article hasn't been updated for over 5 years. The information below may be obsolete.

How to setup ZFS Storage Appliance iSCSI luns on Ubuntu Server

Updated 11 May 2020 | 12 minute read | 6896 views | Storage

Both Ubuntu Server 20.10 and Ubuntu Server 21.04 come with the iSCSI initiator package open-iscsi installed out of the box allowing for easy configuration for iSCSI attached storage.

This article describes how to integrate iSCSI LUNs presented by an ZFS Storage Appliance (ZFSSA) onto the Ubuntu Server environment using the command-line interface.

This article has been updated to show an installation on Ubuntu Server 21.04, thus your output may differ from that below.

Prerequisites

This article assumes the following:

For the Ubuntu server:
- You know the root password.
For the ZFSSA:
- You know the hostname/IP address.
- You know and root password.
- Pools are configured.
- The network has been configured to present LUNS.

Installing the Ubuntu iSCSI Initiator

The open-iscsi package is installed by default on both Ubuntu Server 20.10 and 21.04. However, if you have removed the package for some reason you can simply reinstall the iSCSI initiator package using the apt(8) command. For example:

# apt install open-iscsi
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libisns0 libopeniscsiusr0.2.0
The following NEW packages will be installed:
  libisns0 libopeniscsiusr0.2.0 open-iscsi
0 upgraded, 3 newly installed, 0 to remove and 15 not upgraded.
Need to get 437 kB of archives.
After this operation, 2176 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
   :
Preconfiguring packages ...
   :
   :
Preparing to unpack .../open-iscsi_2.1.2-1ubuntu1_amd64.deb ...
Unpacking open-iscsi (2.1.2-1ubuntu1) ...
   :
   :
Setting up open-iscsi (2.1.2-1ubuntu1) ...
Created symlink /etc/systemd/system/sockets.target.wants/iscsid.socket → /lib/systemd/system/iscsid.socket.
Created symlink /etc/systemd/system/iscsi.service → /lib/systemd/system/open-iscsi.service.
Created symlink /etc/systemd/system/sysinit.target.wants/open-iscsi.service → /lib/systemd/system/open-iscsi.service.
   :
   :
update-initramfs: Generating /boot/initrd.img-5.8.0-36-generic

iSCSI Initiator auto startup

The iSCSI Initiator should automatically start on each reboot of the Ubuntu Server. However, we can confirm here that this is the case.

Check that the iSCSI Initiator will start when the system is rebooted:
```
# systemctl is-enabled open-iscsi.service
enabled
```

If the iSCSI Initiator service is disabled, we can simple enable it and start the service as follows:

# systemctl enable open-iscsi.service
Synchronizing state of open-iscsi.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable open-iscsi
Created symlink /etc/systemd/system/iscsi.service → /lib/systemd/system/open-iscsi.service.
Created symlink /etc/systemd/system/sysinit.target.wants/open-iscsi.service → /lib/systemd/system/open-iscsi.service.
# systemctl start open-iscsi

Ubuntu Server Configuration

Identifying the Host IQN

The iSCSI Qualified Name (IQN) of the Ubunutu server is used to identify the host to the ZFSSA. The IQN is automatically generated when the iSCSI initiator package is installed and is stored in a file in /etc/iscsi. To find the IQN of your Ubuntu server, simply type:

# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1993-08.org.debian:01:b96d30143d8e

The IQN for our Ubuntu Server is iqn.1993-08.org.debian:01:b96d30143d8e.

CHAP Authentication

In this article, we are using the Challange Handshake Authentication Protocol (CHAP) to authenticate communication between the Ubuntu server and the ZFSSA. The IQN is used as the CHAP name for the iSCSI initiator and a secure password is used for the CHAP secret.

To configure CHAP authentication, use your favourite text editor and edit the file /etc/iscsi/iscsid.conf

Remove the # character from the beginning of the following line:

#node.session.auth.authmethod = CHAP

to:

node.session.auth.authmethod = CHAP

Set the CHAP username and password, using the following steps:
1. Edit the lines that set the CHAP username and password to remove the # character from the beginning of these lines:
```
#node.session.auth.username = username
#node.session.auth.password = password
```
  to:
```
node.session.auth.username = username
node.session.auth.password = password
```
2. Change username to the IQN of the Ubuntu server. For example, the username is:
```
node.session.auth.username = iqn.1993-08.org.debian:01:b96d30143d8e
```
3. Change the password to a reasonably secure password to be used for authentication. For our example, the password CHAPsecret11 is used. The line now look like:
```
node.session.auth.password = CHAPsecret11
```
To set the username and password for all targets, use the following:
1. Removing the # character in front of the following lines:
```
#node.session.auth.username_in = username
#node.session.auth.password_in = password
```
2. and also set the username and password as shown:
```
node.session.auth.username_in = chapuser
node.session.auth.password_in = CHAPsecret22
```
Save the changes and quit your editor

Configure the ZFSSA using CLI

Access to the ZFSSA CLI is possibe using any SSH client. For example:

% ssh root@zfssa
Password: ************
Last login: Sun May 10 19:39:23 2020 from ubuntu
zfssa:>

Define the iSCSI Target Group

A target group is created on the ZFSSA to define the ports and the protocol by which the LUNs to be presented to the Ubuntu server is accessed.

In our example, an iSCSI target group is created that contains the LUN as an iSCSI target that is identified by the default IQN for the ZFSSA and presented over default appliance interfaces.

zfssa:> configuration san targets
zfssa:configuration san targets> iscsi
zfssa:configuration san targets iscsi> create
zfssa:configuration san targets iscsi target (uncommitted)> set alias=iSCSI-Targets
           alias = iSCSI-Targets (uncommitted) 
zfssa:configuration san targets iscsi target-001 (uncommitted)> commit
zfssa:configuration san targets iscsi> list
TARGET     ALIAS
target-000 iSCSI-Targets
           |
           +-> IQN
               iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881
zfssa:configuration san targets iscsi> select target-000
zfssa:configuration san targets iscsi target-000> list
Properties:
           alias = iscsi-Target
             iqn = iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881
            auth = none
  targetchapuser = (unset)
targetchapsecret = (unset)
      interfaces = nge0
zfssa:configuration san targets iscsi target-000> set targetchapuser=chapuser
  targetchapuser = iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881 (uncommitted)

zfssa:configuration san targets iscsi target-000> set targetchapsecret=CHAPsecret22
targetchapsecret = ************ (uncommitted)
zfssa:configuration san targets iscsi target-000> commit
zfssa:configuration san targets iscsi target-000> list
Properties:
           alias =
             iqn =
            auth =
  targetchapuser = \
      iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881
targetchapsecret = ************
      interfaces =

zfssa:configuration san targets iscsi target-000> cd ..
zfssa:configuration san targets iscsi> list
TARGET     ALIAS
target-000 iSCSI-Targets
           |
           +-> IQN
               iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881

zfssa:configuration san targets iscsi> group
zfssa:configuration san targets iscsi groups> create
zfssa:configuration san targets iscsi group (uncommitted)> set name=iscsi-TG
name = iscsi-TG (uncommitted)
zfssa:configuration san targets iscsi group (uncommitted)> set targets=iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881
targets = iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881 (uncommitted)
zfssa:configuration san targets iscsi group (uncommitted)> commit
zfssa:configuration san targets iscsi groups> list
GROUP      NAME
group-000   iscsi-TG
            |
            +-> TARGETS
                iqn.1986-03.com.sun:02:e8589be5-144d-c9b2-89d4-f7fe4e887881
zfssa:configuration san targets iscsi groups> cd /
zfssa:>

Defining an iSCSI Initiator

An iSCSI initiator is defined to restrict which servers have access to a particular volume. If more than one host can write to a given volume concurrently, inconsistency in file system caching between hosts can cause corruption in the on-disk image. Typically, a single initiator is given access to a volume, unless a specialized cluster file system is being used.

The iSCSI initiator serves to define the “host” from the point of view of the ZFS Storage Appliance. To identify the Ubuntu server to the ZFS Storage Appliance, the Ubuntu iSCSI IQN must be registered with the ZFS Storage Appliance as shown in the following example.

zfssa:> configuration san initiators
zfssa:configuration san initiators> iscsi
zfssa:configuration san initiators iscsi> ls
Children:
                            groups => Manage groups
zfssa:configuration san initiators iscsi> create
zfssa:configuration san initiators iscsi initiator (uncommitted)> set alias=Ubuntu
                            alias = Ubuntu (uncommitted)
zfssa:configuration san initiators iscsi initiator (uncommitted)> set initiator= iqn.1993-08.org.debian:01:b96d30143d8e
initiator = iqn.1993-08.org.debian:01:b96d30143d8e (uncommitted)
zfssa:configuration san initiators iscsi initiator (uncommitted)> set chapuser= iqn.1993-08.org.debian:01:b96d30143d8e
chapuser = iqn.1993-08.org.debian:01:b96d30143d8e (uncommitted)
zfssa:configuration san initiators iscsi initiator (uncommitted)> set chapsecret=CHAPsecret11
chapsecret = ************ (uncommitted)
zfssa:configuration san initiators iscsi initiator (uncommitted)> commit
zfssa:configuration san initiators iscsi> ls
Initiators:

NAME          ALIAS
initiator-000 Ubuntu
              |
              +-> INITIATOR
                  iqn.1993-08.org.debian:01:b96d30143d8e

Children:
              groups => Manage groups

zfssa:configuration san initiators iscsi>

Defining an iSCSI Initiator Group

Related iSCSI initiators can be combined into logical groups to allow single commands to be executed on multiple iSCSI initiators, such as, for example, assigning LUN access to all iSCSI initiators in a group with one command. In this example, the iSCSI initiator group will contain one initiator, but in a cluster, where multiple servers are treated as a single logical entity, the initiator group may contain multiple initiators.

zfssa:configuration san initiators iscsi> groups
zfssa:configuration san initiators iscsi groups> create
zfssa:configuration san initiators iscsi group (uncommitted)> set name=ubuntu-servers
name = ubuntu-servers (uncommitted)
zfssa:configuration san initiators iscsi group (uncommitted)> set initiators=iqn.1993-08.org.debian:01:b96d30143d8e
initiators = iqn.1993-08.org.debian:01:b96d30143d8e (uncommitted)
zfssa:configuration san initiators iscsi group (uncommitted)> commit
zfssa:configuration san initiators iscsi groups > ls
Groups:
GROUP      NAME
group-000  ubuntu-servers
           |
           +-> INITIATORS
               iqn.1988-12.com.oracle:5fa4af61c4a0

zfssa:configuration san initiators iscsi groups> cd /
zfssa:>

Defining an ZFS Storage Appliance Project

A project can be defined in the ZFS Storage Appliance to group related volumes. A project allows inheritance of properties for file systems and LUNs presented from the project and also allows quotas and reservations to be applied.

zfssa:> shares
zfssa:shares> project ubuntu-project
zfssa:shares ubuntu-project (uncommitted)> commit

Defining an Oracle ZFS Storage Appliance LUN

A LUN must now be created from an existing pool of storage resources, which the Ubuntu server will access. For this example, a thin-provisioned 32 GB LUN called Demo will be created. The LUN will be set up for an EXT4 file system.

The target group will be the iSCSI target group created in the section "Defining an iSCSI Target Group" to ensure that this LUN can be accessed using iSCSI protocol. The initiator group defined in the section "Defining an iSCSI Initiator Group" will be used to ensure that only the server(s) defined in the ubuntu-servers group can access this LUN. (In this example, this group contains only one server.)

zfssa:shares> select ubuntu-project
zfssa:shares ubuntu-project> lun Demo
zfssa:shares ubuntu-project/Demo (uncommitted)> set volsize=32G
volsize = 32G (uncommitted)
zfssa:shares ubuntu-project/Demo (uncommitted)> set targetgroup=iscsi-TG
targetgroup = iscsi-TG (uncommitted)
zfssa:shares ubuntu-project/Demo (uncommitted)> set initiatorgroup=ubuntu-servers
initiatorgroup = ubuntu-servers (uncommitted)
zfssa:shares ubuntu-project/Demo (uncommitted)> list
Properties:
               checksum = fletcher4 (inherited)
            compression = off (inherited)
                  dedup = false (inherited)
                 copies = 1 (inherited)
                logbias = latency (inherited)
         secondarycache = all (inherited)
           volblocksize = 8K (default)
                volsize = 32G (uncommitted)
                 sparse = false (default)
               exported = true (inherited)
            targetgroup = iscsi-TG (uncommitted)
         initiatorgroup = ubuntu-servers (uncommitted)
               lunumber = (default)
         assignednumber = (default)
                 status = (default)
            fixednumber = (default)
                lunguid = (default)
         canonical_name = (default)
              nodestroy = (default)

Children:
              snapshots => Manage snapshots
            replication => Manage remote replication

zfssa:shares ubuntu-project/Demo (uncommitted)> set volblocksize=16k
           volblocksize = 16K (uncommitted)
zfssa:shares ubuntu-project/Demo (uncommitted)> commit
zfssa:shares ubuntu-project> cd /
zfssa>

Configuring a LUN for Use by the Ubuntu Server

Now that the LUN is prepared and available to an authenticated iSCSI initiator, the LUN must be configured for use by the Ubuntu server by completing the following steps:

Initiate an Ubuntu iSCSI session with the ZFS Storage Appliance. Since the LUN was created prior to initiating the iSCSI session, it will be automatically enabled.

# systemctl start open-iscsi
# iscsiadm -m discovery -t sendtargets -p zfssa
192.168.0.73:3260,2 iqn.1986-03.com.sun:02:012213d6-826b-47b3-a438-f336cf0f8c01
# iscsiadm -m node -p zfssa -l
Logging in to [iface: default, target: iqn.1986-03.com.sun:02:012213d6-826b-47b3-a438-f336cf0f8c01, portal: 192.168.0.73,3260]
Login to [iface: default, target: iqn.1986-03.com.sun:02:012213d6-826b-47b3-a438-f336cf0f8c01, portal: 192.168.0.73,3260] successful.

(optional) Review /var/log/syslog and check for any errors associated with iSCSI initiator and connectivity. For example:

# tail /var/log/syslog
May 10 12:36:16 ubuntu kernel: [70844.682726] scsi host3: iSCSI Initiator over TCP/IP
May 10 12:36:16 ubuntu kernel: [70844.693372] scsi 3:0:0:0: Direct-Access     SUN      Sun Storage 7000 1.0  PQ: 0 ANSI: 5
May 10 12:36:16 ubuntu kernel: [70844.695880] sd 3:0:0:0: Attached scsi generic sg2 type 0
May 10 12:36:16 ubuntu kernel: [70844.696616] sd 3:0:0:0: Power-on or device reset occurred
May 10 12:36:16 ubuntu kernel: [70844.700720] sd 3:0:0:0: [sdb] 67108864 512-byte logical blocks: (34.4 GB/32.0 GiB)
May 10 12:36:16 ubuntu kernel: [70844.701208] sd 3:0:0:0: [sdb] Write Protect is off
May 10 12:36:16 ubuntu kernel: [70844.701211] sd 3:0:0:0: [sdb] Mode Sense: 53 00 00 00
May 10 12:36:16 ubuntu kernel: [70844.702200] sd 3:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
May 10 12:36:16 ubuntu kernel: [70844.729760] sd 3:0:0:0: [sdb] Attached SCSI disk
May 10 12:36:17 ubuntu iscsid: Connection1:0 to [target: iqn.1986-03.com.sun:02:012213d6-826b-47b3-a438-f336cf0f8c01, portal: 192.168.0.73,3260] through [iface: default] is operational now

Locate the new LUN device on the Ubuntu server. The quickest method I find is to grep syslog for Attached SCSI disk. For example:

# grep -i "attached scsi disk" /var/log/syslog
May 10 11:27:39 ubuntu kernel: [66727.428119] sd 3:0:0:0: [sdb] Attached SCSI disk
May 10 12:36:16 ubuntu kernel: [70844.729760] sd 3:0:0:0: [sdb] Attached SCSI disk

From the about output we can determine that our new lun is sdb. Let's confirm:

# ls -l /dev/sdb
brw-rw---- 1 root disk 8, 16 May 10 12:36 /dev/sdb

Prepare the lun by creating a Linux partition table using the fdisk(8) command:

# fdisk /dev/sdb

Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognised partition table.
Created a new DOS disklabel with disk identifier 0x1581ec2b.

Command (m for help): p
Disk /dev/sdb: 32 GiB, 34359738368 bytes, 67108864 sectors
Disk model: Sun Storage 7000
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 32768 bytes / 32768 bytes
Disklabel type: dos
Disk identifier: 0x1581ec2b

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-67108863, default 2048): 2048
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-67108863, default 67108863): 67108863

Created a new partition 1 of type 'Linux' and of size 32 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Synching disks.

#

Configure the device as storage for a large EXT4 file system without any intervening virtualization as shown in the following example:

# mkfs.ext4 -L Demo /dev/sdb1
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 8388352 4k blocks and 2097152 inodes
Filesystem UUID: c103d253-5861-4800-9d6e-086fa8712b80
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Setup the new filesystem for mounting at system boot time:

# echo "LABEL=Demo /demo ext4 defaults 2 2" >> /etc/fstab
# mkdir /demo
# mount /demo
# df -h /demo
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1        32G   49M   30G   1% /demo

The output from the df(1) command shows that we now have 32 GB of new space is now available.