CAM Target Layer

Earlier this week, Ken Merry committed CTL to HEAD for testing.  From the commit message:

CTL is a disk and processor device emulation subsystem originally written for Copan Systems under Linux starting in 2003. It has been shipping in Copan (now SGI) products since 2005. It was ported to FreeBSD in 2008, and thanks to an agreement between SGI (who acquired Copan's assets in 2010) and Spectra Logic in 2010, CTL is available under a BSD-style license. The intent behind the agreement was that Spectra would work to get CTL into the FreeBSD tree.

We spoke to Ken about the benefits of CTL and this is what he had to say:

CTL offers a number of benefits, but here are a couple of ways we can use it:

• It provides the missing piece needed to turn a FreeBSD box into an external RAID array.  All you need is a Fibre Channel card (Qlogic 4Gb and 8Gb cards work now), and some disks, and with ZFS managing the disks and CTL providing the target interface, you've got an external RAID array.  End users can do it, or companies can use it as the  foundation for a FreeBSD-based storage appliance.

• CTL provides a test framework for CAM.  When we implement new features and command support in CAM, we can immediately test out the new features in CTL.  For instance, when I implemented SCSI descriptor sense support  last year, I actually first implemented descriptor sense in CTL.  That way, when I implemented it in CAM, I had a way to test everything out.  I was able to test, through sense data injection, scenarios that would be impossible to trigger using an ordinary hard disk.  You can't make a hard disk return every possible error and combination of errors, but with CTL, you can do that.  So I was able to more fully test everything out and gain confidence that the descriptor sense code worked properly.

For people who want to test it, you don't need a Fibre Channel card, you can actually create LUNs and use CTL without any new hardware.  With the CTL CAM SIM, the LUNs are visible on the internal 'ctl2cam0' bus.  Here's what the output of camcontrol devlist -v looks like:

scbus6 on ctl2cam0 bus 0:
             at scbus6 target 1 lun 0 (pass56,da48)
             at scbus6 target 1 lun 1 (pass57,da49)
             at scbus6 target 1 lun 2 (pass58,da50)
<>                                 at scbus6 target -1 lun -1 ()

The da(4) driver is attached to the CTL LUNs, and you can do normal I/O to them just as you would any other disk.

One thing to watch out for is that if you use a block device (as opposed to a file or a ramdisk) as your backing store for CTL, you will need to disable synchronize cache support with ctladm realsync off.  The reason is that g_dev_strategy() does not support the BIO_FLUSH command, and will panic with a KASSERT.  That is something that needs to be fixed.

Files and ramdisks work fine without disabling flush support.

There is a brief description of how to create LUNs and enable ports in the CTL README.ctl.txt file in sys/cam/ctl, and the ctladm(8) man page also describes all of the options and provides some examples.

CTL is pretty functional, and should work well in most cases, but I am certainly interested in any feedback on it.  The README has a to-do list, and I'm also planning on doing some performance optimizations as well.

One of the next things we need is more hardware support for various boards that support target mode.  (e.g. other Fibre Channel, iSCSI and FCoE boards)  It would also be nice to get CTL working with the Firewire target driver.