Recently I migrated core functionality from my home Windows 2000 Server to a new host running ESXi 4.0 with three Ubuntu Server 9.10 VMs. If you want to see a simple diagram on my set up, view my article on Linux Backup Shell Script. One of the core functionality that I migrated was my internal DNS services. Hence the title of this article, DNS Server with Bind9.

bind9

I am impressed, once again, with Linux and the services residing within this amazing operating system. The most amazing part about Linux services is that many of them have been around as long as I have — where have I been?

Before I begin rambling too much, let’s get started on creating DNS forward and reverse zones for your local network …

First ensure you have bind9 installed by running the following command:

whereis bind

If the results is blank, such as just “bind: “, then you will need to install bind9. On Ubuntu, I would imagine the command would look like this:

sudo apt-get install bind9

Forward Zones

We need to configure your DNS forward zones, which will provide name to address resolution in your internal network. As we progress the configuration, keep in mind that your specific configuration will be slightly different than mine; adapt as needed.

For simple networks, such as mine at home, there are only a few changes that you will need to make for Forward lookup zones. The first file is /etc/bind/db.local.

/etc/bind/db.local

The changes are fairly easy because we going to use most of what is provided in the original file. Change the Start of Authority (SOA) to be the domain environment for your network. My domain is dt.local and my primary DNS server is dtsfile.dt.local. Change the SOA to reflect your choices and also change the nameserver (NS) line to be your primary DNS server.

Also, you will want to add A records for your various servers/computers on the network. For this example, I added my Asterisk server:

dtsvoip.dt.local.	IN	A	192.168.0.11

The next file to change, which we will also make changes for reverse DNS at the same time, is the /etc/bind/named.conf.default-zones file.

/etc/bind/named.conf.default-zones

The line for the primary zone, which references the /etc/bind/db.local file must state your local domain in the quotes following the zone directive:

zone “dt.local” {
  type master;
  file “/etc/bind/db.local”;
};

As we have more changes in this file, leave it open and continue to the next section.

Reverse Lookup Zones

As you probably know, a reverse lookup provides a name to an IP address. In Windows you would find the name of 192.168.0.10 by typing “nslookup 192.168.0.10” from a command prompt. If you have configured reverse DNS properly, you will see output similar to this:

C:\>nslookup 192.168.0.10
Server:  dtsfile.dt.local
Address:  192.168.0.10

Name:    dtsfile.dt.local
Address:  192.168.0.10

You may be wondering why the entry appears twice. This is because the Server and the name that I am looking up is the same server. If I were to locate my Asterisk server, it would look like this:

C:\>nslookup 192.168.0.11
Server:  dtsfile.dt.local
Address:  192.168.0.10

Name:    dtsvoip.dt.local
Address:  192.168.0.11

On with the configuration…

/etc/bind/named.conf.default-zones

If you were paying attention in the previous section you would still have that file open. Regardless, let’s add another zone to the file that represents our reverse lookup for the IP subnet in your network. In my network I use 192.168.0.0/24 which is the same as saying 192.168.0.0 with a subnet of 255.255.255.0 (192.168.0.0 to 192.168.0.255).

Immediately after the zone directive for your domain, add the following text for your reverse lookup:

zone “0.168.192.in-addr.arpa” {
  type master;
  file “/etc/bind/db.0.168.192”;
};

If you’re sharp, you’ll immediately know that the file db.0.168.192 doesn’t exist. We’ll create it next. And yes, it’s backwards; in reverse DNS lookups the IP address is reversed as part of the requirements set in the RFC and obviously for functionality pointing back to the host name of the IP. Read more: http://en.wikipedia.org/wiki/Reverse_DNS_lookup

Save changes to named.conf.default-zones.

/etc/bind/db.0.168.192

Next we’ll create a new zone db file for our newly created reverse lookup. Start by copying db.0 into a new file named db.0.168.192 (or whatever your local subnet IP address is).

cp /etc/bind/db.0 /etc/bind/db.0.168.192

Just like in your db.local file, let’s change the SOA to reflect your domain and nameserver. This includes the NS line that should already exist in the file. Now let’s add pointer (PTR) records for your servers/computers on the network. I’ll use mine for examples:

10	IN	PTR	dtsfile.dt.local
11	IN	PTR	dtsvoip.dt.local

Save changes to db.0.168.192.

Forwarders

The last section, assuming you want to use this DNS server as your primary DNS on all computers, is to set up a forwarder for all names that are not a part of your network. You will need to edit /etc/bind/named.conf.options.

/etc/bind/named.conf.options

The change is really simple, uncomment the forwarders directive and modify the IP address within to be your local router or your ISP DNS servers. Mine is similar to the following:

forwarders {
  192.168.0.1;
};

Local Name Resolution

The final step is to change your /etc/resolv.conf file to point your DNS server and to set the domain and search realm. This is what mine looks like:

domain dt.local
search dt.local
nameserver 192.168.0.1

Restart the bind9 daemon

After making all of these changes, the final is to restart the bind9 daemon. Oh, one other step is to change your computers to use this DNS server as the primary.

Configuration Files Examples

/etc/bind directory listing

/etc/bind# ls -la
drwxr-sr-x   2 root bind  4096 2010-08-01 17:52 .
drwxr-xr-x 141 root root 12288 2010-08-01 17:54 ..
-rw-r--r--   1 root root   237 2009-08-19 15:00 db.0
-rw-r--r--   1 root root   271 2009-08-19 15:00 db.127
-rw-r--r--   1 root bind   295 2010-08-01 17:22 db.0.168.192
-rw-r--r--   1 root root   237 2009-08-19 15:00 db.255
-rw-r--r--   1 root root   353 2009-08-19 15:00 db.empty
-rw-r--r--   1 root root   316 2010-08-01 17:14 db.local
-rw-r--r--   1 root root  2940 2009-08-19 15:00 db.root
-rw-r--r--   1 root bind   463 2009-08-19 15:00 named.conf
-rw-r--r--   1 root bind   573 2010-08-01 16:50 named.conf.default-zones
-rw-r--r--   1 root bind   165 2009-08-19 15:00 named.conf.local
-rw-r--r--   1 root bind   570 2010-07-16 11:58 named.conf.options
-rw-r-----   1 bind bind    77 2010-01-30 11:50 rndc.key
-rw-r--r--   1 root root  1317 2009-08-19 15:00 zones.rfc1918

./db.0.168.192

;
; BIND reverse data file for broadcast zone
;
$TTL    604800
@       IN      SOA     dt.local. dtsfile.dt.local. (
        1         ; Serial
   604800         ; Refresh
    86400         ; Retry
  2419200         ; Expire
   604800 )       ; Negative Cache TTL
;
@       IN      NS      dt.local.
10     IN      PTR     dtsfile.dt.local.
11     IN      PTR     dtsvoip.dt.local.

./db.local

;
; BIND data file for local loopback interface
;
$TTL    604800
@       IN      SOA     dt.local. dtsfile.dt.local. (
        2         ; Serial
   604800         ; Refresh
    86400         ; Retry
  2419200         ; Expire
   604800 )       ; Negative Cache TTL
;
@       IN      NS      dtsfile.dt.local.
@       IN      A       127.0.0.1
@       IN      AAAA    ::1
dtsvoip.dt.local.       IN      A       192.168.0.11

./named.conf.default-zones

// prime the server with knowledge of the root servers
zone "." {
  type hint;
  file "/etc/bind/db.root";
};

// be authoritative for the localhost forward and reverse zones, and for
// broadcast zones as per RFC 1912

zone "dt.local" {
  type master;
  file "/etc/bind/db.local";
};

zone "0.168.192.in-addr.arpa" {
  type master;
  file "/etc/bind/db.0.168.192";
};

zone "127.in-addr.arpa" {
  type master;
  file "/etc/bind/db.127";
};

zone "0.in-addr.arpa" {
  type master;
  file "/etc/bind/db.0";
};

zone "255.in-addr.arpa" {
  type master;
  file "/etc/bind/db.255";
};

./named.conf.options

options {
directory "/var/cache/bind";

// If there is a firewall between you and nameservers you want
// to talk to, you may need to fix the firewall to allow multiple
// ports to talk.  See http://www.kb.cert.org/vuls/id/800113

// If your ISP provided one or more IP addresses for stable
// nameservers, you probably want to use them as forwarders.
// Uncomment the following block, and insert the addresses replacing
// the all-0's placeholder.

forwarders {
  192.168.0.1;
};

auth-nxdomain no;    # conform to RFC1035
  listen-on-v6 { any; };
};

/etc/resolv.conf

domain dt.local
search dt.local
nameserver 192.168.0.1

Onward and upward, today I will share a script I use for backing files on my Ubuntu Server. To start off and to help explain what the script does, let me first explain my configuration. I have an ESXi 4.0 server with 3 VMs, all Ubuntu Server instances for Core services (DNS, DHCP, Directory services), Email (Yahoo’s Zimbra), and File (SAMBA shares, LAMP). All of the servers have 30GB primary partitions, provisioned through VMware and located on an internal 500GB SATA. The File server has a additional partition on the 500GB drive for all SAMBA shares and a partition located on an internal 1TB SATA drive that is used for backup.

The script provides backup functionality for my music, pictures, user’s folders, et cetera to the backup drive. The secondary partition on the 500GB drive is mounted to /datashare/ and the tertiary partition on the 1TB drive is mounted to /databackup/.

The Script

#!/bin/sh
#####
#  backup files script, version 1.
#
#  this script keeps one tar file per month for 12 months and rsyncs the entire contents
#  to $destination/daily_replica.
#  the idea is to run this script at least once per day to ensure proper sync and monthly tar.
#
#  created by Aaron @ www.devtrends.com
#####

# location of backup files (recursive sub-folders).
backup_files="/datastore/share"

# location to place tar files and /daily_replica/ directory.
destination="/databackup/share"

#### no editing beyong this line is required!
#### function for TARing
funcTar()
{
 options="--create --file="
 echo " -- tar'ing up $1 to $2/$3"
 echo "   \ creating new archive file: $3"
 tar $options$2/$3 $1
 echo "   \ tar backup completed."
}
#### end funcion

#### CREATE MONTHLY TAR FILE
# Create new archive filename.
month=$(date +%m)
year=$(date +%Y)
archive_file="backup-$month.tar"
full_path_archive_file="$destination/$archive_file"

# do I need to create a new monthly archive file?
# check if the file exists
if [ -f $full_path_archive_file ]; then

 # get file date
 filedate=$(stat -c %y $full_path_archive_file)
 # extract only the year of the file
 filedate=${filedate:0:4}

 # check if the file year is not the current year
 if [ ! $filedate == $year ]; then

 # remove old file
 rm $full_path_archive_file
 # create new tar
 funcTar $backup_files $destination $archive_file

 else

 echo " -- no tar'ing required today."

 fi

else

 # create new tar
 funcTar $backup_files $destination $archive_file

fi
#### DONE WITH TAR

#### rsync time...
echo " -- rsync $backup_files to $destination/daily_replica"
rsync -a $backup_files $destination/daily_replica
echo "   \ rsync completed."
####

If you hate copy and paste, you may download the script here.

Explanation

For those that care, let me explain the script. The first two variables, $backup_files and $destination, should be the only variables you will need to change if you wish to use the script as I do.

# location of backup files (recursive sub-folders).
backup_files="/datastore/share"

# location to place tar files and /daily_replica/ directory.
destination="/databackup/share"

The next block of code is the function used to create the tar file. The reason I made it into a function is because I use the same block of code twice in the main section of the script. No reason to duplicate code. However, the way I implemented the function may cause a problem if your $backup_files or $destination variables contain spaces. If anyone would like to revised, please share. The $1, $2, et cetera, are the argument variables as passed by the calling statement.

funcTar()
{
 options="--create --file="
 echo " -- tar'ing up $1 to $2/$3"
 echo "   \ creating new archive file: $3"
 tar $options$2/$3 $1
 echo "   \ tar backup completed."
}

Next I define some variables that I will use throughout the script. The first two are date strings that contain the month (e.g. 11) and the year (e.g. 2009). The third and fourth variables hold the location of the tar file. The tar file name is comprised of the word “backup-” and then the variable of the month.

month=$(date +%m)
year=$(date +%Y)
archive_file="backup-$month.tar"
full_path_archive_file="$destination/$archive_file"

The main section of code is next and consists of a nested if statement. The first if statement checks if a current backup tar file exists, if not, then it will create one, otherwise it will check the status of the current backup tar file. If the current tar file modified date is not in the current year, then it removes that tar file and recreates it, otherwise there is no need to tar anything. The most interesting piece is the substring command, ${filedate:0:4} which only returns characters 0,1,2,3 from the variable $filedate. You will also notice the use of stat, which, depending on your distribution of Linux, you may need to manually acquire.

# do I need to create a new monthly archive file?
# check if the file exists
if [ -f $full_path_archive_file ]; then
 # get file date
 filedate=$(stat -c %y $full_path_archive_file)
 # extract only the year of the file
 filedate=${filedate:0:4}

 # check if the file year is not the current year
 if [ ! $filedate == $year ]; then
  # remove old file
  rm $full_path_archive_file
  # create new tar
  funcTar $backup_files $destination $archive_file
 else
  echo " -- no tar'ing required today."
 fi

else
 # create new tar
 funcTar $backup_files $destination $archive_file
fi

The last statement is the rsync command which synchronizes the entire content of the $backup_files location to the $destination/daily_replica/ location.

echo " -- rsync $backup_files to $destination/daily_replica"
rsync -a $backup_files $destination/daily_replica
echo "   \ rsync completed."

Use this script at your own risk. If it turns out it didn’t back up your stuff, that is entirely your problem, not mine.

-Aaron Gilbert

VGA Mode

For those DOS’ers, do you remember “mode 80″ or “mode 40″? If you know of a quick command for Linux consoles let me know … otherwise, you can use the tip below to modify your screen resolution.

/boot/grub/menu.lst

The key to changing the screen mode / screen resolution is to modify the menu.lst file and add the vga=[mode #] some where in the kernel call line – or at the end of it. After searching the Internet for some time, I compiled a list here of many of the vga modes available:

VGA MODE          MODE#
80x25              3840
80x50              3841
80x43              3842
80x28              3843
80x30              3845
80x34              3846
80x60              3847
320x200x8bit        816
320x200x16bit       782
320x200x32bit       783
320x240x8bit        820
320x240x16bit       821
320x240x32bit       822
640x480x8bit        769
640x480x16bit       785
640x480x32bit       786
800x600x8bit        771
800x600x16bit       788
800x600x32bit       789
1024x768x8bit       773
1024x768x16bit      791
1024x768x32bit      792
1280x800x8bit       864
1280x800x32bit      865
1440x900x8bit       868
1440x900x32bit      869

On my Ubuntu Server, the kernel line was:

kernel         /vmlinuz-2.6.28-11-server root=/dev/mapper/ubuntuserver-root ro quiet splash

and I added vga=792 to the end of it:

kernel         /vmlinuz-2.6.28-11-server root=/dev/mapper/ubuntuserver-root ro quiet splash vga=792

I’m out…

SUN iSCSI Initiator

Obviously you need to have your network configured properly prior to working with the iSCSI initiator or targets. In this example, I was working with the iSCSI initiator and target in a lab environment, with the host (target) being 192.168.0.18 and the client (initiator) being 192.168.0.19. Secondly, you will need the iSCSI initiator software to be installed from Package Manager – easiest way is to search for iscsi and ensure all four of those packages are installed.

For this example we will use Send Targets for discovering iSCSI targets/paths:

iscsiadm add discovery-address 192.168.0.18:3260
iscsiadm modify discovery --sendtargets enable

Now that you have added the discovery address of the iSCSI target and enabled the discovery “Send Targets”, check to see if your target was discovered:

iscsiadm list target

If you saw the targets you were expecting, the next step is to create the device links between iSCSI and your system – so you can see the iSCSI LUN as a disk on your system.

devfsadm -i iscsi

Check the available disks on your system using the format command. After running the format command, hit CTRL-C to bounce back to the terminal prompt.

Using / Mounting the iSCSI LUN

Now that you have the iSCSI initiator working properly and the iSCSI targets discovered, lets actually use the new iSCSI disk with ZFS.

First we need to create the ZFS pool using the new disk. If you forgot the name of the new disk, you can learn it using the format command. If it is anything like the one on my OpenSolaris box, the name is rather large, (c0t600144F0F9A00C0000004AA9510A0001d0):

zpool create diskpool c0t600144F0F9A00C0000004AA9510A0001d0

To verify the pool was created, use:

zpool list

Now we can create a ZFS file system on top of that pool:

zfs create diskpool/test

Check your new ZFS filesystem was created:

zfs list

And now you can create files within the mount point:

root@opensolaris:/diskpool/test/#

Everything you write to that mount point will be written to the iSCSI target, as one would hope.

Good luck with your performance!

Credits

http://wikis.sun.com/display/OpenSolarisInfo/How+to+Configure+an+iSCSI+Initiator

I owe some of my new-found knowledge to my friend Chuck Hechler and the remaining to Mike La Spina (http://blog.laspina.ca/). I don’t pretend to be anywhere as knowledgeable about OpenSolaris, ZFS, iSCSI, SANs, or storage in general as Mike or Chuck, however, I can create SharePoint workflow actions in VisualStudio, can they? =)

The Equipment

If you read my other posts on the IET and RedHat implementation, you would know that I created a custom server housed in a SuperMicro case. During the testing phase of that project, I learned that the IET product does not work well with the LSI MegaRAID 84016 controller, specifically when using RAID5. Unsure why, however, in OpenSolaris, I had an issue with our Adaptec 31605 controller and ended up using the LSI MegaRAID 84016E controller. Works like a charm. I mention this as you will need to verify that the equipment you are going to use with OpenSolaris should be supported by OpenSolaris (http://www.sun.com/bigadmin/hcl/).

If you want more information on the hardware specifications for the system I used, please review my Custom Server Case article.

OpenSolaris Configuration

Using OpenSolaris to serve as an iSCSI target is surprisingly easy to configure. I say surprisingly because the same implementation in RedHat was thoroughly difficult, both due to a learning curve and due to many complications with functionality. On to OpenSolaris!

As I designed this system, I was focused on using ZFS as the sole solution for disk management and the file system. In other words, for my boot disks, I created a ZFS mirror and for the iSCSI LUN disks I created a ZFS raidz2.

Installation

For the most part, the default installation settings were used. Too be honest, the options during installation of OpenSolaris are limited. If you do not want the graphical interface, see my short article, Disable GUI, for disabling it after installation.

ZFS

Boot pool – mirror

Mirroring the boot partition is about as difficult as it gets with this article. I did not figure out how to mirror the boot disk on my own. Using the steps in the following blog post, I managed to mirror my ZFS boot partition, rpool.

http://darkstar-solaris.blogspot.com/2008/09/zfs-root-mirror.html
If you do not know the devices names of your drives, you can use “# format” to list the drives/devices on your system.

After the mirroring has been completed, display the status of the pool using the command and output below:

# zpool status
  pool: rpool
 state: ONLINE
 scrub: none requested
config:

 NAME        STATE     READ WRITE CKSUM
 rpool       ONLINE       0     0     0
   mirror    ONLINE       0     0     0
     c8d0s0  ONLINE       0     0     0
     c9d0s0  ONLINE       0     0     0

errors: No known data errors

iSCSI pool – raidz2

After reading quite a few posts on best practices for ZFS pools, I propose that you use a raidz set per controller. In my instance, I had a 16 port controller and all 16 drives for the iSCSI pool were connected to this controller. Using the following commands I created a raidz2 pool with 2 spares:

# zpool create diskpool raidz2 c7t0d0 c7t1d0 c7t2d0 c7t3d0 c7t4d0 c7t5d0 c7t6d0 c7t7d0 c7t8d0 c7t9d0 c7t10d0 c7t11d0 c7t12d0 c7t13d0
# zpool add diskpool spare c7t14d0 c7t15d0

That is it; we just created a large disk pool of 16 disks, 14 in a dual-parity set for volumes and 2 for spares. Easy? Yes.

Volumes

Best practice for creating ZFS volumes depends on your perspective. In my case, the iSCSI target will be used with VMware and I will create a new ZFS volume and iSCSI view for each VMware machine (not host). Follow the command below to create the base mountpoint in the ZFS pool:

# zfs create diskpool/iscsi

Next we will create a volume for sharing as an iSCSI target. As I stated above, I create one volume per VM. Also, notice the “-b 64K”, which helps with partition alignment in VMware. The command “-V 40G” creates a volume with a maximum size of 40 gigabyes – keep in mind that ZFS will thin provision the file system.

# zfs create -s -b 64K -V 40G diskpool/iscsi/lun0_vm

Display the pool and the ZFS volumes as shown below:

# zpool status
  pool: diskpool
 state: ONLINE
 scrub: resilver completed after 0h32m with 0 errors on Wed Sep  2 10:52:54 2009
config:

 NAME           STATE     READ WRITE CKSUM
 diskpool       ONLINE       0     0     0
   raidz2       ONLINE       0     0     0
     c7t0d0     ONLINE       0     0     0
     c7t1d0     ONLINE       0     0     0
     c7t2d0     ONLINE       0     0     0
     c7t3d0     ONLINE       0     0     0
     c7t4d0     ONLINE       0     0     0
     c7t5d0     ONLINE       0     0     0�
     c7t6d0     ONLINE       0     0     0
     c7t7d0     ONLINE       0     0     0
     c7t8d0     ONLINE       0     0     0
     c7t9d0     ONLINE       0     0     0�
     c7t10d0    ONLINE       0     0     0
     c7t11d0    ONLINE       0     0     0�
     c7t12d0    ONLINE       0     0     0
     c7t13d0    ONLINE       0     0     0�
 spares
   c7t14d0      AVAIL
   c7t15d0      AVAIL

errors: No known data errors

  pool: rpool
 state: ONLINE
 scrub: none requested
config:

 NAME        STATE     READ WRITE CKSUM
 rpool       ONLINE       0     0     0
   mirror    ONLINE       0     0     0
     c8d0s0  ONLINE       0     0     0
     c9d0s0  ONLINE       0     0     0

errors: No known data errors

# zfs list
NAME                             USED  AVAIL  REFER  MOUNTPOINT
diskpool                         9.9G  6.76T  51.1K  /diskpool
diskpool/iscsi                   9.9G  6.76T  48.5K  /diskpool/iscsi
diskpool/iscsi/lun0_vm     9.9G  6.76T   9.9G  -
rpool                           10.8G  62.1G  77.5K  /rpool
rpool/ROOT                      3.18G  62.1G    19K  legacy
rpool/ROOT/opensolaris          3.18G  62.1G  3.04G  /
rpool/dump                      3.75G  62.1G  3.75G  -
rpool/export                     114M  62.1G    21K  /export
rpool/export/home                114M  62.1G    21K  /export/home
rpool/export/home/vcssan         114M  62.1G   114M  /export/home/vcssan
rpool/swap                      3.75G  65.7G   101M  -

Network

There are a few approaches to spreading the load across multiple links, my favorite is LACP, which is what I used – see my article on LACPBONDING. The other option that may arise in the planning phase is IPMP. However, I was unable to overcome the issue of having more than one inbound interface. If you are knowledgeable with source addressing, you will have better luck than I did.

I aggregated 4 gigabit links together using LACP and a Dell PowerConnect 5448 switch.

iSCSI

During the design phase, I tested both of SUN’s iSCSI targets, the standard iscsitadm and COMSTAR iSCSI. At this point, you can choose one based on word of mouth, or you can try both and use the one that works the best for you. I ended up using COMSTAR.

iscsitadm

The first configuration change we will make defines which interfaces the iSCSI target software will use. We accomplish this by creating a target group:

# iscsitadm create tpgt 1
# iscsitadm modify tpgt –i 192.168.0.101 1

Because I am using link aggregation there is only one IP address to add to the group. If you decide to use multi-pathing as provided by VMware, you will need to add all of those IP addresses to the group.

Are you ready for the next statement, it is very complex…well, maybe not.

# zfs set shareiscsi=on diskpool/iscsi/lun0_vm

That’s it; the ZFS volume is now served through the iSCSI target.

COMSTAR

On a default installation of OpenSolaris, you will need to install the COMSTAR iSCSI software package. Open the Package Manager and search for iSCSI, choose to install all of them, specifically the SUNWiscsit package.

After you have installed the software package, you will need to enable the COMSTAR iSCSI service:

# svcadm enable -r svc:/network/iscsi/target:default

I am having a vague recollection that you may need to restart OpenSolaris before continuing…

The next configuration we will make defines which interfaces the iSCSI target software will use. We accomplish this by creating a target group and target:

# itadm create-tpg iscsi0 192.168.0.101
# itadm create-target –t iscsi0

Now, we will share the ZFS volume through the COMSTAR iSCSI target – this is a two step process:

# sbdadm create-lu /dev/zvol/rdsk/diskpool/iscsi/lun0_vm

Now let’s check for the new logical unit and remember the GUID… as we will use it in the next command:

# sbdadm list-lu

Found 1 LU(s)

       GUID                    DATA SIZE           SOURCE
--------------------------------  -------------------  ----------------
600144f08e6c0f0000004a8331d70002      42949607424      /dev/zvol/rdsk/diskpool/iscsi/lun0_vm

Now we will add the view to share the ZFS volume through iSCSI:

# stmfadm add-view 600144f08e6c0f0000004a8331d70002

That’s it!

I’m out….

Creating the Link Aggregate

If your configuration was similar to mine, you will already have your interfaces plumbed – which means they cannot be added to the link aggregate until they are unplumb’ed.

Checking for Plumbing

If you try to create an aggregate with plumbed interfaces, you will receive a “dladm: create operation failed: link busy” error. You can check if they have been plumbed as they will show in the list for “ifconfig -a”:

e1000g0: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 8
 inet 0.0.0.0 netmask 0
 ether 0:15:17:b8:47:a8
e1000g1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 9
 inet 0.0.0.0 netmask 0
 ether 0:15:17:b8:47:a9
e1000g2: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 10
 inet 0.0.0.0 netmask 0
 ether 0:15:17:b8:47:aa
e1000g3: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 11
 inet 0.0.0.0 netmask 0
 ether 0:15:17:b8:47:ab

If they appear, as shown above, unplumb them using the following commands:

ifconfig e1000g0 unplumb
ifconfig e1000g1 unplumb
ifconfig e1000g2 unplumb
ifconfig e1000g3 unplumb

Create the Aggregate and Plumb It!

Using the dladm tool, you can easily create the aggregate usign the following command:

dladm create-aggr -d e1000g0 -d e1000g1 -d e1000g2 -d e1000g3 1

You’lll notice a -d with each interface and a number 1 at the end. The number 1 references the aggregate number and it must start at 1, not 0.

Next we will plumb the newly created aggregate with an IP address:

ifconfig aggr1 plumb 192.168.0.101 up

Show me the Aggregate

If you are like me, then you will want to see the aggregate configured:

dladm show-aggr

You should see an output similar to the following:

LINK            POLICY   ADDRPOLICY           LACPACTIVITY  LACPTIMER   FLAGS
aggr1           L4       auto                 off           short       -----

Modify Aggregate for LACP

To ensure that LACP is functioning as I would hope, I modified the aggregate, changing LACPACTIVITY to active instead of off. Use the following command to modify LACPACTIVITY:

dladm modify-aggr -L active -T short 1

Now run the following command to verify the settings took:

dladm show-aggr

You should now see an output similar to the following:

LINK            POLICY   ADDRPOLICY           LACPACTIVITY  LACPTIMER   FLAGS
aggr1           L4       auto                 active        short       -----

Setting Aggregate IP Configuration as Persistent

To ensure that your IP address configuration for the aggregate is persistent across reboots, modify the /etc/hostname.aggr1 file and add the desired IP address. The /etc/hostname.aggr1 should contain one line consisting of the IP address:

192.168.0.101

Switch Configuration

Obviously, you must configure the switch for LACP otherwise your aggregate will fail. Each switch is different, refer to the manual for your switch.

I’m out…

Disable GUI

To disable the GUI, type the following command in a terminal window:

pfexec svcadm disable gdm

Enable GUI

To enable the GUI, type the following command at the terminal prompt:

pfexec svcadm enable gdm

I’m out…

This article discusses issues I have come across while building or testing opensource iSCSI target software with various iSCSI initiators. I began testing the IET product on Ubuntu Server 8.04 with VMware ESXi 3.5, which worked well in a segregated lab. Then I moved forward to testing a “production” IET target with Microsoft’s iSCSI Software Initiator, which also worked exceptionally well. Finally, I moved into the VMware ESX 3.5 realm, which did not go well at all, initially.

Test 1: VMware ESXi 3.5 and Ubuntu Server 8.04, iSCSI Enterprise Target (IET)

The test was a combination of feasibility and reliability, with an emphasis on the feasibility aspect. For the Ubuntu Server that served as an iSCSI target, I used an older 1U Dell PowerEdge 850 server. The network fabric was a HP ProCurve 48 port switch, which was used for both the standard network environment in the lab and the iSCSI network.

The test went well with a new VM build to the iSCSI target of Windows Server 2003. Heck yes. The feasibility was there as it took an afternoon to make it happen. The reliability was also there, especially since everyone touts Linux as the most solid operating system in the world.

Test 2: Microsoft iSCSI Initiator and IET

http://iscsitarget.sourceforge.net/
http://www.microsoft.com/windowsserver2003/technologies/storage/iscsi/default.mspx

Although I am a Microsoft guy by trade, when it comes to new technology I am generally weary of Microsoft’s implementation as a solid product. However, during my testing of the iSCSI Enterprise Target (IET) opensource target software (0.4.17), I have had absolutely no issues or concerns. After testing in a production lab environment, we moved to testing with our backup to disk project. In this instance, the iSCSI target is used for backups to disk and is hammered almost 24/7 by EMC Retrospect through the Microsoft iSCSI Software Initiator. The product has been running strong for 3 months as of 8/1/2009.

Test 3: VMware ESX 3.5 to 4.0 and IET

If you notice the title of this third test, you may gather that the initial testing did not go as well as was anticipated. In fact, I had a heck of a time getting the ESX 3.5 software iSCSI to even see the IET target from the start. At first I though the configuration was invalid, however, the Microsoft iSCSI Software Initiator worked exceptionally well with the same target with no reconfigurations.

To FINALLY get to a semi-solid state, I began with simplification…

Simplify

As this was the third test, following two successful tests, I decided that I would enter the test with more features enabled, such as multipathing or bonding. This ended up over-complicating the project as there are some known issues with multipathing IET with VMware. To help resolve the issues with connectivity and failures, I went back to one gigabit link, no bonding or multipathing.

Upgrade ESX host from 3.5 to 4.0

Following a technical support call with VMware, I decided to upgrade ESX to vSphere 4. The technical support person wanted me to update ESX 3.5 to the latest build, so I figured, let’s jump to the next level (should I have revisited “simplify”??).

Immediately following the ESX 4.0 installation, I was able to see the iSCSI LUNS. However, I was not able to use them as I kept receiving an error message about not being able to get disk information. If I remember correctly, this was resolved by using “blockio” type on the IET LUNs instead of “fileio”.

Modify IETd.conf to match ESX iSCSI settings

This change was to ensure that the VMware iSCSI software initiator shared the same “mindset” as the IET target. If I remember correctly, only ESX 4 hosts can modify the iSCSI initiator settings.

Conclusion

After running into a list of complexities getting IET to work well with VMware, I finally got it to play nicely for almost 2 weeks. And then…I received the an “unknown command” error in /var/log/messages and the IET target software jammed up causing the VMware virtual machines to lock up.

From my perspective, enough time has been given to IET and I do not feel as though it is ready for VMware – or that VMware is ready for it. I am, therefore, moving forward with OpenSolaris for two reasons, built in iSCSI target and ZFS. More to come…

The project behind this article was to create a cheap iSCSI SAN for a backup to disk solution. The requirements of the project were foremost price, second were space, and third were performance. Amazingly, or should I say surprisingly, all three requirements were met, and in a short amount of time! In another article I will explain the custom server and the issues I had building a custom server system that could support large (10TB+) disk arrays. On to iSCSI…

Installing RHEL

For those that have installed RHEL, it is a fairly simple process, easy enough for any end user really, with the exception of partitioning if RHEL can’t figure out the correct disk order. Do what you will, however, for this project I installed RHEL with no firewall and disabled SELinux. If you are building a test machine, make sure you have enough space available for an additional partition to be served by IET.

Note: if you have no Internet connection while installing RHEL or you do not want to enter your installation key (subscription code), you may activate the RedHat Network (RHN) connection after you have completed your installation. To do so, from a terminal window, run rhn_register.

Configuring and Preparing RHEL for IET

In order to compile IET, you must configure RHEL with additional applications, another download repository, and kernel development files. First, I would suggest running a “yum update” from a terminal window to update your RHEL. As a side note, all of these commands assume you are root. Be sure to configure network connectivity, especially to the Internet.

Extra Packages for Enterprise Linux (EPEL)

Next, let’s configure RHEL to use an additional download repository, EPEL. From the web browser in RHEL, browse to the following link:
http://fedoraproject.org/wiki/EPEL

Click on 1.1 Getting Started with EPEL, then click on How to use EPEL. Then under the Using EPEL heading, click on the RHEL 5 link which allow you to download an RPM to configure the EPEL on your RHEL.

For those that are unaware, EPEL is a repository that contains considerably more applications ready for installation through “yum” or the GUI package manager. The EPEL project is maintained by Fedora, which is essentially RedHat’s open source Linux.

Installing Development Files

In order to compile IET on your RHEL, you will need the following development files installed. This task is easy as “yum” will do all the work for you. From a terminal window, run the following commands:

“yum install openssl-devel”
“yum install kernel-devel”
“yum install kernel-PAE-devel” (32bit RHEL only)
“yum install gcc”

Although not required, since I am a Windows administrator by trade, I have been rebooting my RHEL server after I completed these updates.

Compile IET

Using your web browser, visit http://iscsitarget.sourceforge.net/ and download the latest version of IET, which at the time of this article is 0.4.17. Once downloaded, extract the files to your Desktop and open a terminal window to that location. Such as /~/Desktop/iscsitarget/.

From the iscsitarget folder in the terminal window, run “make”, verify for success and then run “make install”. Assuming all went well, you should now have the IET installed on your RHEL.

Configure IET

Let’s configure the IET application to serve a partition. Although in this example I will use a SCSI disk device, you can use the LVM to create your own unique disks and serve those through IET.

The configuration is fairly simple, replaced with your favorite editor tool from the terminal window, type “nano /etc/ietd.conf”. Until you feel like diving deeper into the IET world, you will only modify two lines in this file, the Target and Lun lines.

Modify the file to look similar to the following:

Target iqn.2009-07.test:iscsi.disk
     Lun 0 Path=/dev/sdb,Type=fileio

You may modify the iSCSI target name, just make sure it follows the iSCSI standard. Additionally, you will need to replace the path statement with the path to your disk. To add more than one Target to the same server, just duplicate those lines, such as:

Target iqn.2009-07.test:iscsi.diska
     Lun 0 Path=/dev/sdb,Type=fileio

Target iqn.2009-07.test:iscsi.diskb
     Lun 0 Path=/dev/sdc,Type=fileio

Starting IET

As you probably know, in RHEL “services” reside within the /etc/init.d/ directory. Using the following command from a terminal window we will start the iscsitarget service:

/etc/init.d/iscsi-target start

To stop the service, run:

/etc/init.d/iscsi-target stop

Testing IET

For simplistic testing of the iSCSI target connectivity, I would suggest using the Microsoft iSCSI software initiator found at: http://www.microsoft.com/iscsi/. Within the Microsoft iSCSI Initiator, add the IP address of the RHEL server to the discovery list and then refresh the available targets. Once you connect to a target/LUN, you will need to open Disk Management to initialize and/or format the disk.

Depending on the hardware set up, you should see fairly quick speeds. For a simple speed test, copy a large file, such as a 3GB file to the iSCSI disk from the Windows machine. In my tests, using an Adaptec 31605 with 1.5TB Seagate drives, a 3GB file would take around 50 seconds to complete – roughly 1GB / minute.

Auto Start IET

If you want the IET to automatically start at system boot up, use the following commands from a terminal window:

“chkconfig –add iscsi-target”
“chkconfig iscsi-target on”

That is it for now…if you have questions, let me know.

As time progresses, I will update an article with troubleshooting tips I have learned while configuring around 5 of these systems. Additionally, if you are really interested in IET, I would suggest joining the Nabble forum group at: http://www.nabble.com/iSCSI-Enterprise-Target-f4401.html

The specific configuration was that all print jobs are sent through a Windows print server, serving Windows natively and Mac through Unix services Line Printer Daemon (LPD).

For Windows, I was able to control the color output and banner pages through features in the driver configuration on the server, which propagated to all Windows machines. Obviously, this propagation does not happen for LPD on a Mac. Even if the specific queue in Windows is set to BW only and no banner, a color job from a Mac still produces color and the banner is still produced.

The resolution? Modify the Mac PPD for each print queue.

PPD?

A PPD is a PostScript Printer Definition, a text based configuration file that defines the scope of the printer, such as the capabilities, defaults and restrictions.

Since PPD files are based on PostScript, how a PPD is defined is determined by Adobe. Please visit the following link for more information: http://partners.adobe.com/public/developer/en/ps/5003.PPD_Spec_v4.3.pdf

Xerox PPD and Color Control

The Xerox printer driver for Mac has a PPD that defines that printer. Inside the PPD I modified two lines to disable color output:

*ColorDevice: False
*DefaultColorSpace: Gray

Although I changed both, it was only at the time that I changed the DefaultColorSpace that the color output stopped. Even though the Mac user still has the option to specify color at print time, the output will remain BW.

Xerox PPD and Banner Page

In the same PPD, I modified one tag that permanently removed banner pages:

*DefaultJCLBanner: False

PPD Location

In the Mac environment, custom English PPD files are located in:
\Hard Drive\Library\Printers\PPDs\Contents\Resources\en.lproj

To copy files in to this folder you will need to authenticate using a username with Admin permissions.

All tag information, such as * DefaultJCLBanner, is in the document provided by Adobe, as was provided above.