NetApp Create a junction path on a CDOT vserver / SVM

Create a junction path manually on a NetApp Cluster-Mode/C-Mode/CDOT System

If you use the GUI to set up the SVM, it does not prompt you to create a data volume the same way as the CLI.  Therefore, the "/vol" directory is missing and you are unable to mount volumes on a junction path /vol.

This is as simple as creating a directory, unfortunately it's not not easy.

The only way I've found to do this is to re-run the vserver setup wizard on the command line, use the same vserver name (don't worry it won't affect the running vceter) and then accept all the defaults, and create a data volume as /vol/volname.

Once it has created the volume, CTRL-C out of the wizard, delete the temporary volume, and mount your volumes on the "/vol/" junction path!

cluster::> vserver setup
Welcome to the Vserver Setup Wizard, which will lead you through 
the steps to create a virtual storage server that serves data to clients.

You can enter the following commands at any time:
"help" or "?" if you want to have a question clarified,
"back" if you want to change your answers to previous questions, and 
"exit" if you want to quit the Vserver Setup Wizard. Any changes 
you made before typing "exit" will be applied.

You can restart the Vserver Setup Wizard by typing "vserver setup". To accept a default 
or omit a question, do not enter a value.

Vserver Setup wizard creates and configures only data Vservers.
If you want to create a Vserver with Infinite Volume use the vserver create command.


Step 1. Create a Vserver.
You can type "back", "exit", or "help" at any question.

Enter the Vserver name: [CURRENT-VSERVER-NAME]
Choose the Vserver data protocols to be configured {nfs, cifs, fcp, iscsi, ndmp} [nfs,cifs,fcp,iscsi]: [HIT ENTER]
Choose the Vserver client services to be configured {ldap, nis, dns}: [HIT ENTER]
Vserver SVMNAME's allowed protocol list has been modified to nfs,cifs,fcp,iscsi


Step 2: Create a data volume 
You can type "back", "exit", or "help" at any question.

Do you want to create a data volume? {yes, no} [yes]: [HIT ENTER]
Enter the volume name [vol9]: [HIT ENTER]
Enter the name of the aggregate to contain this volume {AGGR1, AGG2} [AGGR2]: [HIT ENTER]
Enter the volume size: [SOME SMALL SIZE LIKE 2G]
Enter the volume junction path [/vol/vol9]: [HIT ENTER]
It can take up to a minute to create a volume...


Volume vol9 of size 2GB created on aggregate AGGR2 successfully.
Do you want to create an additional data volume? {yes, no} [no]:[HIT CTRL-C] 

Exiting the vserver setup wizard. Any changes you have made so far have been saved.

Warning: You have exited the vserver setup wizard before completing all
of the tasks. The vserver is not configured completely. You can complete setup
by typing "vserver setup" in the command line interface.

cluster::> vol mount -volume VOLNAME -junction-path /vol/volname
  (volume mount)

NetApp: LUN Migration - 7-Mode to C-Mode/C-DOT

CDOT/C-Mode LUN/block FC/FCoE/iSCSI storage migration

NetApp made a huge improvement in their software moving to C-Mode, however they did not make the process to move block storage LUNs easy.  Of course, they would like you to buy their DTA2800 appliance, which is licensed per TB migrated, and/or use their professional services.

Like most of us, we more than a couple TB to move, and I don't feel I should have to pay for the migration since I bought both systems in the first place.  The good news is you don't have to....if you can deal with a little downtime.

The solution is to use the NetApp Workflow Automation tool or "WFA".  You can do offline and online block LUN replication, and with the online mode - the cutover period can be very short.

Unfortunately, the pieces needed to easily complete this task are not readily compiled and comprehensively documented.

So far, the best document is NetApp TR 4314 which is not publicly accessible - except from this scribd post.  I will not post the actual PDF - but if you have access to the NetApp field portal - you can get EVERYTHING you need to complete the migrations.

Resources:

• A minimum of a 7-Mode System and a C-DOT system
• This is your source and destination system, if you don't have these, why are you reading this article? ;-)?
• An intermediary / staging system
• Ideally another 7-mode system with a bunch of disk to handle the intermediary work, although technically this could be handled by the second half of your origin HA pair.
• A fast network
• Ideally, 10GbE to minimize the bottleneck
• A working installation of NetApp OnCommand Workflow Automation (WFA) 2.2RC1 
• WFA is great but going over the installation process is beyond the scope of this document
• http://mysupport.netapp.com/NOW/download/software/ocwfa/2.2.1RC1/
• OnCommand Unified Manager 6.1 for C-Mode
• You should already have this, hopefully
• http://mysupport.netapp.com/NOW/download/software/oncommand_cdot/6.1/
• OnCommand Unified Manager 5.2 for 7-Mode
• You should already have this, hopefully
• http://mysupport.netapp.com/NOW/cgi-bin/software?product=OnCommand+Unified+Manager+Core+Package&platform=Windows
• A copy of NetApp TR 4314
• This is available via the NetApp Field Portal
• https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=130681&contentID=249731
• A copy of lunmigrate.dar Workflows
• This is available via the NetApp Field Portal
• Release Notes https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=130979&contentID=250745
• Workflow Archive https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=130977&contentID=250744
• A copy of NFShardlink
• This is available via the NetApp Field Portal
• https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=131057&contentID=251066

Instructions:

1. Install OnCommand Workflow Automater
2. Acquire all the items listed above
3. Follow the instructions in TR 4314 to set up WFA
4. Build a test LUN on your 7-Mode System
5. Wait a minute for OCUM to update, and...

Recommendations/Notes:

• Make sure the administrative interfaces, intercluster interfaces, and replication IPs are all on 10-gigabit networks so the network is not your bottleneck in the migration process.
• If you have a spare NetApp, use this system as a staging system between the source and destination systems.  I happened to have a spare system with a couple hundred 15k drives and multiple 10GbE interfaces which worked great as an intermediary.
• There are a bunch of workflows, but don't be overwhelmed.  You really only need one for the initial setup, and three for most migrations.

Steps to Migrate:

Set up the igroups.  You should only need to run this once per source controller/HA pair if your igroups were set up correctly to begin with.

1. Run the "7toC LUN Migrate igroups" workflow
2. Select the source array
3. Select the target cluster
4. Select the target SVM
5. Click Execute
6. Wait for the workflow to execute, make sure it completes successfully
7. Check the CDOT system

Pre-Migration - "Baseline"

1. Run the "7toC LUN Migrate Job Baseline" workflow
2. Select the source system (your 7-Mode controller)
3. Select the volume, when selecting you can see how many LUNs are on that volume.
4. Select the replication IP - this IP needs to be on the same VLAN or routeable to the staging and target systems.  Hopefully it's a 10GbE interface as well.
5. If there is more than one LUN per volume, you can use a regex to describe the LUN(s) you wish to migrate.  If you want to do one LUN at a time (I recommend this) just put the LUN name in the "LUN Pattern" field.
6. Select the staging system, replication IP, vfiler and NFS access IP.
7. Select the target cluster, SVM, aggregate
8. Enter the volume name...I like to name my block volumes LUNNAME_vol but your mileage may vary
9. Click Execute
10. Wait a while - if you can, keep the workflow execution status window up so you can familiarize yourself with the process
11. Hopefully it completes successfully.  If not look at what step of the workflow failed and troubleshoot it

Migration - "Cutover"

IMPORTANT: Leave all the checkboxes at default unless you are going to read the documentation and see what they all do.

1. Run the "7toC LUN Migrate Job Update + Cutover - Rapid" workflow
2. Leave the offline source LUNs box checked
3. Select the target cluster
4. Select the target SVM
5. Select the volume - in the dropbox, it will show you the status of the volume - it should be in the state"baselined"
6. Leave the remap LUNs box checked
7. Shut down the source system (if this is a boot LUN) or stop all applications using the LUN(s)
8. Unmount the disks in the source OS
9. Click the "Execute" button
10. The workflow will typically take about five minutes to execute on a fast system - unless there is a substantial amount of change since the initial baseline.
11. If the LUN migrated was a boot LUN, make sure to update the boot targets in the BIOS (or boot policy in UCS)
12. If the LUN migrated was a data/application LUN, mount the LUN in the OS and test the application

Post-Migration - "Cleanup"

IMPORTANT: Leave all the checkboxes at default unless you are going to read the documentation and see what they all do.

1. Run the "7toC LUN Migrate Job Cleanup" workflow
2. Select the target cluster
3. Select the target SVM
4. Select the target volume - the state should be "cutover"
5. Click Execute

Manual Post-Migration Steps

The original 7-mode LUN should be offline, and the LUN and volume will be intact.  Once you are satisfied with the migration, you may remove the source data.

1. Delete the source LUN
2. Offline the source LUN volume/container (make sure it's the only LUN in the volume)
3. Delete the source LUN container

NetApp: Unable to resize volume - error: Volume has the fixed filesystem size option set

Sometimes you are unable to resize a NetApp volume.  Usually this is because the volume used to be a snapmirror destination, and it was broken off - but it also could have been some cursed sysadmin that preceded you.  At any rate, when you attempt to resize the volume, you're given this error:

"Volume has the fixed filesystem size option set."

Well, NetApp doesn't have a checkbox for that in the GUI.  But, you should be using the command line anyway :)

Here is the command to fix it:

Cluster Mode:

cluster::> volume modify -filesys-size-fixed false -volume <volumename>

7-Mode:

vol options <volumename> fs_size_fixed off

NetApp Service Processor (SP) crashing when connecting using putty

I recently brought up a four node 8040 cluster.  Unfortunately, the service processors (SPs) crashed putty every time I attempted to SSH into them.  Openssh on Linux works, however.

After an upgrade to firmware 3.0.1P1 this is no longer an issue.

This appears to be a bug with the firmware version on 3.0.1, which (as of this writing) comes from the factory on Clustered Data ONTAP 8.2.1 on all FAS 8000 systems (8020, 8040, 8060)

To see what version you are running, type the following command:

cluster::> system health controller sp config show

               Firmware Booted               SP       Link
Node           Version  Version   Configured Status   Status Status
-------------- -------- --------- ---------- -------- ------ ---------
node-01
               3.0.1P1  primary   true       online   up     ok
node-02
               3.0.1P1  primary   true       online   up     ok
node-03
               3.0.1P1  primary   true       online   up     ok
node-04
               3.0.1    primary   true       online   up     ok
4 entries were displayed.

As you can see, I have already updated 3 of my four nodes, verifying this is hte fix.

You can download the latest version here:

http://mysupport.netapp.com/NOW/cgi-bin/fw

Alternatively, you can upgrade to the latest version of Clustered Data ONTAP which includes the 3.0.1P1 version as of this writing.

NetApp Cluster-Mode Takeover of epsilon node

If you attempt a manual takeover of a node marked as epsilon, you will receive the following warning:

warning: Takeover of the node holding epsilon can result in data serving failure of the entire cluster. Should an additional node go out of service while the node holding epsilon is out of service, a cluster-wide data serving failure will cause client disruption.

This isn't as scary as it seems...it's important to do correctly, but it's also easy.  To change the node marked as epsilon, do the following:

Change to advanced privilege level:

cluster::> set -privilege advanced

Change to advanced privilege level:

cluster::*> cluster show -epsilon * 

Node            Health  Eligibility   Epsilon
--------------- ------- ------------  ------------
node-01         true    true          false
node-02         true    true          false
node-03         true    true          false
node-04         true    true          true

4 entries were displayed.

Next, mark the current node as false
cluster::*> cluster modify -epsilon false -node CURRENTNODE  

Lastly, mark the new node as true
cluster::*> cluster modify -epsilon true -node NEWNODE

Verify that the change worked:

cluster::*> cluster show -epsilon * 

Node            Health  Eligibility   Epsilon
--------------- ------- ------------  ------------
node-01         true    true          false
node-02         true    true          false
node-03         true    true          true
node-04         true    true          false

4 entries were displayed.

After this, you should be able to take over the node previously holding the epsilon.

OnCommand Unified Manager - Active Directory Integration Error

OnCommand Unified Manager - Active Directory Integration Error

One of the first things you want to do when setting up the NetApp OnCommand Unified Manager is set up AD integrated authentication.

Unfortunately, this doesn't work out of the box, with the correct settings.

Upon testing, you receive the following error:

"Unable to communicate with the authentication server because the authentication server is not configured properly. Verify your authentication server configuration."

Yeah, I know how to set up LDAP.  I read the documentation on this, and I'm following all the rules.

So, after some experimentation, I discovered that removing the extra attributes from the "member" field, and just leaving "member" - as well as using the CORRECT LDAP syntax (the documentation is wrong) fixes the issue and allows you to successfully authenticate.

Well, now it works!

As I was saying, the authentication documentation is wrong.  They state that the OU should be referenced by DC=OU, but this is incorrect.  It should be OU=OUNAME,DC=DOMAIN,DC=TLD.

NetApp CDOT/C-Mode: Domain Authentication

NetApp CDOT / C-Mode / Cluster Mode: Domain Authentication

There are several long guides on how to link cluster-level authentication to AD, however if most of you already have the most basic prerequisites satisfied, you can just bang out a few commands.

You need the following items completed already:

• SVM / VServer created
• CIFS licensed
• CIFS set up and joined to domain on SVM / VServer

Once that is complete, you only need to run a few commands:

1. Create a domain security tunnel

cluster::> security login domain-tunnel create -vserver SVMNAME

2. Allow SSH connection (for us leet folk)

cluster::> security login create -vserver CLUSTERNAME -username DOMAIN\USERNAME -application ssh -authmethod domain -role admin

3. Allow HTTP connection (for all the n00bs)

cluster::> security login create -vserver CLUSTERNAME -username DOMAIN\USERNAME -application http -authmethod domain -role admin

4. Allow ONTAP API connection - most people forget this from their guides...it's for NetApp System Manager

cluster::> security login create -vserver CLUSTERNAME -username DOMAIN\USERNAME -application ontapi -authmethod domain -role admin

5. If you want a list of security permissions, here's the command and sample output:

cluster::> security login show

Vserver: SVMNAME

Vserver: CLUSTERNAME
                             Authentication                  Acct
UserName         Application Method         Role Name        Locked
---------------- ----------- -------------- ---------------- ------
DOM\n00b1        http        domain         admin            -
DOM\n00b1        ontapi      domain         admin            -
DOM\n00b2        http        domain         admin            -
DOM\n00b2        ontapi      domain         admin            -
DOM\l33tadmin    http        domain         admin            -
DOM\l33tadmin    ontapi      domain         admin            -
DOM\l33tadmin    ssh         domain         admin            -

And that's all, folks!

Quickly provision new NFS storage on NetApp for VMware ESXi

If you provision a lot of storage, without orchestration, you can speed this up:

On the NetApp ONTAP Cluster-Mode CLI:

For a 20-terabyte volume, with 25TB max growing in 1TB increments, default snapshot policy, and 20% snap reserve:

cluster::> volume create -volume VOLNAME -aggregate AGGRNAME -size 20TB -state online -type RW -policy default -unix-permissions ---rwxr-xr-x -space-guarantee volume -snapshot-policy default -foreground true -percent-snapshot-space 20 -max-autosize 25TB -autosize-increment 1TB

Modify the volume to delete snaps first @ 90% full

cluster::> volume modify -volume VOLNAME -space-mgmt-try-first snap_delete -space-nearly-full-threshold-percent 90  

Turn on dedupe:
sis on -volume esx_template

Mount the volume at the desired junction path:
volume mount -junction-path /vol/VOLNAME -volume VOLNAME


On the NetApp ONTAP 7-Mode CLI:

For a 20-terabyte volume:
vol create <volumename> -s none aggrname 20t

Export the filesystem:
exportfs -io sec=sys,rw=nfssubnet/24,root=nfssubnet/24,nosuid /vol/volumename
exportfs -w /etc/exports

Turn on dedupe
sis on /vol/volumename

Turn on snap auto cleanup @ 10%
snap autodelete volumename on
snap autodelete volumename target_free_space 10

On the VMware PowerCLI:

connect-viserver vcenter.domain.tld
Get-VMHost | New-Datastore -Nfs -Name volumename -Path "/vol/volumename " -NfsHost ontap-nfs-ip