iSCSI

Unable to Resize iSCSI LUN Using SnapDrive on Windows Server 2003 R2

I recently had another one of my weird Snapdrive issues while trying to resize an iSCSI Lun on a 2003 server. The server is a VM that is using the Microsoft iSCSI initiator and Snapdrive to manage the Netapp provisioned Lun. Re-sizing a lun using Snapdrive is normally very simple but of course on this particular day it was not behaving for me.

Snapdrive appeared to be running ok and didn’t seem to have any issues at all that day. The problem came when I attempted to re size the lun, Snapdrive re-sizing process would fail halfway through. The failure to complete the re-sizing left me puzzled since all connections to the filer appeared to be fine. There was plenty of space left on the volume so it wasn’t a space issue.

Since we were dealing with Windows here we rebooted the server just in case it was pending a reboot or it just needed to “clear it’s  head”. After the reboot I attempted to re-size the lun again and again it failed . The actual failure message was that it was unable to connect to the disk. Odd…It’s connected in Snapdrive , it just won’t resize.

The next thing I thought of was to force a disconnect on the iscsi lun, this way it would forces a disconnect on all connections. The downside to the disconnect was that the Lun would be lost and the SQL databases would need to be stopped. After getting approval to take the server down again, I then proceeded to force a disconnect of this lun. Once all connections were stopped and confirmed they were gone, I then reconnected the iSCSI Lun using Snapdrive.

After the re-connection was completed, I continued with trying to re-size the Lun. BAM! It worked. All it took was a force disconnect , reconnect, then I could re-size. To be honest ,  I wasn’t in the mood to go further digging into a root cause for the failure, especially since I got it working now. I suspect it had something to do with Snapdrive and the iscsi connection it was using since a brand new connection seemed to clear any issues that it had previously. So, if you run into something like this, it might be worth a force disconnect to solve your re-sizing problem.

 

 

 

 

Snapdrive services failing to start on Windows Server 2008 x64

Snapdrive for Windows  is Netapp’s storage management software that allows you to easily provision storage, backup and restore your data on a Windows server. It’s a great tool when it works but when it doesn’t it’s a bear. I just recently had the experience of troubleshooting some of our servers that had some Snapdrive issues connecting to our filer. The server’s iSCSI connection was not affected so the issue went unnoticed for some time until a request to expand luns was made….That’s when it was discovered that the Snapdrive service was not running and failing to start.

When Snapdrive was opened the mmc would crash which then resulted in the following error in the Snapdrive MMC:

Web Service Client Channel was unable to connect to the LUNProvisioningService instance on machine ServerName.
Could not connect to ‘net.tcp://ServerNameSnapDrive/LUNProvisioningService.’ The connection attempt lasted for a time span of 00:00:00. TCP error code 10061: No connection could be made because the target machine actively refused it 

The event that appeared in the application logs:

Description:
Log Name: Application

Source: SnapDrive
Date: 1/05/2013 10:41:33 AM
Event ID: 101
Task Category: Generic event
Level: Error
Keywords: Classic
User: N/A
Computer: myserverxxx.com
Description:
SnapDrive service failed to start.
Error code : SnapDrive Web Service failed to start Reason: ‘The TransportManager failed to listen on the supplied URI using the NetTcpPortSharing service: failed to start the service. Refer to the Event Log for more details.’

I immediately jumped onto Netapp’s support site and starting searching for known issues. One post had indicated to check the permissions of the account accessing the filer and make sure it had local admin rights to the server, I knew that wasn’t issue because the account already had local admin rights. Plus, Snapdrive was working up until recently so permissions would be on the bottom of the list of culprits.The next few hits on the forums indicated that IIS admin needed to be enabled and ensure that the .NetTCPSharing service was enabled. When I checked for the services , IIS admin wasn’t even installed  and the .NetTCPPortSharing was in a disabled state.  I attempted to re-enable the service but it failed as I expected it too. Odd, I thought, Where is the IIS admin service?  What would prevent these services from starting?

Since IIS admin wasn’t available I went to Server Manager and confirmed it wasn’t installed and installed the feature through server manager. After the installation was completed I attempted to start the .NetTCPSharing server and the Snapdrive services again but all of them failed. Back to scratching my head again.

It took some digging but eventually I came to Netapp KB2013168 . The article noted  the following “.NetFramework and the Net.Tcp PortSharing Service. If .Net is not properly installed or the Net.Tcp PortSharing Service service are not functioning correctly, SnapDrive will not be able to connect to the LUNProvisioningServices and the ability to manage LUNs via the MMC can be impaired.”

Oh Snap! Anybody that knows me in “real” life knows how much the word .Net just gets under my skin. I’ve had to deal with so many issues that involved corrupted installs of .Net or some sort of Microsoft patch that would  “break” .Net and the application that depended on it, that I’ve grown a hatred for the word .Net.

Now that I’ve something to go on,  I followed the steps in the KB article for issue #2  and issue #3 ( the symptoms I was experiencing);

Issue 2:
Directory permissions to C:\WINDOWS\Microsoft.NET\Framework\v3.0\Windows Communication Foundation\SMSvcHost.exe.
For the NT Authority\Local Service account to be able to start this service, users must have read and execute permissions to the above path.

Resolution to Issue 2:
Incorrect permissions where configured on the C:\windows directory.
Verify that users have read and execute permissions to the path C:\WINDOWS\Microsoft.NET\Framework\v3.0\Windows Communication Foundation\SMSvcHost.exe.

Well, permissions wasn’t it because everything was there. Now onto issue #3

Issue 3:
SnapDrive 6.x service did not start because the ‘Net.Tcp Port Sharing service’ will not stay started. This is a dependency SnapDrive 6.x has that earlier versions do not.

Resolution to Issue 3:

Reinstall Microsoft .Net.

Reinstall .Net? Great , this should be fun  I thought to myself. I confirmed via Add/Remove Programs that the .Net 3.5 was installed but  the document referenced that Snapdrive required .Net 3.0  sp1 and that particular version was not listed anywhere. On a hunch, I went to server manager > Features > to see if the .Net 3.0 framework features were installed and Yes it was! Using the Server Manager wizard I removed the .Net 3.0 Framework Features, which requires a reboot to complete.

Once the uninstall was completed I re-installed the .Net 3.0 Framework using the same Server Manager wizard.When the installation completed I rebooted the server for good measure, once the server came back online the Snapdrive service was running again. Whew! What a morning now onto expanding the Luns as the applications owner requested.