Friday, June 05, 2009

Windows 2008 Desktop hangs (RDC Will not connect)

So my SQL Cluster had an odd issue. Every so often (about every 2 weeks) I would attempt to log into a node of the cluster just to check on things. Well, no dice. I would get the login prompt, supply my credentials and then the Windows 2008 sever would start to do its thing.. all the fancy 'vista' like things it does... and finally I would get 'warned' that it was preparing the desktop.

The desktop would never show... after 5 minutes the session would 'lock' and ask me to provide my credentials. Still no desktop.

We tried the console, no dice. We tried the DRAC, no dice. Force a reboot, try again. Look at the logs nada. nothing.

It happened on both nodes.. no clue why. So, I called up Microsoft and they did some stuff (looked at my logs and ran a couple of other diag programs. They found nothing. So they had their SQL guy and their Cluster guys comb over my config. The servers were setup exactly right, no issues. So they had me setup some perfmon traces, I captured one, but it was way too big (20GB).

I was instructed to let it crash again and get a memory dump... however the server was rebooted every time it happened.

Well my Microsoft tech guy called but I missed the call. Good thing! Why? Well we scheduled a call for the next afternoon, and when I was preping for the call I tried to login to the server and BAM, we were in the problem state. The server wouldn't launch a desktop. However the file system still responded, as did many other things (such as perfmon) and captured logs.

The Tech looked at the logs and found out it was an Exhausted Desktop Heap. Hmmm, what could that mean? I researched that on my own and found that there are some tools that could show me that info.. only not for Windows 2008, only 2003 or earlier. Crap. We got the Kernel dump as the last piece of data. I shipped the dump to Microsoft and waited.

I got the news back today. My NIC driver for my Intel NICs was causing the Kernel to run out of resources. Who knew.

So, lesson time. Even tho Windows 2008 is much better at session management it still has the ability to have all the resources eaten up and not allow any sessions to start.

So we walked through a bunch of stuff and got

Labels: , ,