We had a problem on Friday. It was first noticed on Thursday. I check the Administrator account's e-mail and on Thursday I was denied access due to log-in problems. I thought nothing of it, figuring the Network Admin had changed the password (we change the Admin password monthly) and hadn't notified me yet.
On Friday morning when I still couldn't access e-mail for the Administrators account, I asked Jeff if he had changed the password. He said he hadn't and tried to login as the Administrator to one of our servers and couldn't. It appeared as if the Administrator password had been changed, but none of us had changed it. My first thought is "Uh-Oh, we've been hacked."
We have three servers. One is a file server and anti-virus server. One is the Web-server and the third one is Exchange server and domian controller. The file server and the web server both had been logged off, while the Exchange server still had the Adminsitrator logged in to it. We attempted to login to the file server with no avail. We tried to use our accounts (we are both in the Domain Admins group) and had been denied Log-on Local rights.
So, we attempted to reset the Administrator's password on the file server using the Linux Boot disk trick. This is a trick we have used in times past when we inheritied some Windows 2000 workstations that no one knew Administrator passwords for. We have also used the trick a time or two to assist people with their home XP machines who had forgotten thier admin password. We reset the local Admin password but still were denied log on access because the account didn't have Log-on local rights.
Well, we had never logged off the Exchange server. We had tried to get to Active Directory to reset the password, but the server denied us access to AD. We couldn't change any of the policies to allow log-on local rights. We were in a condundrum. We called some really, really smart people and they said it sounded like a hack to them and pointed us to this great free tool. Its called xScan and is from a company called xFocus and you point the utility to an IP address and scans it for vulnerabilities. It found quite a number of ports open that we didn't realize weren't shut down. It pointed out some user accounts using unsafe password. It was a very thourough diagnostic of areas needing attention. This is a tool I have added to my CD of network utilities. I will use it anytime I am doing a consulting job to perform a security audit.
Anyway, while we found vulnerabilities, we did not find any evidence of being hacked. No backdoors were open. No viruses were found. No spyware was lurking in memory. In other words, we could find nothing wrong... So, we coughed up the $245 and called Microsoft support.
The long and short of it is that a secure channel used by Windows to verify account logins had become corrupted. We had to use some command-line tools, NLTEST and NETDOM to reset the channel and reset the password. This worked and we were back on, logged-in, and had all the appropriate rights. The Microsoft dude couldn't provide an explaination of what happened to cause the problem, but he was able to fix it.
Anyway, Jeff and I spent an extra 2 and 1/2 hours at work on Friday getting the mess straightened out and will now have to spend some time patching holes and repairing vulnerabilities. It does appear like we weren't hacked, but we see some oppertunities to better our defenses anyway.
No comments:
Post a Comment