Now that everything is back to where it should be, they are sharing details of what happened. According to Microsoft, the problem was due to a script gone wrong. Microsoft runs automated test to check the health of Hotmail and, ironically, the problem started was because of this.
To test the health of the Hotmail email service, they create different user accounts with different configurations and uses scripts to simulate a normal user behavior and to report problems. After the test, the accounts are again deleted - this was where the problem arised.
Since the number of test accounts created are large, it is not practical to manually remove them. So, Microsoft uses scripts to delete them. However, there was a bug in the script and it resulted in the deletion of valid user accounts (around 17,000) along with the test accounts. Hence, the reason the people affected got the "Welcome to Hotmail" message.
However, when they found the cause of the outage, Microsoft was able to restore the messages. This was possible because of the way the script deletes the test accounts. When an user account is deleted, the data is not actually deleted but rather its record is removed from the directory server. So, when the problem was identified they were able to restore the messages to the affected users. However, emails sent to the affected accounts would have bounced.
[via: Inside Windows Live]
Related Posts by Tags: email, explain, hotmail, Microsoft, outage