Explaining Service Disruptions
Due to a Twitter error, our Followers list have stopped updating again since the 8th of Jan (see Disruption 1). Do ping @al3x and @ev to fix it, so we can service you folks! =)
Updates for (Disruption 1): We have received the followers list from Twitter for folks who joined between Dec 5th to Dec 11th, and will be serving all of you like, now
We listen to comments very closely on timelines, DMs, and GetSatisfaction. The overwhelming reason why users are sometimes unhappy with our service is because of service disruptions. As such, we want to explain the most severe disruptions that had occurred, and the reason they did, and what we are doing about them.
*Some errors listed below are due to Twitter, but 1) the Twitter staff (especially Alex) have been incredibly helpful through the process, and 2) we are a “special” case because we have been working their system in so many different ways!
1) Twitter Error: Followers List was not updated From Dec 5th to Dec 11th
What happened: Followers list essentially did not budge as a (severe) Twitter error prevented the list from being updated.
Impact: Folks who followed us were successful in following us, but they did not show up anywhere in our system. A couple of nice folks also had difficulty unfollowing us. HEHEHE (dream feature) *evil grin*
Working On: We are waiting for Twitter to fix the error on their end ASAP, so we can know who these followers are, and serve them accordingly!
2) Twitter Error: Approximately 900 users who signed up in a period between Nov 27 / Nov 28 did not get their DM
What happened: Twitter accidentally deleted a bunch of names from their developers list. Unfortunately we were one of them. :( Hence our DMs were sent, but not received by our dear users.
“…looks like some database maintenance inadvertently truncated our table of whitelisted users” - Alex (Twitter)
Impact: Lots of users did not get their DMs from us, even those whose reports are actually done up already. We were not aware that such a problem existed until we received lots of people who told us that they have not received their DMs. These comments usually came with a big ” =( ”
Working On: 1) Being responsive to people who complained of weeks-long delay, and sending DMs to them manually. 2) Working on identifying who these people are, and notifying them accordingly!
3) We are sorry, something went wrong….
What happened: There are a couple of cases whereby either we were processing someone like Jason Calacanis or Robert Scoble, each of follows about >20,000 followers, and servers would crash. Either that, or we are deploying some fixes.
Impact: Server required an reboot. Which means I will wake @ambivalence up at 3am*
*Like you know, the famous 3am phone call. Only that it is a Scoble and not a nuclear disaster. Oh wait… there is a difference?
Working on: Identifying and minimizing these errors. We put on hold folks who follows more than 10,000 people, so as not to disrupt service for the rest. In fact, we process them manually. =(
Alright, that’s all folks, hope we shared some insights into what and why these errors occurred, and how we are handling them. =)
Category: Service Updates