Code katas: summary exercises leading developers through the analysis and motions of solving hypothetical problems.
For this set, I want to provoke the minor and greater disasters that web developers might face; katas which assume hostility and bring an unwelcome challenge. Because disasters (we hope) occur infrequently, it’s all the more valuable to confront them in the form of a kata so that when adversity strikes you can respond with practiced foresight.
The value in code katas is not in the nuance of repetition but in the awareness they build. The range of responsibilities a web developer serves makes these exercises more koan that kumite, requiring strategy and wisdom as the operative weapons.
Like regular code katas the framework and language you imagine during these exercises is irrelevant. For the sake of argument, assume your victim site is running the environment most relevant to your current work but these disasters will threaten most any platform equally.
For general practice I can’t fail to mention that Jeff Atwood has a canonical post full of katas, and in the continuing spirit of the redoubtable list format I want to consider the following scenarios.
But good practice begins with a warm up. . .
Warm Up Kata. Gracefully go offline for upgrade, or after hack
Whether you’re updating a site, or need to take temporarily take a site down in response to a hack or critical bug, there will be dead zone for your site’s visitors.
What are some tactics for seamless deployment? What happens to user sessions during your deployments? How quickly can you replace the sites you manage with a downtime notice? Would your approach differ if your site was running on distributed web servers? And are these processes which can be automated?
Kata 1. Rollback website to last working version
The latest deployment broke your website and now you must immediately restore the prior code.
You probably use source control, but can you match repository numbers with deployments? What would be required to systematically rollback through each of your site’s deployments in turn? Are there any tools that could help you? If so, are there any cases where you would still have to deploy manually anyway?
Kata 2. Restore database from 1 day ago; from 1 week ago
The database has been polluted with corrupt data, or else countless important records have been accidentally removed.
How would restore the database to its timestamped version from 24 hours ago? How about from 1 week ago? What sort of backup strategy do or would you use to prepare for this? Imagine that only part of the database needs restoration, say, only user records and logs – how would you write a script to restore only the required data?
Kata 3. Mailer starts flooding system emails
Something is wrong with the system mailer and it is blindly flooding admins and site users with junk mails.
How quickly can you disable your system mailer. Can you do so gracefully while still leaving the site online? What steps can be taken when designing an internal mail system so that it can easily be killswitched?
Kata 4. Keep website online during traffic spike
While scalability is a huge subject, and our response to its problems often in the form of hardware and hosting solutions, there should be at least a few actions developers can take to respond to traffic spikes and keep a site receiving visitors.
Can you disable analytics logging, error reporting, or any other resources that bring logistical overhead to the site? Can you easily prevent images from being served? What ways can you facilitate these emergency measures in the system architecture design? Are there other ways you can drastically and temporarily reduce the HTML being served?
Kata 5. User passwords/emails are stolen
You found evidence that many of your users have had their login credentials stolen, but you don’t know where the breach occurred.
What are your first steps? Are there any immediate lockdown measures you should employ? Would you bulk reset all the users passwords and If so, how? What attack vector(s) would you suspect could be employed on the systems you’re responsible for in order to steal login credentials?
Kata 6. Locate and cleanse XSS SQL injections in database
Cross-site scripting attacks are appearing in various sections around your website’s dynamic pages; it’s clear that XSS code has been injected into your database somehow. Before you audit for the vulnerability, you must first scrub the polluted data.
How would you go about examining database text fields for the presence of encoded Javascript? How would you remove the injected code if the original text must remain intact? Assuming you only suspect that your data has been tainted, what measures could you take to confirm or deny the fact?
Kata 7. Mail domain blacklisted
Your mail domain has been blacklisted and your system mails are landing in your users’ junk boxes or outright being refused delivery.
Where do check to confirm that your domain has in actuality been blacklisted, and which to which blacklists has your domain been added? What red flags might your emails content and structure be raising? What red flags might your mailer’s behavior be raising? Can you verify that your mailer is not an open relay? Can you determine whether there is a virus propagating through your system’s emails? Once you’ve made the corrections, what steps can you take to repeal the blacklisting?
Having practiced this series, rest assured you can battle the dreadful caprice of fate in some of its more common forms. But what else? Of what other lurking disasters provoke your apprehension? Which disasters posses causes or solutions that remain deep mysteries to you? Anyone else have suggestions to further hone the web developer’s disaster response skills?