You can find a list of the European talks on the developer portal. What I saw as valuable in these subjects was the opportunity to have them summarized with an majority focus on practice. Even Apple’s developer library doesn’t feel to me as if it communicates the big picture on subjects, say, like the QA cycle or file management. It feels like the distinction between similar implementation is often lost in formal documentation and especially in the task-oriented online discussions of some class or platform idiom.

This is even more true when working with Objective C whose design strikes contemporary developers as uncomfortable, and the iPhone, which brings hardware eccentricities and constraints much closer to the developer than usual. Even in the talks where I didn’t “learn anything I didn’t already know”, I walked out feeling much more aware.

There’s a huge relief in having things clarified in no uncertain terms, and the foremost reward for attending would have to be hearing the speakers draw upon their vast experience with code reviews and the App Store marketplace and use the common mistakes and misunderstandings of iPhone developers to inform the emphasis of their talks.

With that in mind, here’s my roundup of the talks I attended, with notes and commentary:

TALK 1: iPhone Tech Talk Kickoff

This was the opening state of iPhone development address, if you will. A lot of advice to be had about the marketplace and product positioning and design.

• Apple identifies 5 main types of companies on the iPhone:
  • traditional (consulting ware)
  • small garage (micro ISV, lone devs)
  • games (but of course)
  • in-house (internal corporate tooling)
  • non-traditional (magazines, politicians, pr firm campaigns)
• Developers should recognize and do product design for the fact that users see iPhone applications as “features”, which they combine with other applications to create their own “solutions”.
• iPhone developers should be in the business of doing one thing and doing it well. This adage was repeated many, many times throughout the day.
• Simplicity is important in light of change. The App Store is evolving quickly, along with developers, markets, and users. Technically, simple things support change better than complex ones. Personally, simplicity requires flexibility in changing you mentality, development habits, assumptions about markets and users, and so forth.
• Great applications almost follow an ingredients list. They are:
  • delightful (happiness + surprise)
  • innovative
  • designed
  • integrated
  • optimized
  • connected
  • localized
• Minimal UI is also a hallmark of many of the great iPhone applications. Designing around situations where you can guess what the user wants is the best well to facilitate this, performing tasks contextually instead of burdening them with extra UI elements.
• Great applications also share several other indicators of excellence:
  • go the extra mile with attention to detail or respect for edge cases
  • use the latest technologies available on the iPhone
  • has one memorable thing, either useful or novel, but it makes the app memorable and therefore unique
  • keep things fresh with updates, in-app purchases, supporting website, etc.

TALK 2: Effective iPhone Development Part 1

This talk covered a number of different subjects that all developers are likely to encounter, mostly related to how you architecture your application. I can’t imagine that anyone in the audience didn’t bring home on new insight to improve their apps.

• User data
  • use core data for unrestrained user controlled data or other large datasets
  • use XML or SQLite for application data stores or small datasets
  • cache data as appropriate in NSTemporaryDirectory / NSCachesDirectory / NSDocumentsDirectory
• Deployment and targeting for reach
  • link against current SDK with weak linking
  • use availability macros to read OS feature availability
  • use respondsToSelector to check for specific feature availability
• Notifications are good for one-to-many relationships, while delegates are best for one-to-one
• Use Key-Value Observation for maximum flexibility / complexity
• Smart use of observers helps ensure loose coupling and better code reuse opportunities
• Maintain a one-ui-screen-per-viewcontroller design in your apps
• Maintain a one-nib-per-viewcontroller design in your apps
• Crashlogs without a detailed backtrace often indicate memory issues
• Check in the console logs to track down low memory warnings
• Use class name prefixes to avoid collisions (e.g. NSBlah)
• Avoid underscores in names (they are potentially reserved)

TALK 3: Effective iPhone Development Part 2

A continuation of the previous talk’s litany of advice and insights. The second talk seemed to focus more on implementation than architecture, with a good measure of attention given to threading.

• Stay focused, i.e. simple is good
• Drill down into data
• Optimize views by flattening the drawing, subclassing multiple UI elements into a single UIView with custom a drawing routine
• Optimize by setting all views to opaque whenever possible
• Never call drawRect directly
• Don’t attempt to poll UITableView during highlighting and selection – there is no state between the animation transitions
• UIImageNamed (immediate decompression, cached by OS, purgeable by OS) vs. ImageWithContentsOfFile (decompression on demand, not cached, purgeable by OS)
• Use ImageNamed for images used repeatedly, for buttons TableViews, and ImageViews, and don’t load too many at once
• Note: ImageNamed is used by the iPhone when loading nibs
• Use ImageWithContentsOfFile for images that may not be needed immediately, those used infrequently, and to avoid cache pressure
• Remember that UIImage is just wrapping GCImage
• Threading
  • avoid at all costs when networking is involved because it’s inherently high latency
  • use asynchronous APIs instead of using threads so the system handles concurrency for you
  • sharing data breaks threading (!!!), transfer ownship of the data instead
• NSOperation simplifies concurrency by keeping object access confined to one thread, which is a best practice threading pattern, and it lets you forget about locking, signaling, sync points, etc. Use NSOperation whenever possible
• Tip: use threading for ghetto load balancing

TALK 4: Mastering OpenGL ES Part 1

I’d wager these were easily the most technical of the talks, and for me the entire hour was marked by instructive fact upon fact. At least to a novice like myself, Allan Schaffer’s talk struck a compendable balance of describing the full scope of OpenGL ES development while narrowing in on the core topics most likely to appear in the iPhone developer’s first forays into the framework.

• Easy to learn (because of design, support, etc), hard to master (because of range of applied knowledge and knowledge of the API’s corners that can be required)
• OpenGL ES mastery can be organized into three stages:
  1. 3D FFX programming, geometry
  2. hardware complexities, optimizations, data structures
  3. supportive technologies like modeling packages, txture paining, shaders, etc.
• OpenGL ES 1.1 vs. 2.0 – conditionally build for ES versions
• In the iPhone display system core animations acts as iPhone display manager and layers are the building blocks
• Understanding layering, CALayer and CAEAGLLayer, and how layers are animated and composited atop other layers, is fundamental to ES development
• Test with Animation Instruments to flash updated layers and determine how to best optimize performance based on compositing views
• Avoid pitfalls in loops with NSTimer by using CADisplayLink instead
• Optimize render loop by limiting frame rate and drawing on demand
• Xcode OpenGL ES template contains defensive code that is redundant if you’re not using multiple contexts and color buffers and such
• Build against ARM based on whether you need floating point maths or not
• Invest time in reading chipset specs, and the OpenGL ES specs in full

Talk 5: Testing and Debugging

The average iPhone application isn’t known for stability, and this appeared to be one of the most popular talks. I assume most developers attended already understanding the bulk of the material but thinking that even if they gleaned only one meaningful insight or tip the talk would be worth it.

That was my reasoning anyway, but it turned out that the very nature of having the full debugging and QA cycle explained by an Apple developer was reassuring and instructive, and cast some much-appreciated context to the regularly painful process and somewhat disconnected documentation. I didn’t take many notes for this reason, though I’d recommend the talk to others attending the World Tour simply for its walkthough value.

• Beta Testing – the full process of creating a build for beta testing was covered and there don’t appear to be any special steps or tricks on top of what’s commonly known
• Make sure to get crash logs from testers; the logs can be symbolicated by organizer effectively providing you with a backtrace of the crash
• The three most common types of crashes are: low memory, application errors, and timeouts
• iTune Connect allows you to see crash logs submitted by customers to be the App Store
• Springboard will always be shown in the logs to consume the most memory, but that’s just a representation of your app

TALK 6: Performance

This turned out to be my favorite talk of the conference. There was a tremendous amount of material covered so the pace was galloping and, as my friend Arthur observed, although it was the end of a tiring day we left energized and awoken. It’s fitting anyway for a talk on performance to speed along at a hurried clip. Of all the talks this was also the one that made me most want to get back into my code to refactor suspected mistakes.

• Use shark and instruments for profiling speed
• Memory concerns are fine to test in the simulator
• But timing or GFX or any hardware involvement should be tested on a device, not only the simulator
• Drawing optimizations
  • call setNeedsDispaly
  • check the rectangle passed to drawRect
  • mark views as opaque
  • invalidate timer before creating new timer
  • use PNG files as preferred image type
• ScrollViews
  • use opaque views
  • avoid allocating views during scrolling because allocating views is expensive
  • reuse table cells
  • collapse cell view hierarchy with lots of views, turning a parent uiview with many subviews into one big view with no subviews and use custom drawing routines for top performance
• Application launch
  • consider user’s immediate needs and load to that
  • make app launch and quit fast
  • load data lazily
  • use small nib for initial UI that is presented to the user
  • don’t perform network operations or expensive operations in ApplicationDidFinisLaunching
• Memory usage
  • use thumb if possible (floats), thumb 2 / arm if not
  • smaller executables (not bundles) make a difference to launch time since the app exe is loaded to memory
• Static memory
  • disable in compiler options
    • avoid c++ exceptions
    • avoid runtime type identification (RTTI)
  • prefer direct allocation over convenience methods
  • release directly over autorelease
  • declare properties as nonatomic
  • image and layers might themselves be small, but could have huge backing store
  • every uiview has a CALayer backing it
• Dynamic memory
  • 4b of data per pixel, RGBA, so big images have exponentially worse performance when they must be decompressed
  • Load UIImages with imageNamed for small, frequently used images; the method is also used by initWithNibName, and automatically caches uncompressed images
  • Load UIImages with initWithData or initWithDataOfMappedFile for large files, and release such images on memory warnings
  • Load UIImages with initWithContentsOfFile for all other (normal) circumstances
  • Use NSAutoReleasePool, especially in loops

Technical Q&A

As a final note, throughout the day those speakers who weren’t presenting at the moment were available for one-on-one Q&A and code review. I managed to grab a slot in order to pose a question that has recently demanded my attention: how, when using a UIScrollView, and zooming in, scrolling to a corner, and zooming back out, can you prevent artificial margins appearing on two edges of the contents?

The answer: you can’t. However, it’s known, noted, and the SDK team would like to fix the problem. Nobody there thinks this is an acceptable UI design. :)

]]> http://www.travisdunn.com/iphone-tech-talk-world-tour-2009-hamburg/feed 1 http://www.travisdunn.com/new-jobs-new-gods http://www.travisdunn.com/new-jobs-new-gods#comments Mon, 19 Oct 2009 12:29:05 +0000 Travis http://www.travisdunn.com/?p=923 Career leaderboards are misleading, indulgent, and hopelessly alluring. But if the details of individual professions are normalized away, strong trends become clearer, as in CNN’s Money Magazine survey of the Best Jobs in America 2009 where Software Developer holds the #12 position and numerous related careers cluster nearby.

It’s not unexpected to see IT rank high, but it is noteworthy that it remains so dominant on the boards after the financial crisis, a decade of outsourcing, and considerable layoffs from industry leaders. Looking at just six of the top twenty jobs, it’s obvious where the value is coming from: problem solving.

• #1 Systems Engineer
• #5 IT Project Manager
• #8 Network Security Consultant
• #12 Software Developer
• #16 Software Project Manager
• #17 IT Business Analyst

The term “programmer” doesn’t appear in the list because the technologists and technicians that fill these roles bring talent to their job that is abstract, analytical, instructive, and oftentimes preeminently human. These are qualities that transcend things like frameworks and languages and demand a much more informed and opinionated view of one’s problem domain. It’s difficult to train for these roles because by their nature they demand a diversity of experience in workplace culture and technology that’s quite difficult to build expediently.

The collective “career” here is one of information engineering, which includes everything from specialty problem solving (deep technical knowledge), human concerns (user experience, productivity), and systems engineering (architecture, security). The importance of this knowledge to the success of a project cannot be understated, where the potential for savings and innovation on one hand or expensive mistakes on the other is usually disproportionate to the recognition actually given these engineers.

The appearance of so many analytical careers in the job rankings is a sign that this is changing, and despite the rise of the global marketplace and the offhand rounds of “unavoidable” corporate layoffs, the skills necessary to really drive technology are being better understood and proportionally valued than in years past.

As a developer, you want to be thinking along the same lines. A computer is no more necessary to “development” or even “programming” than a calculator to calculus. The place where thoughtful developers can really strike an advantage is in extending their problem solving skills to encompass other areas of related business. Moving from from programmer to developer. Then from developer to engineer. And then, ultimately, to a place of abstraction where you are most effective at conceiving, evaluating, and building solutions.

These aren’t new ideas about the IT field, and they have long been understood well by practitioners, but my sense is that this is the first time that the issues are being well understood by the business world at large. If the dotcom bubble is an example of the public embracing technology and casting it in a powerful role, then it was done without appreciation for the parts that were valuable and the parts that were vestigial. Having spent the better part of a decade trying to regain the respect that was lost by way of shallow, grasping hipster tech, I hope that as the recession eases the market emerges with an increasingly realistic view of complexities and value in technology and information systems.

]]> http://www.travisdunn.com/new-jobs-new-gods/feed 3 http://www.travisdunn.com/web-service-api-design-by-example http://www.travisdunn.com/web-service-api-design-by-example#comments Wed, 07 Oct 2009 18:03:56 +0000 Travis http://www.travisdunn.com/?p=835 API design is one of the more important skills possessed by web developers today. The reason behind this fact is that web applications are increasingly unable to live in isolation while at the same time providing the scope of features and web awareness users have come to expect. Developing for the web means, to a greater or lesser degree, being a productive citizen in a kingdom of online data exchange; and whether by formal engineering or organic growth, this means building an API.

But API design isn’t limited to pursuing the affable integration promises of the open web. Understanding the power of an API in relation to application design is just as valuable to business needs as sharing data across services is to user desires. The power, in fact, stems from the same source as every other factor in systems design: managing complexity.

Unless you’re writing a Turing Complete API then your guiding objectives will be to simplify the intricacies of your application down to a suitable input/output abstraction, and describe the application’s behavior with as natural and predictable language as possible. The API is a white box with many different black boxes inside. Its design should generally represent a boundary point after which any further encapsulation of logic would require unproven assumptions, or else seriously violate the separation of concerns, and a boundary point before which any more flexibility and power requires special operating knowledge of the system because the API is too dense to be self-descriptive.

This is a razor-fine line and easy to slip from when moving within the system, but an API that’s clear and approachable to strangers serves in an equally valuable capacity at documenting behavior and intent. In the same way that TDD promotes the ancillary yet no less important benefit of forcing the developer to conceive of their code’s structure, methods, organization, etc. in terms of real-world usage, an API can also provide a sort of blueprint allowing for a theoretical rehearsal of the design impossible with strict bottom-up development.

None of these notions are particularly controversial or open to much debate; the tension is drawn between the gap of theory and practice, and for that I want to look at some examples from some of today’s prevailing web services. Reflecting on the conclusions reached by web leaders should provide us with insight into our own design efforts, and read the collective pulse of the open web.

Twitter

http://apiwiki.twitter.com

Positive: Twitter has become the gold standard in API adoption among the developer community, and much of this owes to a lucid API design. The organization of methods under loose “resource” groupings is easy to understand, and the individual method documentation is clean, forthright, and contains exactly the information you would expect including usage examples, authentication details, parameter constraints, response data, and more. What’s more, you can usually expect the resources to answer in XML, JSON, RSS, and Atom formats, another area Twitter stands out.

The documents do an excellent job of providing implementation these details, alongside top-level summaries of the general API design and conventions.

Twitter’s greatest accomplishment, though, is managing information overload from the potential expressiveness of the system. To point: you can almost immediately write code to begin talking with the API after reading only a few pages of documentation, and that’s as good a litmus test as any.

Negative: There’s not much to dislike about Twitter’s API and documentation, but one uncomfortable area is the quasi-RESTful model it implements. While the resource structure is clear, Twitter’s naming conventions are arbitrary and depend on the developer’s learning custom vocabulary for utility methods. This is a challenge failed by most APIs, though, and while Twitter is exceptional in many positive ways, here the API is no exception.

 

Basecamp

http://developer.37signals.com/basecamp/

Positive: The Basecamp API and documentation matches the 37 Signals adherence to simplicity, and you can familiarize yourself with the whole system in short order. The introduction covers implementation details like status codes, content types, and file uploads. If there’s anything to like about the API, it’s that you can learn it over a cup coffee.

Negative: For all its virtues of simplicity, the API lacks the cohesion found, say, in the only slightly more verbose Twitter API. The resources are well-named but the URLs are subject to unguessable nesting and inconsistent patterns. Searching and filtering data is not easily done, and the usage notes for those few methods which permit manipulation are hastily explained. As far as formats, only XML is supported.

 

Facebook

http://wiki.developers.facebook.com/index.php/API

Positive: Facebook’s API sits at the opposite end of the design spectrum, with a rich, intricate framework behind it. The power of that framework and data stores gives the API its strength and more than justifies its complexity.

Facebook deals with this by progressive disclosure in the documentation, and a small ecosystem of sub-APIs for permissions, application authorization, database queries, a markup and widget language, and many things in between. If the learning curve is steep, the documentation does a respectable job of guiding the developer down through specific methods and use cases as needed, and everything is well-organized, indexed, and linked.

Negative: The downside of the breadth, depth, and power is obvious. To use the API you must essentially learn a new framework, along with its own pitfalls and nuances. The entire nature of the design revolves around specialist knowledge, with heavily parameterized methods and enough usage exceptions to make casual experimentation and rapid development less inviting than alternative APIs.

 

Flickr

http://www.flickr.com/services/api/

Positive: Flickr supports a tremendously flexible API with an impressive number of both response (REST,XML-RPC,SOAP, PHP, JSON) *and* request (REST,XML-RPC,SOAP) formats. The documentation is descriptive, consistent, and concise, and the brief overviews on dates, encoding, tags, urls, and suchlike make further explanation in the method unnecessary.

Like many APIs, Flickr’s authentication is the most complex area of the API, which Flickr eases with instructive guides to authenticating from the web, desktop, and mobile platforms. The method naming is sensible, if not intuitive, while the method grouping is generally both.

Negative: Between support for SOAP and RPC calls, and a prescriptive approach to method naming conventions, Flickr demands slightly more special understanding than we would prefer, but with this as its greatest deficiency I find it to be on the whole well designed.

 

Digg

http://apidoc.digg.com/

Positive: Unlike other APIs here, Digg is essentially an endpoint to a single resource: stories. The heavy parts of the API revolve around filtering the stories on multiple criteria, as can be seen from the route index. The API’s basic concepts are standard enough, and while Digg expects its consumers to cache and chain data requests to prevent system overload, the documentation clearly explains the request patterns intended for talking to the API with these constraints.

Data can be requested in a commendable variety of formats, from XML, JSON, Javscript, or serialized PHP. Each documented method also contains plenty of illustrative example response data.

Because of the focus on stories, there are fewer API methods and each method call can be customzied and purposed with a variety of optional arguments. The documentation systematically explains the exceptions and conditions under which parameters are used, and the design gives far more power to arguments than to routes, in other words more RPC than REST.

Negative: On the other hand, the argument-laden nature of the API methods require careful review and implementation, overhead that could have been avoided by consolidating the filtiering logic either in the documentation or the API itself. This also makes it difficult to grasp the full range of the API without serious review of all of the its corner cases.

The flood controls on API requests are explained well, but irritating to accommodate in contrast to something like Google Data which has a more passive approach to imposing usage restrictions on its consumers. Further requirements like, for example, a User Agent header, make Digg’s API feel less inviting and fault-tolerant than alternative designs.

 

eBay

http://developer.ebay.com/developercenter/rest/

Positive: eBay is firmly in the RPC design camp, and frankly, aside from desirable auction data, there’s not much to recommend the API. If the documentation seems exhaustive – and it most certainly is, see the guide to REST, for example – then it’s an honest emmisary of the beast of a system which sits behind an ostensibly friendly API frontend. Although there are detailed code samples and a development sandbox, these are more necessary complications than any kind of extra, facilitating support. The API does support XML, SOAP, Name Value, and JSON output, though, which makes sense beside the seeming design ambition of leaving no possibility unimplemented.

Negative: Challenges for the developer abound in the API. Issues in the documentation from PDF-only guides, broken hyperlinks, poor IA for a system large enough to clearly deserve it, inconsistent method implementation and naming schemes, duplication of functionality, you name it! If there’s a potential roadblock to eBay integration, you’re sure to find it somewhere in this API.

The documentation is a mix of structured, heavily formatted schema descriptions with confusing conventions on when to bold/italic, change header size, change fonts, etc., and writing that ranges from conversational to clinical in enough voices to give the impression of design by committee. The jargon required to follow the nine-plus API products (searching, shopping, matching, feedback, trading, etc, etc) is everything from dependency and monkey-patching, and nothing from an open, agnostic web. Individual method writeups are an embarrassment of riches with ample reference but no progressive disclosure and little indication as to why one thing is more important than another, or suddenly described at a different level of abstraction than the rest of the page, even digressing into legal or business discussion in the middle of a technical section.

Perhaps the most damning complaint is that there is no suitable top-down entry point for acquainting yourself with design of the API and the extent of its power, no way to understand the goals and constraints of the system or even a clear idea of what exactly it’s modeling.

 

Technorati

http://technorati.com/developers/api/

Positive: Technorati answers the same needs as Digg, but does so with a considerably more elegant (if a bit less flexible) API. In fact, it’s elegant enough that no introductory document is needed, and there are only a handful of methods you need to access its full functionality. Responses are formatted as XML or RSS, and further documentation is provided for setting up custom pings to the service.

Negative: Although the superficial depth of the API makes for swift development, API influence is largely relegated to reading, not manipulating, data. And while a simple system might otherwise lend itself to descriptive structure and resource names, Technorati is not much better at its taxonomy that much more complex APIs.

 

Google Data

http://code.google.com/apis/gdata/

Positive: The Google Data API is difficulty to appraise simply because it’s so diverse. Google Data imposes the governing design for dozens of Google services that each depart from, reduce, or extend the Base API to varying degress. While there is a solid conceptual overview, introductory video, and plenty of examples and framework-specific libraries, the price of admission to Google’s power is study and specialization.

In addition to the resources mentioned above, Google’s documentation acquits itself well in presenting information on the daunting scope of functionality and flexibility found in its APIs. Progressive disclosure, intuitive taxonomy of services, methods, and overall IA, and concise, consistently formatted documentation are the winning conditions that keep the API accessible and friendly.

Negative: It’s hard to fault google for complexity while understanding the design challenge, and especially in consideration of the fact that large but comparatively smaller APIs than Google (e.g. eBay) fail much more spectaularly at intelligble design and documentation.

 

Netflix

http://developer.netflix.com/docs

Positive: The Netflix API is very RESTful with a convincing explanation (and diagrams) behind how resources are organized and manipulated, and some clever tricks like an “expand” parameter for retrieving associated model data.

Documentation takes an instructional approach, walking through the various aspects of the API with perhaps too much depth at places, a distraction otherwise compensated for by the outstanding layout, formatting, example code, and succinct writing style. The extensive annotated screenshots are well chosen and never seem to clutter your search for information. A more formal reference for the API is available as well, where you’ll find the Netflix design to be quite respectful of RESTful principles. There’s even a lightweight Javascript API for basic Netflix actions.

In fact, the documentation is so excellent, if a little verbose, only because it reflects a tight, descriptive API design. In this case,I don’t mind the API writers erring on the side of explanation because it’s always clear how what you’re reading relates to the API as a whole.

Negative: Netflix stands out among rival APIs, both in thoughtful documentation and technical design. Perhaps its weakest point is the use of OAuth for authentication, considering the added complexity. XML is also the only supported format outside of their Javascript API.

 

]]> http://www.travisdunn.com/web-service-api-design-by-example/feed 2 http://www.travisdunn.com/9-ways-developers-fail-to-accommodate-business http://www.travisdunn.com/9-ways-developers-fail-to-accommodate-business#comments Tue, 15 Sep 2009 06:37:56 +0000 Travis http://www.travisdunn.com/?p=747 Developers tend to advertise themselves, indeed, their entire profession, as a pragmatic yet sophisticated participant in business, creating real human value while neatly abstracting away the complexities of technology. They often see themselves in opposition to other participants, engaging in heroics, or protectionism, or just toiling unsung for the higher good.

It’s a flattering portrait that manages to combine superiority, rationality, and integrity in the developer’s character, appearing all the more inviolable when compared to abusive, thoughtless management or the gasping demands of the customer.



These are gross straw-men, however. Even if, as developers, the relationship between business and technology, between corporate and creative, places us under the thumb of clueless decision makers, and even if those authorities are wielding capricious power that can entangle and maim a project, we still have much to answer for ourselves, nor are we as martyred as we like to believe.

If management abuses power, then developers abuse knowledge. It’s around this fact where developers most often fail to support business needs and the “outsiders” who control them. We act as the privileged gatekeepers for warrens of technological complexity, and it’s our responsibility to examine, understand, distill, and communicate the features and tradeoffs of these systems. When we hoard knowledge, saturate our colleagues with irrelevant details, or refuse to take responsibility outside of our technological boundaries, we fail business.

And it’s not always a failure of omission or laziness, because by deliberately hiding or revealing technical facts, developers have the power to steer business decisions and often use this power to a greater or lesser degree in fighting back against perceived corporate hostilities or mistakes. Such influence may be well-meaning or even necessary, but often it’s the result of tensions appearing in a project or a developer who miscalculates their role.

With that said, here are some of the ways that developers fail their businesses.

1. Justification Fail: a failure to rationalize our technical decisions to others and adequately explain our decision making process. As much as software development is about managing tradeoffs, it’s important that we remain aware of the reasoning behind founding technical decisions we make, and likewise important to be able to account for our rationale to laymen.

2. Dependency Fail: a failure to understand, lucidly explain, or else explain at the right moment the deep dependencies between technical components, and the ramifications these relationships impose on system design or change requests. It’s the absolute responsibility of the developer to inform others when outside influence is threatening a dependency such that subsidiary work will be required or other unintended consequences may occur.

This is one of the most serious fails a developer can make, since we’re normally the only individuals with a clear understanding of a system’s composition and behaviors and the ways these drive each other. We’re also a vanguard, expected to remain vigilant to dependency problems that emerge in the course of development, and only we have the power to alert the team to forthcoming dependency tension or else fail to do so.

3. Transparency Fail: Due to the arcane nature of development work, it’s easy for our habitual work efforts to become unintelligible, maybe even invisible, to outsiders. It’s our failure that we don’t make some effort to communicate our activities at a high level, so that colleagues feel everyone is moving forward together.

4. Scheduling Fail: a common failure, partly due to the nature of software development, but also because developers are prone to confusing known and unknown elements, or mistaking the later for the former. When estimating development schedules, we ought to provide solid estimates for tasks of known scope while allowing for sufficient leeway in undertaking tasks of unknown scope.

5. Knowledge Transfer Fail: accumulating vast reserves of formal and informal intelligence about a system, developers often fail to share that knowledge, and become human silos for some of the most core competencies of a company. A developer should be to some extent documenting their design and decision making process in respect to current development work AND to how it may impact future development. Transferring knowledge means allowing other developers to quickly adopt the insights and proceedings you’ve recorded, while failing to do so means making yourself both an essential resource but also an extreme liability.

6. Platform Fail: a failure to select a platform or implementation for ANY reasons other than it’s inappropriateness to the project on hand and direct development productivity. Playing favorites with technologies is a selfish and chauvinistic fail, and developers ought to be expected to learn new platforms when it makes sense to do so, and to evaluate them so they recognize that sensibility in the first place.

7. Security Fail: while security is often overlooked by all sectors of business, a developer should always be passively aware of vulnerabilities in the systems they command, and go out of their way to ensure that even if these aren’t addressed that the risks and consequences are understood by everyone accountable. Only a developer can evaluate this accurately, so they must be a vocal advocate of security (or lack thereof) awareness.

8. Priority Fail: failing to prioritize workloads based on productivity, business requirements, or known scope is inimical to a project. Approaching development on the basis of personal or professional interest undermines the already fragile development cycle, and is by contrast a demotivating factor when it comes time for the priority work to be tackled.

9. Social Fail: there are plenty of fail-heavy stereotypes about developer personalities, and they aren’t worth rehearsing because by social failure I don’t mean likability, but rather failing to integrate with a team, as if you yourself were some closed social format and the rest of your office was talking in open source protocols.

I wouldn’t call unfriendliness or cynicism a fail, but not sharing your insights and teaching things to your team, not advocating for best practices or introducing ground-up improvement in your team’s spoken or unspoken development methodology, refusing to support incidentally related issues with your expertise, neglecting all efforts to advocate richer and more interesting possibilities for technology, and proudly shunning compromise during interpersonal conflict, these all constitute grounds for social failure where a developer, a considerable repository of knowledge and opinion, neglects to participate in their community or decides that other humans must implement implement their jobs to the developer’s “proprietary, social specifications”.

---

However true it is that development is often scapegoated, mistreated, distrusted, overruled, or abused by Corporate Masters, this shouldn’t be cause to retreat from a wider prospective or shut ourselves off to the subtle or reactionary or purely unintentional ways we fail to contribute larger project goals. It should never be an excuse for our own incompetency or negligence, and I think the developer community would be much better served by relaxing some of its criticism of other vocations and tightening the criticism of its own habits.

Software projects rarely fail for technical reasons, but developers tend to accept this as evidence that they have no done wrong, when they are just as human and answerable as anyway else involved. Our value as developers derives heavily from our wisdom, insight, and expertise. When we miscalculate the importance of that, or selectively use it to exert control, we are failing the businesses, and people, that we are supposed to help.

]]> http://www.travisdunn.com/9-ways-developers-fail-to-accommodate-business/feed 1 http://www.travisdunn.com/website-disaster-katas http://www.travisdunn.com/website-disaster-katas#comments Sun, 16 Aug 2009 22:48:13 +0000 Travis http://www.travisdunn.com/?p=371 Code katas: summary exercises leading developers through the analysis and motions of solving hypothetical problems.

For this set, I want to provoke the minor and greater disasters that web developers might face; katas which assume hostility and bring an unwelcome challenge. Because disasters (we hope) occur infrequently, it’s all the more valuable to confront them in the form of a kata so that when adversity strikes you can respond with practiced foresight.



The value in code katas is not in the nuance of repetition but in the awareness they build. The range of responsibilities a web developer serves makes these exercises more koan that kumite, requiring strategy and wisdom as the operative weapons.

Like regular code katas the framework and language you imagine during these exercises is irrelevant. For the sake of argument, assume your victim site is running the environment most relevant to your current work but these disasters will threaten most any platform equally.

For general practice I can’t fail to mention that Jeff Atwood has a canonical post full of katas, and in the continuing spirit of the redoubtable list format I want to consider the following scenarios.

But good practice begins with a warm up. . .

Warm Up Kata. Gracefully go offline for upgrade, or after hack

Whether you’re updating a site, or need to take temporarily take a site down in response to a hack or critical bug, there will be dead zone for your site’s visitors.

What are some tactics for seamless deployment? What happens to user sessions during your deployments? How quickly can you replace the sites you manage with a downtime notice? Would your approach differ if your site was running on distributed web servers? And are these processes which can be automated?



Kata 1. Rollback website to last working version

The latest deployment broke your website and now you must immediately restore the prior code.

You probably use source control, but can you match repository numbers with deployments? What would be required to systematically rollback through each of your site’s deployments in turn? Are there any tools that could help you? If so, are there any cases where you would still have to deploy manually anyway?



Kata 2. Restore database from 1 day ago; from 1 week ago

The database has been polluted with corrupt data, or else countless important records have been accidentally removed.

How would restore the database to its timestamped version from 24 hours ago? How about from 1 week ago? What sort of backup strategy do or would you use to prepare for this? Imagine that only part of the database needs restoration, say, only user records and logs – how would you write a script to restore only the required data?



Kata 3. Mailer starts flooding system emails

Something is wrong with the system mailer and it is blindly flooding admins and site users with junk mails.

How quickly can you disable your system mailer. Can you do so gracefully while still leaving the site online? What steps can be taken when designing an internal mail system so that it can easily be killswitched?



Kata 4. Keep website online during traffic spike

While scalability is a huge subject, and our response to its problems often in the form of hardware and hosting solutions, there should be at least a few actions developers can take to respond to traffic spikes and keep a site receiving visitors.

Can you disable analytics logging, error reporting, or any other resources that bring logistical overhead to the site? Can you easily prevent images from being served? What ways can you facilitate these emergency measures in the system architecture design? Are there other ways you can drastically and temporarily reduce the HTML being served?



Kata 5. User passwords/emails are stolen

You found evidence that many of your users have had their login credentials stolen, but you don’t know where the breach occurred.

What are your first steps? Are there any immediate lockdown measures you should employ? Would you bulk reset all the users passwords and If so, how? What attack vector(s) would you suspect could be employed on the systems you’re responsible for in order to steal login credentials?



Kata 6. Locate and cleanse XSS SQL injections in database

Cross-site scripting attacks are appearing in various sections around your website’s dynamic pages; it’s clear that XSS code has been injected into your database somehow. Before you audit for the vulnerability, you must first scrub the polluted data.

How would you go about examining database text fields for the presence of encoded Javascript? How would you remove the injected code if the original text must remain intact? Assuming you only suspect that your data has been tainted, what measures could you take to confirm or deny the fact?



Kata 7. Mail domain blacklisted

Your mail domain has been blacklisted and your system mails are landing in your users’ junk boxes or outright being refused delivery.

Where do check to confirm that your domain has in actuality been blacklisted, and which to which blacklists has your domain been added? What red flags might your emails content and structure be raising? What red flags might your mailer’s behavior be raising? Can you verify that your mailer is not an open relay? Can you determine whether there is a virus propagating through your system’s emails? Once you’ve made the corrections, what steps can you take to repeal the blacklisting?



Having practiced this series, rest assured you can battle the dreadful caprice of fate in some of its more common forms. But what else? Of what other lurking disasters provoke your apprehension? Which disasters posses causes or solutions that remain deep mysteries to you? Anyone else have suggestions to further hone the web developer’s disaster response skills?

]]> http://www.travisdunn.com/website-disaster-katas/feed 0 http://www.travisdunn.com/tripling-programmer-estimates http://www.travisdunn.com/tripling-programmer-estimates#comments Sun, 09 Aug 2009 17:16:22 +0000 Travis http://www.travisdunn.com/?p=318 Conventional wisdom has it that when asked for an estimate, a programmer will recede into fitful contemplation, perhaps aided by private notes scribbled in parallel, only to produce a number that should dutifully be multiplied by three for the sake of reality. Nobody can say where this constant came from, or why it so often hits the mark. Excluding some kind of the anthropic principle, I can imagine an answer.



1st Factor: Accurate in Theory

When a developer provides an estimate, it’s usually quite accurate, derived from a history of similar tasks and projects, as applied now on the platform in question. If development transpired without challenges and unknowns, there would be no need for further multiplication, but this estimate tends to be generous on two points: (1) that any obstacles which arise can be overcome quickly, and (2) that the scope of the details can be foreseen.

A chaste estimate from a developer represents the time it would take to complete the task if they remained in an uninterrupted state of 100% productive time, all the time.

2nd Factor: R&B (Research/Refactoring and Bugs)

The second factor comes from a collective yet measured cynicism towards bugs. It’s never the bugs you expect that kill productivity, and so it’s hard to anticipate the cumulative damage they will cause a schedule.

Estimates often overlook the research necessary before implementing new technologies, techniques, or integrating with other services. Every developer will invest time in researching an optimal approach for any number of project scenarios that require such analysis, but it’s easy to overlook the process in the course of initial estimation.

Combined, research and bug hunting/refactoring out of unfeasible solutions should at worst grow to the size of the normal development schedule itself, hence the multiplier. It’s a guard limit which, should you reach, you know your project’s in trouble, but if you don’t, then you understand as a natural appendage of development.

3rd Factor: The first 90% and the last 90%

This is about overlooking details which enlarge the scope of a task from beneath. When estimating, it’s easy to mentally do class modeling or construct a theoretical workflow or protocol, but it’s difficult to envision the extent of scaffolding that will be needed to facilitate the actual task. This will take the form of helper functions, refactoring and code hygiene, design improvements, and so forth, and are the tale of elegance and quality a good developer brings to the code.

As developers we sense the presence of this work, but during the estimation process it’s much more natural to think from higher levels of abstraction without realizing what’s required to build that abstraction in the first place.

This factor also represents the strange heuristic that developers must complete 90% of a project twice before the project is really complete. When an irrational exuberance starts signaling in the back of our mind that we’re only a few methods and tests away from delivery, we can recognize this as an emotional vanguard meaning we’re, in fact, approximately halfway from having actually completed our work.

]]> http://www.travisdunn.com/tripling-programmer-estimates/feed 7 http://www.travisdunn.com/testing-a-dependency-silhouette http://www.travisdunn.com/testing-a-dependency-silhouette#comments Mon, 03 Aug 2009 07:30:37 +0000 Travis http://www.travisdunn.com/?p=285 Automated testing confers a lot of advantages, many of them indirect. One of the advantages of the process that’s rarely discussed but which I find to play a daily role in my understanding of tested source is how failing tests outline the dependency streams in your code, revealing the organic shape of complexity that we usually confront in the isolated methods and classes which compose it.



Developers must be comfortable quickly moving up and down levels of abstraction in their mental models of the code, but for any sufficiently sized project it will become increasingly difficult to analyze any full branch of execution, let alone to recline and mentally survey the interconnections of the involved objects on the whole.

Class diagrams and UML are supposed to help with this, but they’re just forms of documentation, typically designed to express intent and not reality. Testing eclipses their purpose in this regard, by (formulaically!) expressing intent and then comparing it to the reality of the code. As a developer, your integration knowledge comes from running breaking tests with changing methods or changing classes, which then sends a distinct cascade of dependency failures throughout the body of the code.

Automating this process over your entire source, along with the logical visualization provided by the test results, can be instrumental in pushing at the edges of the code, finding the boundaries between objects, and heuristically bringing architectural issues to the surface. In fact, it’s a trick you’ve probably already used if, out of perverse or practical curiosity, you ever changed a method or class name and compiled just to get a documented list of the failing locations. As a byproduct of testing, this greater awareness comes naturally every time you run your tests, and especially when a cascade of failures are triggered.



Code is a restless, shadowy creature of our our making, into which our work is cast to be consumed by the darkness of time and intricacy. Testing is the angel-light which falls against its back and reminds us of its shape, how many limbs it has, and how it moves.

]]> http://www.travisdunn.com/testing-a-dependency-silhouette/feed 0