Clinton Montague

Developer, learner of things, functional programming enthusiast, hacker, and all round inquisitor.

Web development is a sack of shit

August 25, 2009

There are some developers out there who feel like they need to know the answers to everything before they can get started on even thinking about starting a project. For the most part, they’re right – but there are times in development when you just don’t know. Is it important to know that there will be exactly 7 links on a page? I argue that (from a development point of view), the only thing you need to care about is that it’s more than one and less than ‘lots’.

Sack of shit?

A few days ago I was talking to a designer and we realised that in a lot of cases, it’s not really important to know the exact technical details of how it will be done, just that its shape is roughly X. As she quite nicely put it, they’re just sacks of shit (shit meaning stuff in this instance rather than bad) – and that in order to design, she just needs to know that it’s possible. She doesn’t care about, need to know or even have the ability to know whether the thing will get its content via XML or JSON – just that she can get the data. Should we be applying the same thinking to development? Or more specifically, do we need to quiz designers to death about the exactnesses of their widget when all they know is that it will grab something from a Twitter feed? I don’t think we do.

As another example, let’s think about building a site in (say,) WordPress which will let the user add images to their page in some place that’s not the content area (hence ruling out the option of placing the images using the page editor). All I need to know before building this is: Will it be one image or will there instances where there might be more than one? It doesn’t matter to me whether the more than one is two or thirteen, just that it’s more than one. This lets me decide which of the sacks of shit I can put the problem into; should I use a string or an array?

A big sack is made of smaller sacks.

When a project is starting, all you really know is that there are pages, and some of the pages might do something special. Some of these something specials might need to talk to each other somehow. All of those somethings are important to know, but the somehows are irrelevant; you just need to know the general shape of the things, not the exact details of what makes them tick. At this stage, you need to be thinking on a level similar to this:

Refine

The general shape of the things helps you to then organise your big sack into a set of smaller sacks which live inside the big sack. A little later in the project, a time will come where you need to then open each of the smaller sacks and peer inside, investigate, have a play and see exactly what the bag contains – or sometimes just add some teeny-weeny bags into the small sacks. You might now be getting to the stage where you can think about breaking high level items down into implementation details:

If you want to go with the list analogy, each bulleted point is a sack. Each sub-bullet is a sub-sack. (It’s also easier to present to clients and bosses if you present it as a list rather than taking the metaphorical sacks to a meeting.)

Recycle

Don’t discard the larger sacks – you may no longer need them, but other people will. Does your client need to know that you’ll be caching their tweets on the file system or database and the ramifications of each? Or do they just need to know that you’ll be doing something clever to make the website faster, and that might mean that new tweets can take a few minutes to appear? The choice should be obvious – the bigger and more loosely defined sacks are for clients/designers, and the smaller bags are for developers – and should be completely invisible to everyone else. A designer simply doesn’t care that in order to have two you need to use an array; they just need to know that it’s possible. And it’s our role as developers to do our job, do our job well, and do the irrelevant technical wotsits without inundating the designer or client with irrelevant and boring details. When was the last time your mechanic took apart the gear box to explain what’s inside?

There are different kinds of sack

After a while, you’ll start to notice that some of these sacks start to have a similar smell. Some might not, and some might only have a faint whiff of their neighbours – but this is ok – knowing that one of the bags contains nothing similar to the others is just as, if not more, useful than knowing that two bags are the same.

Siamese sacks of shit

Often, you’ll find that some of these bags have a magical (and sometimes invisible) tie to each other – and finding a solution to one problem can accidentally solve another. One of the most important traits of a good developer is that he can see these loose connections – even though he doesn’t know the exact implementation details. The realisation that both Twitter and Flickr can provide data through the API in the same way should make you want to do the woohoo dance. Why? DRY, that’s why. Write it once, use it twice. Which brings me onto the next topic…

Long lost identical sacks of shit

You should build up an arsenal of sacks which you keep with you in your head at all times. Remember that Twitter and Flickr go into the same top-level sack – they’re the same, but don’t bore yourself with the nitty-gritty of each service. Remember the top level stuff, remember that the lower level bits exist, and then research/revise the techniques required for the really deep lower level technologies as and when you need them. At this stage, the sacks are the same.

Also remember the ways that the bags link. Remember that PHP has some library for dealing with XML. Remember that jQuery deals with XML really nicely. Remember that although these two technologies are in different sacks, there is a strong tie between them, but again, don’t get too detailed with the implementation.

Black bags of shit

Of course there are also bags which won’t fit into any of the sacks no matter how hard you try. Maybe the bag is bigger than the sack. Maybe the bag is square but the sack is round. Maybe even one day the bag will get upgraded to a sack (AJAX, anyone?). And that’s ok – sometimes things are totally unrelated to anything else you do and you just have to learn new things. But remember the sack and bag analogy – take out the contents of this black bag and cut it up until the peices fit into well defined sub-bags.

Back to reality

Apologies to the metaphorical wankery and no doubt disappointment of the content of this post – I just needed an attention grabbing title. There are too many developers out there who insist on telling everyone everything about what they’re doing in so much detail that most people don’t have any understanding of what they are being told. Too many projects are delayed or built in a rush badly because a developer must know every exact detail before he will begin. It’s not necessary. Granted, in some cases you will need to know everything, but leave that bit of the project until later – build the independent pieces while you are waiting for the requirements of the data-sensitive parts.