Building a large Drupal site may be a complicated task, but building small parts of it as prototypes helped us to build Go Ask Alice!, a flagship site of the Columbia University in the City of New York. If you are about to build such a large site, then continue reading why, what and how you should build using prototypes. (This is a transcript of my part from our presentation at Drupalaton 2015 with Kristof Van Tomme.)
To validate design ideas: Projects rarely begin with a clear understanding of needs, or at least what is worth building. Prototyping allows everyone to understand the difference between what they think they need and what they (really) need.
To validate implementation ideas: Once the developers have at least some grip on how to build what the client needs, they can try out different tools for reaching the goal. Most often the tool that seemed to be the best at first sight is not the one that makes the most sense.
To decrease risk: If developers start building based on a misunderstanding of project needs, all parties involved will be disappointed, as they will need to rebuild a large part of the work. This means additional time, additional costs, and lower confidence level in all parties. Some of the developers' ideas may not be as clever as others, so identifying which ones are the most trustworthy allows them to recognize early what’s worth pursuing.
To facilitate communication: Most of the time clients and developers are from a different planet, or at least speak different languages. Proper understanding of each other is a key for effective communication, which builds trust between the parties.
To minimise custom code: Developers can show customers what is already available as a community contributed solution, which is good for at least two reasons. First, when more of the site uses existing code, less development is needed. Second, building more of the site on community-supported solutions lowers maintenance and security risks.
To focus in on minimal viable product: If the scope gets too wide, the outcome may be nice and shiny, but the project in general may still be an epic fail. Stephenson's steam locomotives have validated that public inter-city transportation can be beneficial – and only later iterations introduced some advanced technologies on railways like TGV. Stephenson's project could not have been a success if he had wanted to build TGV.
What are you doing when prototyping?
Prototypes give everyone a hands-on example of a proposed solution, and take very little time to prepare. Concrete examples replace abstract ideas about needs and implementations. Using them in a review session (not just presenting them) facilitates discussions of limitations and opportunities. We build several quick, minimalistic prototypes of alternative approaches, then share our recommendations as clients explore them.
When prototyping, you could for example create a site for every possible type of contact-like form: first with core contact.module, second with Webform, third with Entityform, fourth with MailChimp, etc.
…and these are only two example areas with six and ten prototypes, respectively (on our project). Usually there are many-many more of them.
How to create initial prototypes?
Use free hosting: There are a ton of providers (visit this LevelTen blog post for a comparison). There is no point in paying for sites that the developers will not want to maintain and will not want to use longer than needed. On the other hand, it is worth keeping those prototypes for reference until the end of the development. Finally, developers will not have to explicitly delete the files as most free Drupal hosting providers will delete them after a year or so.
Basically it means developers should fire up a new site for:
- each (new) idea
- each feature (or set of features later on)
Prototypes are simple throw-away Drupal sites following the fail early, fail fast principle. Following this principle leads to lower risks and lower cost, as it will reduce the amount of work to be thrown away once other ideas are validated.
You have built the prototypes. Now what?
Once we start getting close to the understanding of what the Minimal Viable Product means, we can start integrating the already-validated ideas into a cleanroom. (Don't forget that when enthusiastic project participants speak about the MVP, they most likely mean the whole polished product. That is not minimal at all: check Kristof's related presentation.) Cleanroom is considered to have only the validated ideas where developers are 100% certain that they will be part of the final deliverable (let it be the MVP or the full product).
When there is a skeleton, developers can start adding muscles to it to make it move, then flesh to give it a momentum and finally a skin to make it look better. Basically the skeleton is the data model, the muscles are the core functionality that makes the website work, the flesh is additional nice-to-have features like social sharing, and the skin is the theme of the Drupal site.
With a cleanroom already committed to a git repo, the most meaningful way of additional development is creating the new prototype sites as a fork off of that cleanroom. This approach ensures two things:
- Firing up forks of cleanroom for the new prototypes not only lets you validate those ideas, it also validates that the tools used for them are compatible with the already incorporated prototypes. If there is a compatibility problem, it's visible fairly quickly, so developers can adjust. It is pretty easy to merge the new functionality back to the cleanroom later. This is only possible if the developers share a common, clear and documented process for building feature modules.
- If the forks are simply branches on the same (origin) repo, then merging them can be as easy as a git merge. However, when you encounter a conflict, then keep an experienced senior Drupal developer handy, as resolving merge conflicts in automatically-generated code (like feature modules) without understanding what they actually do would be a nightmare.
What are the downsides of prototyping?
As with each new method, there are dangers.
Clear and documented understanding of featurization is a must (we are talking about D7). All developers must share a common model for that. If this document is missing or somebody is not following it, then merging the prototypes might become a nightmare (or a rebuild of the combined stuff from scratch, which is time-consuming and error-prone). If you don't have such a document, read Kerasai’s article on the topic.
This whole process of prototyping yields a considerable number of sites (mostly throw-away ones, but anyway). After a certain period of time and/or above a certain number of them there is not a single human on Earth who can keep track of them. The solution is a catalog, which can be either as simple as a (Google) spreadsheet, or just another dedicated prototype as a Drupal site with a content type. The minimal information about a single prototype is:
What are the benefits of prototyping?
- Prototyping makes it easy (or at least possible) to work on different parts of the project different teams: the development team can work on the contact prototype while the PM team discusses the workflow with the client and the graphical team is working on how the already-validated parts of the site should look.
- These efforts can have different velocity: for example it is possible to spend only 2 hours a week on the contact feature and 20 hours a week on the workflow feature.
- Additionally, these efforts can have different completeness level: most of the developers can focus on the feature with the highest lead time, while a single junior can work on the contact form that's almost complete.
- Finally, prototyping makes it easy to work on some dark areas of the project even after launching the MVP. Later on, if you have to maintain the site, the maintenance changes can be implemented as new prototypes.
In general, prototyping allows making decisions as late as you want, or at least as late as possible – which is the core premise of agile development.
And there are two more outstanding benefits:
- With this approach, you can already start working on real, paid Drupal 8 projects: you can work on the parts that are already covered by D8 core and the possibly available contrib, while you can postpone working on the parts that would rely on not-yet-existing D8 contrib modules.
- This method is more generally applicable, for example to business methods. You can give some new methods a try, while still keeping the old balls in the air.
It was a really good experience to be the pivot point between such experienced Drupal companies like Pronovix and OwnSourcing. This technique of prototyping was brought in by Kay VanValkenburgh, the lead at OwnSourcing, and proved to be a really helpful method while we were building Go Ask Alice!
The fact that I could attend Drupalaton 2015 and do a presentation about this approach with Kristof Van Tomme, Pronovix’ CEO – days before launching such a large project shows that prototyping works.
And some personal words: I would like to thank both companies that I could attend this event, first time as a member of the Pronovix team. When I first saw the site saying Drupalaton is a "perfect balance of work and leisure, sessions, sprints, beach sports, great food", etc., I thought this was just marketing eyecandy. Then it turned out to be the reality: I could do a presentation (despite the fact that I didn't have too much time for preparing for it, this close to the site launch), I could read my book in the morning on the hotel's private island while sunbathing, I could go out paddling with my friend and boss, I could attend others' presentations which I was interested in, I could have great food and free beer on the party boat. It couldn't happen without the help of all parties involved – thanks!