Quality Assurance for Complex Drupal Implementations
Over the past few years at Phase2, we've been working on progressively larger and more complex web applications and CMS implementations. As a result, I've been increasingly involved in developing a reliable and repeatable testing methodology to support this growth.
As the layers of complexity continue to increase, there are more bases to cover and different user roles in play -- third-party integrations/dependencies to consider, etc. Quality assurance on these types of projects involves a lot more than just clicking around and making sure all of the links in the navigation are hooked up, or making sure the Javascript-based carousel on your home page doesn't crash your browser in IE6.
Another reality is that over the course of implementing a solution course corrections need to be made here and there to get things done. There is a fairly common misconception that you can install modules and the resulting features "just work", which is hardly ever the case.
A fair amount of "glue" needs to be coded in to fill the cracks and smooth things over - particularly for sites requiring a large amount of custom development. This code might not have the eyes of an entire community of developers and testers all over it -- therefore, a close level of scrutiny is needed from a QA team.
In many cases, the team of developers and analysts working on building a given implementation end up being closer than anyone else in terms of the "ins and outs" of how the system is actually fulfilling the initial requirements. It can be challenging to get a distributed team of stakeholders on the same page about how a system is intended to work without tight coordination.
Additionally, many large-scale systems are highly visible and recieve an immense amount of traffic. This obviously exacerbates any slip-ups made during the QA process by an order of magnitude.
For these reasons, I tend to employ a multi-faceted approach comprised of several principles and artifacts when defining a QA strategy for a particular implementation.
Scripted Testing
We have found in many situations that writing a series of test cases over the course of feature development is a great way to ensure that no stone is left unturned from a testing perspective.
There are many different schools of thought regarding the proper format/structure/level of detail for a test case. Generally speaking, I have had the most success:
- keeping them short and concise,
- starting with the most critical and highly visible site features and functions, and
- iterating from there onward as timeline and budget allow.
This process continues to help us get on the same page with customers about how their system is/will function, and provides an opportunity to revisit/adjust requirements as needed in some cases. It encourages the broader group of stakeholders to get involved in the testing process and provide valuable/unique perspectives that might not otherwise be heard. Test cases can be an invaluable tool for onboarding and workload distribution amongst project team members.
I'll also identify the most important test cases and run through them each time code changes are deployed to an existing site.
Cross-Browser Testing
It's important to set expectations early in the project surrounding which browsers/versions need to be supported. Whenever possible, I review a client's analytics to have a better understanding of what their visitors are using.
Depending upon the number of layouts in play and the complexity of the javascript / CSS employed, an implementation can require a decent amount of iterative tailoring to look presentable in older browsers.
You should start cross-browser testing fairly early on in the theming process (in case any major issues), and ramp up significantly towards the end of the project as changes to the design are less frequent/substantial. Your site might not look exactly the same in all browsers, but it should look professional.
Organic Testing
There is some degree of ongoing "fuzzy testing" that has to take place during the development lifecycle. Generally speaking, common sense prevails. There are some basic litmus tests that should administered perpetually during the development lifecycle:
- Does this feature make sense to end users?
- Can someone understand how to use it without instructions?
- Can users get where they need to be in a minimum number of clicks?
- Is it fairly easy for an administrator to utilize this feature?
None of these questions are particularly revolutionary or complex - you just need to ensure that you are constantly asking them.
Accessibility
Develop a shared understanding of accessibility requirements with your stakeholders, particularly if 508 compliance is a requirement.
Some common considerations include:
- Does all of your content have a text equivalent (i.e. alt text for images, transcripts for audio/video, etc)?
- Can your navigation menus be traversed by someone using a screen reader?
These are just a few facets to consider when developing your quality assurance strategy. Iterative/agile development demands iterative/agile testing -- requiring patience, dedication, perseverance, an eye for detail, and a personal investment in producing something that you can truly be proud of.
