Sure you’re agile, what about your QA department?
For the past 14 years I’ve been a software developer for money, I’ve noticed a trend. The QA department finds the most bugs towards the end of the release cycle. Even when I’ve worked in departments that are supposedly following an iterative development process.
A lot of the “bugs” revolve around “fit and polish” issues. Making sure the font and wording is correct in all locations. Making sure the application looks and behaves the same across all platforms,browsers in the case of web applications. Most all requirements documents or stories include a note for the QA department to perform these checks well in advance of the final launch, yet these bugs always seem to come up and they usually are not regression bugs.
The places that follow an iterative cycle I’ve found don’t really deploy after each iteration to any meaningful customers. They are all still tied to the “big launch”. I think that getting the product into the hands of people who ACTUALLY care about it, rather than the ones that are just concerned about the launch, is crucial to delivering a high-quality product.
I’m wondering if the traditional QA department is a relic that we need to get rid of? If we move all of the testing/bug finding to the interation and fix them immediately, isn’t QA relegated to mainly verifying bugs reported by customers/users?
So the question is: How do we get QA more involved in the process?

September 30th, 2009 - 08:20
I think the QA team has to be part of your daily standup meetings, you have to let them know what kind of new features are being build so that they can be part of the group to know what is coming down the pipe.
And in daily standup they need to let you know what worked and what didnt work, so you as a developer could right away fix the bug. This is very tough in real life since usually QA is stuck at the end just like performance.
In realistic terms if you can get maybe QA to be behind one iteration then you are still good at least in your new iteration you can actually allocated to have some bug fixing time that QA finds, but the entire iteration process needs a strong architect and PM to govern for.
Its all about the process, it becomes harder when you have lots of people but it definitely could work and it does require for someone who is experienced it. It definitely its going to be a person full time job to actually make this work, which most shop’s lack unfortunately.
September 30th, 2009 - 11:32
Taswar:
I agree, QA should be part of an agile team.
http://www.lazycoder.com/weblog/2009/07/31/structure-and-composition-of-agile-teams/
The question is how to get them more involved. Often they don’t have the same skillset as the other team members (often developers). So if something goes wrong (a server is misconfigured, their workstation acts funny, etc…), they have to get a dev to stop whatever they are doing and work on it. Also, at least with web applications, there are a large number of different combinations of browsers and operating systems that they have to test. It’s hard to test all of those combinations in a standard one or two week interation.
September 30th, 2009 - 12:30
Here are some recent thoughts along those lines:
http://blog.componentoriented.com/2009/09/the-role-of-qa-its-not-just-testing-anymore/
I think QA has to become more proactive about building testability into the application. If QA really adopts this stance, they’ll be forced to engage in design in a way that will force bugs out early.
September 30th, 2009 - 12:49
Scott in order to do so you need something from top that supports it. Bring the QA lead to your daily standup.
Traditional models of developer departments and seperate qa departments….. that’s not going to work in an environment where communication is key which is agile.
And a lot of automation by good qa coders, qa should not be looked at weak coders there are some really good ones out there who know how to do lots of automation with ruby, watir etc.
I have actually heard the reverse where the QA is using SCRUM but the dev team is just doing stuff i.e feature building monkey
September 30th, 2009 - 12:52
I was wondering, how did an Agile approach get started in the places you’re describing? It sounds like it started in development presumably working with business but excluding testers. Were other areas excluded as well?
I tend to see testers, themselves, willing to engage in as agile a way as possible, but test management being concerned about “independence” and “due diligence.” Having spent a lot of time talking to test folks, it is clear they are sensitive to complaints targted at them if defects escape to customers. Since they get the system last, they are viewed as responsible for quality and accountable to keep problems from escaping.
Thus, test management is not likely to want to give up on strong, independent, end of development testing as the approach to take. They are also suspicious of testing done during development as they are used to it being poorly done, e.g., getting defects that decent unit testing should catch, tests done only on “correct” input, and testing done to confirm belief in correctness not disprove it.
September 30th, 2009 - 13:11
Scott -
“They are also suspicious of testing done during development” — that’s where a really good QA team that’s involved during development can start to engineer their tests to ensure that the delivered system can pass the strong tests they want, but do it in a way that they can automate reliably.
Think about the QA team using unit test-like techniques, but applied to the whole system — not just low-level routines. If done correctly, this lets the QA team create “strong, independent” tests earlier in the process, and implemented with automated tools.
September 30th, 2009 - 13:28
Just tweeted this today! The flipside of this coin!
@kschlab #Agile – why QA / testing resources shine in the spotlight on agile projects [Johanna Rothman] http://bit.ly/tevLe
October 2nd, 2009 - 09:37
One trend I’ve noticed is that sometimes a lot of time is spent coming up with a testing framework to help get comprehensive coverage of all possible combinations.
If you have 5 week iteration, this framework tends to come online after about 3-4 weeks, at which point people start actually writing tests.
For check in the box coverage this is ‘good’, but for picking up bugs early…
not so much.
October 5th, 2009 - 02:21
At my office, we follow SCRUM quite strictly and QA team is always in the daily standups. They are also an integral part of the planning meeting and they have their own section for the review meeting. Their comments in the planning and review are independent of any influence and they are very rigorous in setting a test as ‘test failed’ and sending feedback emails.
Yet, they do fall into the same category more or less that they are finding more important bugs at the end. But that is not because of something inherently flawed in the team, its jsut that at the end of a release cycle more extensive testing is going on. So a solution to this that we’ve found is the all-hands test days where everyone tests a segment of the product rigourously (2-3 hours, not exactly a whole day) and then the QA team then consolidates the defects and isssues found. This is turning out to be a lot better approach.
On a side note, i think the agile developer’s responsibility is to make QA team out of business. They should write unit tests, integration tests and much more to minimize workload on QA team. Only then can you ensure product quality.
October 5th, 2009 - 13:12
I think QA has to be taken seriously.
What I mean is that often in the first few iterations, if there’s misspelling or a formatting problem, no one takes fixing it seriously, and it stays around until the end. In a lot of cases, QA will not report it because programming staff will think they are picking on unimportant things. I often see programmers say – “yes, we know that’s not working”.
When nothing concrete is working for QA people and all they can constructively contribute are UI, usability issues, that needs to be respected. If you are serious about delivering a great product, that’s what it’s going to take.
I understand that work needs to be triaged, but the work will have to be done eventually, and easy fixes are good to intersperse a developer’s workload.
All the TDD and automated testing is about programmers catching mistakes, and has a limited effect on overall QA which (if programmers are catching their mistakes themselves) should essentially then be freed up to ensuring that the product is meeting the stakeholders goals and finding design flaws and conceptual errors which are going to make the software harder to use or accept.