March/April issue of acmqueue

The March/April issue of acmqueue is out now

Quality Assurance

  Download PDF version of this article PDF

ACM DL 500 Error - Server Error

We are sorry ...

... but the URL you have requested has resulted in a Server Error.
It is possible that this was a temporary problem and is already corrected so please try to refresh this page.

We apologize for this inconvenience.

If the problem persists please contact us:

The ACM Digital Library is published by the Association for Computing Machinery. Copyright � 2010 ACM, Inc.
Terms of Usage   Privacy Policy   Code of Ethics   Contact Us


Originally published in Queue vol. 13, no. 1
see this item in the ACM Digital Library



Robert Guo - MongoDB's JavaScript Fuzzer
The fuzzer is for those edge cases that your testing didn't catch.

Terry Coatta, Michael Donat, Jafar Husain - Automated QA Testing at EA: Driven by Events
A discussion with Michael Donat, Jafar Husain, and Terry Coatta

James Roche - Adopting DevOps Practices in Quality Assurance
Merging the art and science of software development

Neil Mitchell - Leaking Space
Eliminating memory hogs


(newest first)

govind | Mon, 23 Jan 2017 16:12:28 UTC

is MBT going to integrate with open source tools , like selenium

Sandy | Thu, 19 Mar 2015 18:53:47 UTC

Thank you Bob! Sorry for comming back to the issue with such a delay! It is high time I poked my nose into the add. resources you offer here. I understood that it needs a deeper dive into test design - not only one aspect like TDD. I think I should also continue reading "The art of softwaretesting" by Glenford J. Myers, Corey Sandler, Tom Badgett to gain a broader notion of test design. As you mention the importance of separation of logical from physical details in modeling - and the Adaptor Pattern as a framework - is there as well a good intro resource re. this Pattern? I'm looking forward to ongoing topics in the area of modeling and testing from ACM. Thank you!

Bob Binder | Thu, 12 Mar 2015 18:32:16 UTC

Hi Sandy - great questions - here are our answers.

1. What do you think about my idea on making TDD a partner of MBT? Bruno Yes, good idea. But, more a partner of ATDD. Bob MBT may be used to generate tests at any level, from a class interface to a Web 2.0 application UI. So, yes, you can practice Test Drive Development (TDD) with MBT. However, TDD is a weak approach to testing, so you will need to think more deeply about test design if you apply MBT when following a TDD approach. The same holds for BDD and ATDD. Please see

2. Braking the model into re-usable components (for other dev. phases)? Bruno: Yes, braking the model in re-usable components is a good practice, but not for other dev. phases, more for other testing projects, because the MBT model is really testing oriented.

Bob: I agree with Bruno. The rationale for the parts and structure of a test model are somewhat different than for other system models. For example, test models are often organized around test purposes, which are not always the same as the functional hierarchy or interface structure of the system under test. It is also important to separate the "logical" aspects of the test model from the "physical" details of interfaces. The Adapter Pattern is typically used for this.

3. If we depart from (the codex of) clean code could it be helpful to establish a clean modeling code to foster the gains of MBT? Bruno: Yes, for sure. We have to establish that. MBT clean modeling ! Bob: Agree. Keith Stobies talk on this has some excellent advice for good test modeling practices.

4. And again: How much time do we and our stakeholders need to be able to harvest the entire MBT- benefits on the whole engineering lifecycle (what might 80hrs as a median mean in terms of proficiency level across the lifecycle?)?

Bruno: The learning curve to be proficient depends of many parameters (background of the team, tooling, commitment of the stakeholders, &). So 80 hrs is an average consistent with what we can see in organizations that deploys MBT.

Bob: 80 hours (about two working weeks) is often adequate to learn the basic usage of an MBT tool, but mastering a tool and its modeling concepts takes more time.

Our study didn't directly answer the question of benefits versus life-cycle. However, modeling almost always results in more discussion and deeper understanding of the system under test. This usually has immediate positive effects. Other studies found that, overall, MBT produce test suites in about half the time it takes to program equivalent test suites by hand (for details, see

Sandy Flath | Thu, 12 Mar 2015 09:15:06 UTC

One critical surveyee points at the fact that MBT gets tricky when the iteration of functionality results in big changes. Id claim here: MBT has to go hand in hand with the iteration of functionality and maybe Test Driven Design (TDD) could be a good partner of MBT in that sense.

What do you think about making TDD a partner of MBT?

The usefulness of MBT seems to be proven - at least among the practitioners surveyed - it plays a major role in functional testing  and this even across all test levels (unit, integration, system and acceptance). Unfortunately, an effortless adaptability of the model into other development phases is not given. Do you think it is possible braking the model into re-usable components (for other dev. phases)?

I can make out some typical pain points mentioned by the rather critical respondents. Those comprise sth. I would call The art of taking and being given enough time to focus on quality that pays off. Components of  time for quality ¬ are: Communication with stakeholders across the engineering lifecycle. Plus: The modelling against real world problems which means nothing else than continuous reasoning over the impact of functionality changes, adaptations  a sense for clean modeling. Akin to clean coding, clean modeling might be the right attitude.

If we depart from (the codex of) clean code could it be helpful to establish a clean modeling code to foster the gains of MBT?

Now which time is needed to become proficient in MBT? The answer to that question is  sorry for being that frank: Hilarious  the expectations among the interviewees range from 0 to 2,000 hrs. which lets the authors conclude to a 80hrs (two work weeks median). A figure that must be further explained in terms of proficiency levels and the according skills on those levels. Maybe we could approach the time frame from a different angle. Let me tweak for a moment the question on MBT-proficiency:

How much time do we and our stakeholders need to be able to harvest the entire MBT- benefits on the whole engineering lifecycle?

Leave this field empty

Post a Comment:

© 2017 ACM, Inc. All Rights Reserved.