“There is nothing divine about quality; it is a purely human affair.”
– modified Einstein quote overhead at QA Lustre
Contents
“There is nothing divine about quality; it is a purely human affair.”“We’re playing in an international field. We need to get into the mentality of doing it right the first time. To progress as a country, the industry has to set standards and demand quality: educations – academia – also has to respond, producing better graduates. Testing competence, and indeed, QA, is a valuable career path and people in this country need to realize that. That’s why we’re here today.” “I killed a patient. I got a call from the hospital asking “Are you Per Sletmo?” I said yes. They said, “Your system has just caused the death of one patient.” A lot of trouble, police investigations, everything. This is why QA is important. In the medtech industry, proper QA is the difference between life and death.”Let’s look at some great products and how long they too to reach 50,000,000 users. Television took 13 years. The internet took 4 years. Facebook took 1,096 days. Google+ took 88 days.“We need create a mentality where QA is a support structure for the development, where you have proactive QA – for example, rather than the number defects found, what if we looked at how many defects are not found? – that might be a better metric.”“Think of a house,” he says. “What is the critical point of building a house?”Silence from the audience. Then a few answers, tentatively spoken. However, they’re all surprised by the answer he gives:“The toilet.”“One famous bank in Indonesia once made a small mistake in 2002, when they introduced Internet Banking. Hackers took them to the cleaners and drained them of every cent. They failed to test properly and as a result, because of two people, they had to shut down their Internet banking. Period.”“Remember: software is logical rather than physical. You cannot see it. You cannot feel it. In a house, you can see and feel the build quality as you build it. In software, it’s not so easy. Testing does not imply the absence of errors: it merely indicates the presence of errors.”Per Sletmo recounts how he was invited to the house of a Test Manager at Microsoft. The house in question occupied an area of 400 square meters by the seas: he drove one Porsche, one Ferrari, four motorbikes, two boats, and all bought for his earning – and at the age on 32. “I don’t think this can be done in Sri Lanka or Sweden, but this shows how much potential there is in QA.”
Quality Assurance is not something we hear a lot about. Nevertheless, it’s perhaps one of the most critical components of creating anything. It’s a huge effort all over the production process, and despite the large conferences we see happening all over the place, we’ve yet to here here much about the whole QA process – something that’s rapidly evolving from simply bugtesting to best practices integrated into the Agile methodology itself.
Hence, QA Lustre 2014, held at the BMICH on the 4th of July. Some introduction is necessary here. This event, held at the BMICH and organized chiefly by Cambio Healthcare Systems, a Swedish outfit that develops medical and healthcare systems. As of the time of writing, they’re the only players in the medtech field in Sri Lanka.
Fair warning here: QA Lustre, like the Colombo Agile Conference, the various JavaScript meetups and so on, is a niche event. It concerns a particular area in IT. If you’re anyway near that field – either a QA professional, a student hoping to enter QA someday or a project manager who wants to catch up on the latest in the field, you’re going to find the following text very interesting. If not…well, read on and find out.
Let’s begin.
“We’re playing in an international field. We need to get into the mentality of doing it right the first time. To progress as a country, the industry has to set standards and demand quality: educations – academia – also has to respond, producing better graduates. Testing competence, and indeed, QA, is a valuable career path and people in this country need to realize that. That’s why we’re here today.”
This is Per Sletmo, Quality Manager for Cambio, Sweden, the first keynote speaker – he’s speaking on “why test is the most important function in a multi-team Agile development and release management”. There’s a good reason Sletmo is speaking today. It’s not just that he’s very high up in Cambio – he’s got over 20 year’s experience in QA and has worked with pretty much all the methodologies used in the industry. He works in an industry where proper QA is not a luxury, but is critical part of the dev process – you’ll see why.
He begins by sharing a little bit of background and a couple of horror stories, including why, due to poor QA, a project he was working on has to be delayed 9 months as unfound scaling issues crept in and they basically had to redo the database design all over again.. The client promptly hit them with penalties for each week that they delayed. In another situation, something more alarming happened.
“I killed a patient. I got a call from the hospital asking “Are you Per Sletmo?” I said yes. They said, “Your system has just caused the death of one patient.” A lot of trouble, police investigations, everything. This is why QA is important. In the medtech industry, proper QA is the difference between life and death.”
Pers’ speech stresses in the importance of QA and the practical implementation of QA. He points out that the traditional method of QA involves waiting for 90% of the work to be done and then testing.
This is wrong, he states, pulling up studies showing that some 56% of bugs are actually introduced in the requirements phase. There are ways and means of mitigating this – starting out with development methodologies, product management, right down to sprint planning, which he believes should be done by the system architect and the test lead rather than the teams themselves. Most of his speech, however. points to one unavoidable fact: QA needs to be integrated right into the development process itself.
“At Microsoft, we had test teams coming in every night and a Change Control Board to decide what to fix in the day,” he recounts. “You see, if you start to build a car, and you put a really strong engine, and the team designing the brakes put in bicycle brakes, it would be great to find that out in the beginning rather than the end. You need to find problems early. When you find problems late, you do a quick and dirty fix instead of fixing it right.”
“And of course, developers need to test, also. They can’t just code it and hand it over to someone who will try to find bugs. That will never fly.”
The second keynote is Chadana Ranasinghe, Head of Process Improvements, on the “Latest Trends in QA and Testing.”
Chandana is funny. “The joke is that I went from software to underwear,” he begins, referring to his position at Brandix.”But before we talk about QA and testing, what are today’s business demands?
Let’s look at some great products and how long they too to reach 50,000,000 users. Television took 13 years. The internet took 4 years. Facebook took 1,096 days. Google+ took 88 days.
What’s common with all these new giants? They provide high-quality services at no charge. Innovation, efficiency and agility are requirements for survival. We’ve gone from waterfall development with weeks to provision, static resources, limited number of users in business hours to agile development, continous deployment, data centers and clouds and millions of users with 24×7 access. This is the new requirement.”
He goes on to stress how testing centers of excellence are a growing priority to make this real. “Only 60% of companies have proper QA planning in place, and most of the money on QA budgets are still spent on infrastructure – 40% on testing hardware and infrastructure, 20% on tools and software, 23% of HR.
Mind you, the QA process – as being practised today – isn’t perfect, either. QA is still focused around finding bugs and tracking the numbers, he points out.
“We need create a mentality where QA is a support structure for the development, where you have proactive QA – for example, rather than the number defects found, what if we looked at how many defects are not found? – that might be a better metric.”
There’s a certain amount of structural work that needs to be done outside of merely changing the mindset. QA testers themselves need to change and build new ways of thinking. They need to be domain experts or technical experts – and they need to be thinking how to make the tests as realistic as humanly possibe. “Also, we need to consider: can we test it in conditions as close to real-world as possible? For example, a mobile app might cause a device to heat up and the battery to die quickly. Are we testing this? Are we testing the value we’re giving to the users?
“Think of the cloud. All of us talk about the cloud, but few of us know what we’re talking about. 28% of testing is now carried out in the cloud and this figure is going to increase – it was at 16% in 2013. This naturally brings its own concerns. Data security, for instance? QA needs to be proactively looking out for each and every one of these concerns.”
So what’s the solution?
1) Incorporate QA as a formalized step in the applicaiton lifecycle, evaluate new models such as DevOps. “We really have to get down from our ivory towers and figure out how to help each other and give better value to the customer.”
2) Fully integrate mobility into testing process – almost everything we do is and will be mobile in the future.
3) Evaluate the breadth of cost effective operating models to see if it a good fit. “For example, we can say we have 1:N developers:testers. What are these people going to do? We need to see what they’re doing, what skills we need and how we need to operate.”
4) Pay close attention to demonstrating QA’s business value for money. “Nobody will give you money unless you show what value you’re giving to the business. As QA engineers, nobody’s going to buy your stories until you clearly show what inroads you can make into improving productivity. If you can’t draw the relationship between what we do with business results, hard luck.”
5) Re-evaluate the diversity of skills required. “What made a good QA engineer 5 years ago may not be relevant now. Are we keeping up to speed ? Have we acquired to tools and the tech and the talent necessary?
Convening after a brief break, we take our seats again as KV Kuganathan, Treasurer of BCS’s Sri Lanka Section Committee, takes the stage. He begins by showing a cup of milk tea on the presentation screen. “Just now, they served milk tea here. But if I was a diabetes patient, would I have gone for the milk tea? No. Would you prefer a single choice – which may not fit – or a lot of choices?
Customers expect many choices these days. How do we do this? Keep improving yourself. Don’t have a wall around you: don’t think “this is my QA process” and finish that and go home. Bring in new ideas. Bring in improvements. Look at the big picture – look at the product from 30,000 feet and see how it can be improved.”
He uses Virgin Atlantic as a case study. No lost or damaged baggage, clean toilets, comfortable seats… “Connect yourself in your testing and identify the critical points.
“Think of a house,” he says. “What is the critical point of building a house?”
Silence from the audience. Then a few answers, tentatively spoken. However, they’re all surprised by the answer he gives:
“The toilet.”
Just think. That is a critical point. The toilet, the kitchen are critical points. Just like in software you bring together many modules, here you bring together the user, the water, the water-proofing, the soundproofing, the ventilation – you have to get this right. If you get these wrong, you’re going to have a bad time.
The same thing happens in software. You need to spend more time on your planning and devote a lot more attention to the critical points. This comes from experience. The difference in cost between finding a defect at the beginning of a process compared to the end is a factor of 100,000. You can’t go an tell your customers “I’m sorry.” You have to understand the endpoint. Think twice, thrice, four times and test properly.”
“One famous bank in Indonesia once made a small mistake in 2002, when they introduced Internet Banking. Hackers took them to the cleaners and drained them of every cent. They failed to test properly and as a result, because of two people, they had to shut down their Internet banking. Period.”
Interspersed between the this and the next came an extensive introduction to automated testing, covering both the process as well as popular OTS tools used in the process.
Chrisantha Silva, representing Sri Lankan airlines and CSSL, same next. His speech served as a general introduction to QA, compared to the very specific points raised by the previous speakers.
“IT has become an integral part of our lives. We need to be in control of the software rather than letting it govern us,” he began outlining the traditional bane of software – cost and schedule overruns and customer dissatisfaction, the ever-present clash between requirements specifications and actual deliverable. We feel that his speech would have served much better at the beginning, and the concepts expressed should be pretty easily found on Wikipedia, so in the interests of maintaining article lengths, we’re not going to replicate it all here. We will however share this bit of wisdom from his speech:
“Remember: software is logical rather than physical. You cannot see it. You cannot feel it. In a house, you can see and feel the build quality as you build it. In software, it’s not so easy. Testing does not imply the absence of errors: it merely indicates the presence of errors.”
Up next, Cambio has an interesting drama: a customer being served by a lady waitress illustrating usability – in short, how to be usable and how not to be usable. UX, basically. It’s also laced with a bit of innuendo to keep us awake. You’ll see a few snapshots for it when we plug in the photos.
Sisira Kumara comes next, introducing SLSTB (built upon ISTQB guidelines) and the need for it – one of the problems being the lack of education geared towards software testing. His comprehensive introduction to the ISTQB, its qualifications, criteria, workings is an excellent point that every QA tester should know of – drop by www.istqb.org for the details.
The day ended with a panel discussion based around busting the myth of ‘if you can’t do development join QA” and outlining career paths.
Per Sletmo recounts how he was invited to the house of a Test Manager at Microsoft. The house in question occupied an area of 400 square meters by the seas: he drove one Porsche, one Ferrari, four motorbikes, two boats, and all bought for his earning – and at the age on 32. “I don’t think this can be done in Sri Lanka or Sweden, but this shows how much potential there is in QA.”
Recommended reading (from the panel): 1) How we test software at Microsoft and 2) How Google tests software. Hugely informative.
GIPHY App Key not set. Please check settings