About breakingembedded

Jon Hagar is a software engineer, tester, and manager supporting software product integrity—with a specialty in embedded software systems. For more than 30 years, Jon has worked in many areas of systems and software engineering, specializing in testing/verification and validation. Projects he has supported include: the space shuttle, large rockets, interplanetary spacecraft, mobile devices, and ground systems (IT). He has experience in the software domain of real-time, reactive embedded control systems as well as test labs and tool development. Jon is a member of the Association of Software Test (AST) and IEEE as well as being an editor, voting and author member on numerous ISO and IEEE standards committees. He is past member of professional society and certification boards: Software Quality Users Group of Denver (SQuaD), and American Software Test Qualification Board (ASTQB). In addition, he frequents forums where he constantly learns, grows, and experiments in first-of-a-kind ideas and adventures. Testers will want to have Jon's book "Software Test Attacks to Break Mobile and Embedded Devices" from CRC Press, due to be published in September 2013.

Wages and High Tech

High Tech is changing the world. The recent Brexit and US election votes show that many voters are unhappy because the think their jobs are or have been taken away, usually going out of country or by some “cheap” labor.
Well, those of us in software/tech are in part to blame and the trend will get worse. Robots, automation, software doing more, etc are taking many jobs away and have for several decades now. This trend will continue.
100 years ago, people worked on farms or product lines as crafts people. The work week was 60 hours (standard) or more. This changed companies like Ford and others that slowly went to better pay, more benefits, and 40 hours (standard).
I believe we need to think about a similar change in the next few years. Maybe we should have 20 or 30 hour work weeks. We should pay “more” for various “service industry” jobs where people want to interface with people and not machines.
This will be a slow change. People will need to be retrained. Countries will need to make sure all workers and voters feel valued (compensated fairly)
On the people side, we will all need to learn what to do with our “free” time. People will need to learn things like art, music, sports, travel, and how to have “fun” when they are not providing value to society.
Some may see this as the “StarTrek” society. It would be exactly like a TV show. There will be issues and change. However, humans are good at dealing change.

Last Conference of the 2016 – Dev focus

http://www.oredev.org is on my schedule for next month. It is in Malmo Sweden and is my last formal conference for 2016. The conference has a developer focus, but has a test section, as any tech conference should. I look forward to talking about what is hot in IoT and embedded testing. IoT is getting a lot of press, and ideas are growing quickly particularly in computer power and how we keep humans safe. I am talking on embedded/IoT safety, general challenges, concepts in Math, Modeling, and AI as applied to testing. I’ve been talking safety, security, and testing for many years, but we all have things to learn, so while I talk, and I hope to learn more than I give. There are a lot of good sounding sessions. Maybe they can help my on my IoT project I’m developing.

Have fun.

Testing and Self driving cars per USDOT

I have written about self driving and embedded software in cars. Now the US Government is stepping up via the USDOT (dept of transportation) on auto-drive cars. And much to my happiness, Validation (and testing and Verification) made their list of things the gov cares about as “must do” to companies. Now there is a more detailed guide from USDOT and a lot of work to happen before we have “smart” cars on the roads generally for the public, but they are coming and getting smarter.

Such cars use embedded, mobile, IoT, AI, and other concepts along with sensors to work. We have questions about using “traditional” life cycles or Agile/Dev-Ops to develop cars. partnerships between hardware (transportation companies) and software (think high tech) are forming so each brings expertise.  I don’t have big preferences between these life cycles and companies. I more support a hybrid approach (use the right parts from each them to form an organization building “smart”), but the measure (covered by USDOT) will be to see if these smart cars reduced accidents and death. A measure has been defined, but will take time to be reached.

I think concepts coming from groups such as ISO and IEEE standards should become part of the USDOT details. This is not to say that standards are the complete answer, but they may be a beginning rather than another branch of the government defining their own standard from nothing (recreating the wheel). ISO and IEEE standards may have concerns surrounding them, but they do have some history. I see companies already having “test drive program”, which make the news, but testing/V&V is so much more than just field testing. The complexity and data space of these system is large. We will see what HAV testing/V&V industry comes up with.

So what standards to consider?

Here is my list: IEEE1012, ISO12207, ISO15288, ISO/IEEE29119 as starters. They all need to be tailored to be useful but again, they are a start.

There also concept like test automation, test modeling, math based testing, and exploratory testing with tours and attacks done by skilled testers, which need to be consider too. Then there is the testing/V&V of qualities such as trust, reliability, safety, security, system interaction, etc. Just today we saw hackers taking control of a Telsa remotely (with a new software patch from Tesla pushed right away to fix the hole).  There will be more of this, so rapid software updates should be part of the life cycle.

So if you are skilled and thinking tester, the future looks bright. But even if standards are used, we can NOT be script bound testing robots where we check only requirements. People will die (have died) in these “smart” cars.  We must use many testing strategies and basis concepts.

We’ll see where things go.

Software Testing Schools: All we need is world peace.

At a conference last month Julie Gardiner did a “lightening talk” about the so called schools of software testing (STAREAST 2016 lightning talks). In Julie’s talk, she advocated that the test industry needs “world peace” between what some call the “schools” of software testing. This call arises from debates I have been witness to, some negative postings to social media, and a certain amount of “ranker” between members of the software test community.
For those that don’t know these test schools by name, I currently classify them as: the test process school (a focus on written standards, certifications and defined test practice concepts), the quality assurance school (a focus on checks, audits, and policing that project plans/procedures are followed), the academic research school (a focus on research and a belief that test is a technology problem that can be solved by tools and techniques), and the context-driven school (a focus that test is a human intensive intellectual activity and that the approach to testing will be different for each effort with no “best” practice). There may be other names or divisions that some might choose. These are my names because some of the names used originally and more recently are not all that complementary to members of a school. A bit of reading and research will enlighten you about each school, the identified characteristic, and membership.
At conferences when the subject has come up, I hear “regular” testers (not affiliated consciously with any school) say “I do not care about school classifications. I just want to do a better job of testing”. However, in part this is why each school’s classification exists, because what each school puts forth are in part their answers to the desire of “doing a better job”. The approach of each school can be 50% or more in common with other schools while the real differences can be subtle, these differences are out of the scope of this blog posting.
I do agree with Julie in large part. The testing world needs “world peace” and what I would classify as civility. The test industry needs communication, discussion and even debate between the schools as all mature (we are a young profession). On the topic of being civil, name calling and glittering generalities solves nothing—let’s leave that for the current U.S. political contests. We are a profession, and disagreement–while a healthy part of discourse–should be considered with professional curiosity. We do need “test world peace”.
Julie proposed having the “all schools” school which could be comprised of people that move back and forth in the schools, while learning and helping each other be better testers. I agree as this has been mine and Julie’s practice for years. This has earned us the “privilege” of becoming a target of some members of schools who brand us various non flattering names, which I will not grace here. Name calling and personal attacks are unprofessional and certainly not peaceful.
Julie, I, and the many possible members of the “all schools” camp, would love to have you join us, but you really don’t need to if you just want to “get better at testing”. We support, discuss, learn, and hopefully advance each school we travel to and within. I regard the “all-schools” camp as a step in world peace, yet recognize that each school still has room for improvement. There are good and not so good testers in each school. Wanting to have exclusive membership in a school where members must adhere to the “rules” of a school to be a member in good standing sounds like an ideal that I would violate. There are schools in music: such as: classical, rap, blues, jazz, rock, and many others. The artists I most like and respect are those musicians who know the rules of a music school, play in that school but may then violate many of the rules by playing within another school. They and I recognize that it makes them better musicians while not being stuck in one genre. I think professional testers can learn from their example.
Come join us or not, but don’t listen exclusively to any negative rhetoric coming any school. This has been just one testers opinion.

Standards and their use

With all the upset a few years back about ISO 29119 software testing standard with words about it being a money making venture (see earlier post), it being to “old” school, heavy weight (documentation, process, etc) and other issues. I finally heard another good point I thought was possibly valid. It was stated that the standard lacked “accountability” with what it required. This is true and something for me to think about, but I wonder if a standard should require accountability. The idea of the standard is to provide a baseline set of ideas about testing (not best practice, not state of the art, not bleeding edge art, and not even average-poor practice). Once you have a baseline it can be subjected to review, analysis, and improvement since baselines are never perfect.
However the issue of accountability is interesting and maybe does not belong in the standard itself but to users of the standard. For example, a company might require the use and that a project be accountability to the baseline once tailored. This would be true of regulatory and contractual (company to company) use. These are done outside of the standard by interested stakeholders. I agree our software industry needs accountability but I am not sure if a standard-baseline should require accountability in it directly but should be done from the outside mechanisms.
More thought is needed.

Bugs in map programs still can be dangerous

A few years back, a navigation program on a hand held device had a bug.  I waited until things appeared to be fixed before I blogged about it, and I will not mention names nor devices here, but my wife and I saw problems happening several times over a year or more.

First, here is some background.  Our house sits in the mountains of Colorado, is well off the main highway on a 4-wheel drive road that in the winter can have snow drifts of 1-to-3 meters and it has drop offs down a cliff where a car would roll several hundred meters.  About 4 km from our house is a hot springs resort that is very popular with tourists but the road to the hot springs is in town and is definitely not our road.  The road to the hot springs has different turns (both turn to the right off the main highway) but the distance between the turn offs are about 4 km.  The road to the hot springs is very safe and well maintained—even partially paved; our road is dirt and often it looks as though no one lives on it since it is not as well maintained as city streets.  We DO NOT advise people to come up our road unless they have a 4-wheel drive vehicle (with very good snow tires in the winter) and they are comfortable driving “off road.”

For about a year, we would find people coming up our road, which essentially dead ends at our house in winter, and they would be looking for the hot springs.  After the first time or two, we asked how they got to our house.  The tourists said the navigation program told them to turn on our road.  This can be very dangerous. We have seen people in 4-wheel drive vehicles become stuck on our road during snow storms and had to spend the night in subzero temperatures inside their vehicles.

One time, several Japanese tourists drove up our road following the directions from the navigation program and even came into our house looking for the hot springs!  This greatly upset our 2 large guard dogs who might have attacked the tourists—as well as my wife who owns a shotgun.  All very dangerous.

So a simple embedded navigation device and the supporting map data caused more than just a bug.  It could have cost people their lives.  (What if my wife had shot the Japanese tourists as intruders?)  We believe that the problem has been fixed, but as we move into IoT, big IoT data, and device dependency, testers (and developers) need to keep in mind:  IoT BUGS can KILL.

ISO 29119 and rent seekers

I have already written a few things on the ISO 29119 software testing standard. I am the IEEE project editor for this actual suite of testing standards.

One item I have not addressed that the loyal opponents complain about is what they call “rent seekers”. As I understand it, these are people whose primary motivation for doing something is because it can make them money with minimal (or no) benefit or it can have a negative impact on the industry as a whole. Various standards writers, training providers for certifications, tool vendors, and others get this “label” . While it is true many people do these things to make money (we all have to make a living), in my opinion, there is nothing wrong with making money if there is an overall positive benefit from the activity being performed.

Now it is possible to debate if ISO 29119 in the long run will have a positive benefit to industry. Studies and many project data points are required before the pro or con of the “benefit” of 29119 can be determined. My estimation is that this will take a decade or more to happen.

I can say that my knowledge of the writers and voters on ISO 29119 tell me that many of them have yet to receive any positive “rent” income. In fact, many members of the ISO working group have spent far more money to attend meetings and do the work than have actually gained. As for me, I have spent thousands of my personal monies to write, present, and vote on the standard than I have received in compensation. I do not expect this inequality to change any time soon nor am I looking to cause my income to change with the implementation of this suite of standards. My work on ISO 29119 has simply been to provide some (not all) in the test industry a worldwide standard and a beginning point to improve the industry.

Note, I did not say there would not be people and companies that do not make money from 29119 or other standards. I am sure there will be training, audits, and consulting as a result of the adoption of ISO 29119. There will also be people that make money “fixing things” when part of a standard such as 29119 goes wrong, as every “ideal” can be subjected to misuse (in the wrong hands or with misbegottten intent).

We all must make a living, but I for one won’t be making much money of ISO 29119.