Software development is a challenge. Although passionate developers enjoy the process, it’s time-consuming, frustrating, and consumers expect perfection. There’s immense pressure placed on DevOps teams to ship the product in record time, so they often launch too early, before performance testing is complete.
Launching software too early can have dire consequences. Consumers are forgiving of minor bugs as long as the software performs its core promised functions. When the bugs encountered are security problems or flaws with core features, consumers aren’t so forgiving. They might give you one chance to fix things with your first released update or patch, but if their experience doesn’t dramatically improve, they’ll move on.
Broken core features and security flaws are results of limited or no testing
It’s no secret that testing takes up the majority of time in the development process. It’s often seen as the bottleneck that causes delays. That may be true, but testing can’t be skipped.
According to the Computing DevOps Review 2017, only 21% of people surveyed were meeting or exceeding delivery expectations. The majority of delays in software production occurs at the Test/QA state of the cycle. A number of organizations take security shortcuts to bypass the test/QA state to maintain continuous delivery.
Summarizing a report by Sogeti, ContinuousTesting.com shares that DevOps teams spend more than 30% of their time just getting testing environments right. The Sogeti report reveals that most organizations have difficulty orchestrating the testing process and aren’t realizing the full benefits of continuous testing.
Continuous testing is the foundation for a successful product launch
Continuous testing is the process of automating tests as part of the software delivery process to generate immediate feedback on risks and flaws in the product. It’s the opposite of what many DevOps teams do when they set aside testing as a specific task to perform at certain intervals.
The most beneficial way to test a product is to apply all testing to each sprint. For instance, if your sprint is four weeks, your goal should be to complete all testing within those four weeks so your software release is fully tested before it ships.
Regardless of how long or short your sprints are, you can apply continuous testing by embedding it directly into the software delivery cycle.
Know what bugs are acceptable in a software release
No software product will be perfect right out the gate. You have to release the first version in an imperfect state. However, major problems should be fixed before you ship your product. Otherwise, people will be mad, and your business could be ruined. For example, the Iomega Zip Drive went to market with a major flaw: the drive head would become misaligned and clip the end of the disk, destroying all stored data. That’s not a small flaw, nor an acceptable flaw to go to market with.
The “click of death” Iomega Zip Drive disaster wasn’t a fluke – many people experienced this problem. Releasing a data storage device with that kind of flaw is unacceptable. Either the company knew about it and pushed the Zip Drive to market anyway, or they didn’t do enough testing.
How do you know if you’ve released software “too early?”
The litmus test to know if you’ve released software too early is simple. If you’re not utilizing continuous testing, you’re releasing too early. The easiest way to implement continuous testing is to follow the best practices for continuous release.
Don’t rush a release to meet financial goals
Major software companies release flawed products to meet sales goals and figure they can fix things later. For example, Ubisoft released Assassin’s Creed: Unity with flaws that decimated the game’s playability.
Game reviewers with advanced copies were not allowed to publish reviews until 12 hours after the game was officially released, which some say indicates foreknowledge of the flaws. As customers discovered the glitches, Ubisoft’s stock dropped 9.12% within 24 hours of the game’s release. Had the company waited to release the game, they would have missed out on millions of sales over the holiday season.
The solution is to prioritize testing
It’s tough figuring out how to release your product on schedule with the quality expected by consumers. The answer is knowing how much testing you need to do.
Since testing takes up a majority of the process, it needs to be prioritized and written into the process from the beginning. When you plan for testing, your delivery timeline becomes more predictable and your releases become more stable.
