Balancing Act: Software Quality Vs. Time To Market
Financial firms often rush software products to market with less regard for quality than for time to market. In their haste to release new technology firms focus first on development, and then hurry the quality assurance (QA) testing at the end. Testing tasks are allocated based on the time available instead of the time needed to do the job well. But inadequate testing can have a detrimental impact on business.
Releasing improperly tested systems can open firms up to disastrous trading errors or outages that may jeopardize their reputations and even their viability. It becomes very expensive to fix these errors after damage has been done. That's why thorough quality assurance (QA) testing is important.
Automating testing using commercially developed automated testing platforms can eliminate the need for trade-offs between quality and time to market. Automation speeds up the process, provides the ability to do more extensive testing, and gives QA departments a greater opportunity to catch defects before new technology is released to customers.
But some firms remain skeptical about implementing automated testing technology, because of the initial upfront investment involved. They think it's more affordable to simply hire a team of testers to handle the job. "It's true that in the short–term, manual testing is the lower cost option. However, the overhead involved in sustaining this approach over time is significantly higher when stacked up against automation," said Josh Tolman, Deputy CEO at CameronTec Group, a provider of FIX connectivity products. CameronTec recently acquired Greenline Financial Technologies.
[Cloud computing certainly has benefits, but how can financial firms and other highly regulated enterprises meet security and compliance demands? Learn how enterprises can structure a hybrid cloud to gain efficiencies, while also meeting compliance goals. Attend the Reigning in the Cloud – Solving Identity, Access, Compliance and Governance Issues in a Hybrid Cloud World session at Interop 2014 in Las Vegas, March 31-April 4.]
Likewise, building testing tools in-house can present considerable long-term expenses. "For example," said Tolman, "IT staff will need to devote a lot of time to not only building, but also keeping these systems up-to-date as regulations and other requirements change. That can become quite arduous. Furthermore, if building testing software is not within their area of expertise, then the quality of the system may suffer. Think about it like this: Do you really want the same people who are building your software to also be building the testing tools? In addition, what quality assurance measures are taken to ensure that the in-house tools are functionally sound?"
According to Tolman, the average firm will see a 155% return on investment (ROI) after the first year using a third party testing tool, and a 327% ROI by the end of year three. The total cost of ownership (TCO) attached to automation is not only lower because it demands fewer employee resources, but it also minimizes the possibility of costly mistakes related to lack of domain knowledge or human error. Tolman used Greenline's ROI calculator to arrive at these numbers. Of course, results will vary by firm.
Direct Edge, a US-based stock exchange operator which recently merged with BATS Global Markets, has automated its entire testing process across all functionality including its matching engines, market data, and 12 messaging protocols. The firm's results validate Tolman's estimates. Rene Ovalle, Head of Quality Assurance at Direct Edge, commented, "A lot of our ROI can be seen in our improved productivity rates and how much more efficiently we're handling testing. Prior to automation, all testing had to be completed during business hours, but now it can be done 24 hours per day, 7 days per week," said Ovalle. To learn more about Direct Edge's use of testing automation, register for a webcast where Ovalle will be discussing his Direct Edge's results.
Fierce Competition
Exchanges like Direct Edge operate in a fiercely competitive marketplace. A large part of their success is based on their ability to deliver innovative products and order types in a reasonable timeframe. But launching new solutions, new functionality or major upgrades is complex, requiring extensive development and thorough QA.Direct Edge is automating testing to improve time to market and to maintain quality. "Introducing greater sophistication into QA has helped us reduce the regression testing cycle from months down to just 33 hours," said Ovalle. "Not only is the process exponentially faster now, but it is also far more thorough." According to Ovalle, automation has given Direct Edge the ability to deliver new releases in weeks, instead of months.
While many market participants agree that automating QA testing could help them, they are reluctant to work with a technology vendor, believing it to be more expensive than handling QA internally. But the reality is that testing complex financial technology requires specialized knowledge and experience. It's neither simple nor inexpensive to find this skilled labor pool.
When you compare it to the cost of recruiting, training and retaining this type of talent; leveraging a vendor with proven knowledge in this space and skilled resources can ultimately be more affordable. Josh Nardo, Global Head of Services at CameronTec Group, comments, "Firms that outsource automated testing to an experienced provider will typically see an overall 30 to 60% in savings. Not only is the process less expensive, but subject matter experts can also deliver more sophisticated tests."
Direct Edge agrees. Reducing overhead and getting access to domain experts were the primary reasons they chose to outsource. "When setting up our automated testing environment, we elected to use Greenline's professional services. As a result, we've been able to get our testing environment up and running more quickly and cost-effectively than if we had managed the entire process in-house. Greenline's professional services team has also been very helpful in sharing their knowledge with our internal staff," said Ovalle.
In combination with competitive and business-related demands, market participants are also dealing with major compliance pressures.
Compliance is further complicated by principles-based rather than prescriptive regulations. "While regulating bodies do set forth guidelines for compliance, they have not supplied precise rules. This potentially leaves room for interpretation and a range of testing standards," said Ovalle. "At Direct Edge, we've made it a point to be a model in this space. Across our organization, we're trying to create repeatable, sustainable, adaptable, demonstrable processes that help simplify the compliance function while holding testing to the very highest standards."
For a more in-depth look at the ROI Direct Edge achieved using Greenline's automated QA testing solution, be sure to register for this live webinar.
About The Author:
Candyce Edelen is CEO of PropelGrowth, an agency helping financial technology companies develop and implement go-to-market strategies. Prior to PropelGrowth, she co-founded two financial technology companies including Selero, a company that provided order routing gateways, direct market access and FIX connectivity. She led teams in delivering FIX gateways to NASDAQ, Boston Stock Exchange and National Stock Exchange as well as dozens of other venues and brokerages. Edelen also was a conference committee co-chair for a recent FIX Trading Community regulatory briefing.
Candyce Edelen is CEO of PropelGrowth, an agency helping financial technology companies develop and implement go-to-market strategies. Prior to PropelGrowth, she co-founded two financial technology companies including Selero, a company that provided order routing gateways, ... View Full Bio
Tweet This