AI and the new testing challenge

The potential of AI is at the same time tantalizing and terrifying, and an ever-growing line of businesses are stepping up to explore its possibilities.

In the last week alone, Ford has announced a tie-up with Dominos for self-driving pizza delivery vehicles, while in China, Pony.ai has become the first company to operate an autonomous ride-hailing service on public roads.

But how are companies testing their AI initiatives to make sure that they work from a technical standpoint, but also to ensure that they are ticking the boxes in terms of safety, security and the thorny issue of ethical behaviour. 

This challenge has come under the spotlight in the last week, with the European wing of Softbank Robots announcing a deal with Cognizant for the latter to provide quality assurance for its Pepper and NAO humanoid robots. Softbank, which is headquartered in Japan, owns the Boston Dynamics business whose quadrupedal robot SpotMini set the Internet on fire last week.

A Cognizant team in Grenoble will develop a standardized approach for testing Softbank’s NAOqi operating system, as well as the applications that support speech recognition, movement perception and obstacle and collision avoidance. The vendor will also provide testing services for the development toolkit and for applications developed by third party partners. 

The reasons for working with a external partner are clear. Softbank appears to be growing at speed – although it doesn’t break out financial results for its robotics business units – and using a partner that can scale up an industrialized approach to testing will be faster than building it off its own bat. 

Accenture has also emerged as a front-runner with the launch of a new set of services for testing AI systems, underpinned by a methodology it calls “teach and test.”

PAC caught up with Accenture’s global testing head Kishore Durg, who said that one of the biggest problems facing early AI initiatives has been the way that agents proliferate a bias in the way that data is analysed and leveraged.  Facebook and Google are two of the highest profile companies that have fallen foul of this issue. 

Durg said that the first stage of the approach is to “teach” the agents to work on the right data sets in the right way. This involves helping clients to use the most effective algorithms and checking the training data to ensure that there is no or minimal bias, and that ethical and regulatory compliance boxes are ticked. 

Under the “test” phase, Accenture validates both the data in the test phase and in production to check accuracy, and to continuously adapt the AI model based on learning from new data. Durg cites an example where Accenture supported the development of a financial services client’s virtual agent that was designed to field and process customer enquiries. Using the “teach and test” approach, Durg claims the agent was trained 80% faster and achieved an 85% accuracy rate on customer recommendations.

AI clearly poses new testing challenges and presents a major opportunity for the professional testing community. The AI engine developers will need help in scaling their operations and ensuring that they fall in line with regional and industry specific regulation. And as Durg highlights, the potential damage to a brand that a biased or misguided use of AI will also be a powerful motivator for organizations to invest in the test and validation phase.