Report comment

They need to check the AI system by running in parallel with the 'old-fashioned' way of doing things in a few cases. Only then will they be able to tell if the AI system is doing all that it could and should. There is no point in it being able to work 2,000 times faster than human beings if it misses vital aspects of the process.

The trouble with AI systems that learn from experience is that those running the systems may be entirely unclear about how the learning process works. Programmers investigating an AI system that was supposed to distinguish wolves from dogs in photographs, but which had misidentified a dog as a wolf, found that, when they wrote a programme telling the AI system to identify in detail what it was looking for when it misidentified the dog, the system was ignoring the the part of the photographs containing the animal and focusing on the background. It turned out that, because wolves were mainly pictured in snowy landscapes, it had 'learned' that the key issue was the background snow and the misidentified dog happened to have been pictured in a winter landscape.

We need to be very careful with placing too much reliance on AI systems when the reality is that they have no human experience to rely upon and cannot appreciate how human beings view things (including our being able to spot when a system is churning out garbage). Their learning processes are functions of programmers' algorithms that may, or may not, work well. For a start, even the best of programmers needs to understand what the task being performed entails from the point of view of the end-users. Without that understanding (which experience suggested is often lacking), the AI system being programmed has little chance of meeting the requirements expected of it. Critical aspects of the criminal justice system may not be the best place to start using such systems.