Preventing Robo Debt Happening Again

Preventing Robo Debt Happening Again

The evidence given by a former member of the AAT is pretty damning

We can’t claim that mechanising the Social Services legislation as a way of avoiding the Four Pieces Limit will prevent the behaviour described. What we can claim is that creating an authoritative source, where all the meanings of all the words have been clarified and agreed to when the legislation is written, will reduce the possibility of such behaviour occurring in the future. The legislation becomes activatable, so it is no longer just words on a page, the words become nodes and operators in an undirected network – an Active Structure. It can make calculations or deduce conclusions exactly according to the legislation – there is no human involvement by someone insufficiently au fait interpreting the words or trying to turn the words into a program. 

The machine reads the legislation, makes all the necessary connections, and turns it into an active network. The approach is not suitable for handling millions of accounts, but it is suitable for testing theories, handling contested accounts, or debt calculations.

This is not Machine Learning – there is one document to be read, not thousands. The machine has to know the meanings of the words before it started. It is using definitions from a well-known dictionary, extensively curated where the dictionary is inadequate or suffers from circularity.

The techniques of ML, DL, LLM are suited for regurgitation, without any understanding of what the words mean, only that words follow other words on a statistical basis in a particular sort of text. In an Active Structure approach, each word has all the meanings that English gives it – 62 for “set”, 77 for “on”, 82 for “run”. A little tedious to set up, but then visible to anyone who needs help in understanding the legislation – particularly useful where the line staff have little or no legal training.

Robo-debt Problems

Robo-debt Royal Commission

(based on a report in The Saturday Paper, by Rick Morton, published December 10)

The RRC heard evidence that the person in charge of Robo-debt had never read the Social Services Act, which is quite clear that payments should be based on a particular fortnight, and not averaged over a longer time, as Robo-debt sought to do. Several people strongly suggested this was so, making the robo-debt scheme unlawful, but each time they were overridden. The opinion of a consultant was sought, by colouring the meanings of words, as one means of discrediting internal dissent. The Ombudsman’s efforts in response to concerned politicians and Social Service groups were thwarted, by allowing the people being reviewed to alter the report on their activities. It took three years for the question to reach the Solicitor-General, who immediately declared it unlawful, with several suicides and much distress among impoverished people having occurred in the interim.

What to do? A person keeping up to date as several Acts evolve over time is a large commitment. The commitment can be considerably reduced by having a machine read and “understand” each of the Acts, including their glossaries, their inline definitions of terms, and references to each other, potentially having different meanings for some words across the connection. It would mean that everything is linked together as it should be. If there is dissent over the wording, it can either be resolved, or left open for review. Someone coming cold to the Act in its Active Structure form can then be sure that all connections have been made, and they can confidently rely on what they read in a local area of the Act, and don’t need to read the whole thing carefully, together with its history of revision, to make sure they haven’t missed anything. Such a tool would be a godsend to tyro public servants or politicians.

This sounds easy, with the rise of technology such as Machine Learning, Deep Learning, and Large Language Models. Unfortunately, the goal of having the machine “understand” the meanings of words has largely been abandoned as too hard, and most current methods use statistics – this word is associated with that word or group of words – a method completely unsuited to the complex text found in legislation or specifications.