Preventing Robo Debt Happening Again

Preventing Robo Debt Happening Again

The evidence given by a former member of the AAT is pretty damning

We can’t claim that mechanising the Social Services legislation as a way of avoiding the Four Pieces Limit will prevent the behaviour described. What we can claim is that creating an authoritative source, where all the meanings of all the words have been clarified and agreed to when the legislation is written, will reduce the possibility of such behaviour occurring in the future. The legislation becomes activatable, so it is no longer just words on a page, the words become nodes and operators in an undirected network – an Active Structure. It can make calculations or deduce conclusions exactly according to the legislation – there is no human involvement by someone insufficiently au fait interpreting the words or trying to turn the words into a program. 

The machine reads the legislation, makes all the necessary connections, and turns it into an active network. The approach is not suitable for handling millions of accounts, but it is suitable for testing theories, handling contested accounts, or debt calculations.

This is not Machine Learning – there is one document to be read, not thousands. The machine has to know the meanings of the words before it started. It is using definitions from a well-known dictionary, extensively curated where the dictionary is inadequate or suffers from circularity.

The techniques of ML, DL, LLM are suited for regurgitation, without any understanding of what the words mean, only that words follow other words on a statistical basis in a particular sort of text. In an Active Structure approach, each word has all the meanings that English gives it – 62 for “set”, 77 for “on”, 82 for “run”. A little tedious to set up, but then visible to anyone who needs help in understanding the legislation – particularly useful where the line staff have little or no legal training.

Robo-debt Problems

Robo-debt Royal Commission

(based on a report in The Saturday Paper, by Rick Morton, published December 10)

The RRC heard evidence that the person in charge of Robo-debt had never read the Social Services Act, which is quite clear that payments should be based on a particular fortnight, and not averaged over a longer time, as Robo-debt sought to do. Several people strongly suggested this was so, making the robo-debt scheme unlawful, but each time they were overridden. The opinion of a consultant was sought, by colouring the meanings of words, as one means of discrediting internal dissent. The Ombudsman’s efforts in response to concerned politicians and Social Service groups were thwarted, by allowing the people being reviewed to alter the report on their activities. It took three years for the question to reach the Solicitor-General, who immediately declared it unlawful, with several suicides and much distress among impoverished people having occurred in the interim.

What to do? A person keeping up to date as several Acts evolve over time is a large commitment. The commitment can be considerably reduced by having a machine read and “understand” each of the Acts, including their glossaries, their inline definitions of terms, and references to each other, potentially having different meanings for some words across the connection. It would mean that everything is linked together as it should be. If there is dissent over the wording, it can either be resolved, or left open for review. Someone coming cold to the Act in its Active Structure form can then be sure that all connections have been made, and they can confidently rely on what they read in a local area of the Act, and don’t need to read the whole thing carefully, together with its history of revision, to make sure they haven’t missed anything. Such a tool would be a godsend to tyro public servants or politicians.

This sounds easy, with the rise of technology such as Machine Learning, Deep Learning, and Large Language Models. Unfortunately, the goal of having the machine “understand” the meanings of words has largely been abandoned as too hard, and most current methods use statistics – this word is associated with that word or group of words – a method completely unsuited to the complex text found in legislation or specifications.

A Few Colours

Are a Few Colours Really Going to Help?

Humans have a very powerful visual channel why not emphasise that? We can colour the words wordgroups are one obvious candidate we can indicate where there are complex objects in the text objects like

same-institution person-to-person electronic funds transfer instruction

We can do more with a wordgroup that has a definition we can show the complex objects in the definition – like same-institutionor bank account or carry out.

That is perhaps 5% of the words what do we do with the rest?

We could go through the document, work out what is the most common use of the word, then have a colour range for meanings ranging from the most common (white) to the least common (say purple). But this would look like a mess. It would be better only to colour words having an unusual meaning, or words where there is dissension over the meaning, or words in the glossary, or words that are unusual for the persons specialty (so for a lawyer, all the legalese would stay black on white, whereas technology terms would stand out), or where the surrounding text does not give a clear demarcation between a meaning in the readers specialty and other meanings.

As the intention is to deal with large, complex documents, the ability to flit through the document, looking at things that may be hard for the person to understand, should be welcome.

This facility is not a game-changer by itself, but its completion does allow turning overa piece of machinery while it is still made only of words.

What Does “Turning over a piece of machinery” Mean?

It means that words, when connected together, can be pieces of machinery, and when activated, can perform similar (although abstract) functions to the real objects they represent. So a funds transfer instruction can transfer funds from one bank account to another. The text of legislation allows a programmer to write a program, but it also allows a person to simulate the operation in their head, if it is not too complicated, or learn a complex operation, like piloting a plane, at an unconscious level. Having the machine read complex text allows it to simulate what the text says, without being troubled by the Four Pieces Limit. Getting the text cleanis an important step.