Kingsley Hayes writes in ISBuzz news regarding NHS waiting lists and AI

Partner Kingsley Hayes discusses the legal obstacles the NHS will need to overcome before it uses algorithmic decision making to tackle record waiting list backlogs, in ISBuzz news. Kingsley’s article was published in ISBuzz news, 1 July 2021, and can be found here.

The NHS plans to deploy AI to tackle the backlog of waiting lists exacerbated by the coronavirus pandemic. A £160 million scheme announced last month plans to implement some radical new technology-driven measures in order to reduce the NHS’s record 4.7 million waiting list.

For example, more GP patients will be given the option to be assessed by AI chatbots which can refer them on to other services such as physiotherapy. In Bristol, elderly patients are already trialing support robots which enable them to be assessed remotely. Multidisciplinary weekend clinics are also planned.

Needless to say, a careful data protection risk assessment of all such schemes will need to be conducted. The most concerning scenario would be where incorrect AI decision making led to a patient receiving the wrong treatment or prioritisation, which ultimately led to a worsening of their conditions or to death. AI does not have a great reputation for dealing with the complexities of healthcare. For example, it was reported that the QCovid algorithm designed to identify priority groups for vaccination was likely to miss those with rare illnesses.

Aspects of the scheme have been roundly criticised by the BMA. BMA council chair Dr Chaand Nagpaul said that “The Government is already aware that many of the technological innovations, such as ‘pop-up clinics’, previously launched during this pandemic have ended up actually increasing workload”. Mr Nagpaul went on to suggest that, “Technological innovation should be used to address some really basic and simple issues, which the BMA has been calling for, such as enabling hospitals to be able to prescribe electronically for patients and to have direct access to community diagnostics”.

The decision to adopt AI carries with it significant consequences in terms of data protection law. Article 22 GDPR has rules designed to protect individuals where automated decision making is being used. In this context, the informed consent of patients is key. There is as yet little detail as to how the NHS plans to secure consent. Nor do we know how such requests for consent might be framed. Nor do we know if any future use of the patient’s consent to processing by AI will be made once the processing of waiting list data is completed.

In a time of increasing public concern around the misuse of personal data, a further risk to the success of such schemes is that many patients might simply refuse to consent to their data being processed by AI.

Too often organisations see AI as a panacea against all ills. It is often touted as a miracle cure for supposedly insurmountable administrative obstacles. The result is often that organisations rush headlong to implement algorithms without adequate development and testing.

The NHS needs support to deal with the effects which the coronavirus pandemic had on waiting lists. Before AI becomes part of the solution, the NHS will need to ensure that appropriate data protection processes and technical security are implemented with great care. If this is seen to be the case, the public’s confidence in such schemes will grow, thus ensuring that the public are willing to consent to participate. In order to be certain that consent to AI processing is valid, it is also vital to ensure that patients fully understand what they are consenting to and that it is written in simple terms.

The NHS should also be careful to avoid seeing unduly broad consent as where an organisation refuses to provide services unless the consumer gives it more data than is required to provide those services, the person’s consent to process such data may be deemed invalid. Recital 43 GDPR states that “Consent is presumed not to be freely given… [if] the provision of a service, is dependent on the consent despite such consent not being necessary for such performance.”

The NHS should avoid any suggestion that consenting to having personal data processed by AI would result in patients having quicker access to services. Where consent to process data is obtained under the shadow of a suggestion that something detrimental may happen if consent is not given, such consent could be argued to be invalid.

The NHS will have to overcome significant technical legal obstacles before it fulfils the promise of AI.