AI in the healthcare sector
Robotics and artificial intelligence (AI) are rapidly changing not only the work of insurance brokers like us, but also that of our customers, for example in the healthcare sector. In many clinics, hospitals and doctors' offices, AI-supported evaluations, robotic assistants in the operating room and many other technologies are on the rise. But what risks are associated with this and how can users protect themselves and their risks? Michaela Krause and Katja Kreylos, both of whom are in product management and also members of the Ecclesia Group's AI working group, provide answers to these questions.
Can AI tools or robotics really be used in the healthcare sector with legal certainty?
Michaela Krause: At present, yes. According to current case law and legislation, AI is not a legal entity in its own right. Behind the AI, there are always companies or legal entities that are liable. And if nothing is changed in the AI, it is completely unproblematic. The situation is different if our customers change the software. Then they could be liable. For this purpose, we have developed a digital module that specifically addresses digitization and data risks within the framework of normal business liability insurance.
How far has the use of AI progressed in the healthcare sector?
Michaela Krause: The use of AI is still in its infancy, but rapid development is already emerging because the technology is advancing enormously. We have already looked at this in the AI working group: for example, there is already an AI application in radiology that pre-selects images. AI is also already in use in most mammography screening devices. But the majority of devices still do not work autonomously, but still require a doctor to assess or operate on them.
Katja Kreylos: The core area of medical service providers is changing enormously due to digitalization or the use of AI in healthcare. Even though our customers now have texts created using AI, the use of robots in operating rooms is still rather the exception – but it is increasing. These robots are currently still operated by a human, a doctor, and are not working with their own technical intelligence. However, since technology is developing rapidly, they may well be able to take on independent tasks within the operating system in certain sub-areas in the future.
And who would be responsible if an error occurred during the operation?
Michaela Krause: Currently, the doctor is liable because the law of torts under Section 823 of the German Civil Code still applies. If it is purely a software error, the manufacturer is liable. In this case, it can lead to discussions because the doctor has the burden of proof. Therefore, you have to look at the system very carefully and check whether the doctor has actually done everything right. This is definitely a tricky area for our customers.
Katja Kreylos: If something goes wrong, patients usually always approach the hospital as the contractual partner first and then the doctor second, since the error occurred in this area of responsibility. In this case, the hospital has to carefully review all surgical and patient education reports and check which activities were performed by whom. This is because we are talking about joint and several liability here, so that not only the hospital but also the doctor personally can be held liable and made to pay damages.
Michaela Krause: These are currently the existing risks that we can very well cover with our building blocks of the policyholders' liability insurance, primarily of the hospitals. Therefore, our customers do not have to worry at the moment. But we have to continue to follow this topic closely and see which instruments and software our customers use and, if necessary, develop themselves.
Where do you think AI will be used in the healthcare sector in the future?
Michaela Krause: For example, 3D printers are currently being used to create special prostheses that have been developed using the technology itself and are individually tailored to each patient. Large hospitals and university hospitals are pioneers in this area.
Katja Kreylos: This also applies to intramedullary nails, which are used, for example, to treat a broken thigh. There are special cases in which no standards can be used. In this area, 3D technology will open up completely new possibilities in the future. In addition, diagnostic apps with image recognition software and patient apps are conceivable, in which an AI reflects the diagnostic process and makes it available to the patient. This would massively relieve the burden on doctors and their staff.
Are there any highly critical areas in the healthcare industry that require special protection?
Katja Kreylos: One particularly critical area, for example, is bionic prostheses that are firmly attached to the body. They can relatively quickly lead to personal injury. That is why an extra module for extended product liability insurance is required in this area for public liability insurance.
Michaela Krause: This is because a new operation is always required to remove a defective part and to re-implant a defect-free part. Since this is not covered by the patient's consent, he or she can claim damages.
How can these liability risks be minimized?
Michaela Krause: With professional risk management. Clinics and hospitals in particular have already developed an awareness of the problem in this area, which reduces the error rate – also because special attention is paid to specific sources of danger.
Does Ecclesia already offer special insurance solutions for the use of AI?
Michaela Krause: We have developed a digitization and data risk module and, in addition to pure product liability insurance, an extended product liability, which are usually components of our general liability terms and conditions. When new technical risks arise, they have to be taken into account, incorporated accordingly and renegotiated with the insurers.
What conclusion did your working group come to?
Michaela Krause: In the liability area, we are currently very well positioned when it comes to AI and have enough options to offer our customers the necessary security. The technologies and their areas of application are developing rapidly. We are keeping an eye on this.