Fatigue and distraction, says Kyle Wilson, Senior Scientist and Team Lead, Human Factors at Seeing Machines, are “massive contributors to crashes and driver fatalities”. Consequently, fatigue and distraction detection solutions and cutting-edge driver monitoring systems (DMS) are being adopted at pace by long-haul commercial fleet operators.
Camera-based driver safety systems, such as Seeing Machines’ Guardian, which has artificial intelligence built into it to monitor a driver’s attentiveness in real time and intervene appropriately if it detects fatigue or distraction to prevent crashes, “are almost unanimously agreed to be the most effective way to recognise driver distraction and fatigue,” says Wilson.
The benefits to drivers are clear, he says. “The big one is obviously safety. Drivers get home to their families, plus it adds an element of protection for other road users who share the roads with vehicles that have these systems.” But there are a range of other factors that fleet customers value too, including costs associated with damage to their assets – the trucks and goods they are transporting – following a crash. Preventing crashes saves long-haul fleet operators significant amounts of money, and there are insurance benefits to consider too. “Our customers can potentially negotiate better insurance premiums because they have this technology in the cabin as that improves their overall safety rating.”
One of the reasons behind that is the sheer depth and clarity of data and insight that fleets now have when they use a system such as Guardian. “As well as real-time protection in the cab the drivers are getting, the fleet operators are also empowered with data because they can identify trends for fatigue and distraction over time,” says Wilson. “They can identify high-risk shifts and routes for drivers and understand the impact training interventions can have in reducing fatigue across a fleet.”
At the driver level, Guardian often helps to identify drivers with chronic sleep issues, says Megan Mulhall, Human Factors Research Scientist at Seeing Machines. “A lot of fleets that use Guardian talk about drivers having identified sleep apnoea issues [when breathing stops and starts as a person sleeps] that they have. They may have had these issues for years, but until they actually have Guardian in their cabin and begin recognising that they’re having a lot of fatigue events while driving, they never had any impetus to see a doctor. Drivers often get diagnosed with sleep apnoea, as a result, and can get help for it.”
Having these devices is often an important catalyst for conversations with drivers as it helps customers to get to know their team better, says Wilson. “What works for one individual might not work for another. This way, you can tailor shifts for drivers. Better managing shifts to suit individuals is something that is a definite advantage.”
The real value of real-time data
The real-time aspect of Guardian’s technology is a significant advantage. Many competitor systems simply make a recording and provide post-event analysis for operators. “Having Guardian in the cabin is not just about detecting a fatigue event and reporting it to a customer; it actually intervenes in real time. Whether that’s a driver who is fatigued and having a microsleep or a driver who is distracted, the in-vehicle system is not only a sensor that sits on the dashboard of the truck and monitors for events such as this; it also has a human machine interface [HMI] that delivers alerts to the drivers to prevent a crash – for example, waking a driver up via a vibration motor positioned under their seat, along with an audible alert used to varying degrees depending on the severity or type of event happening to each driver,” says Wilson.
Real-time interaction also helps the drivers directly in that it can work like a biofeedback mechanism, says Mulhall. “Even if every in-cab alert is not necessarily preventing a crash, Guardian can make the drivers more alert to their own fatigue. It can help them recognise a pattern in their personal symptoms of when they’re getting fatigued, to the point where they might be becoming at risk, so they can then choose to intervene themselves, before the device needs to.”
That point was communicated in a research report for the National Heavy Vehicle Regulator (NHVR) in Australia in 2019. The regulator tested various devices, including Guardian, and the feedback saw many drivers praising the system’s ability to function like a biofeedback mechanism. “They found it helped them to identify their own cues around fatigue. That’s pretty significant for a driver,” says Wilson.
Following the science
It is rigorous scientific studies such as these that have helped Seeing Machines develop a deeper understanding of human behaviour, helping them to further shape how Guardian operates. “Guardian is based on decades of research into drowsiness and driver behaviour,” says Wilson. “That research shows there are observable, head and eye, and facial characteristics that are closely linked to fatigue and distraction. But it’s also extremely important that we do further research to help us to understand what genuine fatigue and distraction look like in a real-world, trucking and driving environment.”
To do that, Seeing Machines has partnered with universities globally, including MIT in the US, the University of Leeds in the UK, and a number of leading Australian universities. One such example was the CANdrive project, a A$1.35m industry development and research program funded by the Australian Capital Territory (ACT) government and led by Seeing Machines, partnering with University of Canberra and University of Newcastle. CANdrive was the world’s first automated vehicle (AV) trial to focus on driver behaviour, using Seeing Machines’ DMS technology. The project successfully uncovered key findings in the interactions between people and machines as increasingly automated vehicles are developed.
Another example is the Advanced Safe Truck Concept (ASTC) study, which was funded by an Australian Government Cooperative Research Centre Projects (CRC-P) funding scheme. “We were able to investigate fatigue from simulator studies, where we can get people to the absolute extremes of drowsiness but in a safe environment. In a driving simulator, we monitor drowsiness through cameras, and systems such as Guardian, adding a range of measures – including ECG heart-rate monitoring – that you couldn’t practically include in a real driving environment,” says Wilson.
“We did another part of the ASTC project on road, where we had over 25,000 truck trips recording the drivers the whole time. That gave us more than 2 million kilometres of on-road, live operations data with [leading Australian transport company] Ron Finemore Transport. That is an absolute goldmine of data, from a world-leading study. It allows us to understand how fatigue and distraction actually manifest in the real world and ensures the algorithms we develop have robust scientific underpinnings.”
Another important source for Seeing Machines is the real world, naturalistic driving data collected from Guardian itself – and the 10+ billion kilometres it has now travelled, keeping thousands of commercial fleet drivers safe in more than 24 countries. “That data is just so important for us. It’s critical for enhancing the algorithms we have to detect distraction and fatigue and this breadth of real-world data is key for helping to inform regulation,” says Wilson.
To that end, Seeing Machines is in communication with the NHVR in Australia, as well as other regulators around the world, to discuss how regulations might change in the future. “Current regulations are pretty standard globally too. They generally rely heavily on knowing the characteristics of a driver’s shift. But we know from the data we’ve collected that there’s significant variability and drowsiness across shifts. A lot of that depends on the time-of-day drivers are working, and that’s something that isn’t well-addressed within current global regulations,” says Wilson.
Wilson, Mulhall and their colleagues are getting significant traction with regulators, who are listening closely to their findings. “We’re looking at driver monitoring and supporting hours of service regulation from the perspective that Guardian can support, rather than necessarily fundamentally change, the driving experience – but the two can be quite complementary,” says Mulhall. “A driver might be able to have more flexibility in their driving hours if they have the support of a DMS. But we’re not going to suggest that if you have a DMS, you can drive indefinitely. It’s more of a complementary system to provide flexibility without compromising safety.”
Understanding those ‘human factors’ behind the driving experience is crucial for Mulhall and her team. “Guardian is backed by a lot of research and data, but it’s also about understanding people too. We have the 24/7 Guardian Centre that uses humans to review all fatigue and distraction related events. We know humans are good at telling whether somebody has their eyes closed because they are actually fatigued, for example, or having a microsleep. We have the real-time in-cab alerts, so that the driver is alerted straightaway, but we also have the human review, to confirm that. It really helps to complement the technology with a true understanding of other humans to provide the best possible outcome for drivers.”
Feedback from the fleets
The impact Guardian has had on commercial fleet operators – whose drivers have been known to exhibit scepticism to other DMS iterations in the past – has been impressive. “There is often apprehension from drivers in the trucking industry about the introduction of devices like these,” says Wilson. “That’s one challenge customers have. But there are certain things Guardian does that really helps with driver acceptance.”
The biggest of these is that Guardian is highly effective. “A camera-based detection system is the best way to recognise fatigue and distraction. When drivers actually experience it waking them up in the cab, that’s often a turning point for them,” says Wilson. “There’s families of these drivers who appreciate knowing their partner, mother or father is in the cab of a truck with this system.”
Another key acceptance point is that, as Wilson asserts, “this is not CCTV”. Only suspected fatigue or distraction events detected by the in-vehicle sensors are sent through to Seeing Machines’ trained analysts as video snippets, where they are then classified. When there is a confirmed fatigue event, an action plan is carried out according to a customer’s pre-arranged, bespoke plan. “Once the fleet operator sees evidence that it works for them, they don’t want to let it go,” says Wilson.
Across the 10 billion kilometres of on-road data Seeing Machines has analysed, there have been more than 13 million distraction events detected and more than 240,000 fatigue events in the last 12 months alone. While it is impossible to quantify exactly how many accidents have been averted due to the technology, it is certain that Guardian has saved countless lives.
“It brings peace of mind – it’s that additional layer of protection, but it also demonstrates to drivers that their safety is being taken seriously,” says Mulhall. “It can be a really valuable thing for the communication between drivers and fleet operators and gives everybody a better understanding of the fatigue risk management framework as well.”
Customers value Guardian because it is backed by robust evidence. “We engage in a significant amount of research to improve our algorithms and ensure we have this deep understanding,” says Wilson. “We’ve also conducted studies which have shown just how credible Guardian is. One of those was conducted in 2017 and published in a peer-reviewed international journal. We compared a baseline period where we had the sensor in the vehicle, but it was functioning silently and not alerting drivers. We compared it later to a period where the alerts were being delivered. The study was run over four years and it involved multiple trucking companies. We found that there was a 66% reduction in fatigue events alone after the alerts were turned on. Beyond that, when you run the product as the full service, and have in-cab alarms, with Seeing Machines contacting the customer to take action, that fatigue intervention plan reduced fatigue events by more than 94%. That definitely adds credibility to the effectiveness of Guardian.”
DMS technology evolves
So, where does the technology go from here? “We’re always looking to improve how advanced our system is and to capture other behaviors that fleet operators recognise pose a safety risk to their drivers. Fatigue and distraction are very complex and detecting these states in the real world across a variety of people and situations make it even more complex. So, we’re doing a lot of research that is very forward-looking. We’re well down the path of a next generation of Guardian that will be physically more compact and will bring in the automotive grade algorithms to improve the detection of fatigue and distraction, as well as improving the customer experience, and meeting compliance in Europe,” says Wilson.
“We’re well ahead of any other option in the market to provide that. A lot of our competition is telematics companies, or companies that provide post-event analysis around fatigue and distracted driving. The beauty of Guardian is that it works in real time.”
