Fighting for Intelligent Solutions that Save Lives
The 1,600 miles of shoreline around Lake Michigan are idyllic. Dotted with beach towns, sandy dunes and resorts, the region is a popular vacation destination. But it’s also very dangerous, says Nick Pabon, a firefighter at the Michigan City Fire Department in Indiana.
“We get called out for missing kids, drowning victims, boat wrecks. We get all sorts of calls,” he says. “There have been many calls where the description that we get is very vague and we don’t know what we’re going into. Had that been different, there is a possibility that outcomes could have been different.”
When outcomes are determined by minutes or even seconds, quick, accurate information is critical, he underscores, which is why his fire department and others are requesting to implement technology created by Notre Dame Professor Jane Cleland-Huang.
Cleland-Huang is the Frank M. Freimann Professor of Computer Science and the department chair of computer science and engineering. Since 2016, she has been the lead researcher on DroneResponse, an AI and software project that can manage, control and monitor cohorts of semi-autonomous drones.
“To be semi-autonomous, it means that they’re able to make a lot of decisions for themselves. For example, they can use their on-board computer vision to detect people in the water, they can plan their own flight routes and they can figure out how to coordinate joint tasks together with other drones,” Cleland-Huang explains. “Instead of requiring a human to plan absolutely every action that the drone takes, the drones can be dispatched really quickly when an emergency occurs. Even as they’re taking off they are starting to engage in the mission.”
The goal, Cleland-Huang underscores, is not to create a mob of independent drones like in a sci-fi movie, but to create extra members of an emergency response team. These smart drones can provide valuable information to people on the ground, like the potential location of a victim, without taking away valuable people and resources.
“Combining drones and artificial intelligence can be disconcerting, but with the right technology we can use them to help people and to save lives,” Cleland-Huang says. “The way we think about it is that they should be behaving as partners in a team. So we want them to collaborate with people. Not just do their own thing, but to have the capability of acting autonomously whilst serving as real partners alongside human firefighters or other emergency responders.”
For Pabon, the chance to add drones to his team, all of which can be controlled by one person on the ground, means better chances for the people he serves.
“More drones in the air is more eyes in the sky. More eyes in the sky is more information. And more information for us is a better response,” Pabon says. “It means an increase in survivability for the people we’re responding to.”
Given Notre Dame’s proximity to the Great Lakes and rivers, water rescue has been top of mind for application purposes, but Cleland-Huang also mentions missing people and fire rescues as programmable mission sets. The work has caught attention nationwide and is being funded by the National Science Foundation and NASA.
Cleland-Huang is acutely aware she isn’t alone in the drone industry. But she believes her software experience has carved a niche area that makes her work and its applications unique.
“Many people that are working in the drone space are more interested in the robotic side of it, but one of the things that’s different about our team is we’re a software engineering team and so we think about the much larger picture. We not only think about the technology for getting drones in the air, how those drones will interact with people, but we also think about the ethical and societal impacts of releasing or putting drones into emergency response scenarios,” she explains. “I think that that is something that Notre Dame is particularly good at and cares about.”
With the help of Notre Dame’s IDEA Center, the technology is now commercializing as Drone Response, a software company led by Bill Reh and Brig. Gen. Shannon O’Harren of the U.S. Air Force Reserves.
O’Harren has worked with both the Air Force and NASA as an intelligence officer and is now responsible for intelligence, surveillance and reconnaissance application for active and reserve troops worldwide. Reh has a background in AI technology and has helped launch six startups. He was pulled onto the project by his son, Hunter, who just graduated from Notre Dame’s ESTEEM program. His hope is to expand the market and missions of Drone Response.
“Drone Response is a next-generation AI UAV software company building off of the original idea, but also adding a lot of new technology to it. We’re adding an augmented reality simulator that features meta learning as well as adaptive swarm learning. This is really an evolutionary technology,” Reh says. “Our aim is to make this technology the universal standard for all drones, all robots and all submersibles, so that they can interoperate seamlessly with one another. We’re designing this advanced software to be a very easy-to-use platform, presenting a common operational picture to deliver life-saving technology that can also protect and preserve the environment and collect unique data at the edge through all the applications that we’re bringing to market.”
While the product will be demo-ready by the end of the year, Reh says Cleland-Huang will stay on board to advance the drone capabilities like computer vision, automatic air leasing and software compatibility for drones of varying manufacturers. As the capabilities expand, so too do its applications. Reh cites Department of Defense, energy sector and climate change initiatives as aspirational partnerships in the near future.