Dark caves and caverns without any GPS make high demands on sensors. | Photo: Jérôme André

Autonomous drones have problems negotiating their way over grid patterns, shiny surfaces and even ordinary wire-mesh fences because these are difficult to identify in flight, says Teddy Loeliger, the head of the Sensor Electronics Group at the Zurich University of Applied Sciences (ZHAW). Drone sensors must measure their distance from their surroundings on a constant basis if the drone itself is to avoid any and all obstacles. And they have to be able to do it, even when contours are blurred or objects disappear completely in fog or darkness. “Conventional optical systems soon reach their limits in such situations”, says Loeliger.

This is why Loeliger is testing innovative sensors in a small hall in Zurich. They emit infrared light, the brightness of which can be altered using frequency modulation. This infrared signal is reflected back, and the sensors evaluate it in order to determine the drone’s distance from the objects around it. This technology is called 3D time-of-flight (3D ToF).

The sensors can detect grids, fences and glass surfaces by working with multiple frequencies; they then use an algorithm to separate out disruptive interference. This results in precise 3D images. “The sensors are cameras that also provide precise spatial information”, says Loeliger. When combined with radar or with conventional colour sensors that use AI to help the drones to understand their environment, 3D ToF proves particularly useful in dark or unstructured environments.

But 3D ToF has one problem: energy consumption. The sensors have to emit light constantly, which drains their batteries. So Loeliger and his team are attempting to reduce the data rate – and thus the sensors’ energy requirements – by only using parts of the infrared spectrum. This works well for distances of just a few metres. But beyond ten metres, the infrared sensors require too much power.

Davide Scaramuzza is a drone researcher at the University of Zurich. In order to get around the problems faced by the ZHAW team, he’s relying on so-called ‘event cameras’, whose sensors are modelled on the human eye. “Our eyes don’t send complete images to the brain all the time. They mostly register changes”, he says. So one pixel in his sensor monitors only the brightness in its field of vision, and when this changes significantly, it reports a so-called event, providing the relevant information about the position and time, down to the microsecond.

Its data evaluations are based on neural networks, and they’re trained in a simulated environment. “Every single pixel is intelligent, so to speak”, says Scaramuzza. Event cameras work both in bright sunlight and in almost complete darkness. And they don’t require much data. The drones using these cameras react with less than a millisecond delay – faster than any human reaction.

Scaramuzza and his team have been setting records, and their autonomous drone has won races against professional pilots. He acknowledges the strengths of 3D ToF technology, but he thinks that event cameras have an advantage over them when it comes to long distances and high speeds. Problems arise when drones fly very fast, close to obstacles. This results in a flood of changes to the brightness – and the data rate accordingly explodes.

Replacement systems are ready

Ultimately, what matters is choosing the right guidance system for every individual application. While event cameras are ideal for long survey flights, 3D ToF has advantages in dark, confined spaces. “3D ToF is highly suited to GPS-free, tightly structured environments such as tunnels, caverns or industrial facilities, especially when it’s combined with other sensors”, says Christian Bermes, a robotics expert at the University of Applied Sciences of the Grisons. Those other sensors include optical cameras, radar, and laser light for distance measurement (lidar). If one system fails, the others can take over.

There are many potential areas of application: search and rescue missions, inspecting bridges, tunnels and power lines, forest surveillance and wildlife monitoring. Loeliger has also tested his drones in conjunction with humans. His 3D ToF sensors can be controlled with hand signals. This offers an ideal combination: humans can coordinate operations while the drone flies into dangerous, hard-to-reach areas – such as close to power lines, through narrow passages in chemical plants, or along barrier fences.