How to make a ship see and hear?

Researchers tested how different sensors on board can automatically detect objects at different distances to bring more safety in seafaring. A combination of different sensors and artificial intelligence can detect a ship or a small buoy from few meters to few kilometres away under different visibility conditions.

Laserscanning data featuring a cruising ship.
A lidar can detect well objects that are close by.
Photo:
Tuomo Malkamäki

The Finnish Geospatial Research Institute (FGI) of National Land Survey, Aalto university, and Fleetrange Ltd joined forces in a research project to increase the situational awareness of maritime vessels. This brings more safety and can increase automatization in shipping. The goal of the project was to automatically identify and recognize objects, such as navigation aids, other vessels and boats around the ship, and provide improved situational awareness information through sensor fusion.

Researchers combined data from visual images, environmental sound recordings, radar and lidar ranging, satellite navigation, and vessel transponders. The tests were conducted on the Megastar ship of Tallink Group in December 2020. Sensors were installed on Megastar and the data processing was performed using innovative machine learning and sensor fusion software.

AI and camera detect other ships and navigation aids up to 5 kilometres away

Tests showed that processing camera images with artificial intelligence can help detect ships and navigation aids up to 5 kilometres away.

‘Our tests onboard Megastar showed that camera-based detection and ranging of ships and navigational aids is currently effective up to 5 kilometres. While the results are already promising for autonomous navigation, the beauty of this is that the process can be scaled up well. With more training data and training, our artificial intelligence algorithm is sure to reach even better performance,’ says Research Scientist Toni Hammarberg from FGI.

Microphones and lidars can help detect other ships when visibility is poor

Microphones help detect the presence of other ships at close range even under low illumination conditions. The direction of the horn sound from another vessel can be estimated with 3 microphones. A fourth microphone will help in estimating the distance to the sound source.

Lidars detect objects by sending laser pulses towards them and measuring the echo intensity, direction, and time-of-flight of the pulse, allowing the construction of a 3D point cloud of its surroundings. 

‘Lidars have been proposed as a short range and more accurate situational awareness sensor to augment the shortcomings of maritime radars. Our feasibility studies in maritime environment highlighted both the promises and challenges of lidar technology. Current generation of lidars is generally inadequate in detection range, pulse rate, and angular resolution. This limits visibility of targets generally below 1 kilometre. In other words, lidars are suitable for bringing more safety in harbours but otherwise they do not help much in detecting navigation aids or other ships early enough,' says Research Group Manager Tuomo Malkamäki from FGI.

Currently available lidars are also poorly suited to harsh maritime conditions. These issues can be addressed by careful optimization of lidar implementations. In the future, vessels equipped with such lidars could form a significantly better situational awareness, as lidars provide relatively accurate 3D point cloud of the immediate surroundings.

‘Lidar data is also indirectly beneficial, as it can be used to aid AI training for other sensors. For example, it can improve short range target classification and verification,' Tuomo Malkamäki continues.

GNSS and other sensors combined bring more safety

Combining GNSS and environment perception sensors will make ships safer and cheaper to operate. Research is trying to develop vessels which will autonomously sail on high seas, navigate difficult and crowded coastal waters, and berth at the right pier, all without human intervention. This requires the right combination of sensors which can see and hear at close-range as well as detect other ships at a distance. 

This technology allows the vessel to take decisions and change its route well before a possible collision. Ultimately, it is hoped that technology will help reduce shipping accidents considerably. Passenger ships like Megastar will probably never be fully autonomous but their crews can benefit from the same technology future autonomous ships use.

‘A single sensor is never sufficient for providing complete safety-critical information to the crew. They always refer to multiple devices providing overlapping information so that defects in any one device can be easily identified and excluded. An autonomous navigation system should work on a similar principle,’ says Research Group Manager Sarang Thombre from FGI.

The Maritime AI-NAV research project was supported by the European Space Agency (ESA). ESA’s overall objective is to improve European know-how in the field of autonomous transports and to study how European space-based positioning and navigation infrastructure, such as Galileo and EGNOS, can contribute to enhancing scientific innovation and consequently business opportunities in this field.

 

Please note that the views expressed in this press release reflect the project partners’ opinions and can in no way be taken to reflect the official opinion of the European Space Agency.

More information

Research Scientist Toni Hammarberg, +358 50 409 1507, firstname.lastname@nls.fi

Research Group Manager Tuomo Malkamäki, +358 50 353 1887, firstname.lastname@nls.fi

Research
Sensor technology
Navigation

News and articles