Field Testing – the Ultimate Test for Autonomous Driving Engineers

A vehicle drives along a straight road. Suddenly, a pedestrian steps off the sidewalk, with no intention to give way. Do they not see the approaching vehicle? Fortunately, the vehicle brakes immediately and comes to a stop. The pedestrian remains frozen in place as a test engineer in a yellow reflective vest comes up to measure the distance between the pedestrian on the road – a test dummy – and the front bumper of the vehicle. Sufficient distance is required to allow for a proper margin for safe stopping.

This is a pedestrian test where an autonomous vehicle observes the road ahead and, if needed, brakes to avoid collision with an obstacle. The autonomous vehicle also needs to understand the difference between a pedestrian on the sidewalk and one on the road. A pedestrian keeping to the sidewalk is no reason to slow down.

A test such as this where a pedestrian dummy is moved from the sidewalk onto the road is one of the most important tests for an autonomous vehicle – it’s not allowed to hit an obstacle, whether it is on the sidewalk or not.

Safety Is the Number One Priority – Even in Tests

For traffic safety and the safety of our test personnel, these tests are not conducted on open roads. We found a suitable closed environment at the Nummela airport where straight runways and taxiways are perfectly suited for testing vehicle automation. Today, light aircraft are giving way to the vehicles of the future.

A versatile team is overseeing the tests: Sensible 4 personnel are seated in the vehicle, including the human driver who is ultimately in charge of the vehicle’s safe movement. The back seat is occupied by a group of engineers more interested in the computer screens in front of them than anything taking place outside the vehicle.

One of these engineers is Sensible 4’s Joonas Humina. He has prepared the vehicles for the tests together with another engineer, Tomas Saaristola.

“First, we mapped out the area for autonomous driving”, Humina says.

Sensible 4’s autonomous driving technology relies on precise pre-drive mapping. Advanced map matching allows the vehicle to locate itself on the map in all weather conditions without satellite navigation or support from other infrastructure. Mapping is the foundation of autonomous driving.

Test Environment Models an Actual Traffic Environment

Next, the team sets up the test cases. A walkway, sidewalks, an intersection, and a bus stop are laid out on the runways and taxiways. These are situations where a vehicle must be able to react the right way, with cars and pedestrians coming and going in several directions and even violating traffic rules.

Finally, everything is ready for the actual testing. Tests are important as they go a long way in determining how Sensible 4 could expand outside of Finland. There are two international pilot projects in the pipeline in Norway, one near Oslo and another in Gjesdal, near Stavanger. These pilots will not get off the ground, however, if the vehicles don’t get an approval from Vegvesen, the Norwegian road authority.

This approval requires a dependable third-party assessment of the vehicles’ safety.

Testing and verifying an autonomous vehicle is far from simple. Any number of situations may present themselves in traffic, limited only by your imagination. Poor weather and bad driving conditions further complicate matters.

However, there are no common, internationally accepted testing practices for autonomous vehicles.

“Unified testing would be great, and it would also make it easier to compare various systems”, says Humina.

Good Expertise in Vehicles and Traffic Automation in Finland

Fortunately, we have good autonomous driving know-how in Finland. The VTT research institute, owned by the Finnish government, has been developing and researching autonomous driving technology for several years. This research culminated in Martti and Marilyn, a pair of vehicles used in developing and testing various traffic automation solutions.

Now, VTT’s autonomous driving know-how has been transported to Nummela.

VTT’s test engineers are observing the test drives and measuring results. A vehicle takes to the road, a pedestrian is pulled in front of it, the vehicle stops, distances are measured, over and over again.

Repeatability is important. It demonstrates that a correct reaction is not a random occurrence, that the performance of the autonomous vehicle software is stable and predictable.

“All the tests were challenging, but the most difficult one to set up was probably the T-intersection where the vehicle makes a left turn while watching for traffic from the right”, Humina describes.

Just under ten different test cases were carried out in Nummela, modelling common traffic circumstances, including entering and exiting a bus stop.

Some of the tests also required the use of assistant vehicles. For example, for the purpose of trailing a car and reacting to its emergency braking, another vehicle was needed to measure the behaviour of the vehicle equipped with Sensible 4 software. This was VTT’s Martti’s time to shine.

All Set for Norway

“Once we got the map working and the test cases set up, we were feeling good about running the tests,” Humina adds.

And indeed, the test cases were passed successfully, repeatedly, and safely.

The VTT report was ready a few weeks later, and we had what we needed for applying for approval with Norwegian authorities. And we launched our first driving pilots in Norway in January 2021.

Sensible 4 runs vast amounts of tests on the autonomous vehicle software. Most tests never see an actual vehicle, as they take place in various simulators and other software test environments.

However, vehicle tests are also performed extensively. Various teams build and test their components in vehicles for smoother driving and improved performance in increasingly difficult weather and traffic conditions.

This time, however, the complete system was tested to ensure that all software components work together with the vehicle technology and sensors.

“An autonomous vehicle is a complete system where all parts must work together. Therefore, these tests on the complete vehicle are very important to us”, says Humina.