Applied Intuition participated in the Online Map Validation and Road Model Creation (MaVRoC) workshop at the Institute of Electrical and Electronics Engineers (IEEE) Intelligent Vehicles (IV) Symposium this year. Our team led a discussion to share challenges and jointly gain new perspectives on offline map and online road model creation and validation for autonomous driving (Figure 1).
Many autonomy programs face map curation challenges that map validation can help mitigate:
To help autonomy programs address the above challenges, Applied’s workshop presentation focused on the following key topics.
Maps are a key component of localization systems. They provide a priori information about landmarks that enable an autonomous vehicle (AV) to place itself in the world. Incorrect maps can cause the AV stack to provide poor localization estimates, which can cause disengagements during on-road testing or, in the worst case, accidents. Teams need to validate their stack continuously and update their maps whenever they receive new data, therefore addressing and mitigating the challenges of map curation.
Different map validation methods exist—each with its own advantages regarding speed and effectiveness and different levels of fidelity and cost:
Algorithmic map validation requires teams to write unit tests that verify map data correctness and completeness. This method is relatively inexpensive and fast to carry out. It catches errors that may result from improper data cleaning or missing data. Teams can also use algorithmic validation to check for semantic relationships and geometric issues that may exist in the map (e.g., invalid lane widths).
Simulation-based map validation requires teams to define map-agnostic test cases relative to the AV, map points, and actors. Teams will also query the map to find various road features and generate scenarios at scale. This method helps teams find errors in the interaction between the map and the autonomous planning system.
Data-in-the-loop map validation (or real-world data validation) requires teams to compare ground truth maps to information that fleets of AVs collect on the road. This method uses ML to perform feature extraction from sensor data, creating a “lightweight” map. Teams can perform “diffs” between this lightweight map and the real world to determine where the real world has changed. This can happen using ID-based checks or geometric heuristics. Real-world data validation is usually sufficient to find most changes, and it is especially effective with large fleets.
Applied’s map editing and validation tool Meridian enables autonomy programs to create maps, validate them in the real world, and address the key challenges of map validation by combining all of the methods mentioned above. Applied’s platform automates geometric and semantic checks and uses continuous integration and continuous deployment (CI/CD) processes to help prevent regressions. Furthermore, Applied’s solutions integrate with a variety of different map formats to ensure that errors found in the validation process relate to true map errors (Figure 2).
Contact our team to learn more about Meridian and how it can help validate maps for autonomy applications in the real world.
