Robustness Against Unkown Noise for Raw Data Fusing Neural Networks

written by: Mario Bijelic, Christian Muench, Werner Ritter, Yuri Kalnishkan, Klaus Dietmayer

Adverse weather conditions are extremely challenging for autonomous driving as most state-of-the-art sensors do not function reliably under such circumstances. One method to increase the detection performance is to fuse the raw data signal with neural networks that learn robust features from multiple inputs. Nevertheless noise due to adverse weather is complex, and in addition, automotive sensors fail asymmetrically. Neural networks that perform feature level sensor fusion can be particularly vulnerable if one sensor is corrupted by noise outside the training data distribution compared to decision level fusion. The reason for this is that no built-in mathematical mechanism prevents noise in one sensor channel from corrupting the overall network even though other sensor channels may provide high-quality data. We propose a simple data augmentation scheme that shows a neural network may be able to ignore data from underperforming sensors even though it has never seen that data during training. One can summarize this as a learned “OR” operation at fusion stage. This learned operation is also generally applicable to other noise-types not present during training.