Jinho Park

Data-driven visual-inertial odometry (VIO) has received highlights for its performance since VIOs are a crucial compartment in autonomous robots. However, their deployment on resource-constrained devices is non-trivial since large network parameters should be accommodated in the device memory. Furthermore, these networks may risk failure post-deployment due to environmental distribution shifts at test time. In light of this, we propose UL-VIO – an ultra-lightweight (<1M) VIO network capable of test-time adaptation (TTA) based on visual-inertial consistency. Specifically, we perform model compression to the network while preserving the low-level encoder part, including all BatchNorm parameters for resource-efficient test-time adaptation. It achieves 36X smaller network size than state-of-the-art with a minute increase in error – 1% on the KITTI dataset. For test-time adaptation, we propose to use the inertia-referred network outputs as pseudo labels and update the BatchNorm parameter for lightweight yet effective adaptation. To the best of our knowledge, this is the first work to perform noise-robust TTA on VIO. Experimental results on the KITTI, EuRoC, and Marulan datasets demonstrate the effectiveness of our resource-efficient adaptation method under diverse TTA scenarios with dynamic domain shifts.

Spectral-domain convolution engines can effectively reduce the computational complexity of convolution operations. In these engines, however, element-wise multiplications of the spectral representations dominate the multiply and accumulate (MAC) operations. In light of this, we propose bin-specific quantization (BSQ), which is to judiciously allocate varying bit width to each spectral bin in overlap-save. This allows efficient computation of the Hadamard product since the magnitude of the high-frequency components in image features is significantly smaller than that of the low-frequency counterparts. Using the statistics from spectral representations of feature maps, we also delineate methods for properly allocating bit precision to those spectral bins. When BSQ is applied, the average bit precisions of the arithmetic operators in spectral-domain convolvers, without the requirement of network re-training, were lowered by 24% (AlexNet), 20% (VGG–16), and 22% (ResNet–18) while having no significant reduction (< 1%) on classification accuracy on the ImageNet dataset.