Dual-Stream Spatiotemporal Networks with Feature Sharing for Monitoring Animals in the Home Cage

Dual-Stream Spatiotemporal Networks with Feature Sharing for Monitoring Animals in the Home Cage

Blog Article

This paper presents a spatiotemporal deep learning approach for mouse behavioral classification in the home-cage.Using a series of dual-stream architectures with assorted modifications for optimal performance, we introduce a novel feature sharing approach that jointly processes the streams at Servo regular intervals throughout the network.The dataset in focus is an annotated, publicly available dataset of a singly-housed mouse.We achieved even better classification accuracy by ensembling the best performing models; an Inception-based network and an attention-based network, both of which utilize this feature sharing attribute.

Furthermore, we demonstrate through ablation studies that for all models, the feature sharing architectures consistently outperform the conventional dual-stream having standalone streams.In particular, the inception-based architectures showed higher feature sharing gains with their increase in accuracy anywhere between 6.59% and 15.19%.

The best-performing models were also further evaluated click here on other mouse behavioral datasets.