Wearable units, from smartwatches to health trackers, have grow to be ubiquitous, repeatedly capturing a wealthy stream of information about our lives. They document our coronary heart charge, depend our steps, observe our health and sleep, and way more. This deluge of data holds immense potential for personalised well being and wellness. Nevertheless, whereas we will simply see what our physique is doing (e.g., a coronary heart charge of 150 bpm), the essential context of why (say, “a brisk uphill run” vs. “a aggravating public talking occasion”) is usually lacking. This hole between uncooked sensor information and its real-world which means has been a significant barrier to unlocking the complete potential of those units.
The first problem lies within the shortage of large-scale datasets that pair sensor recordings with wealthy, descriptive textual content. Manually annotating tens of millions of hours of information is prohibitively costly and time-consuming. To resolve this, and to actually let wearable information “converse for itself”, we want fashions that may study the intricate connections between sensor indicators and human language immediately from the information.
In “SensorLM: Studying the Language of Wearable Sensors”, we introduce SensorLM, a household of sensor–language basis fashions that bridges this hole. Pre-trained on an unprecedented 59.7 million hours of multimodal sensor information from over 103,000 people, SensorLM learns to interpret and generate nuanced, human-readable descriptions from high-dimensional wearable information, setting a brand new state-of-the-art in sensor information understanding.
