Automatic toothbrush timer

This toothbrush timer tracks how long you spend on actively brushing your teeth. It uses an accelerometer to measure motion, and analyzes it using a simple machine learning model. The device helps you get to the 2 minute mark, which is the recommend duration!

This is a demo project for emlearn-micropython, a library efficient Machine Learning and Digital Signal Processing for MicroPython. The intent is to be a fun and simple, but realistic application, of machine learning on microcontrollers, in the area of Human Activity Recognition.

In use

Here is a short demo video.

Status

Proof of Concept. Tested working on device.

• Firmware is complete and functional
• Basic documentation on how to reproduce exists
• Software and training pipeline needs some cleanup

For more details, see TODO.md

License

MIT

Hardware

To assemble the device, one needs the following.

• M5Stick C PLUS2 from M5Stack
• 3d-printed toothbrush holder. .STL export | FreeCAD project
• Zipties
• Double-sided tape

.

Stand

There is also an optional stand for a magnetic charging connector. It uses a Plexgear Magnetic USB-C adapter. Hopefully something similar can be found online.

FreeCAD project

.

How it is built

The code that runs on device is found in firmware/. The code used on PC to train machine learning model is found in software/

Data recording

To record data we used the har_record.py script from emlearn-micropython har_trees example. We also recorded video of the brushing activity using a mobile phone. This was used to establish the ground truth. See doc/data_collection.md for details.

The raw data from device is stored in data/jonnor-brushing-1/har_record/.

Data labeling

We use Label Studio to annotate the videos. The labeling annotation definition can be found in doc/labeling.xml.

The labeled data should be exported as .CSV file.

Data set preparation

When the data recording and data labeling has been completed, the data can be combined into a dataset.

FIXME: clean up these steps. Move from notebooks into .py files

The output goes to data/jonnor-brushing-1/combined.parquet.

Model training

To train the model we used the har_train.py script from emlearn-micropython har_trees example.

MIN_SAMPLES_LEAF=0.20,0.30 python har_train.py --dataset toothbrush_jonnor --window-length 50 --window-hop 50

Firmware

The firmware has the following overall architecture:

Here is the states of the state machine, that defines the overall behavior of the device:

Installing

Development environment

You must have Python 3.10+ installed. Using a virtual environment is recommended.

Install project dependencies

pip install -e .

For M5Stack M5Stick C PLUS 2

Flash device with MicroPython for ESP32

mpflash flash --version 1.24

Copy dependencies to device

mpremote mip install https://emlearn.github.io/emlearn-micropython/builds/master/xtensawin_6.3/emlearn_trees.mpy
mpremote mip install github:jonnor/micropython-npyfile
mpremote mip install github:jonnor/micropython-mpu6886
mpremote mip install https://github.com/emlearn/emlearn-micropython/raw/refs/heads/master/examples/har_trees/timebased.py
mpremote mip install https://github.com/emlearn/emlearn-micropython/raw/refs/heads/master/examples/har_trees/recorder.py

For XIAO BLE Sense NRF52840

Flash device with MicroPython

TODO: provide pre-built firmware

Copy dependencies to device

mpremote mip install --target lib github:jonnor/micropython-npyfile
mpremote mip install --target lib https://github.com/emlearn/emlearn-micropython/raw/refs/heads/master/examples/har_trees/timebased.py
mpremote mip install --target lib https://github.com/emlearn/emlearn-micropython/raw/refs/heads/master/examples/har_trees/recorder.py
mpremote cp firmware/lsm6ds.py :lib/

Copy application to device

Copy the firmware files

mpremote cp firmware/core.py firmware/buzzer_music.py firmware/process.py firmware/brushing.trees.csv firmware/main.py :

Start the application and observe log

mpremote run firmware/main.py

Restart device, will disconnect log

mpremote reset

Developing

Running firmware tests on PC

With MicroPython Unix port, one can run the firmware tests on PC.

micropython firmware/test_toothbrush.py

Running firmware tests on device

TODO: check tests on device, document how to run

Training model

TODO: document how to run
See the Gitlab CI actions

Modifying dataset

See doc/data_collection.md

Porting to other devices

All the hardware-specific code is located in main.py. So modifying that is all that should be needed to adapt to another device. If there is no buzzer with PWM, you may also want to change OutputManager.

Note that the accelerometer driver should use the FIFO, to enable sampling data independently from the firmware execution. More information.