Face Tracking App allows you to control any Calliope mini robot with your face position, rotation or even opening of your mouth.
Supported platforms
- Desktop web browser – Chrome, Edge, Opera
- Android
- ChromeOS
- iPhone or iPad – via Bluefy web browser
How it works?
The app tracks face with such parameters:
- X – horizontal position, range 0-99
- Y – vertical posiition, range 0-99
- Distance (Z) – range 0-99
- Yaw – rotating head left/right, range 0-99
- Pitch – rotating head up/down, range 0-99
- Roll – tilting head, range 0-9
- Mouth – mouth openness, range 0-99
- Left eye – range 0-99
- Right eye – range 0-99
- Smile – range 0-9
- Face visibility – 0-1
The parameters are scaled to fit the range between 0-100 or 0-10 (Roll, Smile) face visibility is 0 or 1.To adapt the code range to your needs you have to recalculate it on the Calliope mini.
Bluetooth communication
The app communicates with the Calliope mini via web-bluetooth over UART service (bluetooth serial port). This method tends to be quite slow so the transmision rate is reduced to 10 updates per second.
Data is being sent as a string (text) containing 19 digits. Each two digits transfer one parameter.
| Character | 0,1 | 2,3 | 4,5 | 6,7 | 8,9 | 10,11 | 12,13 | 14,15 | 16 | 17 | 18 |
| Parameter | X | Y | Distance (Z) | Yaw | Pitch | Mouth | Left eye | Right eye | Roll | Smile | Face visibility |
| Range | 0-99 | 0-99 | 0-99 | 0-99 | 0-99 | 0-99 | 0-99 | 0-99 | 0-9 | 0-9 | 0-1 |
Check out MakeCode template code to see how to decode it on the Calliope mini it might look intimidating at first but in fact is quite simple. Remember that the values are transferred as text so it has to be converted into numbers.
Problems with connecting over bluetooth
If you have problems connecting over bluetooth:
- your board is not discoverable by the app
To make the board discoverable over bluetooth you have to properly set PROJECT SETTINGS in MakeCode (individual for each project!)
- go to (GEAR ICON) -> PROJECT SETTINGS
- Next click EDIT SETTINGS AS TEXT
- scroll down to section about BLUETOOTH and add this line: “open”: 1,
"preferredEditor": "blocksprj",
"yotta": {
"config": {
"microbit-dal": {
"bluetooth": {
"open": 1
}
}
},
