Playful Toothbrush: Ubicomp Technology for Teaching Tooth Brushing to Kindergarten Children CHI 2008, Florence Italy Yu-Chen Chang, Chao-Ju Huang, Hao Chu, Peggy Chi National Taiwan University Jin-Ling Lo, Nan-Yi Hsu, Hisn-Yen Wang, Ya-Lin Hsieh National Taiwan University Hospital
Child Behavior Training • Child behavior training as important but challenging parental responsibility – Potty training, self-dressing, cleaning room, self- feeding, tooth brushing (this work).
Persuade Children to Change Behaviors • The most common form of parental persuasion is verbal persuasion. – “If you don’t brush your teeth properly and thoroughly, you are not allowed to go to sleep”. • Not effective, why not? – Verbal persuasion alone lacks proper incentive to motivate children
A Case Study: Child Tooth Brushing • Target kindergarten children (5-6Y) – Learn to care for their own oral hygiene – Average 5Y children brush only 1/4 of teeth (Rugg-Gunn) – A common scenario: candies → improper brushing → cavities → dental clinic
Play-based Occupational Therapy • Pediatric occupational therapy (O.T.) – Leverage the desire of children to play to induce their behavior change willingly. • Children may not like to brush teeth, but like to play. • Add playfulness (game) into the brushing activity – Effective – full of toys in pediatric O.T. clinic.
Play-Based O.T. Limitations (1) Children treated in clinics during regular office hours (NTU Hospital) – Many child behavior problems not observable to therapists • Eating (dinner time), brushing (before bedtime), etc. • Effective treatment is difficult. (2) Train general performance skill rather than specific functional skill – Hand dexterity skill vs. tooth brushing skill – Improvement in general performance skills may not translate into the target functional skills
HCI/UBICOMP Opportunities • Embed digital technology into a child naturalistic living environment – Sensing to observe child behavior anytime, anywhere – Game playing to influence child behavior anytime, anywhere • Occupational therapist perspective: – From treatment clinic – To the child actual living environment (functional behaviors) • HCI/UBICOMP perspective: – From sense and track behaviors – To engage children to change behaviors • Also cal ed Persuasive technology (by Fogg, King, and others)
Playful Toothbrush Goals • (Child) Dislike brushing – Make brushing more attractive • (Child) Habitual brushing but not properly – Teach proper brushing skill
• Not replace adults’ supervision – Extend their effectiveness – Used together can overcome young children’s limited physical and cognitive abilities, such that they can successfully learn proper brushing
Playful Toothbrush Web camera for tracking brushing • Sensing motions – Camera vision to detect children’s Camera #2 to brushing actions record videos for later human – Brushing actions become game inputs analysis Brush extension • Playing Brushing – An interactive game of brushing teeth game – Start with a mirror image of dirty teeth. – Brushing own teeth maps to cleaning the same virtual teeth in the mirror image. • Demo video
Play-based Occupational Therapy Model • Playful toothbrush is a treatment program / play activity (three steps). Volition Performance Habituation Bring enough Ensure a child wil Apply reinforcement enjoyment to have a successful to reward good attract a child to experience. Set performance, so participate in the appropriate level of increase change of target activity. difficulty. repeating desirable behavior. Enough times to become a habit.
Design Considerations • Enjoyment – Bring enough fun to attract children to brushing – Associate brushing with game playing and having fun • Engagement – Simple Interaction (Not all young children can learn to operate complex devices) – Use their natural brushing actions as game inputs • Automation – Help children learn tooth brushing skill and internalize as habit • In brushing case, internalizing means doing without much thinking) – Advice adults not to interrupt children and their own motor planning, necessary for internalization of brushing motor skill
Playful Toothbrush Prototype (two main components) Vision-based brushing tracker Brushing game
Vision-based Brushing Tracker • Use one camera to detect brushing – Recognize brushing of 24 different teeth surface areas (or granularity of 2 adjacent teeth surfaces) • Mouth closed during brushing – Bristle-teeth contact area not visible to an external camera • Brush extension – Each of four faces has unique LED-pattern – Used as marker to assist camera-vision recognition – Children told not to block brush extension from the camera
How to recognize a brushing stroke on a specific teeth surface area? • Take an example (brushing the outer surface of frontal teeth) • Three features determines this brushing stroke – Bristle rotated toward the face – Brush oriented parallel to the face – Back-and-forward motion vector (parallel to the face)
Computer Vision Technique • Use the brush extension marker to reconstruct z (θ ) yaw – Bristle rotation angle (x-axis) – Brush orientation angle (z-axis) brush extension – Brush back-n-forward motion vector • Infer the brushing stroke and its y target teeth area x (θ ) rol
Accuracy Test (vision-based brushing tracker) • 13 kindergarten children (72-81 months) • Recorded 48 minutes of their brushing videos • Compare human-read brushing strokes (ground-truth) and machine-recognized brushing strokes • Machine-recognized accuracy 90% – Need not be perfect. Occasional error in games has little effect.
Brushing Game • Start with a virtual mirror image of the children’s own uncleaned teeth – Grouped into 24 teeth surface areas • “1” appears next to the first cleaning
target – 7 layers of plaques were drawn, requiring 7 brushing strokes – Each brushing stroke removes one layer of plaque – Combine audio feedback: Do-Re-Mi-Fa .. – No game response for brushing other teeth areas
Brushing Game 2 • After 1st area is complete, “2” appears next to the 2nd target area. • Applause at the game completion – Visual-audio feedbacks provide a sense of accomplishment
• Enforce a brushing sequence 3 2 (Stil man’s brushing method) – Upper right, upper left, the lower right, and lower left • Two benefits of sequencing – Ensure brushing all teeth – Repeat this sequence enough time -> “do without thinking”
User Study • Two questions guided our user study – How effective is the Playful Toothbrush in improving the brushing skills of kindergarten children? – What aspects of brushing behaviors were affected by Playful Toothbrush?
User Study Test Subjects • 13 young children (72 – 81 months) from a NTUH-run kindergarten class – 5 girls and 8 boys • Written informed consent from parents • Tooth brushing is required after meals or snacks • Setup – Install our system at the kindergarten’s restroom – A video camcorder to record children’s brushing sessions
User Study Procedure (11 days spanned over 3 weeks) • Pre-trial (one day) – Familiarize with the therapists and our system • Pretest (two days) – Brushed with their own toothbrushes – Established a baseline behavior • Training (five days) – Brushed with our playful toothbrush – A trained therapist helps children understand the brushing game • Posttest (two days) – Brushed with their own toothbrushes – Measured behavior improvement from pretest • One week fol ow up (one day) – Brushed with their own toothbrushes – Measured behavior retention after a week
User Study: Brushing Effect • Teeth cleaning effectiveness – Red plaque disclosing dye before each brushing session (dye attached the plaque) – Oral exam (counted teeth surfaces with dye before and after brushing) – Plaque index • number of teeth surfaces with plaque / total number of teeth surfaces – Cleaning effect • Reduction of Plaque Index from before to after brushing
User Study: Brushing Behavior • By analyzing/coding brushing videos, measured 3 aspects of brushing behaviors: – Length of brushing time – Number of brushing strokes on each of 24 teeth areas – Total number of brushing strokes
User Study: Cleaning Effect Results Before brushing After brushing Cleaning effect Mean(SD) Mean(SD) Mean (SD) Pre-test Day 3 0.69(0.25) 0.37(0.18) 0.32(0.21) Training Day 4 0.76(0.14) 0.09(0.10) 0.67(0.15) Day 6 0.68(0.28) 0.04(0.05) 0.64(0.26) Day 8 0.79(0.18) 0.11(0.10) 0.68(0.17) Average 0.74(0.19) 0.08(0.06) 0.66(0.17) Post-test Day 9 0.83(0.11) 0.16(0.10) 0.67(0.13) Day10 0.86(0.16) 0.15(0.08) 0.70(0.15) Average 0.85(0.10) 0.16(0.07) 0.66(0.28) Fol ow-up Day 11 0.88(0.14) 0.18(0.10) 0.70(0.14)
User Study: Brushing Behavior Results Number of brushing Number of unbrushed Brushing time strokes Mean(SD) teeth areas Mean(SD) Mean(SD) (sec) Pretest Day 2 212.31(137.77) 11.31(5.25) 84(53) Day 3 168.62(157.03) 13.46(5.35) 67(46) Average 190.46(138.38) 12.39(4.75) 76(45) Training Average Posttest Day 9 281.69(120.66) 7.46(4.89) 137(41) Day 10 214.31( 71.91) 9.46(5.12) 99(31) Average 248.00(87.12) 8.46(4.82) 118(30) Fol ow-up Day 11 239.62(107.48) 8.31(5.07) 120(36)
Result Summary • (Pretest) Child subject failed to clean 37% of their teeth surfaces • After 5 training days, improvements in – Teeth cleaning effectiveness – Number of brushing strokes – Length of brushing time – Coverage of brushed teeth areas
Conclusion • Demonstrate a case study of applying HCI/UBICOMP technology in play-based occupational therapy • For teaching young children proper tooth brushing, user study results were encouraging • Other similar repetitive development tasks for young children (self-feeding, potty training, cleaning room, etc.) – This technique can make training attractive and simple for adults/children.
Examples of Other Persuasive Technologies
Case Study: Playful Tray Encourage good eating habit in young children • Sense to recognize behavior – Weight sensor underneath the tray to sense eating actions – Eating actions as game input • Play to engage behavior change – Interactive games: coloring cartoon character or penguin fishing
Case Study: Mug-Tree Encourage healthy habit of drinking fluid regularly • Sense to recognize behavior – Tilt sensor to detect drinking actions – Drinking actions are game inputs • Play to engage behavior change – Game metaphor: hydrating/dehydrating body -> watering/drying a tree
Case Study: ChroMirror Persuade people to explore more colorful dressing • Sense to recognize dress & dressing color – Camera and Computer Vision • Play to engage behavior change – Playful explore & experiment with how different clothing color look on pe
ople • CHI 2008 poster on Tuesday
Thanks Q & A
Lesson #4 • Mediate, not automate – Mediation: not to replace children’s efforts but make the experience of performing more enjoyable for children – Mediation works better than automation in this case.
Lesson #1 • Unpredictable behaviors from young children – Make activity recognition difficult • Randomly switch from left to right hands, swing brush wildly, head movement brushing the brush … Observe child behaviors carefully before designing and programming the system
Lesson #2 • Activity grading – Children have different physical and cognitive capabilities • Some learned fast (not challenging enough); some learned slowly (frustration) – Different levels of challenges fitting to each child’s ability “Play-based O.T. is about the experience of performance, or the fitness between the level of challenge in an activity and a child’s physical and cognitive capabilities.”
Lesson #3 • Personalization & customization – Easily personalized and customized to environmental or human factors (preferences of children, changing performance of a child, different deployment environments). – Left hand vs. right hand – Lighting condition – Child height – Preferences about game characters
Implementation Details • Toothbrush extension weight – Not too heavy for children to cause inconvenience in brushing • Prevent child from holding the brush extension – Block the brush extension from the overhead camera – Solution: a protruding cap as separator • Camera placement – Not too high such that camera cannot see the brush extension clearly. – Not too low such that children can switch the brush extension outside camera view