In spite of the many efforts to encourage healthier diets, obesity continues to be a serious public health concern in the United States and across the world. Most dietary approaches rely on detailed individual tracking of points or calories, which makes the necessary long-term compliance challenging and ineffective. It has recently been shown that simply monitoring and regulating the number of bites one takes in a day can lead to significant weight loss. While counting bites is in principle as burdensome to individuals as is counting points or calories, existing technology can be leveraged to automate the process. Our Hand-to-Mouth (HTM) bite-counting device consists of a simple wrist-mounted sensor with accelerometer and gyroscopic motion sensing capabilities. The associated HTM algorithm is designed to run continuously, actively counting bites throughout the day. We take a novel machine learning approach to customize the system to each individual user, and achieve an average accuracy of 91.8%, well above the current state-of-the-art. We do this while using a small set of 5 motion features, a Naïve Bayes model, and a streaming rate of only 10 Hz in order to conserve power and efficiency. Our algorithm represents the first viable solution to counting bites. It is lightweight enough to be put in a smart device, and accurate enough to stream data 24 hours a day.