From dumb + bell. Originally an apparatus used for physical exercise or to practice ringing church bells. It was similar to those used to ring church bells, but didn't have a bell, which is why it was called dumb ("silent, mute").
1997, Pete McDonald, Climbing Lessons: Inside Outdoor Education, page 40:
He also had a room strewn with the toys of the trade and what-have-you: ropes and ironmongery, buoyancy aid and bagpipes, fetid socks and half-eaten haggises, dumbbells and ski-instructor sweaters.
2007, “The Years before the End of the War”, in A Girl From Shanghai: The Story of Lillian Hsu, a Medical Geneticist, a NTU Medical College Alumna[1], →ISBN, →OCLC, pages 35–36:
On the map Gansu is shaped like a dumbbell lying obliquely from east to west. Its best known city is Lanjhou, the capital, and also the gateway to the Silk Road. But we did not go there; we were heading for a relatively unknown city a few hundred miles away from Lanjhou, called Tianshuei.