The simple answer to this is that over a period of years the yo-yo effect of going on and off of diets is a slow stripping of lean body mass. Also, if these diets border on starvation levels your body simply reduces your burn rate to compensate for such huge reductions in calorie intake. Your lean mass or muscle has a metabolic need of roughly 25-50 calories per pound per day. If you, over time reduce your lean body mass and burn rate through starvation type diet principals you simply set yourself up for higher and higher percentages of overall body fat even at lower and lower levels of body weight and very real reductions in caloric need. This effectively reduces your total metabolic rate significantly over time.
We all have to learn to reduce calories systematically while maintaining a smart fitness program that helps us retain muscle while we lose body fat. This is how we maintain our metabolic rate while reducing body fat and overall body weight.