Dairy weighing you down? Some researchers tout dairy for weight loss yet choosing high fat dairy could have the opposite effect. Especially if you eat 2 or more servings daily. Reduce calories – choose fat-free milk, yogurt and cottage cheese, or significantly reduce the amount and frequency of cheeses and other full-fat products.