It can be difficult for some people to lose chest fat. Regardless of how much weight is eliminated from the body as a whole, chest fat appears to persist. Gynecomastia is a condition in which there are an abnormal number of glands under the nipples, which attracts fat storage.
The recommendation for folks like the ones mentioned above, as well as anyone who wants to lose chest fat, is to maintain a low body fat percentage (18 percent or so).