If the ultimate goal of life is the freedom from suffering then why do we avoid pleasure?
Why do the wise men tell us to avoid pleasure instead of using it to our own benefit? Is it not ironic to try and escape suffering by making oneself suffer?
Pleasure is a temporary relief from suffering. Once you have gained pleasure from something, the inherent value that it has to you greatly diminishes.
What does this tell you about the nature of using pleasure as an escape from suffering? It only relieves suffering temporarily. It creates attachment to the object that temporarily relieved your suffering. New suffering arises from missing the object that initially relieved your suffering.
In short, temporary relief of suffering is not a solution. It will only amplify suffering in the long run. That's what the wise men are telling us.
For this reason, following the middle-path is important. By using self-control, we avoid dependence on external forces. Simply speaking, with the right mindset, one can learn to derive happiness from acts of self-control. These are not external forces, so we can't miss them.
This is a much surer path to enlightenment.