In a truly free market there should be almost no barriers to trade, I agree. The reality though is the world doesn't work that way so within our current context how do we best promote the movement of goods and capital around the globe? If it means forced agreements where countries will stop with tariffs and allow for trade then that's how we have to do it.
We all know politicians will do what they think is best for their re-election chances even if it is bad economic policy. With trade it's a lot easier to point to a group of industry workers that have been affected negatively than it is to point out the benefit to the economy as a whole. So we know politicians will pander to that group of workers and try and 'protect' them. That's why we won't have truly free trade.