Idealistically, yes. But in reality, people agree to give them to each other, and sometimes they fight until a party not receiving them, finally gets the other party to agree to granting them, or dies trying to procure them, or lives, but fails in their procurement.
For instance, you may think you have the ultimate right to control your property, but, in fact, you do not, due to eminent domain and the kelo decision. You may DESIRE To have that right, and maybe we had it at one time, but right now our actual legal reality is at odds with what we perceive our rights to be, or what they might have been in the past. That right has been taken away.