I disagree that people don't have the right to other people's labor. Society is effectively a system where people pool their labor, resources, ability, etc. for mutual benefit. If people can't reasonably expect to gain some of the benefits that other people can provide, then why would they enter into the social contract in the first place?