I always interpreted it the way I presented it. I really didn't get any sort of reference to financial status out of it. I saw it as saying that the fewer worries one had about losing the valued things in life through ones own actions (because they were already lost), the freer the constraints on those actions became and at the limit, where one has lost everything one valued there were no longer much in the way of limits on what you might or could do (except those imposed by faith, if any)
Freedom has a dark side, the lack of any remaining social or personal connections to the world can free you to bomb a federal building just as easily as it can free you to do something positive. Liberty and freedom are related, but not synonyms
I see your point but you might have trouble selling that to anybody in prison or most north koreans.