Well, when certain races are property, some are citizens, and some are neither this sort of thing was pretty important. It could literally be the difference between a child inheriting the father's land vs the child being the property of the landowner. Remember slavery was big business in the Americas. Indian slaves first, then African slaves (who escaped in large numbers and set up their own "colonies" which then had to be negotiated with as some odd mix of sovereign nations made up of "property". See "Maroon" communities in South America.
So, economics guided the "official race charts" in a way that doesn't enter the discussion today. The easiest way to justify owning another human is to "prove" they aren't really human. Then it's sort of like owning a particularly clever animal, much more morally tidy.
*edit*
Speaking of economics, the book I've been referring to cost $7.50 in 1854. $210 and change in today's money. The cost of textbooks has apparently been insane for longer than I would have thought.
It's a bargain. Using 4% average inflation, rule of 72nds says price doubles every eighteen years
(2018 - 1854) = 164
164/18 = ~9.1
2 ^ 9.1 = ~548.7
548.7 x $7.50 = $4115.25