You obviously cannot reduce Latin America to the Spanish/Portuguese heritage.
ALL Latin American countries share more than language; we share a colonial history, religious beliefs, social structures, and geopolitical struggles. Panama, which your book would probably place under "Middle America" (such a weird term, I prefer Central America), was until the beginning of the XXth century part of Colombia, which is South America. This example is very interesting to show the social nature of territory
If "Anglo-America" was an improper name for North America, I can only jump to conclusions and say it's because Mexico, traditionally included in the North America category, is not Anglo.