As you say. However, this in turn makes it important to distinguish between "Empires" and "empires", since "empire" is habitually used to describe accumulations of territory that don't meet any other definitions but which are nevertheless not technically Empires: Empires being ruled by an Emperor and carrying with it the purported universality (and, at least in Europe prior to about 1800, the implication that that Empire is the legitimate successor of Rome, although the remnants of that fell away when Napoleon and Francis II got into a manhood-measuring contest over their titles).
Among those you mention, for instance, neither the British Empire nor the French Empire were actually Empires in 1900 (nor Italy, Belgium or the Netherlands), and the British Empire never was in its entirety: the title of Emperor held by the British sovereign was Empress/Emperor of India, but that title wasn't applicable to non-Indian territory (hence why the style used was, after Victoria, "King-Emperor"). France had been an Empire from 1852-1870 but thereafter was a republic, even if that republic had an empire attached.
Part of this comes down to sloppy Roman terminology: the Roman emperors accumulated a lot of redundant titles many of which had their origins in the republic and most subsequent Imperial titles originate with them. So the Roman Empire (or imperium) prior to Augustus isn't even technically an anachronistic or incorrect term in the way that, say, "British Empire" is, because imperium was originally a generic term relating to power... but later imperator became exclusive to the ruler and became the root form for "Emperor". Likewise, of course, "Caesar" was a family name which also became a title and that's the one claimed by the Russian and German emperors. Although the Ottomans used Sultan as their primary title they also claimed "Caesar" as a subsidiary title as a successor to the eastern Emperor.
Previous folks have already admirably covered "nation" and so on. I find "realm" is a useful non-anachronistic generic term to use for polities in place of the more modern "state" since with relatively few exceptions it's usually applicable, but it's probably not something that would be used as part of a polity's formal name.
As far as king(dom)s go, there are three usual styles for kings which carry different conotations which may or may not be relevant to mapping depending on various factors. Using France as an example:
King <of France>: This implies that the king is owner of the land and sovereign lord of the kingdom. It's the usual style for absolute rulers.
King <in France>: This indicates that while the king is a king and owns land in a given area in respect of his title he's not the sovereign lord of all that territory and may in fact be almost entirely titular. This style was used by rulers who wanted the dignity of kingship but didn't control all the relevant area, perhaps most notably the early Prussian kings.
King <of the French>: This indicates that the king is lord of the people and may also imply some form of (formal or informal) social contract between the rulers and the people. This was the most common style of kingship before the high Middle Ages (French kings before Philip II were "Kings of the Franks") and was revived in a number of places after the French Revolution (including in France under the Emperors and Louis Philippe). Where you have a king of this style then "nation" would be an appropriate way to describe it, although still probably an anachronistic one unless it's a relatively modern or modern-inspired polity.
Depending on which form of kingship is in play it might affect how the territories are mapped politically. Hungary, for instance, was often mapped showing "Imperial Hungary" under the Ottomans and "Royal Hungary" under the king of Hungary (who was also, somewhat confusingly, Holy Roman Emperor). So if you have a "King in France" the chances are there will be "Royal France" and then "<other> France" depending on the political setup. If the king is "King of the French" then the subnational divisions will probably be more important, and you might want to detail individual duchies, counties etc. within the kingdom more than you would otherwise.