Yeah, it's different for someone who wants to practice. I moved from 'vague idea' to 'draw map' so that I had a physical concept of where the land was in the world I made, and that then made me more excited to think about how those places interacted with each other and how their way of life would have to be to live there.
I wouldn't force it if you hate doing it so much that it saps the rest of your creativity. That would be way too much of a sacrifice for any creative person. If you'd rather plot first, then do that, and leave your map to being a few words on a page that say "big forest" "city" "castle" "ocean" "ice" desert" PERFECT.
Tons of stories don't even need maps. Say you wrote a thriller that takes place in a single mansion and a boat out back, oh, and a scene with a helicopter. Or you wrote a murder mystery in a graveyard and a police station and there's a point where they go into another house to hide from the killer for like half a chapter. Or you wrote a story about a dance festival contest and it takes place exclusively in a fairgrounds and community centre and the protagonist's car which she lives out of, trying to hide this fact from the competition lest it be an embarrassment used to disqualify her. Or a killer clown in a circus gone wrong. Or a short horror story about a grocery store beset by giant bugs. Whatever, you get the point, there's stories set in small locations all the time, and for those, often a written description IS going to be good enough for a reader. In that case, don't stress yourself out about the town or the State or the country if it doesn't matter all that much to what happens in the story.
And well, if you write a long adventure of travel from place to place, once you've written it you might find yourself inspired to map out that journey and it won't hurt your creative juices.
Do what gives you the creative thrills first! I love maps but I'd never say anyone should prioritize them. I want to, but that's just me... you go write those stories.