Setting is a pretty all-encompassing element of your manuscript. It is difficult to talk about any one element in isolation. Here I'll look at some key things to keep in mind when crafting the geographical setting in your manuscript.
Know your audience
Readers of different genres expect different settings. If you're writing any shade of dark fantasy, your settings should lean towards darker imagery. If you're writing serious high-fantasy, consider the impact that weird and wonderful settings might have on the atmosphere of your writing. I'm a big fan of breaking rules. But to break rules well, you've got to know exactly what your readers are expecting and why. That way you can give them just enough of what they want so as not to disappoint them.
Start with your characters
This is especially true of more fantastical fantasy. If you've got a mountain-soul druid in your fantasy, the setting really should include at least one mountain range. If the furthest your pixie has ventured from her flatland swamp house is ten kilometres and they've never seen even a hint of a forest in her life, the forest probably has to be at least fifteen kilometres from her house. And what does that say about the barren nature of the land she grew up in. Every decision you make about your setting will have an impact on your characters and how they interact with the world.
Consider a map
Whether you're writing high fantasy in an entirely imagined world or urban fantasy based on your favourite city, a map might be a good idea. The urban fantasy situation is the easiest to convince you of. Very few readers want a story that reads like your drunk mate giving you directions to the pub they've found themselves in at 3am. But more readers than you'd care to know are going to raise an eyebrow at the character making their way across town in rush hour traffic in ten minutes flat. Pull up your favourite maps app and use the directions function to add some realism to your timelines.
For your imagined world, a map can do great things. There is, of course, the matter of how quickly your characters can traverse your world via different modes of travel. A map will also help you visualise the world from a human perspective. If it is near the mountain range I demanded above, maybe the climate is colder. So maybe the house your characters are in is lit by a fire. Or maybe you're by the sea and the scenes are peppered with the smell of seaweed and whatever your world's equivalent of fish and chips is. A map will see you through from big picture all the way to the finer details.
What year is it?
Even is years don't exist in your world, do consider what year your world is comparable to. Yes, yes, time is a different element of setting. But it is in communication with the physical setting. What year it is may impact the infrastructure in your towns and cities. It is an important thing to keep consistent, or at least plausible. We'll tackle plausibility first. In the foreword RF Kuang's Babel she details each of the intentional historical inaccuracies with the setting. There are obvious ones like "hey the whole world runs of magical silver!" This is pretty unaffected by what year it is. The big one she highlights is the railway line between Oxford and London. She tweaked the timeline for the sake of the pace and basically told any reader that cares to sod off. An admirable tactic! But if you don't think you can pull it off, keep things consistent. You might be saying "well my characters get around on dragons and no modern technology exists." If that is the case, you better think about: whether they've got stables; whether the landing site for dragons is walkable to the village; if it is in the village, how small are the dragons or how big are the streets; how far away are the cities from each other if dragons are preferable to horses? Don't think it matters? Well, it might do when you've got a close combat fight in the village and one character darts in a flash across the street that is supposedly the width of three massive dragons to shove his friend out of the line of an arrow. Unless your character also has super speed. In which case, please continue.
Putting thought into the geography of your world is the first step to creating rich and convincing cultures. You don't need to be crumpling paper and spilling tea over it. But it might be fun...