The triangle-shaped fort stands on a bluff overlooking the small harbor. Old mansions are in the distance. Rusted canons stand guard on the ramparts. A whale breaches the calm surf in the distance. And a wooden ladder leads downstairs to an overheated dungeon. The fort may be quiet and empty, but the air is heavy with sadness and guilt.

You’re at Fort Santo Antonio in the Western Region of Ghana. Axim’s fort was built by the Portuguese in 1515, captured by the Dutch in 1642, and used as a trading post that sent African slaves to Europe and the Americas. When you close your eyes, a breeze sends a chill up your spine. You almost expect to see ghosts when you finally open them.

But Ghana isn’t consumed with its past. The West African nation, whose name means “warrior king,” became the first African country to declare independence from its European colonizers in 1957. Unlike some of its neighbors, Ghana found peace and stability in its sovereignty. It’s now one of the most developed, thriving, and welcoming countries on the continent.

After exploring the fort and the harbor filled with fishing boats—plus admiring the brightly colored dresses that all the women wear—the smell of grilling fish makes you hungry. Follow the rocky coast and the palm-lined road outside of town to the Lou Moon Lodge. It’s located on a quiet bay in between a sandy beach and the thick jungle. The purple-cushioned sunbeds catch your eye on the way to your bungalow, but they’ll have to wait until tomorrow.

The sun is lowering in the sky, so you head to the open-air bar for a sundowner and the perfect view. Three dolphins playing offshore seem just as excited as you are. A candlelight dinner of freshly caught seafood—ceviche, shrimp with champagne sauce, and big crayfish—follows. Don’t pass on the white chocolate mousse, either.

Tomorrow, you’ll hike through the jungle. You’ll paddle a dugout canoe up the Ankobra River to meet villagers panning for gold. You’ll visit the still-active, red-topped lighthouse. And you’ll claim one of those purple-cushioned sunbeds on the beach. But the ghosts of Fort Santo Antonio stay in the back of your mind the remainder of your time in Axim. Like the rest of this trip, they won’t be forgotten anytime soon.