The world is constantly changing. Lately, that change seems to be accelerating. New cities are being created in countries—China and India—with surging populations. War-torn countries—Iraq and Angola—are awakening from years of destruction and neglect. And places that used to be under Soviet control—Mongolia and Azerbaijan—are now flush with cash thanks to their natural resources. Azerbaijan?

Yes, Azerbaijan. Even if you’ve never heard of this country on the Caspian Sea, you’ll be hearing a lot about it soon. It’s capital and largest city, Baku, is changing daily, thanks to the petroleum industry. The Heydar Aliyev Cultural Center, designed by award-winning architect Zaha Hadid, looks like a billowing blanket and is one of the most fascinating new buildings in the world. Baku Crystal Hall, an indoor arena that recently opened, looks like diamonds shining along the waterfront. And the three, curved-shaped Flame Towers are an engineering feat. Add high-end shops located in Parisian-style mansions, an exciting modern art scene, and a pulsing nightlife, and you have a city poised to become the next Dubai.

These may be the reasons you end up visiting Baku, but there’s another side to the city: Icheri Sheher. The walled, medieval town has sandstone buildings, winding streets, and carpet shops. The Fajr prayer—it’s a predominately Shia Muslim culture—starts calling before dawn. And tea houses are the center of social activity.

Start your Old City tour at the Maiden Tower, a 12th-century section of the city’s wall. Its museum provides a history of Baku, but it’s the roof where you’ll spend most of your time. From here, you have views over the Old City’s connected alleyways, Baku Boulevard (a seaside promenade), and the Bay of Baku. Move over to the Palace of the Shirvanshahs, a massive, 15th-century complex with burial vaults, a mosque, a mausoleum, and calligraphed walls. See rare and beautiful rugs at the Carpet Museum. Sip strong tea out of a pear-shaped glass and play dominoes at a tea house. And watch kids kick a soccer ball outside of an old mosque.

You could stay at one of the upscale hotels that recently opened in the city, but a better choice is the Sultan Inn right in the Old City. Pass through the building’s stone arches to find a boutique hotel in which east meets west and elegant meets cozy. The chocolate-and-gold rooms have views of the Maiden Tower, plush sitting areas, ornate ceilings, and bronze-framed mirrors. Relax at Q-Café-Bar, the second-floor lounge, with a Turkish coffee or a Cohiba cigar while you enjoy the view as the wind picks up. Move to the Terrace Garden for a traditional dinner of dushbara, stuffed prunes, walnut kuku, grilled lamp chops, and local pomegranate wine.

As the sun begins to set, Baku’s new architecture starts to light up in the distance. People are swarming to the new sections of town. But you’re quite content stepping back in time in the Old City.