Bornholm was supposed to be your final destination. The Danish island is famous for its dramatic coastline and gorgeous beaches, Nordic round churches and wooden houses. It’s even known for feeling like the Mediterranean instead of the Baltic. So you assumed that it was the easternmost point of Denmark. It wasn’t until you arrived that learned about Ertholmene.

Though only 12 miles and an hour ferry ride from Bornholm, Ertholmene feels much more remote. The tiny archipelago—it’s actually three small islands and a handful of rocks—is a picturesque spot that seems frozen in time. The two larger islands, Christiansø and Frederiksø, surround a well-sheltered natural harbor. A pedestrian bridge connects the two islands and swings open for large boats. Less than 100 people live here, though that number swells during the summer. Græsholm, the third island, is an uninhabited bird reserve full of moss-covered rocks. It’s where eiders (sea ducks) and puffins breed in the spring.

Fishermen from Bornholm started using Ertholmene as a temporary shelter during the Middle Ages. When tensions between Denmark and Sweden increased in the late-17th century, the Royal Danish Navy built Store Tårn (Great Tower) as an extra layer of protection. The islands continue to look much like they did during this time. The remains of the naval fortress still have thick granite walls and cannons pointed toward the sea. The Christiansø Lighthouse, at the top of the tower, was the first lighthouse with a flashing light in Denmark. It continues to act as a beacon in the windswept Baltic Sea.

The bright yellow buildings, where soldiers once lived, are now occupied by year-round and seasonal residents. A three-room schoolhouse, artist studios, and a small grocery store do, too. The old jail has been turned into six hotel rooms, though most overnight visitors prefer to stay on yachts. While Lille Tårn (Little Tower) has become a museum displaying old cannons and fishing gear.

You’ll see all of these sites as you walk around—there are no cars here—Christiansø and Frederiksø. Be sure to try Ruth’s herring and Slåensnaps, both made from old local recipes. It seems like it is possible to travel back in time after all.