A U.S. Virgin Islands day trip. You’re probably staying on St. Thomas, where you’ve been beach hopping and duty-free shopping. Or you’re camped out on beautiful, quiet St. John. Lucky you. Or you’re really lucky, and you’ve been sailing between the islands on a private boat—it doesn’t even matter that it’s not yours. Wherever you are, you’re ready for a change of pace. At least for a few hours.

Christiansted on St. Croix will break up all the snorkeling and the tanning you’ve been doing since you arrived. St. Croix is 40 miles south of St. Thomas. Christopher Columbus landed on the largest of the U.S. Virgin Islands in 1493. He was quickly driven away by the arrows of the Kalinagos. The French later claimed the island, and the Danish bought it from them in 1733. They built forts, plantations, and a capital city for the Danish West Indies.

That capital, Christiansted, is a walkable, picturesque city on the northern coast. It’s full of colorful Georgian buildings, cobblestone streets, and views of the boat-dotted harbor. Arrive early, you’ll want to walk around in the morning when it’s cooler. Start at the Visitors Bureau, also known as the Old Scale House. Sugar from more than 100 plantations around the island was weighed here to be taxed before shipment in the 19th century. The huge scales are still inside the yellow building. The taxes were paid at the Customs House next door. Visitors aren’t allowed in the current headquarters of the National Park Service, though the carved brick staircase outside is a popular spot for photos.

Across the street, Fort Christiansvaern was built to protect the Danish from hurricanes, pirates, and, eventually, slave revolts. The yellow, diamond-shaped fort is a fun place to explore—see the cannons on the ramparts, hear echoes in the dungeon, and enjoy the view of the turquoise water. The admission fee to the fort also includes entry to the nearby Steeple Building Museum. Originally a church, the building was also used as a hospital and a school before becoming a local history museum with plantation artifacts. And the Danish West India & Guinea Company Warehouse—the current post office—was once home to one of the largest slave auctions in the Caribbean.

When you need to lighten the mood, walk to Apothecary Hall. The 19th-century pharmacy is now filled with shops and restaurants. Stop at Luncheria for a margarita in the shady courtyard. Stroll through the peaceful gardens at the elegant Government House. Buy fresh fruit in Hendricks Square, the island’s largest outdoor market. Browse the small shops on Strand Street. Many Hands sells handmade pottery and local paintings, Crucian Gold has popular knot bracelets, and Royal Poinciana stocks local hot sauces and seasoning blends you’ll want to add to your own cooking.

After collecting your goodies, walk back to the waterfront along King’s Alley. The boardwalk is lined with low-key bars and restaurants. Reggae music streams from most of them. Stop at the Fort Christian Brew Pub for a Hammerhead Pale Ale or a Leatherback Nut Brown. You opt for a late lunch when you smell Southern food in the kitchen. And you’re almost tempted to stay in Christiansted for dinner when you hear the people next to you talking about a little bistro that serves tasty Caribbean-Thai-Mexican fusion nearby. Add Savant to your next trip to St. Croix. Maybe you should find a hotel room, too.