Everyone claims that they want to be whisked away to a deserted island. But the reality of an island with just sand, a few palm trees, and the hot sun isn’t nearly as romantic as one would expect. Sure, the view of the turquoise water and the greens islands in the distance are jaw dropping. But what about food and drinks, shade, and, most importantly, shelter to protect you from the crabs, the mosquitos, and who knows what else. Maybe deserted isn’t all it’s cracked up to be.

Cooper Island, one of the Southern Islands in the British Virgin Islands archipelago, is one step up from deserted. Along with white-sand beaches, swaying palm trees, and stunning views, the 480-acre island only has a handful of privately owned homes and a small beach club. There are no casinos, resorts, or even cars. A boat—or a yacht, if you prefer—is the only way to get there. While shipwrecks have become intriguing scuba diving sites just offshore.

As you approach Cooper Island on a boat, it certainly looks deserted. It isn’t until you reach Manchioneel Bay, dotted with sailboats, that you start to see the Cooper Island Beach Club through the dense greenery. It’s early, so the loungers, the kayaks, and the paddleboards are still being dragged down to the sand. You decide to explore the island, before the humidity increases. Hike to Quart-a-Nancy Point. From the island’s northern tip, you have a perfect view of the bay and rocky Cistern Point, with Salt and Peter Islands beyond the calm water. Then head south to the Haulovers, a narrow strip that connects the north and south ends of the island. You find pretty shells on the rocky beach and schools of sergeant majors when you put your snorkel mask on to go for a swim.

By the time you return to the beach club, you’re happy—make that relieved—to see umbrellas and someone serving ice-cold beverages. The informal Beach Club Restaurant is open air with a view of the dock. Grab a table overlooking the beach. Order a pitcher of sangria to share. Then decide to share a few seafood dishes, as well. Homemade conch fritters are served with a spicy Marie Rose sauce. Shrimp and pineapple are skewered between red peppers and grilled limes. Plus a bucket of fish and chips quickly reminds you who rules this Caribbean island.

After lunch, and probably a little nap on the beach, head back into the water. Join a scuba diving group to see the best spots around the island. Squid and eels dance around huge pillar corals at Vanishing Rock. Four deliberately sunk wrecks cover the ocean floor in Wreck Alley. While blue tangs, queen angelfish, and even sea turtles weave through the lattice-like rocks in Devil’s Kitchen.

It’s happy hour by the time you return to the beach club again. Rounds of painkillers, rum punches, and Carib beers are being passed around the bar. People are celebrating the end of a fun day before returning to their boats and, eventually, their hotels on the larger islands. But you’re not ready to leave. Not quite yet. Luckily, the beach club has a few cottages nestled among the cactus garden and the tropical flowers. You’ll have this peaceful island practically to yourself once the rest of the now-tipsy crowd sails away. And you’re more than happy to trade practically for deserted when it comes with a recently renovated room, recycled teak furniture, and a sunset-facing table at your prepared dinner.