The gut is home to trillions of microorganisms that have fundamental roles in many aspects of human biology, including immune function and metabolism. The reduced diversity of the gut microbiota in Western populations compared to that in populations living traditional lifestyles presents the question of which factors have driven microbiota change during modernization. Microbiota-accessible carbohydrates (MACs) found in dietary fibre have a crucial involvement in shaping this microbial ecosystem, and are notably reduced in the Western diet (high in fat and simple carbohydrates, low in fibre) compared with a more traditional diet. Here we show that changes in the microbiota of mice consuming a low-MAC diet and harbouring a human microbiota are largely reversible within a single generation. However, over several generations, a low-MAC diet results in a progressive loss of diversity, which is not recoverable after the reintroduction of dietary MACs. To restore the microbiota to its original state requires the administration of missing taxa in combination with dietary MAC consumption. Our data illustrate that taxa driven to low abundance when dietary MACs are scarce are inefficiently transferred to the next generation, and are at increased risk of becoming extinct within an isolated population. As more diseases are linked to the Western microbiota and the microbiota is targeted therapeutically, microbiota reprogramming may need to involve strategies that incorporate dietary MACs as well as taxa not currently present in the Western gut.