The zinc enzymes carbonic anhydrases (CAs, EC 4.2.1.1) are very efficient catalysts for the reversible hydration of carbon dioxide to bicarbonate and hence play an important physiological role. In humans, 16 different isozymes have been described, some of them being involved in various pathological disorders. Several of these isozymes are considered as drug targets, and the design of selective inhibitors is a long-standing goal that has captured the attention of researchers for 40 years and has lead to clinical applications against different pathologies such as glaucoma, epilepsy, and cancer. Among the different strategies developed for designing selective CA inhibitors (CAIs), the "sugar approach" has recently emerged as a new attractive and versatile tool. Incorporation of glycosyl moieties in different aromatic/heterocyclic sulfonamide/sulfamides/sulfamates scaffolds has led to the development of numerous and very effective inhibitors of potential clinical value. The clinical use of a highly active carbohydrate-based CA inhibitor, i.e., topiramate, constitutes an interesting demonstration of the validity of this approach. Other carbohydrate-based compounds also demonstrate promising potential for the treatment of ophthalmologic diseases. This review will focus on the development of this emerging sugar-based approach for the development of CAIs.