Biomass product 5-hydroxymethylfurfural (5-HMF) can be used to synthesize a broad range of value added compounds currently derived from petroleum. Thus, the effective conversion of glucose or cellulose (the major components of biomass) into fuels and chemical commodities has been capturing increasing attention. Previous studies have been extensively focused on a two-step process for producing 5-HMF from glucose or cellulose, i.e., the isomerization of glucose into fructose and then the dehydration of fructose. We herein discovered that heterogeneous sulfonated poly(phenylene sulfide) (SPPS) containing strong Brønsted acid sites is able to convert glucose and cellulose into 5-HMF with a high yield in ionic liquids (ILs). The optimal activity of glucose conversion to 5-HMF achieves a yield of 87.2% after 4 h reaction at 140 °C. For direct cellulose conversion, a 5-HMF yield of 68.2% can be achieved. The reaction mechanism over the SPPS catalyst in ILs was studied by DFT calculations, and the results indicated that the SO3H group of SPPS plays a crucial role in glucose conversion into 5-HMF, and it acts as a proton donor as a Brønsted acid and functions as a proton acceptor as the conjugate base. Furthermore, the anions and cations of ILs together with SO3H-SPPS helped in stabilizing the reaction intermediates and transition states, which also resulted in glucose facile conversion into 5-HMF. The new catalyst system highlights new opportunities offered by optimizing the production of 5-HMF directly from glucose and cellulose.