Effectiveness is essential for knowledge base question answering (KBQA) to determine whether the query can return the correct answers. Existing works for KBQA mainly focus on converting input questions into corresponding logic formats, such as SPARQL queries. However, since these works are largely decoupled from the knowledge base, the converted query may be ineffective. In this paper, we propose a novel semantic matching-based approach to model the query intention of the input question by extracting the subgraph of the knowledge base. The generation of the SPARQL query is reduced to semantic matching in the knowledge base to solve the ineffectiveness of the query. Firstly, a semantic query graph is proposed to model the reliable query intention of the input question. The SPARQL query graph could be extracted by matching the semantic query graph in the knowledge base. Secondly, an embedding-based method is developed to represent different forms of questions and queries in a common space. It is easy to detect semantic loss between the question and the converted query with the common representation. Finally, a data-driven semantic completion technique is presented to reduce the semantic loss by expanding the incomplete SPARQL query in the knowledge base. The experiments evaluated on benchmark datasets show that the proposed approach significantly outperforms state-of-the-art methods in efficiency and effectiveness. The code is available at https://github.com/offerhub-ht/gMatch.