Employing leaf shape features for plant species recognition, especially for cultivar recognition is very challenging due to the high similarities of leaf shapes across different species and cultivars. In this paper, we attempted a new strategy of depicting leaf shapes by convolving the contour vector functions with Gaussian functions of different widths. The resulting Gaussian convolution vector (GCV) has the following traits that are desirable for shape characterization: (1) It provides an overall description to the shape in which both the curvature features and the proportional relationship of leaf contour are encoded. (2) It is intrinsically invariant to the group transformations including translation, scaling and rotation. (3) It depicts local shape geometry at multiple scales which enhance the discriminative power of the shape descriptors. The 2D histogram that reflects the distribution information of GCV is generated for efficiently yet accurately matching shapes. Two types of leaf image datasets, the middle European Woody plants (MEW2012) and the Flavia leaf dataset that are widely used for plant species recognition, and the soybean leaf dataset we built especially for cultivar recognition, are utilized to examine the effectiveness of the proposed method. The experiments demonstrate that no matter on species recognition or on cultivar recognition, the proposed method achieves higher retrieval accuracies over the state-of-the-art benchmark methods. In addition, a self-overlapped leaf image dataset is built to validate the robustness of the proposed method to self-intersection of leaves.