Pedestrians' road-crossing behavior can often interrupt traffic flow and cause vehicle queueing. In this paper, we propose some moving rules for modeling the interaction of vehicles and pedestrians. The modified visual angle car-following model is presented for the movement of vehicles with consideration of the lateral effect of waiting pedestrians. The pedestrians' behavior is summarized as consisting of three steps: pedestrian arrival, gap acceptance, and pedestrian crossing. Some characteristic parameters of pedestrians are introduced to characterize pedestrians' behavior. Simulation results show that the interaction of vehicles and pedestrians lowers the traffic capacity and increases delays to both vehicles and pedestrians.