This study attempts to analyze how precipitation and vegetation affect water yield in Haihe watershed. Based on the annual runoff and meteorological data, the water balance equation and the calibrated Zhang model were used to simulate the effects of precipitation and vegetation on water yield. The simulated results showed that the effect of precipitation on water yield was linear in mountainous area of Haihe watershed, and a 1 mm change (increase or decrease) in precipitation will cause a 0.51 mm change in water yield; under the mean annual precipitation of 534 mm, the effects of decreasing water yield by per unit cover ratio for forest, grassland, and farmland were 1.26, 0.47, and 0.69 mm, respectively. The decreased water yields for forest, grassland, and farmland were associated with aridity indexes by negative power functions, and indicate that the effect of vegetation on water yield was smaller in drier areas. For three vegetation types the effects of vegetation on water yield were similar for crops and grass, and were much larger for forest.