Linking ecology with evolutionary biology is important to understand how environments drive population and species divergence. Phenotypically diverse Salix species, such as lowland riparian willow trees and middle- to high-elevation multistemmed shrubs and alpine dwarf shrubs, provide opportunities for studying genetic divergence driven by ecological factors. We used amplified fragment length polymorphism (AFLP) to quantify the genetic variation of 185 individuals from nine populations of four Salix species in Taiwan. Our phylogenetic analyses distinguished two riparian species and the separation of riparian species from multistemmed and dwarf shrub species. Variance partitioning for the total data found that environment explained a substantially larger proportion of genetic variation than geography. However, no genetic variation was explained by geography alone when only compared within and between species. Spatially structured regional environmental effects explained more variation than pure environments in most comparisons within and between species, suggesting that unmeasured environmental variables and/or past demographic histories played important roles in shaping population and species divergence. Based on forward selection analysis, annual mean temperature, aspect, and fraction of absorbed photosynthetically active radiation were the most influential ecological factors in shaping genetic variation within and between species. Nevertheless, different combinations of environmental variables correlated significantly with genetic variation within and between species. We identified eight AFLP loci that potentially evolved under selection intraspecifically using different outlier detection methods. These loci correlated with more than one environmental variable, suggesting local adaptation along environmental gradients at the population level.