Polycyclic aromatic hydrocarbons (PAHs) in urban sewage sludge are becoming serious environmental pollution and phytoremediation is a new technology for the remediation of soil organic pollutants. The purpose of current study was to compare the phytoremediation effect on PAHs between Awnless Brome and Alfalfa and to explore the potential mechanism of the phytoremediation effect. By pot experiment of Awnless Brome and Alfalfa in greenhouse, we compared the repaired effect of PAHs in the sludge, soil enzyme activities near plant rhizosphere area and bacterial species diversity. The results showed that PAHs in the sludge can be restored by the two plants directly. Awnless Brome had a 7% higher removal rate for PAHs than Alfalfa in total. The range of removal rates of single PAHs was about 24.69%~97.69% for Awnless Brome, and 29.69%~97.72% for Alfalfa. As compared with Alfalfa, Awnless Brome had a removal rate of 53%, 13% and 9% higher for PAHs with 2, 3 and 4 rings, 5% and 11% lower for PAHs with 5 and 6 rings, respectively. We also found that the soil enzymes activities and microbial diversity were related with the removal rate of PAHs. The activities of catalase and polyphenol oxidase near Alfalfa root zone were significantly higher than Awnless Brome root zone in sludge, which was opposite for dehydrogenase. Using the PCR-DGGE method, we found one common microbe band related PAHs degradation in rhizospheric area of Awnless Brome and Alfalfa, two specific microbe bands for Alfalfa, and six specific microbe bands for Awnless Brome. In conclusion, the two plants, Awnless Brome and Alfalfa, would be directly used for remediation of PAHs-polluted sludge.