A device for detecting polycyclic aromatic hydrocarbon in a water sample on site comprises a sample liquid tube, a cleaning liquid tube, an eluent tube, a regeneration liquid tube and an enhancement liquid tube, wherein each of the sample liquid tube, the cleaning liquid tube, the eluent tube, the regeneration liquid tube and the enhancement liquid tube is connected with a peristaltic pump, a first peristaltic pump is connected with a first three-way solenoid valve, a second peristaltic pump is connected with a second three-way solenoid valve, a third peristaltic pump is connected with a third three-way solenoid valve, a fourth peristaltic pump is connected with the third three-way solenoid valve, the third three-way solenoid valve and the second three-way solenoid valve are connected, the second three-way solenoid valve is connected with the first three-way solenoid valve, the first three-way solenoid valve is connected with an adsorption column, the adsorption column is connected with a detection cell, a fifth peristaltic pump is connected with the detection cell, and the detection cell is connected with a Raman spectrometer. The invention further provides a method for detecting the polycyclic aromatic hydrocarbon in the water sample. With the adoption of the device and the method, analysis is rapid, the sensitivity is high, a few samples are used, the application range is wide, operation is simple and convenient, repeated usage can be realized, and carrying is convenient.