The invention discloses a loading and unloading manipulator system of a broaching machine, which comprises an executing mechanism, a driving mechanism and a hydraulic control mechanism, wherein the driving mechanism comprises a waist seat, a bull gear, a pinion, a stepping motor, a waist revolving shaft, tapered roller bearings, a fixed upright and a rack cylinder; the executing mechanism comprises a rising oil cylinder, a rising oil cylinder piston, a rising oil cylinder piston rod, a rising guide rod, a sleeve, a lifting platform, a rolling bearing, a support plate, a support frame, a translation oil cylinder, a translation oil cylinder piston rod, a guide rod, a guide bushing, an annular sleeve and a manipulator; the manipulator comprises a paw and a telescopic oil cylinder; and the hydraulic control mechanism comprises a three-phase alternating current asynchronous motor, a hydraulic pump, an oil filter, an oil tank, a two-position two-way solenoid directional valve, a one-way valve, a pilot overflow valve, a three-position four-way solenoid directional valve, a speed regulating valve and a hydraulic cylinder. The system can guarantee the positioning control precision, and reduces impact and vibration to the greatest extent, so that the safety of the system is guaranteed.