A unitary graphene matrix composite comprising: (a) a unitary graphene matrix containing an oxygen content of 0.001% to 10% by weight, obtained from heat-treating a graphene oxide gel at a temperature higher than 100° C. and contains no discrete graphene platelets derived from the graphene oxide gel; (b) a carbon or graphite filler phase selected from carbon or graphite fiber, carbon or graphite nano-fiber, carbon nano-tube, carbon nano-rod, meso-phase carbon particle, meso-carbon micro-bead, exfoliated graphite flake with a thickness greater than 100 nm, exfoliated graphite or graphite worm, coke particle, needle coke, carbon black or acetylene black particle, activated carbon particle, or a combination thereof. The carbon or graphite filler phase is preferably in a particulate, filamentary, or rod-like form dispersed in and bonded by the unitary graphene matrix. This composite exhibits a combination of exceptional thermal conductivity, electrical conductivity, mechanical strength, surface hardness, and scratch resistance.