I did something similar in a vice and a long torx drill bit, but still managed to break the replacement perch head. It was a cheapie replacement and not an original part, so the plastic is more than likely of less quality to the OEM and was brittle, in hindsight it was probably best to maybe warm it up a little before tapping with the hammer.
