Roarke finally pushed him off of him, seeing that Fred only wanted someone to blame for his unfortunate circumstances of birth and upbringing.
Roarke felt hurt that the person he had thought of as a friend had harboured such amount of hatred towards him, over something he had no control over.
He understood that he just needed someone to blame and pass his frustrations on. If only he had expressed his feelings a little, perhaps he could have shown him how wrong he was. Maybe things wouldn't have gotten to this extent.
Fred had been so blind in nursing his blames and plotting deceit towards him, that he didn't see how hard he had worked regardless of his background to what and how he was now.