Pollster Ben Ephson has said he is unperturbed about attacks on him following his controversial prediction about the presidential race.