Yes, you do need to go. You are compensated for lost work time only if you are going for a medical exam at the insurer's request and the appointment conflicts with your  scheduled work hours.