Same happens to me, despite of getting all 5 stars, and good private feedback, still the score is dropping. On what basis they are saying negative impact on:
Client satisfaction
Conflict-free orders
Effective communication
Even client is giving 5 stars and client himself saying good communication, and he is very happy, then what you guys want. They don't have proper metrics. I know fiverr is copying upwork, but upwork atleast have a kind of criteria, but fiverr testing things that effecting sellers allot.