Testing improvements to the rating & review system

[visualstudios]

1 hour ago, terrygrantvo said:

I would argue the opposite is true. I don't think most buyers give two giggles what your score is as long as it's not alarmingly lower than everyone else's. They look at the price, and then your work, and decide if you're going to make them look good or not, and that's about it.

Even if you were right about this, the new system isn't a going to be any better a gauge of a sellers value or performance, as the lower ratings we're all destined to have are going to come mostly from buyers who would have normally left 5 stars.

 

This new system is literally designed to make it harder to leave a 5 star review, period. That's it. No matter how Kesha or anyone else tries to spin this, it's pretty clear what they're after here, and it isn't 'transparency'.

Indeed.

Of course, if you have 20k reviews prior to this new review system, you know that you can receive hundreds of 4 star reviews (that seem to be the norm now) and retain your 5 star average for years to come (an average that was obtained when 5 stars reviews were the norm), therefore appearing, to the unsuspecting buyer, to deliver a better service than everyone else.

This new system is a net positive for you, then, since it will give you an unfair advantage versus the people who will be impacted directly (and immediately) by it. Yes, everyone's average score will drop, but some will drop much faster than others, while getting the same review average going forward.

Interesting.

Edited December 14, 2023 by visualstudios

[donnovan86]

2 hours ago, terrygrantvo said:

They look at the price, and then your work, and decide if you're going to make them look good or not, and that's about it.

Well I bought some stuff here and there and the first thing I look for is the star rating. Also, then I narrow down with samples and prices. But whomever tells you they don't give 2 giggles about stars, they are lying. Obviously it's the first thing that grabs your attention, along with the gig thumbnail. 

2 hours ago, terrygrantvo said:

This new system is literally designed to make it harder to leave a 5 star review, period. That's it.

I am pretty sure I was the first one who said it in this thread, clearly they want to differentiate sellers. Because again, there's decision paralysis and many buyers find it hard to figure out what seller is ideal for them. It takes time to go through dozens of sellers with 5 stars. 

I voiced my concerns at the beginning of this thread, I don't think saying anything more about this system will make a difference. Fiverr will implement it, maybe they change it a little bit, but as I said, they implemented the reviews for canceled orders without major changes, and I think that will be the case here. The bar is already lowered to 4.2 stars when it comes to losing levels so.. they already covered that front. At this point, we can expect even more random reviews from people that liked our work, yet they rate it 3 stars. Not sure if you saw above, but I had a 1 star review from someone that liked my work, it was late on their side and just wanted to close the order to go to sleep. Needless to say, that mistake from my buyer haunts me to this date, since the review is there, a 1 star review with a positive comment... I think we can expect even more of these.

Although I will say, the system itself seems more intuitive, I don't see how people leave these random reviews. It even shows the final number of stars at the end before you publish the review. So I think buyers that leave these bad reviews with the new system either don't care, they randomly choose stars, or that's exactly the review they want to leave. 

[lalitsdmittal]

I have mix feeling about this update

[donnovan86]

48 minutes ago, visualstudios said:

This new system is a net positive for you, then, since it will give you an unfair advantage versus the people who will be impacted directly (and immediately) by it. Yes, everyone's average score will drop, but some will drop much faster than others, while getting the same review average going forward.

 

Well those bad reviews also come with a bad private review. So.. the overall performance drops very quickly, regardless of how many thousands of reviews you might have. You have to keep that in mind. Granted, the review average will take a while to change, but people can easily sort based on the number of stars, latest reviews and see if you are receiving very bad reviews in recent months or during the past year.

At least that's what a dilligent buyer will do. Those that just buy when they are in a rush or don't really care about the seller/their work/their performance will always end up leaving a random review. That's my experience anyway. 

Obviously some will have a lower average when compared to others, that's to be expected. It depends on how many orders you had on the platform, when you joined, also the type of work you do. Some niches are more complex, others are simpler. But then again, people will compare you with the average score of people within your niche. And in the case of a person with 100 great reviews and 1 bad review, even if the average is lower, you clearly see it's due to a single review, one that might be from a nutso 🙂 Again, every buyer should do their due dilligence. 

[visualstudios]

45 minutes ago, donnovan86 said:

Well those bad reviews also come with a bad private review

Those are not bad reviews. That's the point - they are normal reviews now.

Before, a 4 star review was indeed bad - because most reviews were 5 stars. If, under the new system, everyone gets 4 star reviews, they are no longer bad, they are normal. Reviews are not bad or good by themselves - they are bad or good by comparison. I'd rather have 4 star reviews in a platform where all my competition has 3 stars, than have 4.7 in a platform where all my competition has 5 stars.

But if the new system is aimed at making the average review 4 stars (or even 4.5), whereas before the average review was 5 stars, those that accumulated a ton of 5 star reviews under the old system have a huge advantage.

That's the problem here. If everyone was starting from 0, it would be fair to have a system that makes the average review be 3 or 4 stars for everyone.

But if some people already have thousands of reviews under a system that made it easier to get 5 star reviews, they'll have an unfair advantage over people who need to start under a system that makes those 5 star reviews much harder to get.

It's like if you were in school, doing easy tests, and getting 100% on all of them. Suddenly, a new teacher comes in, makes the tests much harder, and people start getting 80%, 70% on them. At the end of the year, all the tests are averaged for your final score. The thing is, some students have been there since the beginning, and some students have taken thousands of easy tests, while others have just started, or have taken few easy tests in the same time. Doesn't matter how good the new students, or the students that have taken the easy tests are - their final score will be worse. Not because they're worse, but because the tests are harder, and they don't have the easy tests to average up.

Just a quick thought experiment - imagine if it was the other way around. If reviews on Fiverr had been, for years, averaging 3 stars. The best profiles on the platform were like 4 star average. Suddenly, Fiverr reworks the review system, and everyone starts getting 5 star reviews. New users, or users with fewer reviews, would have higher averages than the people with thousands of reviews. How would they feel about that?

The problem here is simple, and it's easy to see - two different review systems, who lend themselves to different end results in terms of ratings, are being equally counted. There will be no difference, from the buyer's perspective, between old and new reviews. And there should be - because the old 5 star reviews are easier to get than the new 5 star reviews.

The conclusion is simple - you can't change a review system in a way that impacts the average review left by the client. If you do that, you're either invalidating all previous reviews on the platform, or making the system inherently unfair. If you increase the average review score, it's unfair for those with a ton of reviews. If you decrease it, it's unfair for those with few reviews.

Edited December 14, 2023 by visualstudios

[donnovan86]

1 hour ago, visualstudios said:

And there should be - because the old 5 star reviews are easier to get than the new 5 star reviews.

Just because people don't understand the new system yet and leave stars randomly, that doesn't mean it's harder to acquire 5 star reviews though. Buyers that already know the platform very well aren't inclined to lower their review scores. I think the problem stems from new buyers, then again it happened in the past too.

Not sure if you saw when I said it, but I have a 1 star review from a buyer, yet the text was positive and he was happy. But because it was late night for him, he just wanted to close the order and not receive any nagging emails from Fiverr. And I had to suffer due to his lack of attention. And it's not a half a star or a star dropped from the score, it was the most negative score you can give to a seller, all because he was not careful and obviously I couldn't report it due to a potential feedback manipulation warning. Even if it was an obvious mistake.

1 hour ago, visualstudios said:

If you increase the average review score, it's unfair for those with a ton of reviews. If you decrease it, it's unfair for those with few reviews.

As it was said at the beginning of this thread, they are trying to find a way to make it fair for everyone. Not sure how that will work or how will it look. Just like you, I am concerned it will affect my account as well. 

We'll see, I am sure there are lots of calculations happening in the background. For now, we don't have any idea how the final version of this review system will look. As I said, in my opinion, it's not as bad as some people say, it's not armageddon for crying out loud. It's just some buyers showing how much they (don't) care about how they rate their seller. But as I said, it happened to me in the past multiple times and I know from browsing my category that I am not alone. That's why I try to vet my clients first these days, and pick only those that actually seem good to work with. 

I respect your opinion but as always, we are on a different side of the coin. I agree that the new system needs improvements, emoji aren't really the way to go for a professional platform in my opinion. But removing a category and leaving just 2 is a step forward. 4.5, 5 and 4 are better than 4.3 and 4.7. Many times, people were just removing a star from the 3 categories just to make a point, they don't leave perfect scores.. Plus as I said, just like the canceled order reviews, this will go onward regardless, with some slight changes based on our feedback. But the core idea will be the same, more or less. 

If anything, I see this as a normal reaction that people have to change, because we are all creatures of comfort. I would prefer the previous rating system because I am used to it, but I will get used to this one, just like I switched to the star model from the thumbs up/down system. 

For now, this is not the final version, I am sure they wil adjust it, but at least for me, it's not as bad as others make it seem. 

1 hour ago, visualstudios said:

New users, or users with fewer reviews, would have higher averages than the people with thousands of reviews. How would they feel about that?

It would be easier to increase your average since you're getting only good and excellent reviews. Plus, people can still see your seniority on the platform and again, bad reviews will stick like a sore thumb regardless. I don't know what the final version of their math will be, but they said they will try to ensure everything is fair. They switched from thumbs up/down to a star rating and no one really complained that much, even if it was a massive change. Now they remove one of the 3 criteria and it's the end of the world 🙂 

I can only assume that .7 reviews will become .0 reviews and .3 reviews will become .5 reviews. For example, if you had 4.7, it would be a 5.0 review, a 3.3 review would be a 3.5 star review. That makes the most sense to me, and it would also be fair towards sellers too. Other than that, fixed number 1, 2, 3, 4, 5 star reviews would remain the same. Honestly, that would solve the conversion and still be ok for sellers too. Again, my opinion, but I can't think of a better one when it comes to converting old reviews to the new system. Since the problem are those .3 and .7 reviews that can mess up the conversion. Round them up🙂

[rad_graphic]

Fiverr should not try this beta testing with FIRST TIME BUYERS in Fiverr. First-time buyers don't know how the review systems work here. They might leave a low rating without knowing they are giving a low rating. This has actually happened to me today with a FIRST TIME BUYER. That's why I'm posting it here.  

[lamspyrou]

On 12/7/2023 at 12:59 AM, katakatica said:

I think I'll have to echo some of the others - this feels potentially concerning to me. On one hand, I do think that buyers seem overwhelmed by the review system at the moment (it's not a big deal but if you order in bulk it adds up...) on the other, I'm not sure emojis are the solution. 

I know that certain places (like coffee shops / etc. ) use this system, but I'm worried that it's just different on a relatively smaller scale. 

Very good would (in theory) be a 4 star review now. Somehow when I see the four stars, as a buyer I recognize that as imperfect (as in - I could give them more stars if they did better) but very good is already... Very good? 

If we end up punished for it without the buyers realizing that it's, in fact, a bad rating (granted that things stay the same!) I feel a bit worried. 

(Basically, I know it's similar now but when choosing between happy emojis it feels like it'll be easier to rate lower? I'm guessing that's partially the point though...

That being said, I'm not huge on it but I'm sure I'll get used to it! Simplification is a good idea in itself!

You were 100% correct. This happened to me just now.

The buyer was left satisfied with a 'great experience' and said 'great communication' but still left a 4 star review. After completing over 2,500 order on this platform, I am sure that at least the 'Seller Communication Level' would be 5 stars instead of 4 stars in this case. (Great is not very good).

Hence, I agree that the buyer has 'punished' me without him knowing about it just because the 'exceptional' threshold is just too high. 

Someone said above that it would be more consistent to be: 

Very poor -> poor -> average -> good -> very good (or this could just be great and not exceptional).

With 'Exceptional' being a rare occasion and the buyer can, thus, tip. 

Fiverr is just trying to differentiate sellers even more. If more sellers have ratings of 4.7, 4.8 etc, then the 5 star rating will have more value. Nevertheless, this will drive sellers into the ground with much more demand, effort and work. You cannot simply deliver 'Exceptional' work in every single order. Sometimes you just do the work according to the requirements, why do we have to give more and go 'above and beyond' every single time?

This raises the expectations of clients that *every single time* they will get more than they ask for. This is a massive risk for sellers. If clients have such unrealistic expectations then it will definitely drive sellers into the ground with a lot more work.

 

[biggerant]

Now I am afraid of reminding a buyer to leave feedback about my services too. 

[biggerant]

If you guys look at different profiles of any niche, you can clearly notice that now every seller is getting 4.7, 4.5 or even 4 within a week. this new system is already implemented on all categories, this will change the whole scenario of the ranking system. looks like it's bye-bye for a 5-star rating same as a top-rated badge even when you feel you deserve but the system does not.. 😞  

[chrisaxworthy]

Had a perfect experience with a client today, job done within 1 day when they said it could be done within 5, fully communicated every step of the way and did everything they asked and even got a tip but got a 4.5 review and a message telling me how great It was. I gotta say that this I think will be the new normal for sellers going forward to get a great review but a 4.5 rating under this new system. 

[chrisaxworthy]

4 minutes ago, biggerant said:

If you guys look at different profiles of any niche, you can clearly notice that now every seller is getting 4.7, 4.5 or even 4 within a week. this new system is already implemented on all categories, this will change the whole scenario of the ranking system. looks like it's bye-bye for a 5-star rating same as a top-rated badge even when you feel you deserve but the system does not.. 😞  

Yes I just noticed that looking at other sellers profiles that their scores are dropping 

[melanielm]

Emojis are unprofessional. Period.

With the new system, someone (like me) who has a lot of five stars will suddenly start getting 4ish stars. It will look like our quality or service skills have dropped. Buyers will think "Well, they must be slacking off these days. I'll go with someone else!" This is damaging to our business (and Fiverr's!)

The "value for your money" bit makes it almost impossible to raise our prices. Repeat buyer may now mark us down for that because they used to get the service for less. I was planning to raise my prices in the new year again. Not any more!

Instead of getting rid of poor quality and scam sellers, Fiverr is pushing away established ones. How is that good for business?

Edited December 15, 2023 by melanielm

[visualstudios]

4 hours ago, lamspyrou said:

Fiverr is just trying to differentiate sellers even more.

The thing is, if everyone gets 4 star reviews under the new system, it's not differentiating anything. It's just lowering the overall level.

[venonusa]

@Kesha when this new system comes in, all sellers Level is expect to drop, so sellers with first level or second level or Top Rated will move to new sellers level..... or Fiverr will reduce the target that sellers has to keep to maintain its level? ( Since now to be a Level 1 or 2 or Top Rated need to have at least 4.2 stars )

[biggerant]

So here is some screenshot of my few reviews of this week. there are different options going on, in one order it has 2 options with "Quality of delivery" and in 2nd order it has 3 options with "value for money" and in 3rd order it's the old one as before. it means Fiverr is testing more than one thing. interesting

Edited December 15, 2023 by biggerant

[sripra9007]

Testing improvements to the rating & review system

Hi @Kesha
Unfortunately it seems that this "testing" is going to have a long-lasting negative impact on the hard-working sellers (and possibly to Fiverr as well to some extent). Not sure how much thought was given into before proceeding with this testing, but looking at the discussions/after-effects, it seems it's not doing any good to anyone, neither to Fiverr nor it's customers nor the sellers.

After 3.5 years of working here on Fiverr, after getting 1000+ ALL 5* reviews (except just one which was done by a customer by mistake), today I received my 1st 4* review on an order, credit goes to this Testing. As you can see from my profile, I have ALL 1000+ 5* reviews on all orders , if I receive a 4* due to some ongoing testing, it's not something desirable. 

 

 

I take pride in my work, and strive to earn these 5*s. This is a bad spot on my profile (who knows such bad spots may get frequent in future due to this new testing feature by Fiverr).

IMHO, a "Testing" should ideally be done with a limited/selected group of audience, and should not be done after rolling out the implementation in live production environment. 

If you are calling it a "Testing", IF in near future Fiverr decides NOT to proceed further with this change, are you going to change all these ratings (that the sellers have received due to this testing) to 5* ?

If you call 4* a Very Good, think of a seller who gets such "Very Good" on each of their, say, 20 orders. Will they get a level, or get demoted?

If Fiverr's Level cutoff is 4.2 rating, how can a 4* be termed as "Very Good"? The choice of words/verbiage with the emojis (Oh, we are all talking about Fiverr PRO and then comes the emojis on such a professional site) definitely needs to be restructured.

Please rethink about this, if possible.

[venonusa]

4 hours ago, sripra9007 said:

Testing improvements to the rating & review system

Hi @Kesha
Unfortunately it seems that this "testing" is going to have a long-lasting negative impact on the hard-working sellers (and possibly to Fiverr as well to some extent). Not sure how much thought was given into before proceeding with this testing, but looking at the discussions/after-effects, it seems it's not doing any good to anyone, neither to Fiverr nor it's customers nor the sellers.

After 3.5 years of working here on Fiverr, after getting 1000+ ALL 5* reviews (except just one which was done by a customer by mistake), today I received my 1st 4* review on an order, credit goes to this Testing. As you can see from my profile, I have ALL 1000+ 5* reviews on all orders , if I receive a 4* due to some ongoing testing, it's not something desirable. 

 

 

I take pride in my work, and strive to earn these 5*s. This is a bad spot on my profile (who knows such bad spots may get frequent in future due to this new testing feature by Fiverr).

IMHO, a "Testing" should ideally be done with a limited/selected group of audience, and should not be done after rolling out the implementation in live production environment. 

If you are calling it a "Testing", IF in near future Fiverr decides NOT to proceed further with this change, are you going to change all these ratings (that the sellers have received due to this testing) to 5* ?

If you call 4* a Very Good, think of a seller who gets such "Very Good" on each of their, say, 20 orders. Will they get a level, or get demoted?

If Fiverr's Level cutoff is 4.2 rating, how can a 4* be termed as "Very Good"? The choice of words/verbiage with the emojis (Oh, we are all talking about Fiverr PRO and then comes the emojis on such a professional site) definitely needs to be restructured.

Please rethink about this, if possible.

you forgot that you have to maintain 4.2 to be a level 1,2, top..... for who in the past have archived a lot of 5 star reviews "maybe" will remain in their level, but a new seller will not be able to reach 4.2 stars to reach the first level.... 

This may mean that Fiverr is thinking of demoralizing new sellers so that they leave the platform and only "old" sellers with a good history reviews can continue working... or they are thinking to set a new "Seller Level"... example reducing level 2 to 3.2 stars.... @Kesha can you clarify it for us?

[donnovan86]

32 minutes ago, venonusa said:

but a new seller will not be able to reach 4.2 stars to reach the first level.... 

Pretty sure these stats are covered for 60 days. So even if there are issues now, you can recover and regain the level. That hasn't changed, as far as I know.

[venonusa]

Just now, donnovan86 said:

Pretty sure these stats are covered for 60 days. So even if there are issues now, you can recover and regain the level. That hasn't changed, as far as I know.

let's see what @Kesha say

[breals]

@artsbrand

Is this your option? or what ChatGPT is telling you?

[mandyzines]

So, does anyone have something in their order requirements asking whether or not buyers have read and understand the gig description and FAQ's (to help form their expectations based on precisely what you offer)? If so, does this help?

I don't offer screaming, and state that, yet people order with the expectation that they'll get it.

Edited December 16, 2023 by mandyzines

[donnovan86]

29 minutes ago, mandyzines said:

So, does anyone have something in their order requirements asking whether or not buyers have read and understand the gig description and FAQ's (to help form their expectations based on precisely what you offer)? If so, does this help?

 

Ideally, I would recommend to have all the necessary info in the gig description. By the time they get to the gig requirements, that means they already placed the order, so it's a mess. 

I try to share all the info either in the gig description, gig package info, anywhere I can to ensure people read. But even then, you will have situations when buyers just order expecting you to be an expert in anything, as you said in your own message, you don't offer screaming but they expect that. So, you can only do so much. 

[dereck_s]

So here's a crazy idea. Remove the public star rating system that everyone sees and just make it where as long as you're .... I don't know 4.7 or above you keep your level (like how it's done with literally every other metric including seller rating). The star system really is just trust right?

You see someone with a ton of 5 star reviews and you can trust reasonably that they deliver a good product. The issue is that everyone on the first few pages is 4.8-5 stars already so the star system really doesn't serve a function publicly to the buyer.

Edited December 17, 2023 by dereck_s
added "publicly to the buyer"

[donnovan86]

3 hours ago, dereck_s said:

The issue is that everyone on the first few pages is 4.8-5 stars already so the star system really doesn't serve a function publicly to the buyer.

That's the reason why they are changing things. They are making it a bit harder for people to obtain 5 star reviews. That will potentially make it easier for buyers, so there is a focus here on streamlining the buyer experience. However a lot of sellers are disappointed that it's all at their expense and in some cases they might have an unfair number of stars, even if the buyer really liked how they worked.