Testing improvements to the rating & review system

[Kesha]

9 hours ago, vickieito said:

Hi @Kesha, 

This new system for review is repetitive (especially since it has the same questions as the private review) and doesn't make it easier for the buyer to complete a review. Especially since we still have to fill out the private reviews and then deal with the spammy reminders to tip our sellers (even after we've already left a tip).

Things that would make the review system a little bit easier:

1. Let buyers see the total orders a seller has (not just how many reviewed orders they have).

Some gigs already have this, and I don't see why every seller can't have this number right below their seller picture. Some sellers only have 40-60% rated orders, so it looks like they have fewer orders than they really do. New sellers also wouldn't be so devastated when their buyers choose not to leave reviews on their orders.

2. Make the written review optional, especially for repeat buyers or frequent buyers.

After doing about 6 orders with a seller, my reviews start to look the same. It would be nice to indicate what type of buyer I am on Fiverr's platform ("Occasional buyer," or "Frequent Buyer") and if I'm a "Regular Customer" of that seller. That, with my top reasons of why I chose to work with that seller, would be more helpful to prospective buyers than a 5-star rating scale with a generically written review. The same goes for leaving reviews for my buyers. Sometimes I don't have anything new to say, especially to the repeat buyer who just gives me an order and accepts a delivery. I'd just like to rate my experience on a 5-star scale and be done.

3. Allow buyers the option of hiding their IDs when leaving a review. Keep this all in one rating system.

The review could still be shown as a "Verified Order" but buyers could leave a public review without disclosing their IDs to other buyers. This would encourage more buyers to leave a review, especially those who don't want others in their organization/network to know that they are using Fiverr. It would also encourage buyers to be more honest about their experiences (without having a separate rating system like the BSR score). If we want transparency, make it visible and keep it in one place.

4. Stick with the star rating instead of "labeling" buyers' experiences with words.

Not everyone will have the same interpretation of the words you use for seller ratings. Especially if English isn't your first language. It's so much easier for buyers to quantify their experiences qualitatively with stars than with words. 

Many buyers won't know the impact or difference between saying a seller is "Very Good" or "Exceptional." They won't know that sellers with a "Very Good" average rating may be demoted because their average is 4.0 (not 4.2). Other buyers may think that "Exceptional" is a rare thing, and may never give out such ratings. Especially if a seller is exceptional (and then that becomes the norm for that repeat buyer).

 

These are all good points, @vickieito! We appreciate your valuable insights and suggestions and will keep them in consideration for future updates. 

[coerdelion]

Hey, @emmaki - good to see you back here ... always enjoy your posts!

[andywarburton]

Quote

"How was the quality of the delivery in relation to your expectations?"

This seems to be a really silly question to ask because some people in life just have messed up expectations (like for one customer of mine who expected me to impliment the changes in my audit for free even though my gig makes no mention of this). So now they can give me a poor score because THEIR expectations were wrong? That makes no sense.

We see it all the time in life... people expecting gourmet food in Subway or low waiting times for public utility support lines. Peoples expectations rarely make any sense at all.

[vickiespencer]

3 minutes ago, andywarburton said:

(like for one customer of mine who expected me to implement the changes in my audit for free even though my gig makes no mention of this)

I had a buyer for my beta reading gig who expected me to rewrite the parts I told him he needed to improve. When I told him that was not part of the gig, he did not reply, but shortly after that, my CSR decreased. Private reviews already affect the sellers considerably, and now we have an addition to the review system that can easily damage our accounts even more. I feel ill when I think about it.  

[donnovan86]

49 minutes ago, vickiespencer said:

I feel ill when I think about it.  

Hopefully if Fiverr goes through with this change, they remove the need for private reviews. Now buyers have to rate pretty much the same stuff twice? Makes no sense, and a lot of people won't. Aside from those that severely want to damage your reputation. Those will always do 🙂

[venonusa]

15 minutes ago, donnovan86 said:

Hopefully if Fiverr goes through with this change, they remove the need for private reviews. Now buyers have to rate pretty much the same stuff twice? Makes no sense, and a lot of people won't. Aside from those that severely want to damage your reputation. Those will always do 🙂

I thought the same! Many of my clients feel oppressed by this thing ( leaving private reviews and public reviews ) furthermore the private review data is not shared with the seller and this makes everything unclear for the seller and how to improve the service.

 

Since Fiverr are changing the public reviews system I hope that Fiverr team can also change something on the private reviews, by removing them or sharing the data with the seller. ( @Kesha )

Edited December 7, 2023 by venonusa

[mandyzines]

10 minutes ago, venonusa said:

this makes everything unclear for the seller and how to improve the service

And often the seller doesn't know that the reason their impressions have tanked is because of a bad private review. So, they fiddle what seems to be endlessly with their profile and gigs, stressing like crazy-- for months. Then, they become crazy themselves. 

Edited December 7, 2023 by mandyzines

[donnovan86]

17 minutes ago, venonusa said:

Since Fiverr are changing the public reviews system I hope that Fiverr team can also change something on the private reviews, by removing them or sharing the data with the seller.

That won't happen because they are private. If they make these public, they won't be private anymore. So, most likely either those are removed or they remain as always.

[terrygrantvo]

On 12/6/2023 at 2:22 PM, Kesha said:

 Hi @terrygrantvo! This test will not affect the Private Review form. I appreciate your feedback and will pass it along to the product team. 

So it appears that private reviews are staying. They're just going to be there twice now, essentially. First as part of the new public review, and then again privately.

I and struggling to see any reason for this beyond trying to make sure fewer (if any) sellers are able to maintain a 5 star rating.

[donnovan86]

1 hour ago, terrygrantvo said:

I and struggling to see any reason for this beyond trying to make sure fewer (if any) sellers are able to maintain a 5 star rating.

Well most sellers have 4.8 to 5 stars. That makes it very difficult for a buyer to narrow down the best option. 

 

[kawsar_log]

On 12/6/2023 at 8:18 PM, Kesha said:

A streamlined customer review process with an upgraded design and experience.

Emojis have no place on a professional platform, in my opinion. From the perspective of a buyer, it makes the entire platform appear less professional.
Also, using emojis, we are unable to express our true feelings, therefore some buyers may misinterpret your honest reviews!

So Instead of emoji, I would Like to say The Current numeric is as good as it is now!

[kawsar_log]

20 hours ago, vickieito said:

Make the written review optional, especially for repeat buyers or frequent buyers.

I agree with this one; it's a great practical idea that we could need!

[Kesha]

10 hours ago, andywarburton said:

This seems to be a really silly question to ask because some people in life just have messed up expectations (like for one customer of mine who expected me to impliment the changes in my audit for free even though my gig makes no mention of this). So now they can give me a poor score because THEIR expectations were wrong? That makes no sense.

We see it all the time in life... people expecting gourmet food in Subway or low waiting times for public utility support lines. Peoples expectations rarely make any sense at all.

Good point you make there @andywarburton! We appreciate you sharing your thoughts and we will keep them in mind for future updates. 

[Kesha]

4 hours ago, kawsar_log said:

Emojis have no place on a professional platform, in my opinion. From the perspective of a buyer, it makes the entire platform appear less professional.
Also, using emojis, we are unable to express our true feelings, therefore some buyers may misinterpret your honest reviews!

So Instead of emoji, I would Like to say The Current numeric is as good as it is now!

@kawsar_log Thanks for that feedback! I will pass it along to the product team for consideration. 

[Kesha]

10 hours ago, vickiespencer said:

I had a buyer for my beta reading gig who expected me to rewrite the parts I told him he needed to improve. When I told him that was not part of the gig, he did not reply, but shortly after that, my CSR decreased. Private reviews already affect the sellers considerably, and now we have an addition to the review system that can easily damage our accounts even more. I feel ill when I think about it.  

Hi @vickiespencer! Our goal is always to improve the experience for everyone on the platform. We appreciate you sharing your thoughts concerning this new test and we will be sure to keep them in consideration as we work through future updates. 

[emmaki]

55 minutes ago, donnovan86 said:

That won't happen because they are private. If they make these public, they won't be private anymore. So, most likely either those are removed or they remain as always.

An average score of each hidden metric wouldn't exactly give anyone's privacy away, unless the gig/seller was brand new or something. But I can't imagine it's that difficult to program it so that it only displays after 10 private reviews are done. Of course, private reviews aren't about privacy. They were just introduced to save sellers from the agony of their hurt feelings when a buyer publicly gave them any less than 5 stars. Or was it to save buyers from the endless parade of deceased grannies, tragic life stories, and prophecies of doom that would guilt them up into changing that *awful* 4-star review into a 5-star review, saving them from the jaws of demotion?

Could it be that the last big change to reviews - cancellation reviews - which also came with the threshold drop from 4.7 to 4.2 didn't quite make the impression Fiverr hoped it would, so now we're going for this emoji/XP randomizer buttons experience? Will this mean that buyers who actually want to leave a 5-star review now have to play a careful game of whack-a-mole to arrive at precisely the star rating they want to give? 

I'm also willing to bet that we haven't seen how the new review thing actually looks is because nobody can figure out how to display the new stuff without it looking like a dog's dinner of TMI with added emojis because emojis are fun and relatable*. 

*Does not apply to shell-shocked Fiverr sellers who will soon get psychosomatic Pavlovian stress reactions from various common emojis, giving them trauma-based anxiety attacks for the rest of their life.
NEW PRODUCT IDEA FOR FIVERR TEAM: THIS WILL MAKE LOTS OF MONEY!
I propose that the ability to view some sort of obfuscated private reviews be shown to Sellers. This will not only help Fiverr to make more money, but also increase the transparency on-site for The Few - the 25,000+ who, according to Fiverr's latest investors' report are currently coughing up for SP. 

That's far too many sellers, of course, and I feel like SP isn't monetized enough. This new feature could be the analytics of a new, third tier of Seller Plus with a new badge (because badges are free, low cost, and our foolish reptile brains get a dopamine boost from them!), helping Fiverr to wow investors in Q1 '24 and beyond with the take going even higher.

Chuck in something like Upwork's magical feedback eraser too to enhance transparency (it's like private reviews, but for sellers. See? Transparent). And access to an exclusive new feature called "Best Buyer Quotes," which could be like Brief and Match, only there's no notification spam because you go on like, a special board full of demanding quotes for work because they never could figure out how BMs worked and the prospect of crawling through 400,000 5-star sellers for a simple job is giving them hives. Revolutionary new idea. Never before seen or abused on Fiverr.

What's that, you say? It sounds like all my ideas are rip-offs of old ideas? No! I'm just reinventing the wheel, fairly and squarely, like any good product manager does when the old ideas aren't exactly working.

You see, sellers could use pre-purchased Fiverr credits to offer their BBQs. Sellers could grade these offerings using the fire emoji 🔥 and pick the one that's most "fire". If a seller doesn't have enough fires, their BBQ quota depletes faster and they are given a hot dog (🌭) warning, which is essentially telling them their Fiverr career is about to be fried so they need to git gud at making BBQs. Or simply pay for more credits. Which, if I didn't already mention it in this overly-long post, can only be purchased by members of SPPP as a SPPP 'Extra'. 

These Fiverr credits can also be used to purchase Buh-Bye Bad Review tokens to use again buyers who simply don't get your brilliant work. All of this to be locked behind a new level of SP, which I would call Seller Plus Premium Pro because I'm really good at naming things like that. 

Would this be double-dipping with the 20% commission? Absolutely not. These are value-added services for serious Fiverr sellers. The credits could be valid for 30 days only, unused credits will slowly be sucked into in the complex black hole of a unique and opaquely described "60-day rolling action-taker" algorithm (Don't ask me, ask the financiers cackling wildly behind me). 

As you can see, big thinking is at play here with advanced triple-dipping (or maybe quad dips? That's more emojiable with, IDK, 🦵, and sounds better). But I'm sure Fiverr could make concessions, like 10% commissions to offset the reasonable Fiverr Utilized Credit Kinetic Expenses and add a new level of desirability to purchasing SPPP. Buyers can just do whatever so long as it isn't illegal obvs. This includes practicing new and inventive ways to Telegraph their intentions to the unwitting seller. 

We do, after all, need buyers to keep their mental energy for the Trial by Emoji that they will soon endure. 

I'm appalled the best that Fiverr's team could come up with was emoji faces and a few buttons. With a little extra thinking, I've come up with a whole system of pure profit that is also unnecessarily complicated, features emojis, and increases transparency. I accept checks and a tacit admission that the feedback system is a complete mess and you have no idea what to do with it so emoji faces and clicky button to feed Fiverr Neo (wild speculation) it is. 

I am also available for ideas on newsworthy studies. These don't need to include emoji ideas, but they could. You only need to ask 😘

I am not responsible for Fiverr taking any of this seriously. Which they probably won't, to be fair. Will Kesha thank me for my invaluable insights and pass them onto the team? 

[moikchap]

I don't know or care what the rating is of the sellers I've used. I've never made a decision using the number.
The question for me has always been whether their gig description is coherent and the preview images seem to match what I'll be asking for.

I feel like chasing rating accuracy is a fantasy, as if it will be possible for a buyer to find the exact right seller first try.

Maybe get rid of ratings entirely and use Endorsements like LinkedIn does. I'm pretty sure I've seen those tags at the bottom of the review page.
I have basically five adjectives I use for written reviews and every seller gets three. The best thing for me is to show me a list of "Fast, Communicative, Detail Oriented, Hits Spec, Professional" and let me click the ones that I feel match.

Buyers will be able to infer which sellers are good or bad based on how "weak" the endorsed words are. "Okay this fella has ten Fast and eight Detail Oriented but only three Hits Spec, two Communicative, and no Professional". That will very strongly inform me what to expect at a glance where a rating number does none of that.

Edited December 8, 2023 by moikchap

[melanielm]

I'm beginning to feel old. Fiverr commercials with people jumping around and yelling in silly voices and now emoji ratings on a professional platform?

There seems to be a huge number of things buyers have to do to rate sellers. At what point does the annoyance of the rating system start tainting their ratings? 

[emmaki]

1 hour ago, melanielm said:

I'm beginning to feel old. Fiverr commercials with people jumping around and yelling in silly voices and now emoji ratings on a professional platform?

There seems to be a huge number of things buyers have to do to rate sellers. At what point does the annoyance of the rating system start tainting their ratings? 

To be fair, those emojis do translate vaguely into stars, totally not fully explained how in the OP here due to the overwhelming focus on transparency and all of that. 

Speaking of AI, I've been wondering why Fiverr changed its AI policy from "sellers need to disclose upfront" (beginning of year) to "buyers gotta ask or *shrug*" (now). There's quite the uptick of complaints about this on reddit, where buyers have asked and are - according to them, anyway - getting outright lied to their face. Then of course getting artwork that can't really be edited. 

Then there's the buyers who don't know they need to ask and think it's OK because why would a MidJourney artist pretend that they do handrawn art? Can someone from Fiverr possibly address how this TOS helped with their ongoing efforts for transparency/marketplace integrity? I don't get the general impression that these buyers wanted the dancing robots stuff. They just wanted an artist who would like, draw the thing and do it how they said they would do it. What bliss it must be for buyers who don't realize they've been hoodwinked, accidentally fooling everyone else into buying terrible services. You obviously can't trust buyer star reviews anymore.

Still, we may just be old. As an experiment, here's ChatGPT. I asked it to translate "emoji reviews are amazeballs" into emoji: 😃👍📝🔥🎉

So.... "I'm happy to burn my work and celebrate it"... or look on my works, ye mighty, and despair. One of these has multiple spheroid entities, the other doesn't. Hmm. Maybe a 1.8 star review in the end? Emoji math is hard. It's so good that Fiverr is now doing the number-crunching because it's quite clear buyers are unable to review in an transparent, honest and integrity-d way. So Fiverr must use AI to make it better. Dancing robots and emoji reviews for all! 

The good news is that none of this particularly matters in 100 years when we're all dead and our descendants are suffering even more bizarre review systems that go hundreds of levels deep from Fiverr (beyond our comprehension as 21st century tech simpletons). 

Do you remember the good old says when Fiverr focused on heroin(e) chic with swears for their advertising? 

EDIT: This is from the Fiverr subreddit. They're not aware of this test, as far as I know. But this is an interesting comment made today on a post complaining about - you guessed it - reviews.  



 

Edited December 8, 2023 by emmaki
I added a pretty picture.

[lenasemenkova]

It’ll be great. I can feel it. No profile will look suspicious, confusing, or contradictory ever again.

[donnovan86]

10 minutes ago, lenasemenkova said:

It’ll be great. I can feel it. No profile will look suspicious, confusing, or contradictory ever again.

 

That's what I was afraid of, really. Random reviews made by people that have no idea how their small choice impacts their rating.. 

[emmaki]

If only you spoke English better, Lena. Or worse, depending on how fluent the buyer is, perhaps? 

Edited December 8, 2023 by emmaki

[greg_vo2020]

I appreciate the efforts for an improved rating system.  One concern I have is the "value for money" rating.  I consistently have received good ratings for 'the work' over the years.  But that particular rating is not a rating on work.  It is a veeeerrry subjective view of money.  I personally KNOW my rates are lower than other past outlets but that still isn't good enough for people.  In some countries the value is great for vo service.  In some it's not so much maybe. Not to mention the personal opinions about money.  I don't feel this is helpful. 

[moikchap]

So, I just interacted with the new review page, and it opens/shows some tags to select after selecting a smiley.
Then after completing that page, it opened another page of smileys. I was able to exit that page, so I have that going for me at least.

Stacking pages/surprising me with more prompts feels more obnoxious than a dense single page, especially when it's not obviously showing "1/3", "2/3" or however many there are.

Edited December 9, 2023 by moikchap

[donnovan86]

On 12/9/2023 at 2:38 AM, moikchap said:

So, I just interacted with the new review page, and it opens/shows some tags to select after selecting a smiley.

Do they show the final review score before you press Publish? Like it showed in the initial image posted by Kesha?

I mean this: