Jump to content

The Impact of Monthly Evaluations - Four Months In


Recommended Posts

A few months back, I did a comparison of the impact of the first St. Level’s Day, following the first monthly evaluation. We’ve now had four evaluations in total and I thought it would be interesting to see what that has done to the number of gigs on Fiverr, at each seller level.

With this in mind, I captured the number and percentage of gigs by level across 16 popular categories in the Fiverr marketplace - two under each of the eight top-level categories. I measured them on two dates, Friday 12 January and this morning, Thursday, 3 May.

I have shown the “before” and “after” results below, together with the difference between the two days. A few points to bear in mind as you read through the tables:

  • This only measures the total number of gigs by seller level in each category. It doesn’t measure number of sellers - however, number of gigs should be an OK proxy.
  • This is publicly available information that I took simply by visiting each category, filtering by level of seller, and copying the results.
  • Highlights are purely there as a visual aid - I will explain this a little more in the notes under each table.

The Day the Earth Stood Still (Baseline) Table - Friday 12 January

image.png.febdb813f969f82d75cfa25234fc2044.png

The yellow highlighting on this table simply shows what percentage of gigs by level are above the average for that level of gigs as a whole.

Armageddon Table - Four Months In - Thursday 3 May

image.png.10a484ab6d60d1d2a5f2245eed500b7d.png

The highlighting on this table shows us where the biggest changes are.

Deep Impact Table - Changes Between the Two

image.png.4f073d840acb512ff4f8d5524d5b7f0d.png

Finally, this shows where the changes are. Again, the highlighting shows the areas that moved the most. Note that the percentage numbers here are absolute not relative.

This means that if 10% of the gigs were at a certain level before, and only 5% are now, I will have shown a difference of -5% (an absolute value) not -50% (a relative one). This is probably only a definition that matters to stats geeks like me, so you can safely ignore it!

So, what does this tell us? Here are my initial thoughts:

  • The overall number of gigs between the two days went up quite signifcantly, by almost 16,000 gigs, so in aggregate, monthly evaluations are not dirving sellers off of the platform.
  • 89% of all gigs are now in the “New Seller” or “L1 Seller” level, compared to 69% before the change.
  • The “video” and “marketing” categories saw the biggest overall percentage shifts away from L2.
  • There are only about 3% fewer TRS on the platform than before monthly evaluations.
  • The biggest impact has been in demoting L2 sellers.
  • Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

I hope this is useful, and I would be fascinated with your thoughts. Let’s hear them!

Note: My maths is not impervious - if there are any errors in my insights or calculations, please point them out to me in a private message, and I will correct them. Additonally, this is not a thread to complain about losing your level, there are plenty of other places you can do that. Thanks!

Link to comment
Share on other sites

A few months back, I did a comparison of the impact of the first St. Level’s Day, following the first monthly evaluation. We’ve now had four evaluations in total and I thought it would be interesting to see what that has done to the number of gigs on Fiverr, at each seller level.

With this in mind, I captured the number and percentage of gigs by level across 16 popular categories in the Fiverr marketplace - two under each of the eight top-level categories. I measured them on two dates, Friday 12 January and this morning, Thursday, 3 May.

I have shown the “before” and “after” results below, together with the difference between the two days. A few points to bear in mind as you read through the tables:

  • This only measures the total number of gigs by seller level in each category. It doesn’t measure number of sellers - however, number of gigs should be an OK proxy.
  • This is publicly available information that I took simply by visiting each category, filtering by level of seller, and copying the results.
  • Highlights are purely there as a visual aid - I will explain this a little more in the notes under each table.

The Day the Earth Stood Still (Baseline) Table - Friday 12 January

The yellow highlighting on this table simply shows what percentage of gigs by level are above the average for that level of gigs as a whole.

Armageddon Table - Four Months In - Thursday 3 May

The highlighting on this table shows us where the biggest changes are.

Deep Impact Table - Changes Between the Two

Finally, this shows where the changes are. Again, the highlighting shows the areas that moved the most. Note that the percentage numbers here are absolute not relative.

This means that if 10% of the gigs were at a certain level before, and only 5% are now, I will have shown a difference of -5% (an absolute value) not -50% (a relative one). This is probably only a definition that matters to stats geeks like me, so you can safely ignore it!

So, what does this tell us? Here are my initial thoughts:

  • The overall number of gigs between the two days went up quite signifcantly, by almost 16,000 gigs, so in aggregate, monthly evaluations are not dirving sellers off of the platform.
  • 89% of all gigs are now in the “New Seller” or “L1 Seller” level, compared to 69% before the change.
  • The “video” and “marketing” categories saw the biggest overall percentage shifts away from L2.
  • There are only about 3% fewer TRS on the platform than before monthly evaluations.
  • The biggest impact has been in demoting L2 sellers.
  • Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

I hope this is useful, and I would be fascinated with your thoughts. Let’s hear them!

Note: My maths is not impervious - if there are any errors in my insights or calculations, please point them out to me in a private message, and I will correct them. Additonally, this is not a thread to complain about losing your level, there are plenty of other places you can do that. Thanks!

The overall number of gigs between the two days went up quite signifcantly,

I found this the most interesting number actually, from 119,345 to 190,085.

My thoughts to that aren´t as much "

so in aggregate, monthly evaluations are not dirving sellers off of the platform.

as sellers who lost their levels trying to make up for it with more gigs/new accounts.

Thanks for the data!

Link to comment
Share on other sites

I told you 4 months ago that I put your original post in my calendar… This post came as an unexpected Christmas gift. Thanks!

I told you 4 months ago that I put your original post in my calendar… This post came as an unexpected Christmas gift. Thanks!

Sorry I was four months late (or seven months early)

Link to comment
Share on other sites

A few months back, I did a comparison of the impact of the first St. Level’s Day, following the first monthly evaluation. We’ve now had four evaluations in total and I thought it would be interesting to see what that has done to the number of gigs on Fiverr, at each seller level.

With this in mind, I captured the number and percentage of gigs by level across 16 popular categories in the Fiverr marketplace - two under each of the eight top-level categories. I measured them on two dates, Friday 12 January and this morning, Thursday, 3 May.

I have shown the “before” and “after” results below, together with the difference between the two days. A few points to bear in mind as you read through the tables:

  • This only measures the total number of gigs by seller level in each category. It doesn’t measure number of sellers - however, number of gigs should be an OK proxy.
  • This is publicly available information that I took simply by visiting each category, filtering by level of seller, and copying the results.
  • Highlights are purely there as a visual aid - I will explain this a little more in the notes under each table.

The Day the Earth Stood Still (Baseline) Table - Friday 12 January

The yellow highlighting on this table simply shows what percentage of gigs by level are above the average for that level of gigs as a whole.

Armageddon Table - Four Months In - Thursday 3 May

The highlighting on this table shows us where the biggest changes are.

Deep Impact Table - Changes Between the Two

Finally, this shows where the changes are. Again, the highlighting shows the areas that moved the most. Note that the percentage numbers here are absolute not relative.

This means that if 10% of the gigs were at a certain level before, and only 5% are now, I will have shown a difference of -5% (an absolute value) not -50% (a relative one). This is probably only a definition that matters to stats geeks like me, so you can safely ignore it!

So, what does this tell us? Here are my initial thoughts:

  • The overall number of gigs between the two days went up quite signifcantly, by almost 16,000 gigs, so in aggregate, monthly evaluations are not dirving sellers off of the platform.
  • 89% of all gigs are now in the “New Seller” or “L1 Seller” level, compared to 69% before the change.
  • The “video” and “marketing” categories saw the biggest overall percentage shifts away from L2.
  • There are only about 3% fewer TRS on the platform than before monthly evaluations.
  • The biggest impact has been in demoting L2 sellers.
  • Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

I hope this is useful, and I would be fascinated with your thoughts. Let’s hear them!

Note: My maths is not impervious - if there are any errors in my insights or calculations, please point them out to me in a private message, and I will correct them. Additonally, this is not a thread to complain about losing your level, there are plenty of other places you can do that. Thanks!

Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

It’s a bit confusing when you write 20%.

In january, L2 = 69146

Now, L2 = 25951

That means : 62% of L2 have lost their rating.

We could also say that L2 were accounting for 30.3% of the gigs. Now they account for 10.9% (-20 percentage points).

Link to comment
Share on other sites

Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

It’s a bit confusing when you write 20%.

In january, L2 = 69146

Now, L2 = 25951

That means : 62% of L2 have lost their rating.

We could also say that L2 were accounting for 30.3% of the gigs. Now they account for 10.9% (-20 percentage points).

It now appears in my category, astrology, that new sellers and level two sellers are the main levels sellers are in. We have 8 Top Rated Sellers left in the astrology section. About five lost their TRS level. By far the new sellers are the main type. Two of the TRS are new to the category.

It looks like approximately 70% of all gigs belong to new sellers.

Link to comment
Share on other sites

I doubt if the number of TRS that decreased in all other categories is only 17. Of all categories on fiverr that leaves out quite a few.

The total TRS in all other categories is only 18? Even the “writing” category is left out of that. How many TRS are in the writing category?

There are only 4,019 other sellers in all the other categories? This site is about one tenth the size I thought it was then.

Link to comment
Share on other sites

I doubt if the number of TRS that decreased in all other categories is only 17. Of all categories on fiverr that leaves out quite a few.

The total TRS in all other categories is only 18? Even the “writing” category is left out of that. How many TRS are in the writing category?

There are only 4,019 other sellers in all the other categories? This site is about one tenth the size I thought it was then.

It’s not “all other categories” - “other” on the table refers to “Fun and Lifestyle > Other” - https://www.fiverr.com/categories/lifestyle/lifestyle-services

Link to comment
Share on other sites

I doubt if the number of TRS that decreased in all other categories is only 17. Of all categories on fiverr that leaves out quite a few.

The total TRS in all other categories is only 18? Even the “writing” category is left out of that. How many TRS are in the writing category?

There are only 4,019 other sellers in all the other categories? This site is about one tenth the size I thought it was then.

Edit: okay, so, now, I´ve got this re the Writing & Translation category (before the 1st SLD, after the 1st SLD, today):

TRS ----------- 8.1.2018 – 15.1.2018 – 04.05.2018

Translation ------- 27 -------- 21 ------------- 21

Proofreading ----- 30 -------- 22 ------------ 23

Writing ------------ 319 ------ 204 -----------206

(numbers for Proofreading and Writing column 1 and 2 are from @sydneymorgan 's topic, I´m not quite sure about the numbers for Writing, because writing is made up from quite a few subcategories and I assume there are overlaps)

(another edit: just a moment ago, I got 21 for translation, and now that I went back for a screenshot, it´s 20. :woman_shrugging:)

Link to comment
Share on other sites

A few months back, I did a comparison of the impact of the first St. Level’s Day, following the first monthly evaluation. We’ve now had four evaluations in total and I thought it would be interesting to see what that has done to the number of gigs on Fiverr, at each seller level.

With this in mind, I captured the number and percentage of gigs by level across 16 popular categories in the Fiverr marketplace - two under each of the eight top-level categories. I measured them on two dates, Friday 12 January and this morning, Thursday, 3 May.

I have shown the “before” and “after” results below, together with the difference between the two days. A few points to bear in mind as you read through the tables:

  • This only measures the total number of gigs by seller level in each category. It doesn’t measure number of sellers - however, number of gigs should be an OK proxy.
  • This is publicly available information that I took simply by visiting each category, filtering by level of seller, and copying the results.
  • Highlights are purely there as a visual aid - I will explain this a little more in the notes under each table.

The Day the Earth Stood Still (Baseline) Table - Friday 12 January

The yellow highlighting on this table simply shows what percentage of gigs by level are above the average for that level of gigs as a whole.

Armageddon Table - Four Months In - Thursday 3 May

The highlighting on this table shows us where the biggest changes are.

Deep Impact Table - Changes Between the Two

Finally, this shows where the changes are. Again, the highlighting shows the areas that moved the most. Note that the percentage numbers here are absolute not relative.

This means that if 10% of the gigs were at a certain level before, and only 5% are now, I will have shown a difference of -5% (an absolute value) not -50% (a relative one). This is probably only a definition that matters to stats geeks like me, so you can safely ignore it!

So, what does this tell us? Here are my initial thoughts:

  • The overall number of gigs between the two days went up quite signifcantly, by almost 16,000 gigs, so in aggregate, monthly evaluations are not dirving sellers off of the platform.
  • 89% of all gigs are now in the “New Seller” or “L1 Seller” level, compared to 69% before the change.
  • The “video” and “marketing” categories saw the biggest overall percentage shifts away from L2.
  • There are only about 3% fewer TRS on the platform than before monthly evaluations.
  • The biggest impact has been in demoting L2 sellers.
  • Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

I hope this is useful, and I would be fascinated with your thoughts. Let’s hear them!

Note: My maths is not impervious - if there are any errors in my insights or calculations, please point them out to me in a private message, and I will correct them. Additonally, this is not a thread to complain about losing your level, there are plenty of other places you can do that. Thanks!

The biggest impact has been in demoting L2 sellers.

Almost 20% of gigs lost their L2 or TRS rating and were demoted to a lower level.

yes you got it right amazing article and research 💪

Link to comment
Share on other sites

Awesome data gathering and number crunching! I am not sure what conclusion can be drawn from this. Is the new system good for sellers? Is it only good for a few sellers?

with new system they are promoting quality sellers which is good as level increase more orders with less competition

Link to comment
Share on other sites

Awesome data gathering and number crunching! I am not sure what conclusion can be drawn from this. Is the new system good for sellers? Is it only good for a few sellers?

Awesome data gathering and number crunching! I am not sure what conclusion can be drawn from this. Is the new system good for sellers? Is it only good for a few sellers?

The main takeaway for me is that it has hardly impacted TRS at all, so those performing the very best on the platform are maintaining the statistics they need to stay there.

The biggest fall in numbers was at L2 - that tier was likely overpopulated, and the monthly evals reduced those by almost 20%. At L1 the impact was less severe, with only 6% of gigs demoted. Note that all these figures are aggregated, and will be a combination of gigs being both promoted and demoted.

Interestingly, there has been a combined 25% fall in the number of L1 and L2 gigs, and a 25% rise in new, “unlevelled” gigs. The total number of gigs on the platform also increased by 15,000 over that time, or about 4,000 per month (about 250 per category, on average).

Link to comment
Share on other sites

Drawing conclusions is the hard part. When I take translation in my specific language pair, for example, there was 1 TRS, then none, then 1 again but a different person who wasn´t TRS before.
So, I could draw the conclusion that 100% of all TRS got demoted, and that 100% of the people who got TRS were not TRS before and that no TRS who got demoted regained TRS. 😉

The total number of increased gigs is the most striking number for sure, the question here is if that´s simply because numbers of gigs rise consistently since Fiverr exists, could have nothing at all to do with the evaluations as far as I can see. We need more ! numbers, preferably from Fiverr (too)! 🙂

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...