Author Topic: ESPN’s “CARMELO” projections are hot garbage  (Read 1156 times)

0 Members and 1 Guest are viewing this topic.

ESPN’s “CARMELO” projections are hot garbage
« on: June 19, 2018, 07:55:54 AM »

Offline Roy H.

  • Forums Manager
  • James Naismith
  • *********************************
  • Posts: 35611
  • Tommy Points: -27751
  • 33,333 posts and counting . . .
ESPN decided to project the value of certain trade packages, projected over the next five years. Here’s what they came up with:



https://fivethirtyeight.com/features/the-next-phase-of-nba-superteam-technology-creating-one-from-scratch/?addata=espn:frontpage

Does it strike anybody as odd that Lonzo Ball projects to be better than Kyrie and Hayward combined? That Kyrie and Covington have equal value?  That Pascal Siakim projects to add more value than Hayward?

If your model gives results like that, it’s time to scrap your model.


Once a CrotoNat, always a CrotoNat.  CelticsBlog Draft Champions, 2009 & 2012;
DKC Draft 2015 Champions and beyond...

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #1 on: June 19, 2018, 08:00:40 AM »

Offline BringToughnessBack

  • Jim Loscutoff
  • **
  • Posts: 2925
  • Tommy Points: 327
Whoever the data scientist was behind that, I sure would never want them near my company- Those projections are utter garbage! My guess is they are a Lakers fan no doubt.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #2 on: June 19, 2018, 08:36:14 AM »

Offline SHAQATTACK

  • NCE
  • Paul Pierce
  • ***************************
  • Posts: 27333
  • Tommy Points: 2073
Lakers fans dreaming

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #3 on: June 19, 2018, 08:44:56 AM »

Offline hodgy03038

  • NCE
  • Jim Loscutoff
  • **
  • Posts: 2546
  • Tommy Points: 43
  • Marcus Smart #1 Fan
That list is ridiculous. Ball, Kuzma and Deng? So without Deng which brings the value down Ball & Kuzma should be able to get Anthony Davis with their value.


Hey Jaylen. I lost 519 Tommy Points betting on you guys and lost

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #4 on: June 19, 2018, 09:01:56 AM »

Offline ManUp

  • Don Nelson
  • ********
  • Posts: 8309
  • Tommy Points: 259
  • Rondo doesn't believe in easy buckets...
I'm still waiting on Marcus Smart's breakout into the top 10 that CARMELO predicted last year.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #5 on: June 19, 2018, 09:03:44 AM »

Offline gouki88

  • Ray Allen
  • ***
  • Posts: 3761
  • Tommy Points: 63
That is beyond delirious

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #6 on: June 19, 2018, 09:04:04 AM »

Offline saltlover

  • Ed Macauley
  • ***********
  • Posts: 11231
  • Tommy Points: 2418
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.
“When an alien resides with you in your land, you shall not oppress the alien. The alien who resides with you shall be to you as the citizen among you; you shall love the alien as yourself, for you were aliens in the land of Egypt: I am the Lord your God.”

Leviticus 19:33-34

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #7 on: June 19, 2018, 09:57:44 AM »

Offline action781

  • Antoine Walker
  • ****
  • Posts: 4377
  • Tommy Points: 336
Having a computer model predict how good NBA players will be in the future is an insanely difficult task -- especially for young players where their futures end up being so volatile.  Think about how is a computer supposed to know that Oladipo will be a better player than Anthony Bennett?  Or how much the future value of Oladipo as we fans know it has changed in just the last 12 months.   

As far as I know, we don't have any better models out there than CARMELO so I would say we probably should not scrap it altogether.  It is far from perfect, but gives us some insight to what kinds of players have had productive and unproductive careers in the past.  It is entirely probabilistic and is expected to be inaccurate quite often.  But it can be improved over time with more learning data fed into it.

If I'm an NBA GM, I'm not going to be basing my potential trades off of it.
2018 Mock Trade Deadline Philly 76ers

Guards:  A Bradley / Covington / Redick / Fultz / Bayless / Luwawu / Korkmaz
Forwards:  Saric / Simmons (point forward) / Trev Booker / J Anderson
Center:  Embiid / THE BOBAN / R Holmes

Picks: 2018 LAL 1st (if 1, 6-30), 2019 SAC 1st (if LAL 2-5), 2018 2nds from BKN, NYK, HOU, and 2019 SAC 2n

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #8 on: June 19, 2018, 10:07:08 AM »

Offline action781

  • Antoine Walker
  • ****
  • Posts: 4377
  • Tommy Points: 336
The article itself though certainly is garbage.  It appears to have been written by someone who knows nothing of basketball.  It listed CP3, Harden, and Trevor Ariza as a "Big 3".  It put Brandon Ingram as starting point guard on next season's Lakers team. 
2018 Mock Trade Deadline Philly 76ers

Guards:  A Bradley / Covington / Redick / Fultz / Bayless / Luwawu / Korkmaz
Forwards:  Saric / Simmons (point forward) / Trev Booker / J Anderson
Center:  Embiid / THE BOBAN / R Holmes

Picks: 2018 LAL 1st (if 1, 6-30), 2019 SAC 1st (if LAL 2-5), 2018 2nds from BKN, NYK, HOU, and 2019 SAC 2n

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #9 on: June 19, 2018, 10:11:55 AM »

Offline Roy H.

  • Forums Manager
  • James Naismith
  • *********************************
  • Posts: 35611
  • Tommy Points: -27751
  • 33,333 posts and counting . . .
Having a computer model predict how good NBA players will be in the future is an insanely difficult task -- especially for young players where their futures end up being so volatile.  Think about how is a computer supposed to know that Oladipo will be a better player than Anthony Bennett?  Or how much the future value of Oladipo as we fans know it has changed in just the last 12 months.   

As far as I know, we don't have any better models out there than CARMELO so I would say we probably should not scrap it altogether.  It is far from perfect, but gives us some insight to what kinds of players have had productive and unproductive careers in the past.  It is entirely probabilistic and is expected to be inaccurate quite often.  But it can be improved over time with more learning data fed into it.

If I'm an NBA GM, I'm not going to be basing my potential trades off of it.

I think it’s useless as a predictive model, which is how it’s used. Any model that will be “inaccurate quite often” doesn’t serve much good.

Obviously, no model is perfect. They can’t account for flukes, like the 2018 election. However, those models are very accurate overall. Predicting players? That’s an impossible task, because players all develop at different rates.

I agree on your comment above about the “big threes”. Trevor Ariza will never be a member of any big three.


Once a CrotoNat, always a CrotoNat.  CelticsBlog Draft Champions, 2009 & 2012;
DKC Draft 2015 Champions and beyond...

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #10 on: June 19, 2018, 10:25:58 AM »

Offline Big333223

  • NCE
  • Antoine Walker
  • ****
  • Posts: 4644
  • Tommy Points: 445
lol Yeah.

I'm not sure how they look at this outcome and think "let's publish this."
1957, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1968, 1969, 1974, 1976, 1981, 1984, 1986, 2008, 2019

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #11 on: June 19, 2018, 10:33:43 AM »

Offline Roy H.

  • Forums Manager
  • James Naismith
  • *********************************
  • Posts: 35611
  • Tommy Points: -27751
  • 33,333 posts and counting . . .
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?


Once a CrotoNat, always a CrotoNat.  CelticsBlog Draft Champions, 2009 & 2012;
DKC Draft 2015 Champions and beyond...

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #12 on: June 19, 2018, 10:55:30 AM »

Offline slamtheking

  • Dennis Johnson
  • ******************
  • Posts: 18988
  • Tommy Points: 8885
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?
because they produced comparable numbers during that Celtic/Sixer playoff series.  the model just didn't take into account that Kyrie wasn't playing.   ;D

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #13 on: June 19, 2018, 11:04:04 AM »

Offline saltlover

  • Ed Macauley
  • ***********
  • Posts: 11231
  • Tommy Points: 2418
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?

CARMELO uses a blend of Box Plus-Minus and Real Plus-Minus to calculate its wins projection.  I’m okay with this, because no one overall stats is going to be able to work for all players, and so using two stats should work better in the aggregate.  However, RPM gives probably  too much value to Covington, who it rates as the 8th-best player in the league and the best SF, over LeBron and KD.  This value is largely from defense, which is notoriously tricky to quantify — Covington comes in as the 3rd-best defender in the game this season.  Last year he was 4th.  Year-to-year consistency is generally a positive aspect of a statistic.  Maybe Covington is really that good.  Probably not, and CARMELO hedges by including BPM, which rates him as a solid player in the top 60 to 80, but not superstar level.  But average those two together, and also factor in the model is probably a little concerned with Kyrie’s injury, and voila!  It should be noted that entering this year, CARMELO had Kyrie at 20% more value than Covington for the next 5 years.  Even that feels off, but it’s almost certainly due to RPM.
“When an alien resides with you in your land, you shall not oppress the alien. The alien who resides with you shall be to you as the citizen among you; you shall love the alien as yourself, for you were aliens in the land of Egypt: I am the Lord your God.”

Leviticus 19:33-34

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #14 on: June 19, 2018, 11:11:40 AM »

Offline smokeablount

  • Don Chaney
  • *
  • Posts: 1684
  • Tommy Points: 108
  • When my account was born, Mark Blount was our C :(
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?

CARMELO uses a blend of Box Plus-Minus and Real Plus-Minus to calculate its wins projection.  I’m okay with this, because no one overall stats is going to be able to work for all players, and so using two stats should work better in the aggregate.  However, RPM gives probably  too much value to Covington, who it rates as the 8th-best player in the league and the best SF, over LeBron and KD.  This value is largely from defense, which is notoriously tricky to quantify — Covington comes in as the 3rd-best defender in the game this season.  Last year he was 4th.  Year-to-year consistency is generally a positive aspect of a statistic.  Maybe Covington is really that good.  Probably not, and CARMELO hedges by including BPM, which rates him as a solid player in the top 60 to 80, but not superstar level.  But average those two together, and also factor in the model is probably a little concerned with Kyrie’s injury, and voila!  It should be noted that entering this year, CARMELO had Kyrie at 20% more value than Covington for the next 5 years.  Even that feels off, but it’s almost certainly due to RPM.

Glad you’re back Salt.
Give us this pick, Almighty Red
And forgive us our tanking
As we forgive those who tanked against us
And lead us not into the lottery
But deliver us from losing

-Sexyscottish