Author Topic: ESPN’s “CARMELO” projections are hot garbage  (Read 4926 times)

0 Members and 1 Guest are viewing this topic.

ESPN’s “CARMELO” projections are hot garbage
« on: June 19, 2018, 07:55:54 AM »

Online Roy H.

  • Forums Manager
  • James Naismith
  • *********************************
  • Posts: 58549
  • Tommy Points: -25636
  • Bo Knows: Joe Don't Know Diddley
ESPN decided to project the value of certain trade packages, projected over the next five years. Here’s what they came up with:



https://fivethirtyeight.com/features/the-next-phase-of-nba-superteam-technology-creating-one-from-scratch/?addata=espn:frontpage

Does it strike anybody as odd that Lonzo Ball projects to be better than Kyrie and Hayward combined? That Kyrie and Covington have equal value?  That Pascal Siakim projects to add more value than Hayward?

If your model gives results like that, it’s time to scrap your model.


I'M THE SILVERBACK GORILLA IN THIS MOTHER——— AND DON'T NONE OF YA'LL EVER FORGET IT!@ 34 minutes

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #1 on: June 19, 2018, 08:00:40 AM »

Offline BringToughnessBack

  • Don Nelson
  • ********
  • Posts: 8088
  • Tommy Points: 941
Whoever the data scientist was behind that, I sure would never want them near my company- Those projections are utter garbage! My guess is they are a Lakers fan no doubt.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #2 on: June 19, 2018, 08:36:14 AM »

Offline SHAQATTACK

  • James Naismith
  • *********************************
  • Posts: 36776
  • Tommy Points: 2961
Lakers fans dreaming

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #3 on: June 19, 2018, 08:44:56 AM »

Offline hodgy03038

  • NCE
  • Ray Allen
  • ***
  • Posts: 3701
  • Tommy Points: 440
  • Marcus Smart #1 Fan
That list is ridiculous. Ball, Kuzma and Deng? So without Deng which brings the value down Ball & Kuzma should be able to get Anthony Davis with their value.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #4 on: June 19, 2018, 09:01:56 AM »

Offline ManUp

  • Don Nelson
  • ********
  • Posts: 8509
  • Tommy Points: 285
  • Rondo doesn't believe in easy buckets...
I'm still waiting on Marcus Smart's breakout into the top 10 that CARMELO predicted last year.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #5 on: June 19, 2018, 09:03:44 AM »

Offline gouki88

  • NCE
  • Red Auerbach
  • *******************************
  • Posts: 31552
  • Tommy Points: 3141
  • 2019 & 2021 CS Historical Draft Champion
That is beyond delirious
'23 Historical Draft: Orlando Magic.

PG: Terry Porter (90-91) / Steve Francis (00-01)
SG: Joe Dumars (92-93) / Jeff Hornacek (91-92) / Jerry Stackhouse (00-01)
SF: Brandon Roy (08-09) / Walter Davis (78-79)
PF: Terry Cummings (84-85) / Paul Millsap (15-16)
C: Chris Webber (00-01) / Ralph Sampson (83-84) / Andrew Bogut (09-10)

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #6 on: June 19, 2018, 09:04:04 AM »

Offline saltlover

  • Frank Ramsey
  • ************
  • Posts: 12490
  • Tommy Points: 2619
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #7 on: June 19, 2018, 09:57:44 AM »

Offline action781

  • Rajon Rondo
  • *****
  • Posts: 5217
  • Tommy Points: 609
Having a computer model predict how good NBA players will be in the future is an insanely difficult task -- especially for young players where their futures end up being so volatile.  Think about how is a computer supposed to know that Oladipo will be a better player than Anthony Bennett?  Or how much the future value of Oladipo as we fans know it has changed in just the last 12 months.   

As far as I know, we don't have any better models out there than CARMELO so I would say we probably should not scrap it altogether.  It is far from perfect, but gives us some insight to what kinds of players have had productive and unproductive careers in the past.  It is entirely probabilistic and is expected to be inaccurate quite often.  But it can be improved over time with more learning data fed into it.

If I'm an NBA GM, I'm not going to be basing my potential trades off of it.
2020 CelticsStrong All-2000s Draft -- Utah Jazz
 
Finals Starters:  Jason Kidd - Reggie Miller - PJ Tucker - Al Horford - Shaq
Bench:  Rajon Rondo - Trae Young - Marcus Smart - Jaylen Brown -  Peja Stojakovic - Jamal Mashburn - Carlos Boozer - Tristan Thompson - Mehmet Okur

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #8 on: June 19, 2018, 10:07:08 AM »

Offline action781

  • Rajon Rondo
  • *****
  • Posts: 5217
  • Tommy Points: 609
The article itself though certainly is garbage.  It appears to have been written by someone who knows nothing of basketball.  It listed CP3, Harden, and Trevor Ariza as a "Big 3".  It put Brandon Ingram as starting point guard on next season's Lakers team. 
2020 CelticsStrong All-2000s Draft -- Utah Jazz
 
Finals Starters:  Jason Kidd - Reggie Miller - PJ Tucker - Al Horford - Shaq
Bench:  Rajon Rondo - Trae Young - Marcus Smart - Jaylen Brown -  Peja Stojakovic - Jamal Mashburn - Carlos Boozer - Tristan Thompson - Mehmet Okur

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #9 on: June 19, 2018, 10:11:55 AM »

Online Roy H.

  • Forums Manager
  • James Naismith
  • *********************************
  • Posts: 58549
  • Tommy Points: -25636
  • Bo Knows: Joe Don't Know Diddley
Having a computer model predict how good NBA players will be in the future is an insanely difficult task -- especially for young players where their futures end up being so volatile.  Think about how is a computer supposed to know that Oladipo will be a better player than Anthony Bennett?  Or how much the future value of Oladipo as we fans know it has changed in just the last 12 months.   

As far as I know, we don't have any better models out there than CARMELO so I would say we probably should not scrap it altogether.  It is far from perfect, but gives us some insight to what kinds of players have had productive and unproductive careers in the past.  It is entirely probabilistic and is expected to be inaccurate quite often.  But it can be improved over time with more learning data fed into it.

If I'm an NBA GM, I'm not going to be basing my potential trades off of it.

I think it’s useless as a predictive model, which is how it’s used. Any model that will be “inaccurate quite often” doesn’t serve much good.

Obviously, no model is perfect. They can’t account for flukes, like the 2018 election. However, those models are very accurate overall. Predicting players? That’s an impossible task, because players all develop at different rates.

I agree on your comment above about the “big threes”. Trevor Ariza will never be a member of any big three.


I'M THE SILVERBACK GORILLA IN THIS MOTHER——— AND DON'T NONE OF YA'LL EVER FORGET IT!@ 34 minutes

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #10 on: June 19, 2018, 10:25:58 AM »

Offline Big333223

  • NCE
  • Tiny Archibald
  • *******
  • Posts: 7489
  • Tommy Points: 741
lol Yeah.

I'm not sure how they look at this outcome and think "let's publish this."
1957, 1959, 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1968, 1969, 1974, 1976, 1981, 1984, 1986, 2008

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #11 on: June 19, 2018, 10:33:43 AM »

Online Roy H.

  • Forums Manager
  • James Naismith
  • *********************************
  • Posts: 58549
  • Tommy Points: -25636
  • Bo Knows: Joe Don't Know Diddley
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?


I'M THE SILVERBACK GORILLA IN THIS MOTHER——— AND DON'T NONE OF YA'LL EVER FORGET IT!@ 34 minutes

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #12 on: June 19, 2018, 10:55:30 AM »

Offline slamtheking

  • NCE
  • Red Auerbach
  • *******************************
  • Posts: 31869
  • Tommy Points: 10047
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?
because they produced comparable numbers during that Celtic/Sixer playoff series.  the model just didn't take into account that Kyrie wasn't playing.   ;D

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #13 on: June 19, 2018, 11:04:04 AM »

Offline saltlover

  • Frank Ramsey
  • ************
  • Posts: 12490
  • Tommy Points: 2619
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?

CARMELO uses a blend of Box Plus-Minus and Real Plus-Minus to calculate its wins projection.  I’m okay with this, because no one overall stats is going to be able to work for all players, and so using two stats should work better in the aggregate.  However, RPM gives probably  too much value to Covington, who it rates as the 8th-best player in the league and the best SF, over LeBron and KD.  This value is largely from defense, which is notoriously tricky to quantify — Covington comes in as the 3rd-best defender in the game this season.  Last year he was 4th.  Year-to-year consistency is generally a positive aspect of a statistic.  Maybe Covington is really that good.  Probably not, and CARMELO hedges by including BPM, which rates him as a solid player in the top 60 to 80, but not superstar level.  But average those two together, and also factor in the model is probably a little concerned with Kyrie’s injury, and voila!  It should be noted that entering this year, CARMELO had Kyrie at 20% more value than Covington for the next 5 years.  Even that feels off, but it’s almost certainly due to RPM.

Re: ESPN’s “CARMELO” projections are hot garbage
« Reply #14 on: June 19, 2018, 11:11:40 AM »

Offline smokeablount

  • Ray Allen
  • ***
  • Posts: 3103
  • Tommy Points: 628
  • Mark Blount often got smoked
CARMELO is a fun toy.  But he’s really chosen to put his finger on the scale in a major way.

Firstly, I’m not going to get up in arms about Hayward’s number — the model is clearly concerned about his injury.  It felt Hayward was a borderline All-Star for its 5-year forecast entering this year, but missing a year with injury in his late 20s clearly gives Hayward a lot of unfavorable comparables.  We can be more optimistic because we know that Hayward had a bone injury, which, while terrible, is less likely to recur and more likely to fully heal than ligament and tendon injuries, which are going to be the predominant source of lost seasons in the database.

That said, the way in which he’s put his finger on the scale is to use a different kind of projection, which he’s called his “upside projection”.  This favors young players — and the younger and less experienced the better.  CARMELO comes up with its projection by looking at past players and assigning a level for how much they correlate with players, and then weighting the future production of similar players by how more or less similar they are to the player being forecast.  This is perfectly reasonable — it’s probabilistic, and will have it’s misses, but in the aggregate it’s sound data science.  However, for his upside projection, used specially for this article, he excludes from the pool of similar players all those that had a zero or negative value.  For an in-his-prime star like Kyrie, this means virtually no one is excluded.  So we keep a lot of the players who ultimately had a worse performance than we expect Kyrie to have, and they bring down his average.  Ball, of course, has a lot of players as comparables who had a negative value and were out of the league.  Those players are excluded, meaning we’re seeing a weighted projection of only positive outcomes, and not all outcomes.  As a #2 overall draft pick with good measurables, there will be a lot of stars and superstars in this group, including some of their top seasons.  This is why Bam Adebayo is also at a similar tier to Kyrie.

I don’t know if Neil Paine, who’s normally quite good, didn’t think about the bias he was introducing into his calculations, or if the editors at ESPN said “write an article showing that the Lakers can make the best offer for Kawhi.”  I’m disappointed, and he should have been a little more open with the fact that his methodology would clearly favor those with the least experience.

Interesting.  I can understand how, with that adjustment, young guys get vastly overrated.

But even excusing that, and forgiving the model for having an abjectly negative outlook for Hayward, how do we get to “Kyrie = Covington”?

CARMELO uses a blend of Box Plus-Minus and Real Plus-Minus to calculate its wins projection.  I’m okay with this, because no one overall stats is going to be able to work for all players, and so using two stats should work better in the aggregate.  However, RPM gives probably  too much value to Covington, who it rates as the 8th-best player in the league and the best SF, over LeBron and KD.  This value is largely from defense, which is notoriously tricky to quantify — Covington comes in as the 3rd-best defender in the game this season.  Last year he was 4th.  Year-to-year consistency is generally a positive aspect of a statistic.  Maybe Covington is really that good.  Probably not, and CARMELO hedges by including BPM, which rates him as a solid player in the top 60 to 80, but not superstar level.  But average those two together, and also factor in the model is probably a little concerned with Kyrie’s injury, and voila!  It should be noted that entering this year, CARMELO had Kyrie at 20% more value than Covington for the next 5 years.  Even that feels off, but it’s almost certainly due to RPM.

Glad you’re back Salt.
2023 Non-Active / Non-NBA75 Fantasy Draft, ChiBulls:

PG: Deron Williams 07-08 / M.R. Richardson 80-81 / J. Wall 16-17
SG: David Thompson 77-78 / Hersey Hawkins 96-97
SF: Tracy McGrady 02-03 / Tayshaun Prince 06-07
PF: Larry Nance Sr 91-92 / Blake Griffin 13-14
C: Bob Lanier 76-77 / Brad Daugherty 92-93 / M. Camby 06-07