I'm not even exactly sure what you mean when you say that it takes time to rebuild a contender. If you are saying that we need multiple years in the lottery, I would disagree with that.
If you are saying that you don't think it's likely that we'll be contending for an NBA championship by next season, I'd probably agree with that.
I'm saying that there are a few basic steps to building a contender.
a) Put together the key pieces that form your foundation
b) Get those pieces to fit together and assemble a supporting cast around them
c) Wait for the team to learn to win together
Every once in a while, all three of those steps happen within the space of a single season or two. Far more often than that, it takes more like 3-5 years, at least.
I mean, take a look at the upper echelon of teams in the league right now. How many of them managed to get to where they are without spending a few years in the lottery?