I went back and looked at the top 5 picks from the 2003 to 2012 drafts. This analysis will not be scientific enough for many here but what I did is qualitatively determine how many players in the top 5 really became stars, how many ended up really no better than someone who could have been drafted in say the next 5, and how many were outright busts.
On average, less than 2 (1.8 to be exact) became what I called a star. The best draft was 2003 which had 4 become stars (LBJ, Wade, Bosh, and Melo) but even that historic draft had a major bust in the top 5.
In terms of outright busts, on average, the top 5 had 1.3 busts. 2006 was a bad year for busts where I counted Bargnani, Morrison, Tristan Thomas, and Sheldon Williams all as busts (or if not busts by your definition, they were certainly major disappointments and really bad picks).
The point is that drafting has an element of chance in it, especially as the draft has gotten younger. Parker may not come out, Embiid may be the next Hasheem Thabeet and all of the sudden, 1-3 doesn't look all that much different from 5-7. Sure, I would rather pick Parker than say Aaron Gordon but I am not looking for either to be transcendent for the team. No player has shown that they are for sure going to be a star at the next level. Picking will be a crap shoot this year just like most other years.
So no, I am not worrying too much about our draft position and besides, we are due for some luck in the lottery. I think it will turn out to be true of this draft as in past ones that it is better to be lucky than good (or bad depending on how you look at it). Lucky in terms of the lottery and then more importantly, lucky in terms of who we get (or avoid getting).