You rebuild successfully by both getting high picks and hitting on lesser picks to build up a supporting cast. Look at the Penguins rebuild in the early 2000s, it wasn't just "draft Fleury, Malkin, Crosby and Staal". Go ask Buffalo how their rebuild has gone with their inability to build up a supporting cast.
The Penguins rebuild started with the Jagr trade in 2001, and the Penguins didn't really "tear down" the team until 2003. Before that tear down, the Penguins drafted all of Scuderi (5th rounder in 1998), Malone (4th rounder in 1999), Orpik (1st rounder in 2000), Ouellet (4th rounder in 2000), Armstrong (1st rounder in 2001), Whitney (5th overall in 2002), Christensen (3rd rounder in 2002) and Talbot (8th rounder in 2002) that had a significant impact (either direct or indirect) on the Penguins rebounding as a team.
The Penguins are in the stage of "draft the next Orpiks, Malones and Armstrongs" right now. They're building up the supporting pieces of the prospect pool now so that when they truly bottom out (probably in the 2027 timeframe), the top guys they'll be drafting once they suck won't be thrown into a completely helpless situation.