That's one way to look at it. Another would be to say he was a big success at Temple ... with all the disadvantages that came with that. Maybe that says a lot about his skills.
The bottom line is that not enough people pay enough attention to the context in which successes and failures happen. Golden was a terrible fit for Miami. And even though he was able to attract talent there, kids who are drawn to Miami are looking for an experience that is a 180 from the environment Golden looks to cultivate. His approach is much more Midwestern/Northeast which would actually make him a fish in familiar waters at, say ... Rutgers.
There's no way to look at what he did at Temple and say he isn't a skilled coach. What he inherited there made pre-Schiano Rutgers look like Notre Dame. And he made them a winner. So, if nothing else, he's a good program builder, talented when it comes to extinguishing dumpster fires or instilling life into moribund programs.
Another study in context might be Mark Richt. Here is a guy who was given the keys to a program with more built-in advantages than just about any in the country, and hadn't won a blessed thing in the past 10 years. Do you realize how many coaches could've done a better job there than Mark Richt? But people think because he operated in a situation where he had everything at his disposal that he did a great job. He petty much did what any reasonably talented coach would've done. Because that's what he is: reasonable talented. But while he was holding serve, the Nick Sabans and Mark Dantonios were reinvigorating marquee programs that has flattened out (like Georgia) and making them top teams again. Dantonio not as much as Saban, but Michigan State is not Alabama, so it's all relative. It's tough to understand the fascination with Richt.