I felt that after all of my recent arguments on this board I should summarize my stance on the relevance of advanced statistics.
Essentially, I've narrowed my challenge down to these three overarching premises:
1) If advanced statistics demonstrate that a player is on the ice for a negative amount of shots on goal, this information cannot (and does not attempt to) explain "why" this is the case, because it ignores all possible reasons or factors for the data (with the narrow exception of considering the quality of team mates and opposition - "quality" being defined on the same scale and ignoring actual player attributes), accepting simply and without question that the player is "bad".
2) Advanced statistics ignores the facts that the use of players is situational, the impact of a player can extend beyond time on ice and positive puck posession in a given situation does not always amount to a relative success.
3) Advanced statistics can not determine to what extent a player's shot +/- adjusted stats are affected by style of play, overall team quality and all other intangibles. Advance stats do not care to address "what" qualities produce success and only state that because success is achieved something must be going well. The "what" is the most important truth to be determined in team management, since it is already obvious that a team doing everything right will naturally produce, on average, higher advanced statistical ratings for all players.
A short, partial list of key questions advance stats cannot definitively answer:
1. What effect does dressing room presence have on overall team play? 2. What effect does intimidation have on opposing team psychology? 3. How do teams adjust their style of play to perform against certain players? 4. Are all styles of play, at all times of the game equally directed towards achieving high quality shots on goal? 5. What level of impact does physical play have on player's bodies throughout the course of a game? 6. Does physical punishment (such as crosschecks in front of goal, body work along the boards), create hesitation in an opponent's play throughout the course of the game? 6b. If so, do all players react in exact same measure to this factor? 7. Are certain players better suited to certain roles? 8. If an organization were to compile a squad of the highest ranking advanced stat players, with no consideration to the roles they played within their respective teams, would they certainly be successful? 9. What qualities (physical attributes such as size and speed) of opponents are players recording their statistics against? (Note that I did not say "quality", as defined by shots for and against ratios, I said attributes) 9b. Would a 5'9, 170 pound defenseman have different statistical results if regularly employed against a forward line averaging in size at 6'4, 220 pounds, as opposed to typically sized opponents? 10. Does player behaviour outside of hockey affect performance? Do players have an effect on each other's behaviour outside of hockey? 11. Is it possible for players to learn from each other? 12. Is it statistically possible to determine the level of experience and ability to communicate experience and knowledge? 13. When there is a statistical variance over a set period of time, can advanced statistics determine what has caused the increased or decreased results of a player's performance? 14. Do certain players achieve differing advanced statistic results based on the system they are playing within? 15. Can advanced statistics determine what other players would produce in the exact same situation as the player being measured? 16. Is effort and intensity affected by confidence? What affects confidence? 17. Are certain shifts played to produce a "best case scenario" result? In other words, are certain shifts played as survival shifts against top opponent lines, with the intention to frusterate and deny top quality chances, with little thought to counter attacking? 17b. Are top shot generating players typically employed in these situations? 18. Why are the majority of defensive minded defensemen rated low in advanced statistics? 19. Are successful teams traditionally comprised of no defensive minded players? 20. If a team outshot an opposing team, was successful at puck possession and clearing their zone would this automatically result in wins? 20b. If the same team was unable to clear the front of the net and was overagressive offensively, leading to a large number of odd man counter attacks, would they still be likely to succeed? 20c. Which team would be more likely to conceed quality scoring opportunities? 21. Do teams adjust their possessional play based on goaltender qualities? Is it sufficient to give up low % shots in front of a high caliber goaltender? Is it sufficient to take a large # of low % shots against a high caliber goaltender? 22. Why would a player's advanced statistics change when he switches teams, systems, coaches or roles? 23. Are players ever played outside of line combinations or roles that they are best suited for? 24. Can advanced stats measure player potential (potential derived from analyses of player mechanics and best game performances)? 25. What effect does a successful fight have on crowd involvement, team morale and performance?
I stopped at 25 since intangibles are practically infinite and we don't have all day.
Finally, a fictional story to put things in perspective. I'll call it "A Game in the Life of Douglas Murray". Not that I'm a particular fan of Doug Murray, a marginal role playing defenseman who is not the best at his particular craft, but he seems to be the bane of the stat collector's existence, so we'll use him. (I will say however though that I was happy with his acquisition following the embarassing performance of the Canadiens in last year's play-offs after an attempt to use players such as Raphael Diaz in roles needed to be filled by players such as Douglas Murray - I know you're confused stats guys, all this talk of roles, but please read on). Count the intangibles ;)
"Wow, I am ready to give all I have today after the vote of confidence given to me by Mr. Bergeron following the Diaz trade. First shift, here I go. Michel has put me out against the Khadri line. This boy is good, he has me turning in circles out here, but I'll not get drawn out of position, I'll hold my position and keep them to the outside. They are peppering us, but I'll clear the crease and make sure Carey gets a good look at anything coming through. Had a chance to clear it, but fanned on the puck, f@%k. Missed a couple of shifts because of that gaff, must do better. Coach has me out against their fourth line now. Kulemin barely beat me to the puck but a flattened him and got the clear. He was flying like a banshee his first shift, ran over Gallagher. We don't see much of him for the rest of the period after he gingerly skated to the bench following that hit. Didn't create a shot on the counter because Prust and Moen were out here with me. Back out against the top line. JVR tries to plow through the crease and I put him on his ass. He gets up and stumbles into Price as the shot comes in, I dump him again after the play. He'll think twice about trying that again. Next shift the Leafs send out Colton Orr, he takes a few runs at me, ignoring the run of play. I stand him up and Boullion gets the puck out cleanly. The Eller line gets some sustained pressure in the opposing zone. Second Period. Michel tells me to send a message to Orr. We start the period badly, they score, crowd is out of it and coach has me waiting. Emelin made a weak turn over on the goal and has his head down. I give him some advice on the outlet option he missed and punch him hard on the shoulder, tell him I'll sleep with his sister if he doesn't stop sulking. He laughs and refocuses. Orr is on the ice, its my time. I go straight for Orr and he knows the drill. We drop gloves and quickly he has me off balance. I start wondering why Parros wasn't dressed for this game. It's on me now though. He throws a few jabs that connect, but they don't affect me. He gets over confident and opens up, I spin with the haymaker and catch him square. He goes down, the crowd goes nuts. I sit for 5 and watch the team pick up their play for the next few shifts. Deharnais draws a penalty driving to the net, something her normally doesn't do. Another power play later in the period. I sit and watch. Third period, I'm cold. Haven't had a shift since the beginning of the 2nd. Out against the 3rd line this time. Eller line again. They are overcommitting to the forecheck. Wrong play against this line while I'm out here. Mason Raymond is waiting for the clearance. Phaneuf emerges with the puck under possession and I turn to make way back to our end. Raymond is already accelerating though. Perfect pass and I'm screwed. Phanuef hits him nicely, 2-0 Toronto. I'm minus 1. Now we're chasing the game. Next shift I'm out with the fourth line again and they are dumping the puck in. Toronto clears and I regain possession, we dump again, but can't get any meaning full offensive zone control. Next shift with Eller line again. We need a goal so they are pressing high. Get burnt again and we are stuck with a 3 on 2. I make a great interception on the critical pass, but can't clear. They get zone possession, our forwards are tired. I block a point shot, clear the front of the net. A shot comes in from a bad angle, they create nothing dangerous despite their possession and a few shots through on goal. After that shift Michel trusts me against the JVR line again (maybe he's learned not to leave me matched with Raymond). Once again, we face some pressure in our end. JVR playing peripheral now. 4, 5, 6 shots towards goal, a couple that Carey has to kick out harmlessly. The Leaf's top line goes off, on come the Pacioretty line. They burst out against the Leaf's fourth unit and immeidately pot a goal. 2-1! One minute later, Gallagher takes a penalty. Coach sends me out on the second unit. I face JVR again. He makes one attempt to plant himself in front of me and I easily move him out of the way. His heart isn't in it, he chases a puck into the corner. Shot comes, I block it. They aren't getting any good looks. One shot gets through, but Carey sees it all the way. We clear the puck. 5 minutes to play and the crowd is getting excited following our last clearance and PK kill. I sit and watch the team trying to no avail to get the tying mark."
Fictional Murray was -1, -9 in shots for and against, had 1 fight, 2 hits and 3 blocks over 10 minutes of playing time. He did his job perfectly while statistically maintaining his place as "the worst defenseman in the league". If you doubt that, ask yourself the outcome if Diaz was asked to perform the exact same role described above in the Murray story.
That's all I've got for now...