Rugby World Cup: The ref debate

It's been a long time between posts - busy work periods, lack of inspiration, lack of news stories (well, that's not entirely true!), but pick the excuse.  Apologies for the long break. I'm back with a viewpoint on rugby - probably not the topic of interest for most of you reading this in the USA, but as our national sport in SA, felt compelled to put it out there!  I'm planning a series on fatigue, probably as a series of short video posts in the coming week, so hopefully that breaks the silence on sports science!  Join us soon!  And for those who followed the Rugby World Cup, congratulations to New Zealand.  Some thoughts on the refereeing below!


Rugby World Cup: New Zealand's drought ends and rugby's referee problem

So 24 years of waiting is over for New Zealand, who beat France 8-7 in a pulsating and perhaps unexpectedly competitive Rugby World Cup Final today.  It may have been the lowest scoring final ever played, but it was suspenseful and adventurous, certainly more than the previous two finals.  France produced a performance worthy of the showpiece match of the tournament, having come into it with two losses and the anticipation of a blowout victory to New Zealand.  Rather, it was France who played the adventurous rugby, and only some ineffectiveness in attack and New Zealand's resolute defending prevented them from winning their first title.  

Instead, New Zealand won their second, but it was significant in that they have been, for the most part, the best team going into each of the six World Cup tournaments, sometimes by a large margin.  Having failed to win the World Cup on five occasions despite being the favorites had earned New Zealand the tag of "chokers", a team that peaked between World Cups but failed to deliver when it mattered.  Two of those famous defeats came at the hands of France (in 1999 and 2007) and so when this French team stood firm and began to control the match following a second half try that brought the score back to 8-7, a blanket of anxiety settled over Eden Park in Auckland.  

Choking vs panic

There were times when New Zealand appeared close to panic in this final - they were flustered, made unforced errors, chose poor tactical options and generally seemed to be hanging on and defending a one-point lead with desire rather than application.  At this point, it seemed to me that had New Zealand NOT won this World Cup, it would have been because of panic, rather than choking (an explanation that is just too convenient to use, and unfairly earned, not only by NZ rugby by also by SA cricket).  Their composure deserted them, though the injury to their flyhalf, which meant that they played most of the final with a fourth choice pivot, certainly influenced their tactical approach.  As did their lead, and they seemed more concerned with defending the one-point advantage than playing proactively, which set the final 30 minutes up as France with the ball, New Zealand without it.

For an explanation of how choking differs from panic, and why a team that loses a match is not necessarily choking, read this piece by Malcolm Gladwell.  I've never really been fond of simply throwing out the excuse of "chokers" every time the more favored team loses - sometimes you are just outplayed or out-thought by a team who are better than you on the day.  The margins in international sport are so small that this can happen fairly easily, and it's too simple to say "New Zealand choked", when in fact, France may have simply been unbeatable on a given day, as was the case in 1999.  For a comparable case in tennis, Federer's loss to Tsonga in Wimbledon earlier this year is the best I can think of - sometimes, however great you are, the other team/player just rises to a level that no one would match, and it's your bad fortune to be there at the time!  

The influence of the referee in rugby

However, the tactical and technical nature of the game is not what I want to focus on in this post - that is something that rugby websites around the world will do enough of (see this example for a match report).  

Instead, I thought I would give some of my thoughts on a topic that follows every rugby match, and that is the debate and criticism of the referee.  The reality is that the referee in a rugby match has become incredibly influential in determining how the game is played.  The result is that rugby has a growing credibility problem, where every match threatens to degenerate into objections about the performance of the referee, rather than assessment of the relative performances of its players.  Whenever the result on the scoreboard can be dismissed as being the result of someone's opinion or bias, there is a problem.  

And this has happened in virtually every close match in the 2011 Rugby World Cup, which will be remembered not solely for the on-field performances, but for weak referee performances, some of which have been questionable, some outright poor.  The most controversial of these probably came in the Quarter-final, where Australia beat South Africa 11-9 in a match that was later alleged to have been "bent" as part of the condemnation on the performance of referee Bryce Lawrence (more on my views of that allegation later)

Rugby presents a unique challenge in that the referee is required to make a specific decision about a contested tackle almost 200 times a match (once every 30 seconds), and this decision is multi-dimensional, instantaneous and open to interpretation.  As a result, these decisions (and there are so many of them) influence the game to the extent that accusation, criticism and allegation are inevitable.   It's part of sport, certainly, but rugby seems more prone to accusations that "the ref helped ABC win" than any other sport.  The problem is that from this point, it's a short journey to allegations of fixing, corruption and cheating, when the problem may be simple incompetence or interpretation of the tackle rules of the sport.  Either way, the credibility of a result is called into question.

This situation exists because so much of the contest in rugby revolves around competing for the ball after a tackle, in the breakdown contest.  The attacking team needs to recycle possession quickly, whereas the defending team are at worst trying to slow it down to re-organize in defence, at best trying to win the ball on the ground.  The result is a huge contest, the outcome of which goes a considerable distance towards determining the match result, but which is itself determined by how the referee interprets how both sets of players test the boundaries of the law (because this is what players will do, understandably - it's like football players trying to play close to the offside line)

A unique situation?

I cannot think of another sport where the interpretation of the rule by an official so clearly influences the way that teams play the match.  In football (soccer), the most contentious decisions are those when a penalty appeal is made, offsides is ruled, or when foul-play is adjudged.   They are fairly clear-cut, and far less frequent than in rugby.  And certainly, they can influence matches in a big way - I'm not downplaying how significant a referee decision can be.  In the NFL, decisions can be similarly significant, but usually involve clear transgressions of rules.  Tennis, there's no influence, particularly now that television replays are used.  And similarly, cricket umpires are often criticized and single decisions can be very influential, but with TV assistance, the incidence of these has certainly come down.  If there is a sport that I'm missing, please let me know.

The rugby situation - too much interpretation

Rugby is different - the most contentious decision in rugby is one that is made on average twice a minute (five times a minute if you use ball in play time rather than total time), and it influences the next minute, rather than being a decision in isolation.  Consider that a typical match has about 170 rucks (or contests for the ball in a tackle) , and you realize that there are probably 100 decisions (because not all are contested the same way) where the referee must interpret, in a split second, a dizzying array of laws, and where each decision has implications for what follows.  

Different referees have a different sequence or approach to the decision, but they must judge, more or less in order: how the tackler interacts with the tackled player, when the tackle actually occurs, that the tackler releases the tackled player, that the tackled player releases the ball, when the ruck is formed, that players arriving to join the ruck remain on their feet, and that they join from the correct position and do not seal the ball off to prevent the contest.  Add in that there are often multiple tacklers, so the referee has to decide who the tackler is, and you appreciate that within half a second, there's a lot to judge.  Then the next problem is that many times, four or five things happen more or less simultaneously, and so it really is a judgment call.

Ultimately, what the decision comes down to is a) assigning roles to the involved players, and b) deciding on the order in which events occur - every tackle has similar events, and the job of the referee is to sort through the order in which they occur,  and if he sees a different order to you or I, then his decision will be accordingly different.  And this is precisely what happens to make these decisions so contentious.

I've been fortunate enough to work with the SA Sevens team for the last three seasons, and at every tournament, the IRB Head of Referees, all the coaches and technical staff of competing teams, and all the referees have a sit-down meeting a few days before the tournament starts.  The meetings involve discussion around how the referees have been instructed to officiate and usually include clips of tackles and rucks from previous tournaments.  Now bear in mind that this is Sevens, where the contest involves fewer players and with less congestion than you'd see in 15s, and then consider that even so, there rarely agreement in these meetings.  The situation in 15-man rugby is of course even more complex (though the tackle contest may be more significant in 7s, but that's for another discussion!)

For each clip, one coach will point to the tackler, another to the tackled player, another to the arriving player, another to the offside line, each one pointing out a different possible transgression PER RUCK!  Mostly, it boils to disagreement about the order in which events happen, and which player should be entitled to do what.  Eventually, even in slow-motion, it takes consensus or a swing vote to sort through the order of decisions that a referee must make.  Even then, it's often a 50-50 call as to whether a player released the tackled player or the ball and so on (if you are reading this without much knowledge of rugby and you're confused at how complex it sounds, well, that's exactly the point!)

A general approach to the decision and its implications

The reality is that rugby, by design, prioritizes the contest for the ball on the ground, and therefore the spotlight falls squarely on the man who must judge whether players are transgressing those laws.  Simple on paper - there is a very distinct set of rules governing the tackle.  But here's the problem - the rules may be clear, but the judgment of them is not.  So much is open to interpretation, and it is interpretation that happens in an instant, while on the run.  The result is that a match can very, very easily look 'influenced' by the referee, who generally speaking, can take one of two extreme approaches to how they cut through this organized chaos to make a decision.  Call it "conservative" vs "liberal" decision-making, but at its simplest, a referee is going to lean one of two ways.

The first approach is to over-police the contest (the conservative).  The result is that the referee will appear to punish legitimate contesting for the ball, and will reward penalties frequently, forcing players to back right off, killing the contest for the ball.  This favors the team in possession.  Alternatively, the referee can under-police the breakdowns (liberal), and allow much more to go unpenalized.  

Importantly, when this happens, the result is that the defending team will usually be favoured, because the referee will fail to prevent them from slowing the ball down, and slowing it down creates a disproportionate advantage.  I believe this is what happened in the South Africa - Australia match, where the rucks were highly contested and too much was allowed on the ground.  The result is that the defending team is advantaged.  But, significantly, the problem in that particular match is that the defending team was mostly Australia.

The stats reveal this - South Africa had 131 rucks, compared to Australia's 44.  That is, for every one opportunity for South Africa to contest and slow down Australian ball, there were three chances for Australia to do so.  So, by allowing too much contesting, the referee effectively gave Australia three times as many chances to push the limits of what was legal (and some would say exceed those limits).

When one team is as dominant as this (in terms of possession), and the more liberal referee is making the extreme "decision" to under-police and allow more, then it will always appear that he is deliberately biased.  The reality is that if the possession was equal, and if both teams have the same number of rucks, then nobody would really notice the referee because BOTH TEAMS would get away with slowing the other team's ball down!  You'd get a very messy match, but the liberal referee would be far more "anonymous" because his leaning affects both sides equally.

Instead, this match was one-sided, and South Africa seemed to be on the receiving end of an unfair performance.  I do think that Lawrence was poor, and I do think that his poor performance affected SA more, but it wasn't deliberate.  And as for match-fixing?  Not based on decisions that didn't go our way, no.  Rather, I think that the referee was poor and didn't do enough to control the rucks, but my point is that this may be because he was either instructed to allow the contest, and "over-applied" the instruction, or he just has a natural inclination to be liberal towards the contest.

In the case of Bryce Lawrence, it would not surprise me if he was told to allow a contest for the ball, because earlier in the tournament (in the Aus v Ireland match), he was criticized for penalizing Australia TOO MUCH at the breakdown.  I strongly suspect that what happened next is that he was asked to be a little slower on the whistle, and he erred on the other extreme, and didn't do enough.  In the end, it appeared that South Africa were hard done by, but as I have said, that's more because whenever one team dominates play, an error like Lawrence's appears to favour the team without the ball (Australia).

Analyzing referees - navigating with a broken compass

It may not surprise you to learn, for example, that many international teams now attempt to analyze referee trends, so that they can attempt to guess whether a given referee is likely to decide one way or the other.  At the most basic level, for example, you can look at whether a particular referee tends to award a penalty to the attacking team or the defending team to get an idea of that referee's "in-built bias".  This partly reveals whether that referee's priority in assessing the breakdown is whether the attacking team player releases the ball (penalty against the attacking team) or whether the tackler releases the player (defending team).  You can then go further to see whether the referee is more or less lenient on the tackler or the tackled player and the arriving supporting players from either team.

The problem with this approach is two-fold.  First, it's subjective.  When analysing clips, you have to judge not only what the referee does decide, but what he does not.  This means you have to make a call yourself, and this brings us back to the point about disputable situations, especially because on TV, you don't see what the referee does.  

The second problem, more significant, is that the referees, in my experience anyway, are too unpredictable to code in this way.  They are influenced by individual players and teams, and they change their approach too often, probably because they are very susceptible to suggestion and to the instructions coming down at them from their superiors. 

For example, we tried this in the Sevens setup,but it was a futile quest, because the referees changed their approach too often.  We worked out that what was happening was that the IRB were evaluating the referees and providing feedback on their performances (which is a good thing, of course), but this feedback was influencing the way that referee approached their next match.  The result was that for each referee, if you plotted a graph showing how they made decisions, it would look like a zig-zag curve of mountain peaks and valleys - one week they leaned one way, the next week they went the other.  And so trying to pre-empt how they would decide was like navigating with a broken compass.  

Yet again, what this showed is the "unstable" nature of the decision-making process.  Again, 170 decisions per match, each one in a fraction of a second at speed, with five or more variables to assess is going to introduce some "interpretation", and the problem is that this can lean one way or another very easily.

Emotion - the inherent bias when working backwards

The other factor in all this is that emotion and passion are such significant influencers of how we interpret this watching on television.  Fans (and even neutral spectators) have an inherent bias (it's what makes them fans!) and the result is that when they assess a referee performance, they exist in a world of black and white - the referee is either right or wrong.  Unfortunately for rugby, the decision is rarely black and white.  It is grey, because of the previously mentioned decisions around judging the order in which events occur, and who does what in the tackle, and so there is always conflict between what fans see and what is actually happening on the ground.

Consider an example from football (soccer):  A player scores a goal but is offside when he received the pass.  The referee/assistant see this, and the goal is correctly disallowed.  On first viewing, a fan who feels that his team has been robbed can make all manner of accusations including match-fixing and bias, but a replay will prove him wrong in most cases.  Similarly, in tennis, the ball is either in or out, and in the Hawkeye era, there's little dispute over those calls.  NFL, there are debatable calls (pass interference, roughing the passer etc), but they're much less frequent and different in nature to the ongoing, continuous rugby tackle calls.  

Rugby, however, has a much more subjective decision happening 170 times a match, and that's why I laboured the point about how "grey" the decision-making process can be earlier in this post.  The end result is that people who watch matches can make the logic mistake of working backwards.  They then interpret their observations to fit their theory, and of course their desired theory is that their team must win!  

It's a lot like bent science, in fact, in that you start out with the finding already "known" (in a fan's mind, there is only one team that can win - they "know" the result before the match!).  Then you have a series of "experiments", also known as the tackle situation, where the outcome of each must be known too.  The entire match is an observed experiment, and unwittingly, people mix emotion with interpretation and they will come up with accusations of bias because their observation will always fit their model.  This is the danger of looking for proof of what you already believe, because you will always succeed at finding it!

Don't trust the passionate perception

I made this mistake myself when working with the Sevens team.  Every single decision was "wrong" as long as it went against our team!  Such is the desire to win, that I stood on the sidelines and could not believe that a penalty should not be awarded to us.  We lose the ball, it could only be because the other team cheated, and the referee missed it!  

Only in the cold light of day, often the next morning, sitting in the hotel lobby, did I have the opportunity to review the match, sometimes to talk to the referee and he would explain what he was seeing as he made the call, and then it became much clearer to me that what was "obvious" to me was in fact "obvious" in exactly the other direction!  I was wrong, pure and simple.  But at the time I could not see that I was looking at it incorrectly.  I learned to have a deep mistrust of my own perceptions in those emotional, stressful situations, and learned instead to wait, hold the opinion and rather decide when removed from the passion and emotion.  It was a valuable lesson.  

Sometimes, of course, the referees did make mistakes - more than once, I still believe we were wrongly judged and that it cost matches.  Sometimes, referees even admitted it, and apologized.  But we have also been the beneficiaries of the decisions, and that's the result of rugby's tackle rule.  It certainly needs to be fixed, but this was a difficult lesson to learn, but an important one.

The reality is that fans need to step away from the emotion, and if they did, they may, in the case of South Africa anyway, recognize a few other reasons why it was New Zealand, and not us, lifting that trophy in Auckland yesterday.

The solution - analysis and a scorecard

As for the solution, my bias as a scientist is to measure and analyse, so that's where I'd look for rugby's problem.   And transparency would help - no one really knows what the IRB does with referees - they are accused of being a "protected species", which may not necessarily be a bad thing, but I do feel that some more open discussion would help.  At the moment, it's all left to the media, and in this day and age, the "media" now includes social networking, and so the public WILL have their say, and they are rarely going to be diplomatic in the absence of information.  Rather control the perception by making some information available  (it's a lot like the Caster Semenya case - the secrecy around her testing and treatment only fueled the flames and allowed people to make up the "truth".  And that version is always worse than the real truth).

And for rugby, the solution to me is that the performance of referees needs to be evaluated more transparently.  A panel of independent officials could analyze matches, producing a report on the match.  This report could analyze every single one of the 200 decisions a referee has to make in a match.  How many of the 200 were incorrect?  20? 30?  And of those 30, how many were clear, conclusive errors, and how many were interpretive calls?  One has to build in this human interpretation element, because it would be wrong to think that one can accurately judge off TV when the referee is 5m away from the decision he is making.

And of those conclusive errors, do they favor one team?  If you find for example that 30 decisions out of 200 are wrong, and 90% of them go against one team, then you have some weight behind accusations of bias or fixing.  But until that kind of evaluation is done, people speculate, and speculation is almost always worse than the truth.

Especially when the passions of die-hard fans are involved.  Just ask any referee...

Ross

Berlin Marathon 2011: Live

Berlin 2011: Magnificent Makau 2:03:38!

Patrick Makau has broken the world marathon record.  He raced to a 2:03:38 on the streets of Berlin, breaking Gebrselassie's world record by 21 seconds.  It was a terrible day for the Ethiopian emperor - he stepped off the road soon after Makau launched a big surge at around 27km, and while he did resume running, he was not a factor and it seems that he bailed some time later.

The questions will begin again - it is Geb's second DNF if two marathons, after New York.  On that occasion, he retired, and questions will now be asked again.  It's a sad way to go if it's true - two DNFs and the loss of his world record.

But today, it was Makau all the way.  Below are the splits (for the men - I lost track of the women's race as the men's world record became more and more apparent), and my comments as the race unfolded.

For a more detailed breakdown of the race, including a comparison between Makau 2011 and Gebrselassie 2008, click here

But briefly, halfway was reached in 61:45, which projects a 2:03:30, and that was more or less the pace from the start.  Makau and Geb were in the group, 11 strong, until around 25km when things began to fragment.  That's when Makau, perhaps sensing a weakness is Geb, pushed the pace, and under the pressure of his surge, Gebrselassie stepped off the road.  He seemed to be clutching his stomach - it was either a stitch, stomach cramp, or maybe asthma.  We'll find out later.

But the pace of the Makau surge was brutal - 14:20 for the 5km segment from 25km to 30km, and that's what took Makau from being in with a shout to having a real shot at it.  He did pay for that surge later in the race, and when we compare the splits later, you'll see that Makau got slower and slower from that point onwards.

But he had enough of a 'buffer' in hand - he was 49 seconds ahead of Gebrselassie's time at the same point in the 2008 record, and 45 seconds ahead of the pace required to break the 2:03:59.  So even though Makau did slow over the final 5km in particular, he had the record in the bag, and went on to break it by 21 seconds.

That's the short version - below is my coverage of the race as it unfolded, and here is the more detailed breakdown for those interested.

And thanks for following our live coverage of the race!

Splits


Men                                                                  Women

5km - 14:36 (2:03:13 pace)                           5km - 16:37 (2:20:14)
10km - 29:17 (2:03:34)                                  10km - 33:16 (2:20:22)
15km - 43:51 (2:03:21)                                   15km - 49:50 (2:20:11)
20km - 58:30 (2:03:25)                                 20km - 1:06:32 (2:20:22)
Half-marathon - 61:43 (2:03:26)                Half-marathon - 1:10:11 (2:20:22)
25km - 1:13:18 (2:03:43)                               25km - 1:23:15 (2:20:31)
30km - 1:27:38 (2:03:15)                                
35km - 1:42:16 (2:03:17)                              
40km - 1:57:15 (3:03:41)                          
Finish - 2:03:38                                                Finish - 2:19:44

Apologies - in all the drama of the men's world record, I missed a few women's splits, and the truth is, Kiplagat was just churning out consistent kilometers.  I'll see if I can update the splits later.

The men's race deserves more analysis, and so I'll have a closer look at that in a follow-up post


Commentary

Comments in reverse order - most recent at the top

Finish

WORLD RECORD!

2:03:38

Makau has done it, he has held on to break the world record by 21 seconds!  That was a hard effort at the end, Makau was fighting over the final 5km but he did it, and did it in style.  Even vaulted the advertising wedges in the finish straight to do it!

More analysis to come, including a comparison of this race with Geb's 2008 record.  Check in shortly!

40km

Makau is hanging on to world record pace - the last 5km was run in 14:59, and Makau only needed to run 15:20 to get this record.  So he is on course for a 2:03:41.  He has slowed slightly - the last interval was easily the slowest of the race!  But he has a buffer of sorts, and the last 2.2km are a race against the clock!

In 2008, Geb reached 40km in 1:57:34.  Makau reached it 19 seconds faster (1:57:15).  But remember, he was 49 seconds ahead at 35km, and so he is "losing ground" to the virtual figure of Gebrselassie on the road!

6 minutes of running is all that stands between him and the WR.  It's going to be a great finish!

37km

Makau is still on course for a big world record.  It's looking more and more like he is not just going to edge it, he is going to smash this record!  He's looking at a sub-2:03:30 time, and this is history in the making!


5km to go, and Makau needs to run it in 15:20, this is a world record on the way!

35km

Makau is on course for the world record!

Makau is out on his own now - it's a tough, tough ask to race the final 10km at world record pace.  You'll recall that Gebrselassie had James Kwambai with him in 2008.  The time at 35km is 1:42:16.  In 2008, Geb covered it in 1:43:05.

So Makau is still ahead, and he is holding onto the pace he needs.  To put it into context: At 30km, Makau was 47 seconds ahead of Gebrselassie's split from 2008 (1:27:38 for Makau vs 1:28:25 for Geb).  Now he is 49 seconds ahead, and so the world record is on!  Makau has run these 5km as fast as Geb did in 2008 (14:38)!  But the section 35km to 40km is where Geb really picked it up 3 years ago!  Does he have enough in reserve to do this?  Fascinating finish in prospect!

32km

With 10km to go, Makau needs to run 29:49 to break 2:04 and the world record.  It's definitely on, the only question is how Makau recovers on the run from that 14:20, and whether those first 20km were just too quick?

30km

It's Makau's race now - he's out in front with two pace-makers for company, probably over a minute clear of Gebrselassie.  The last 5km were run in 14:20, which is incredible - that was the acceleration that pulled Makau clear of Gebrselassie, just before the Ethiopian stepped off track.  That was very, very quick and now the race, and possibly the World Record's, is Makau's to chase.  His projected time at 30km, by the way, is 2:03:15.

The big question is what the surge takes out - Makau now has to consolidate - it may be "only" 12km to go, but the potential for time losses here are enormous.  He needs to run just inside 3 min/km to get that world record.  That is definitely doable, and this could be a great race to the line against the clock!

Gebrselassie is up and running again.  One of the pacemakers has dropped back and is now pulling him again.  He seems to making ground on those runners between himself and Makau.  He is 1:10 down on Makau, and we'll check that again at 35km to see if Geb has really recovered.  Quite extra-ordinary developments with Geb stepping off the road and now seemingly back, running still well under 2:04:30 pace!

27.2km

Gebrselassie has STOPPED!  He was dropped by Makau and he has stepped off the road, clutching his stomach, bending over and cearly in trouble!  Maybe asthma - he seemed to gesture that he was unable to brath.  Either that or a stomach problem.  Cramp/stitch maybe.  He is crouched over.  Now he is back running again, but he's lost big ground.  What a great pity!

Gebrselassie is now running again, and doesn't seem to be going too slowly.  It is very peculiar because he really did look to be in trouble there.  He was either struggling for breath, or had some kind of cramping or stitch pain.  But to reverse that and resume racing a minute later...very interesting.  We'll get you a split of the gap shortly, and keep an eye on it.

25km

The men have now begun to slow - the last 5km were run in 14:48.  It's the first time that a 5km split has been outside of the pace required for a world record (that's 14:41, by the way).  The projected time is now 2:03:43.  However, I can tell you that if they maintain the 14:48 pace for the rest of this race, they will finish in 2:04:12.

The TV coverage keeps flashing a projected finish time of 2:03:05, which is never going to happen.  not sure where that projection comes from.  Makau and Geb playing games shadowing one another!  A taste of things to come?  The last 10km might be very slow if they start racing and playing tactical battles

On the women's side, no change.  Kiplagat ran the last 5km in 16:43.

Halfway

The men's halfway split is 61:43.  Easy maths - it projects a 2:03:26.  The world record is now a definite possibility (it was at the start, of course).  The key will be after 30km, when most of those pace-makers drop out, and we're left with Makau and Gebrselassie.  Then we'll see if the early pace is costly.  If the pace is going to drop, it's going to be 25km to 40km.  Fascinating race developing though - the possibility of Geb vs Makau needing a 29-min final 10km to break the WR is mouth-watering.

Kiplagat has reached halfway in 1:10:11, 19 seconds ahead of Radcliffe.  If both keep going at this pace, they'll run 2:20:22 and 2:21:10 respectively.  That's probably a "par" for Radcliffe, given the build-up and hear she has had.  For Kiplagat, it's a good comeback after failing to finish Boston.

20km

Still no sign of slowing - after hitting 15km 12 seconds faster than the WR split from 2008, they ran a 2:58 and a 2:55.  So not surprisingly, they hit 20km in 58:30 . The 2008 split at 20km was 58:50, so they're 20 seconds ahead of that.  That's a big improvement - it projects a 35 seconds breaking of the world record.  The last 5km, incidentally, were done in 14:39, so they're holding faster than WR pace.  Every split so far has been faster than the WR pace.  It's still an 11-man group.

Kiplagat has continued to roll - 16:42 for the last 5km, so a small slowing in the pace.  She is 17 seconds ahead of Radcliffe, who has now dropped back to around 17 min/5km pace.  Unless there's a dramatic change of fortunes for one (or both), the women's race is developing into a victory for Kiplagat by just over a minute.  She's on for a 2:20:22 still.

15km

The men have actually sped up on the last interval - 14:34 giving them a 43:51 at 15km.  For comparison's sake, when Geb broke the world record in 2008, he hit 15km in 44:03.  They're actually saying that they ran the 15th kilometer in 2:45, which is unbelievably fast.  I'm more inclined to call that an error in the distance markings than a real time!  But it's very fast.  It now projects 2:03:21, so they're setting up an incredible day.  Or a big meltdown over the final 10km!

On the women's side - a big development - Kiplagat is nine seconds ahead at 15km.  Radcliffe has dropped off the pace somewhat.  Kiplagat has run with amazing precision.  The last 5km were 16:34, for a 15km split of 49:50.  Radcliffe came through in 49:59, so that's interesting.  Kiplagat meanwhile, has produced splits of 16:37, 16:39 and 16:34, and she's on course for a 2:20:11.  Will keep an eye on Radcliffe to see if she's going back, or holding that gap.

10km situation

The men have hit 10km in 29:17, so that's 14:41 for the last 5km.  The projected time now is 2:03:34.  There are still 11 men there, five of them pace-makers.  The other four (excluding Geb and Makau) are running many minutes faster than their bests, so that group could get very thin very quickly once the pace-makers start dropping off.

The commentators are saying that it's unusual for Gebrselassie to misjudge the pace.  I remember the Dubai race a few years ago where he went through 10km in a mid-28 time, projecting 2:02.  And I think the same happened the next year.  So it's not entirely unusual.

In the women's race, Radcliffe and Kiplagat have already opened up a sizeable lead over Mikitenko.  Their 10km split was 33:16, a last 5km of 16:39, so they're rolling along at the same pace.  Projects a 2:20:22.

5km reached

The men hit 5km in 14:36.  That's 2:03:13 pace, so fast, but that's normal for the first split.  There are eleven men in the lead group, five of them pace-makers. Gebrselassie and Makau are there.  They've requested 62 minutes to halfway, so if they get that, they'll be on course for a world record, and a race between Makau and Gebrselassie at that pace will be fascinating.  Early days yet.

Nice slow-motion shot of the elite runners feet landing - notice how they're heel-striking?  Gebrselassie in the yellow shoes - used to be a very clear forefoot striker on track.  It's a quick transition to mid-foot though, so the definition of "heel-strike" is disputable.  Flat at best.  But it sure isn't a fore-foot strike...

The women are on course for a 2:20:14 at 5km - 16:37.  The split was Kiplagat's, with Radcliffe listed at the same time.  So far no coverage of that race.

Shots of Geb training in Ethiopia now.  Nice touch for the human side of the race.

Pre-race

They're saying there are 30 pace-makers in the race today.  Not all for the elite men, but for various groups.  Of course, the build-up to this race included discussion of the role of pace-makers given the IAAF's recent decision that they will not recognize women's records set in mixed races, because this allows women to be paced by men.

I appreciate the reason for this, but I can't help feeling that if men are able to be paced to 30 or 35km, then women should receive at least the same benefit.  Of course, women's running lacks the depth of the men's race, and so they can't find six or seven women to get to 30km in the 1:40 required for a 2:20 marathon.  So women are either advantaged (by having men as pace-makers all the way to the finish), or disadvantaged (by having no pace-makers, or pace-makers to only halfway, for example).  There doesn't seem to be a middle ground.

But there's no question that the men get a large benefit - when the current record was broken by Gebrselassie, he was surrounded by men for 30km and then had Kwambai for "company" over the final 10km.  It's impossible to quantify the advantage this would provide, but it's fair to assume that it does help for a variety of reasons, both physiological and psychological.

To me, the best solution would be to allow men to do a pace-maker job for women, but only up to 30km.  Enforce something similar to the Ironman triathlons, where there is a "no drafting" rule that prevents athletes from riding in groups.  Why not have a rule that says after 30km, any men who are pace-making for women must drop back by a minimum of 50m?  They can still finish, but may no longer support and set the pace for the women.  Seems like the most reasonable compromise, given that it is a tricky situation, where either of the current situations creates advantages and disadvantages.

Anyway, enough of that, the race is about to start...

Pistorius: the "12 sec advantage" and mechanical superiority

Revealing the Pistorius science part 3: "We knew all along", as the debate turns to an argument

"We conclude that the moment in athletic history when engineered limbs outperform biological limbs has already passed"

That was the concluding statement from the first of a series of back-and-forth articles that were published in the Journal of Applied Physiolgy in November 2009.  It was written, remarkably, by two of the scientists who had in fact been involved in the research that was presented to the CAS when they made the decision to legalize Oscar Pistorius' carbon-fiber limbs 18 months earlier.  A research team of seven conducted the testing on Pistorius in Peter Weyand's laboratories in early 2008, and the fact that two of them would come out with a completely different conclusion after the hearing begs many questions - why did the difference in opinion not emerge sooner?  Was that difference deliberately snuffed out and 'hidden'?

Note that neither Weyand nor Bundle were present at the CAS hearing, and that begs a question or two itself.  Why did Weyand not attend the CAS hearing, or at the very least, Bundle, since they were of the differing opinion?  Was pressure applied to ensure that CAS did not hear a word of this possible advantage, and did they rule based on incomplete and 'manipulated' information?

This conclusion is the subject of this final piece in the full review of the Pistorius evidence.

The role of the lawyers, and the scientific integrity of the process

The paper emerged 18 months late, along with a Weyand saying: "From the instant we collected the gait-mechanics data and saw how short his swing times are, we said to the group [of scientists they were working with] that it's really clear he's got an advantage".

So to be clear - the evidence existed, and so did the interpretation that it provided an advantage.  But for whatever reason, this did not emerge until a year and a half later.  The "group" had clearly reached some agreement that this interpretation of an advantage would remain "hidden", because Hugh Herr and Roger Kram went to the CAS and presented a united front, saying that there was no scientific basis for an advantage based on the IAAF testing (if you read the CAS verdict, there is not a single mention of this possible interpretation).  Where were Weyand and Bundle in this process, you may be wondering?

Even if you accept that there are two interpretations of the data, you'd think that this one should at least have presented or discussed.  It was not.  And given that Weyand was convinced about it, you really do have to wonder what happened behind the scenes, especially when you consider that the research and the resultant published research paper was done as a result of a "deal" between the scientists and Oscar Pistorius' lawyers (in all this, don't underestimate the legal and PR machines in the background).  This is the main issue about the scientific process that I have questioned, along with what I explained in the previous post.

Too narrow a question and more selective omission

There is the argument that the CAS-proceeding determined a very narrow approach to the scientific question, which had really only one goal - to disprove the IAAF's findings.  In other words, the CAS process was not interested in the entire truth, but only in evaluating the evidence gathered by the IAAF.  And there's no question that the IAAF started off with a very narrow research question.

By extension, Pistorius' scientific team were interested only in the "truth" that would, among other things, eventually see them add distance runners to the control group until Pistorius looked similar to able-bodied athletes.  Effectively, the previous research had "set the bar" and they jumped over it, using the methods I explained yesterday.  A narrow finding got even narrower, and the whole truth did not emerge when it could have.  In all this, there was reason to suggest an advantage but the scientists did not make it known at the time, even if it was only for the purposes of debate.

Taking this into account, and adding in the fact that the research to clear Pistorius had very obvious omissions and false comparisons with distance runners when they knew what the sprinter-comparison would have revealed, you start to see that things really were not what they appeared to be with this "independent scientific process".  You may make up your own minds about what it means when scientists selectively leave out able-bodied sprinters and compare a sprinter to distance runners?  Or what it means when scientists recognize the possible advantage but fail to mention it at a hearing on advantages?  It strikes me as strange at best, manipulative bordering on dishonest at worst.

For a great read on the process and how twisted it seems to have been, David Epstein of Sports Illustrated provides the information, and I'd highly encourage you to read it here.  When you consider this, as well as the information we discussed yesterday on how the able-bodied group was manipulated to find

It then took 18 months, but when the paper was eventually published, it revealed a big split in the camp, and a group of seven was split into two - Weyand and Bundle on one side, arguing against Herr and co.  Eventually, it got to John McEnroe-like shouts of "you cannot be serious" in a scientific journal, and nothing came of it.  But it is this debate that I will end of with in this look at the science on Pistorius.  And then we can get on with enjoying the IAAF World Champs and on-track action.

The mechanical aspect of the debate

So far, we've seen that the IAAF testing found completely different mechanics and suggested less metabolic energy demand.  The Herr-Weyand study for the CAS hearing managed to find that Pistorius was "similar" to able-bodied elite and sub-elite distance runners.  Had they compared him to other sprinters, he was anything from 2 to 7 Standard Deviations different.

And then came this debate, which moves us from the metabolic to the mechanical - as I said in the first post, I believe the mechanical data provides the explanation for the metabolic and physiological findings of reduced VO2, and the implications for reduced metabolic demand.  And so that's the context of understanding this debate.

The "12-second advantage" - Weyand's calculation and the rationale behind it

Weyand and Bundle produced an article that explained the mechanical basis for an advantage, and concluded by estimating how much slower Pistorius would run if his carbon-fiber blades behaved like normal legs.  12 seconds was their answer, and that conclusion made their paper easy to dismiss for those who wanted to dismiss it.

So with hindsight, it may have been disingenuous to try to estimate the time advantage.  You can appreciate why they'd want to do it - it makes the debate more 'tangible', easier to conceptualize, but the problem is how it was done.  It took a few assumptions, and produced an advantage of 12 seconds, which was very easy for the likes of Herr to dismiss as too large to be realistic.  And that's fair enough.  But it shouldn't stop us from looking very closely at what Weyand is saying, how that number is estimated.  In order to understand where that number comes from, you first have to understand what it was that Weyand was arguing was the reason for the advantage, which is a good place to start.

A word on Weyand to begin with.  For all Pistorius' talk about how Herr and Kram and co were the best in the world, the truth is that Weyand is the world's leading biomechanist in sprint running.  Look him up on Pubmed and it's clear that he has done the seminal work on sprinting mechanics.  He is, you might say, an "F1 Driver" when it comes to mechanics...

And one of the key articles by Weyand is this one - a paper published in 2000 on how sprinters speed up, and it contributed the first key point in the debate.  Below is the key figure from that paper (sorry for not redrawing it - not enough time)

What you are looking at above are the swing times, in seconds, of able-bodied sprinters at a range of different speeds.  Swing time is the time spent by each foot in the air.  That is, it is the time taken from when the foot leaves the ground until when it next lands.

The runners are measured at a range of speeds, from 6m/s up to 11.1m/s.  Weyand adds Olympic runners measured on another occasion for comparison, but I'll disregard these three because there are questions about how accurately they are measured.  Besides, running at 40km/h is pretty much Olympic 200m level, and much faster than a 400m runner would run, anyway.  Pistorius ran at 10.8 m/s, so you don't need those Olympic athletes to make this case. 

The thing that should jump out is that there is no change in swing time as the runner speeds up.  Whether you are running at 6 m/s (21.6 km/h) or 11.1 ms (40 km/h), your swing time is pretty much the same. Even sprinting downhill, swing times don't get faster than this.

It's extremely tight, and that's crucial, because the key point about that is that in able-bodied runners, it is just not possible to swing the leg any faster than shown in the figure above, and so by the time we run 21km/hour, we are already at this "limit" for how quickly we can reposition our legs.  In Weyand's own words:

 "Clearly, with athletes with intact limbs, there's a lower limit to how fast they can reposition their limbs"

As a result, if you want to get faster, you have to do so by applying more force to the ground.  There's a downside to this - more force means more muscular work.  More muscular work means more metabolic energy cost, and two potential limits to how fast someone can run - first is the inability to generate that force, and second is the metabolic cost of doing the work, running at a given speed.  And that brings us to the theory of sprinting, and the Oscar Pistorius finding

Sprinting mechanics, and how Pistorius is "off the charts"

There are three things that limit the speed of a runner: 1) how quickly the limbs can be repositioned for the next step, 2) the distance traveled by the body while in contact with the ground and 3) the force applied to the ground in relation to body weight.

Above, I've shown that the evidence says that the ability to swing the leg is constrained regardless of running speed - 0.373 ± 0.03 seconds was the average ± SD.  Here's what Oscar Pistorius' mechanics look like compared to the able-bodied SPRINTERS when running at 10m/s in the CAS testing (about 400m pace):

So, longer contact times, shorter swing times, shorter aerial times, lower vertical force but not lower peak vertical force.  Using the statistical method that Herr and Weyand used for VO2, where anything more than 2SD different from able-bodied runners is different, it's clear that Pistorius is mechanically very, very different to able-bodied sprinters and crucially, these differences all provide significant sprinting advantages.

First, Pistorius has stride frequencies 16% faster than any athlete ever measured (and 9.3% faster than elite 100m sprinters - data not shown, and again, not worth including because of the measurement method).  It's a mass effect - the carbon fiber prosthetics weigh 3kg less than able bodied limbs below the knee, and so they can accelerate much faster, for less force.

The high stride frequency was the result of the speed of repositioning the limb.  That is, Pistorius' swing times are so much faster than any other runner measured, that Weyand described them as "off the charts".  At top speed, Pistorius has swing times of 0.284 s, which is 21% faster than sprinters - it's also 4 SD different from the other sprinters tested in this study, and 2.7 SD faster than the entire collection of sprinters tested in Weyand's lab in previous studies using the same methods.

Being able to reposition the limb so fast (11% faster than the fastest athlete in the group tested in the laboratory) has some major theoretical advantages.  In Weyand's own words:

"Oscar is off the charts. Clearly, with athletes with intact limbs, there's a lower limit to how fast they can reposition their limbs. With Oscar, if you make that lower limb twice as light, that moves that lower limit.  He repositions his limbs so fast that he doesn't need to get his body back up into the air so high like other sprinters, and that lowers the force he needs to generate.  The muscular forces he has to generate are less than half of what an intact sprinter has to generate to go the same speed." - quoted in SI article

Then there is contact time - Pistorius spends 14% longer in contact with the ground, which means the horizontal distance traveled during ground contact is higher (see the three factors limiting speed above), and it means that the force production required to run at a given speed is substantially reduced - it was 5.2 SD lower than the able-bodied controls in this study.  Interestingly, peak vertical force was lower, but not statistically (2SD) different from able-bodied controls.

How did he estimate 12 seconds?

So the next thing is to take the above measures and ask how did Weyand estimate a 12-second advantage?  There are a few assumptions that have to be made to do this, and as mentioned, I think Weyand did his own argument something of a disservice to not make a more "conservative" adjustment.  It's much the same as when I tried recently to explain how 6.2W/kg is a limit for cycling without doping.  You have to make assumptions, but if you make them sensibly, always making the assumption that goes AGAINST the thing you're trying to prove, then they're extremely valuable (I can elaborate on this more in the discussion if it comes up).

What Weyand did is move Pistorius' mechanics back into the able-bodied group.  That is, he looked at swing times and contact lengths and asked how Pistorius would run if those were "normal", or equal to the average of the able-bodied group.  So, the average swing time for the group was 0.359s so Weyand corrected Pistorius' swing times to that value. He also adjusted the contact length to 1.05m (to match the normal leg length-contact length ratio in able-bodied athletes) and average force produced during the stance phase.

The result was that he calculated that Pistorius' top speed would change from the measured 10.8 m/s to 8.3 m/s.  Next, using a similar method to that which Herr & Kram had used at the CAS to show that Pistorius DIDN'T have an advantage, Weyand estimated a 200m time of 27.3s and a 400m time of 61.7s, which was 12 seconds slower than his lab time-trial at the time of testing (49.8s)

Now there are two reasons why this 12-second estimate may be too high, and both relate to the inputs that estimate the performance.  The first is Pistorius' speed at VO2max.  During the testing, Pistorius' speed measured at VO2 max was 5 m/s (or 18 km/h, which is a sub-elite level), and this is factored into the prediction.  An elite sprinter would arguably reach a faster speed than this, and this increase (say to 6m/s, or 20% faster) would reduce the estimated time and thus the calculated advantage.  There is no question that Pistorius was less than in peak shape at the time of the testing - he fatigued at 15km/h after only 5 minutes, for example.

This has implications for both the findings of the study (Pistorius probably uses even less oxygen when highly trained than in the study, and that makes him even more different than was measured), and for the 12 second advantage.

The actual advantage - smaller than this, but nevertheless present

I think it's important to realize that the problem may be the INPUTS into Weyand's equation, and not the concept of the calculation.  If Pistorius had been 10% fitter, then those predicted times would drop, and the advantage predicted would be much smaller.  But given his state of training, it's actually not an unreasonable projection. 

The other issue is whether it's appropriate to correct Pistorius highly different mechanics to the average of the group.  It would perhaps have been better to correct them to say 2 SD from the average, so that he's still statistically similar, but at the extreme edge of the normal range.  If that had been done, then instead of adjusting his swing time to 0.359s, they would have made it 0.321s.  That calculation would produce estimates for performance that are closer to Pistorius' current times, but still faster.  I don't have the formula (or time) to make all these inputs now, but if the VO2max speed was 10% higher, and if the corrected mechanics were 10% lower, then the calculated advantage over 400m would drop considerably from 12 seconds.  And that is not nearly as easy to throw out as unreasonable.

Weyand didn't do this, and unfortunately the size of the advantage he did calculate gave ammunition to shoot the theory down.  That would wrong - the theory is correct, and the evidence based on mechanics is very strong, all that needed to change were the adjustments, the inputs to the equation.

So how large then might the advantage be?  Obviously, it's impossible to say, unless there is more testing.  That's not going to happen because Pistorius would never agree to it now - the science was only important until it cleared the blades for use.

I'd suggest another method that would give a good indication, using Pistorius' 400m and 200m race times, and then comparing them to other 200-400m athletes.  This works because the performance at one distance is a great predictor of performance at the other, and so when you look at this, compared with pacing, you get a good idea of how much he GAINS overall in a 400m race.  That is is, net loss would be canceled by net gain.  But not to leave you hanging, but I'll save this for another occasion, in the interests of time (it's a lengthy explanation of pacing strategy).

The rebuttal

Herr and the rest of the intial group produced a counter-point, and there were two main "points of attack".  One was video footage of other runners, to show that Pistorius was not really as different as Weyand claimed.  The problem is, you can't use normal video footage to make this case, the resolution is just too low to be accurate - that's the reason why I wouldn't include the Olympic or World Champion level runners in the control group.  Weyand did this in his subsequent response to Herr, and it saw the debate get a little side-tracked - the truth is that every finding in Weyand's initial report, and what I've summarized above, was produced in the laboratory, with very high resolution cameras, whereas Herr's rebuttal was based on inaccurate video footage.

Then Herr also reveals the hypocrisy of the statistical method used when it suited the desired finding earlier in the CAS process.  He writes:  

"Pistorius’ leg swing time of 0.284 s at 10.8 m/s is nearly 3 SD faster than that mean (It's actually 4 SD from the mean, but who's counting?).  However, leg swing times as low as 0.31 s for Olympic 100-m medalists at top speed have been reported"

So firstly, the Olympic runners were measured with low-resolution video, and Weyand responds to this with the example of the same athlete measured in the lab and on TV, and the TV times are 16% faster than the accurate lab times.  Clearly, you can't base an argument on TV footage.  The case I've made in the post above doesn't use TV footage, and is very compelling.

But more than this, having set up the 2SD requirement for difference, Herr now disregards it, and instead wants to compare Pistorius to the minimum value in the control group.   Even here, though, the difference is enormous.  The top graph in the post above shows some of the athletes who can be validly compared to Pistorius - yes, one athlete has swing times of 0.315s at top speed.  But Pistorius was 0.284s, and that is fully 11% faster than the next fastest swing-time ever measured - it's a whole 1 SD between the fastest swing-time and the next fastest. Weyand was quite correct in his response to say that "the double-artificial-limb value is not simply an outlier; it is quite literally off the biological charts".

The force production - advantage or disadvantage, cause or consequence?

The second big point of contention is the importance of the ground forces.  To repeat, Pistorius' average force production is 23% lower (5SD) than that of able-bodied athletes.  Hugh Herr argues that this reduced force production is a disadvantage, whereas to Weyand, it is a huge advantage (See quote above).

In trying to sort out this issue, the important point is that Pistorius' reduced force is present and measured during running at the same speed as the able-bodied runners.  That is, they're all going at 10 m/s when these huge differences are measured (see the graph above), and they persist at top speeds.  When you have fixed the speed, then the ability to increase the force is irrelevant.  In Bruggemann's correct words: "If we look at subjects running at different speeds, it’s logical to say that the higher the force, the higher the speed. But with all subjects running at a given speed, lower force is an advantage.”

On top of this, Herr looked at it 180 degrees backwards.  The 2000 finding by Weyand showed pretty clearly that the first limiting factor to sprinting was the ability to swing the leg faster.  This is "maxed out" fairly early, and then the ability to produce force separates those who can achieve Olympic-like speeds and those who cannot.  So there is an order - first you exceed the limits of moving the limbs faster, and then you have to apply more force to get faster.  And so when you have the ability to move the limb that fast - fully 11% faster than the next fastest person measured, and 33% faster than average), then the changes in force are the result.

As a result, the lower force production measured in Pistorius is the consequence of the reduced swing times.  The muscle force production requirement is thus lower - "half" according to Weyand, and that has some obvious benefits.  Add to this the higher energy return, the fact that less mass is being carried, and that carbon fiber does not fatigue during a 400m race, and you have explained why Pistorius uses much less energy at all speeds than other sprinters.

Advances in technology - have the blades really stayed the same?

The final point, before we get on to the IAAF World champs and leave this for now, is the issue of technological progress.  In a recent interview, Pistorius said the following:  

"My prosthetic legs have stayed the same for seven years, down to the bolts and the lining"

Hugh Herr, meanwhile was interviewed in a piece where he says that the "Cheetah blades have been available to athletes in their current form for 15 years, and that Pistorius has run on the same blades for the last seven years." 

Now, if you do believe that a company like Ossur, which operates five R&D departments, files more than 300 patents per year, and spends millions of dollars every year on product development (6% of total sales in 2007 was spent on R&D) would not change the blades over more than a decade, then you are more gullible than the Pistorius PR team would even have dreamed.

The reality is that technology evolves all the time - and in this particular segment, prototypes are produced, and tested by athletes who then have them customized.  Take the following, from an article where the journalist actually accompanies Pistorius (with his permission) to Reykjavik to test the latest prototypes in 2007:

"But last September [in 2007], Pistorius and Brauckmann went to Reykjavik to test prototypes designed for double amputees. The new ones, which Pistorius hasn’t debuted at a major race yet, make just one smooth curve, an arc of pure engineering.  Ossur’s R&D team met them at the company’s workshop and unveiled the prototypes. Brauckmann attached the blades to the sockets, and Pistorius walked around on them, testing the design".

So, in 2007, a prototype or two was tested.  Read further and you discover that each blade is customized for the athlete.  Add to this that these companies (and there are a few) regularly send their engineers to the athletes to 'test-drive' limbs that have a different stiffness, and you realize that the technology is not as stagnant as you may be led to believe.

Finally, I also know that a simple thing like changing the stiffness of the blade makes a big difference to  performance.  One case - a single-leg amputee tries out a range of different blades.  They are the same product, but with subtle differences in stiffness, curve etc.  The result is a variation in performance that is not hundredths or even tenths of seconds, but seconds.  That's a bigger effect than a year of training, than nutrition, I dare say, even doping will provide.  The simple reality is that the technology IS evolving, and there's no way to police it.

And to say, like Herr and Pistorius are both saying, that the blades are unchanged over 7 years is to say that the BMW that is driven by a friend of mine, a 2004 model, is the same as the 2011 BMW that he would like to upgrade to!  They're both BMWs, they both have four wheels, an engine and a steering wheel, but they're nothing like the same car.  So sure, the blades are the same.  But they're a world apart.

And the implication of this is that performance improvements, so coveted at the elite level, are now accessible through acts of engineering, subtle changes like stiffness, where there are many options within the same product.  Remember that a 0.1 second improvement is valuable in a 400m race.  0.5 seconds can be the difference between being fifth in your own country and top 10 in the world.  The margins are so small that it is unnecessary to talk about revolution - all that is needed is progression and prototypes provide this.  The CAS verdict applied to the blades that were used, but how is that even enforceable when you have subtle changes that can produce


Conclusion: Skillful but with an advantage

One final quote, again from the Wired article:

"To give me a sense of how they feel, Ossur’s engineers bolt a pair of Cheetahs to the back of two rigid plastic-and-leather motorcycle boots. I clamp in and trot across the room a few times. The Cheetahs seem to bounce of their own accord. It’s impossible to stand still on them, and difficult to move slowly. Once they get going, Cheetahs are extremely hard to control."

Legs that "bounce of their own accord" supports the conclusion of a "bouncing locomotion" made by Bruggemann in his research.  They must be very difficult to control and let me emphasize this point - Pistorius' unique performance is the result of his skill levels in using that equipment.  In the way that Roger Federer, Rafa Nadal or Novak Djokovic are elite because of their ability to use their equipment, or in the way that Sebastian Vettel is a superstar because of his skill with his equipment, Oscar Pistorius is skillful.

But every line of evidence - the metabolic, the mechanical, the physiological, the pacing - points to one thing - substantial advantage.  Of course, there is so much that has gone unanswered as a result of the narrow questions asked first by the IAAF and then by the Herr/Weyand research.  Would it not have been great to look at things like how much time may be lost at the start, whether running the bend is a disadvantage (I have data showing that Pistorius runs the bend faster than the straight)?  To test whether increasing the mass of the limb removes these mechanical advantages?  The narrowness of the research means we'll never know and there won't be more testing unless the IAAF demands it, because Pistorius has nothing to gain from discovering the truth, whatever it may be.

But the Weyand research, where Pistorius is off the charts, and the metabolic finding (which is explained by the Weyand conclusion, thus supporting it further) all say advantage.  Again, remember that this begins with the hypothesis that there is an advantage as a result of energy return and mass.  That hypothesis was confirmed by the metabolic measurements, and then explained by the mechanical differences.  It's difficult to see it any other way, as I read it.

Next up:  IAAF World Champs

And on that note, I leave this and look forward to the action on the track - IAAF World Championships start this weekend and there are countless stories that will add to the drama.  There is Semenya in the 800m, returning to World Champs after the 2009 saga.  There is Rudisha.  Bolt vs Powell.  Bekele vs Farah and Kenya. Galen Rupp and the possibility that either Kenya or Ethiopia might not get a medal.  There is Allyson Felix attempting a double.  There are dozens of great match-ups and stories and we'll cover them all in the next few weeks!

Ross

P.S.  A final point on the Herr-Weyand research, which I put as an Appendix of sorts because it's of interest but peripheral to the evidence...

Why was it not peer-reviewed?

One of the most common retorts to the Weyand argument about advantage is that it was not peer-reviewed.  That has come from both Herr and Pistorius, who dismiss it because of this failure to get it peer-reviewed.  The problem is, that research will never be peer-reviewed, because it belongs to BOTH Weyand and Herr.  They collected the data together, and so any research paper that gets written has to have all seven of the scientists agreement in order to be published.

Quite clearly, that was not going to happen - Herr was not going to put his name to a paper he so clearly disagrees with.  And therefore, it is not possible to be peer-reviewed.  That is the reason why the debate took place the way it did, as a point-counterpoint.  The irony then is that Herr (and thus Pistorius) are the very reason it isn't peer-reviewed.

The scientific evidence for an advantage for Oscar Pistorius

The second research study - what it really found about Oscar Pistorius

We may as well jump straight in with the continuation of the scientific summary of the research done on 400m sprinter Oscar Pistorius.  Yesterday, I described how the first round of testing, done at the request of the IAAF in Germany, found that Pistorius used 25% less oxygen during a simulated sprint, and that his running mechanics were vastly different to those seen in able-bodied runners.

That study led the IAAF to ban him from competing, and resulted in the appeal to the Court of Arbitration for Sport.  That brings us to the second research study, led by Hugh Herr, and, at the time, Peter Weyand.  Below follows an explanation of a key part of the paper they produced a few months AFTER the CAS hearing - the oxygen uptake question, which I introduced as being important in yesterday's post.  The timing is important because one of the first key points about the CAS-process is that the research was not peer-reviewed until after the decision was made, which contrasts with claims that the decision was based on peer-reviewed research.

With respect to the CAS hearing, there were without doubt procedural errors in the IAAF process of the scientific testing in Germany.  These are legal issues, pertaining to things like the timing of making documents available, inaccurate reporting of data in an IAAF summary, and dictating testing conditions.  This procedural dispute led the CAS to declare that "the manner in which the IAAF handled the situation of Mr Pistorius...fell short of the high standards that the international sporting community is entitled to expect from a federation such as the IAAF".  Unfortunately, it wouldn't be the first time that this has happened for a major sporting federation...

However, this is a look at the science, following on from yesterday, where I explained the specific scientific issues that were taken to the CAS panel, along with responses by Bruggemann and a few quotes from other biomechanists, as well as my views and interpretation of those points.

Today, I move on to the Herr research, which I believe is equally flawed.  But I'll go through it systematically, building up the main findings and figures, and then allow you to draw conclusions about what was done to come up with this overall finding: "We conclude that running on modern, lower-limb sprinting prostheses appears to be physiologically similar but mechanically different from running with intact limbs." (Weyand et al, 2008)

The metabolic cost of running - similar?

To begin with, one of the key points challenged at the CAS hearing was the measured reduction in oxygen use during a simulated sprint (figure shown to the right).  The problem is that during a sprint, metabolic energy is provided by a combination of oxygen dependent and independent sources, so Bruggemann provided only half the picture.

You would not be surprised to learn that in the CAS hearing, the IAAF focused on the biomechanical aspects of their research, and downplayed this particular finding, because their conclusion based on the measurement of oxygen use in a sprint was always going to be challenged very strongly.

And challenged it was - it would also not surprise you to learn that the response from Pistorius' research was to repeat the above test, but this time to measure oxygen use at sub-maximal speeds, because this provides a better indication of the metabolic cost of running.  And so this is what they did - Pistorius ran on a treadmill at a range of SLOWER running speeds.

The method used was to have Pistorius run for 5 to 7 minutes at a range of speeds, with 3- to 5-minute rest periods.  The starting speed was 9km/h (a very slow jog) and Pistorius reached exhaustion (unable to complete 5 minutes) at 15km/h (which is very slow, tellingly - Pistorius did this testing at a time when he publicly announced that he was untrained and unfit.  That this impacts on the results you'll see below is important).

Using measurements of oxygen use from Pistorius during these runs, they were able to calculate what is called running economy, or the volume of oxygen used per kilogram per kilometer. Think of this as fuel use in a car - the less oxygen you use per kilometer, the more economical you are.  Remember that the hypothesis based on one interpretation of the theoretical knowledge at the time would be that Pistorius would use less oxygen, and hence less energy, at a given speed because of the reduced mass and the increased energy return of the carbon fiber limb (see yesterday's post for the explanations).

(By the way, if you adopted the exactly opposite position based on the theory, then you might hypothesize that Pistorius would use MORE oxygen and more metabolic energy because of the demand for balance and the increased work required to control the limbs - this is what his coach, Ampie Louw, explained in a TV interview in 2007 for a UK news insert.  The point is that either theoretical claim can be tested by measuring oxygen use - it is a barometer for energy use, and will either be similar, in which case the theory is disproven, or will be different, in which case one of the theories will be proven, the other disproven.)

There are a few pertinent considerations here.  First, the Cheetah blades are built for speed (the manufacturer's own claim), not slower running, and so I would argue that testing them at these slow jogging speeds (9km/h is very slow) will provide a false picture, skewed in favour of finding a higher oxygen use.  However, even without that potential confounder, the data still provide some really interesting insight.

What is more crucial, for reasons that will emerge as we progress, is the vital question of who do you compare Pistorius' data to?  The answer must surely be able-bodied sprinters of similar performance levels.  That's because the literature shows that sprinters use more oxygen (or are less economical) than distance runners.  So the "control" group would have to be able-bodied SPRINTERS.  And this is where I begin, with four able-bodied sprinters compared to Pistorius (note - the development of the graphs below (which is Figure 2B in the research paper) does not necessarily follow the timeline of the testing, but it builds the argument and highlights the key issue)

The first graph, shown below, reveals the first finding - Pistorius compared to able-bodied sprinters at the same performance level:

So, Pistorius uses 17% less oxygen than the able-bodied controls, or 2.7 Standard Deviations less.  Without going into the stats, that's a big difference - generally, anything more than 2 SD is considered an "outlier", very different.  And so on the basis of this finding, Pistorius is very different to able-bodied sprinters.  This is not different to the 25% difference that Bruggemann found during sprinting, and it's not inconceivable that the 17% would be higher (approaching 25%, maybe?) at sprinting speeds, where the Cheetah blades are more effective, doing what they were built to do.  This first finding confirms the Bruggemann research.

Yet the paper concluded, you'll recall, that Pistorius is physiologically similar, and Hugh Herr was recently quoted as saying "It is not true that Oscar uses less oxygen than a person with two biological legs, although he is very economical".

The addition of the distance runners

So how do you get to that conclusion?  Answer - you add distance runners to the able-bodied population. Herr and co decided that in order to increase the size of the control group, the next group that should be compared to Pistorius was not more sprinters, but rather a group of sub-elite and elite DISTANCE runners, who had been measured in a study in 1995, thirteen years earlier in a completely different laboratory.

Once these distance runners are added, the following graph can be drawn:

So, with the addition of the distance runners to the control group, the gap between the sprinter Pistorius and the able-bodied runners is coming down - presumably this is a "good finding" if you're involved in this research and "want" to disprove the earlier IAAF study finding.

The difference in oxygen use between Pistorius and sub-elite distance runners was 6.7%, while Pistorius uses 3.8% LESS oxygen than elite distance runners.  However, statistically speaking, the differences are 0.8 SD and 1.3 SD to those groups, respectively, and so it can now be concluded that Pistorius is, in fact, not different to able-bodied runners.  Provided those able-bodied runners are elite distance runners, that is.

For good measure, they also add John Ngugi and Zersenay Tadese to the sample, and the graph is now complete, as found in the research paper (Figure 2B on Page 906):

Just a word on the Tadese value shown above: this is a study that I actually wrote quite a bit about back in 2007, and the main reason was because Tadese was the most economical runner in history, but there was something not quite right with that running economy value.  His economy, measured at 150 ml/kg/km has some bizarre implications - it means that running at world record pace for the marathon (2:56/km), Tadese would be using only 51 ml/kg/min of oxygen.  Take it from me, that's just not possible, especially when you consider that Tadese's VO2max was supposedly 83 ml/kg/min - his marathon world record pace would be an easy jog!  My point is - something didn't add up, and I'd be very cautious about accepting that Tadese value.  And this illustrates just why you should not take data from other labs and use it as part of your data set as was done - you simply cannot guarantee the validity of that data.

To do this, to borrow from other research, where methods are different, equipment is different, and athletes are different, is to violate a key concept in the control of science.  It's the same thing as if I want to test the effects of something like compression socks on muscle pain after running, and I make a few runners do a trial, and then compare them to a study done in a different lab with different methods.  Certainly, you will use other studies to explain your data and provide context in the discussion, but to borrow data like this is extra-ordinary.

Having said this, I can appreciate why it was done - the time was limited, and the research had to be done for the CAS panel.  The control group was small initially, and had to be increased in size.  So given the time constraint, I can appreciate that this would be an exceptional circumstance.  But the question is this:  Why compare a sprinter to a distance runner in one of the key variables (running economy) that is known to be very different between them, when data on sprinters exist for a more valid comparison?  At the very least, show BOTH distance runners and sprinters...

Why compare to distance runners, when data for sprinters is available?

The question is so important, I repeat it - why would Herr and co have chosen to compare Pistorius to elite and sub-elite DISTANCE runners, when the data exist for sprinters' running economy?  It's not as though we don't have the data for sprinters - it exists.  But the research chose to ignore these data, and instead focus on distance runners, both elite and sub-elite.  Why?  I hope it's becoming clear in your minds as you read this.

But before continuing, take a look at this sentence from the research article, in the discussion (pg 909):

"However, his values were 17% (2.7 SD) lower than those of the intact-limb 400-m specialists tested here and two or more SDs below the means reported for four other groups of subelite male sprinters"

In other words, the authors are aware of FOUR studies on sprinters that could have been used to expand on the subject numbers used in the very first graph I showed above.  Four studies that would have increased the size of the able-bodied control group and allowed a VALID comparison of a sprinter (Pistorius) to other sprinters, rather than elite distance runners.

One acknowledged consideration is that the sprinters in these studies are not elite, but then neither are most of those in the distance group used in their place.  The sprint data could, at worst, have been included in the analysis.  But instead, they chose to ignore the sprint data on athletes who were, at the time, at more or less the same performance level as Pistorius.  Rather, they presented the distance data, which conveniently supported the ultimate finding of physiological similarity.  Perhaps the comparison with other sprinters produces a finding that you'd rather not show...

The graph that compares sprinters to Pistorius - the real finding and comparison

So I went and found the four studies in question, and I've redrawn the graph, this time showing the oxygen use of Pistorius compared to other sprinters and middle-distance runners, rather than elite distance runners:

So, that graph looks a little different to the one shown above, the one that was eventually published in the research paper (Fig 2B), comparing the sprinter Pistorius to the elite and sub-elite distance runners.  Here, when you compare Pistorius to sub-elite sprinters, you get a very different picture.  I've shown in red below each bar the difference in percent between Pistorius and these sprinters, and in black, the difference in Standard Deviations.

So 17% or 2.7SD is the Weyand et al finding (comparing Pistorius to performance matched sprinters).  13% or 7.2SD is the difference to 400m sprinters (performance time 50s). It's 3.2 SD to 800m runners, 5.1 SD to 1500m runners, and overall, Pistorius uses 14% less oxygen than able-bodied sprinters and middle-distance runners.  That's 2.3 SD and a difference that, according to the definition set out by Herr, would have led to the conclusion that "Pistorius is physiologically different"  And importantly, it's a difference that strongly supports the hypothesis of advantage.

Again, these are sub-elite athletes (50sec 400m time, for example), but they do include middle distance runners who would be expected to have better economy than sprinters (remember, Pistorius is a 200m-400m sprinter), and they are close to the performance level achieved by Pistorius in the actual testing protocols (they actually exceed his level - remember, he hits a peak sustained speed of 15km/hour - some of the above runners were measured at 20 km/hour).

Conclusion - look at what is included as well as what is not

So the point in all this, I would hope, is obvious.  The selective addition of distance runners to the able-bodied group allowed the physiological differences to disappear (statistically speaking).  Meanwhile, selective omission was happening - there were at least four studies on sprinters that would have confirmed the Bruggemann finding, and even the Weyand 2008 finding that Pistorius has significantly different physiology and metabolic cost compared to other sprinters.  This would go a long way to confirming the initial hypothesis, and so it's not a trivial matter at all - it is in fact the crux of the issue.  The question is why make this selective addition and omission?

One answer may provided by the background to the research.  Pistorius has the Bruggemann research, he knows the methods and the results, and knows why he has been banned.  He now has to cast doubt on the Bruggemann research, showing the CAS that there is insufficient scientific evidence to ban him.

The first point to challenge for the CAS hearing, arguably the "weakest point" in the IAAF case, was the issue of energy use, or metabolic cost of running.  So the researchers Pistorius works with, one of whom is a big recipient of funding from Ossur, who make Pistorius' blades, are tasked with disproving the Bruggemann findings.  That means one thing - show that Pistorius uses similar volumes of oxygen compared to able-bodied runners.

Unfortunately, he doesn't.  At least not compared to sprinters, who train like he does, who share the genetic make-up of being a sprinter rather than a distance runner - he's 17% different despite everything being in his favour - he's untrained, he's running at slow speeds on equipment designed for higher speeds.  I dare say that three months of training would reduce Pistorius' values even further.

But when distance runners are added, the control group changes sufficiently so that even though Pistorius uses less oxygen and energy than them, he is statistically similar, and it is concluded that he is physiologically similar.

CAS accepted this (which defies belief, but anyway), and so the key barometer in the debate, the key variable that allows one to evaluate whether the theory is correct regarding the reduced energy cost of running on the carbon fiber blades, suddenly appears questionable.

Go right back to the hypothesis - either Pistorius has a metabolic advantage because of the reduced mass and the elastic energy return, or he doesn't.  Measure the oxygen used, and it will decide for you - the measurements of oxygen used confirm the metabolic advantage - across a range of speeds, from barely a jog to a 400m race, Pistorius uses significantly less oxygen than able-bodied sprinters.  Even dipping into previously published literature confirms this.  My take - hypothesis proven.

The mechanical debate and the "10 second advantage"

There is still the issue of mechanical advantage.  But it's late here in SA, and that is an issue for another day, when I'll look at the split that occurred between Pistorius' own scientific team, leading Weyand to publish a letter to JAP suggesting a 10-second advantage.  That is the final piece of the puzzle.

Thanks for the comments to yesterday's piece, it's good to inspire debate and as mentioned, in all this, there's allowance for disagreement.  I mean, we can't even agree on something as basic as whether dehydration causes heatstroke, I don't expect consensus on this.  But I've seen more ambiguous data, that's for sure - this is, I suggest, pretty clear cut.

Ross