THE WORLD’S FIRST OCEAN RATING SYSTEM
Every major sport has a rating system that provides an objective way to rank and compare people who compete.
In sailing yachts are handicapped rated (e.g. IRC, AMS) to allow different sized yachts to compete. But there is a void in how ocean sailors are rated and ranked.
Sailor.com.au Ocean Rating (SOR) system has been created to provide an objective way to rate and compare ocean sailors who compete at the highest levels. It has been designed to compare sailors within a race or a race run over many years.
The SOR system is objective, transparent and fair in how it rates sailors.
A core belief of SOR system is that over the long term sailors end-up on yachts that they deserve and yachts end up with sailors that they deserve.
Sailors who are gifted, or with lots of potential usually end up on top-notch yachts and top-notch yachts attract the best sailors. Those yachts that are badly run or not up to scratch lose good sailors, they jump ship for a reason. These yachts, short of crew, attract sailors who can’t get rides on better yachts or don't know any better.
Sailing is a more than a sport it has elements of danger, luck, adventure and skill. Many people win races because they were in the right place at the right time. These observations can be used to develop a way to rate sailors.
Ocean yacht racing is not a zero sum game like tennis or chess. It's an event where yachts place within a race and they are ranked based on their respective times. The winner is determined by their elapsed time or handicap elapsed time.
SOR is an evolving system. We expect to refine it and improve it as more information comes to light and we better understand the insights from racing data collected. As we evolve and improve we will re-rate everyone using the improved models. This will provide a consistency and while also provided progress in the face of change.
HOW SOR WORKS
The way that SOR (SAILOR.COM.AU OCEAN RATING) currently works is that it attributes the success of a yacht to the owners and sailors associated with it. As a sailor goes from yacht to yacht they inherit their past yacht racing results. The theory is that over time a sailors rating reflects the results of all the yachts that they have sailed in races for. SOR also tracks the role of a sailor and determines their rating based on their roles as well.
To be able to rate sailors we must first be able to rate a yacht. This is fairly straight forward as all races have a system in place to score yachts in a race.
This can be done by taking its elapsed time to complete the race and scoring this against the rest of the fleet. However, this "raw" rating is unfair as some yachts are bigger than others.
To deal with yachts of different classes and sizes we can adjust the times by the primary yacht handicapping system in use for the event, for example like IRC, or ORCi, to get close to assessing crew performance.
However many yachts races change their handicapping system over time when a better handicap emerges. For example the Sydney to Hobart yacht race organisers have used a series of different handicapping systems over its history.
RACES OVER THE YEARS
The next issue to contend with is that yacht races are never the same. As an example, the Sydney to Hobart racetrack is the same each year it starts in Sydney and finishes in Hobart, but each year the fleets experience different weather and sea-state conditions. Some Sydney to Hobart races are easy and fast, others are slower, brutal and gruelling. To complicate things even more yachts get faster because of innovation in yachts, materials and technologies. Yachts have become faster, lighter and better. Sailors also get better because of technology and improvement in clothing, nutrition and yachts.
Each ocean race victory is not equivalent, and that some victories contain more information than others.
For example, it would be misleading to give a similar rating score to a yacht winning a hard race as compared to an easy race.
SOR comprehensive ranking method takes into account the quality of a victory when calculating the rating of a yacht and sailors.
To have meaningful ratings we needed to deal with issues of different conditions and innovation to be able to compare results in races for the same event over time.
SOR approach is to "normalize" races held in different years by a factor that adjusts yacht race results to take into account different conditions and innovation.
These “normalisation factors” need to be objective and transparent so that the community can have trust and confidence of the ratings. Most of all they must be easy to understand and use.
To make the system easily used SOR uses three objective measures in its "normalization" factor.
Average speed of the fleet
Number of Retirements in a race
Number of race starters
AVERAGE SPEED OF FLEET
Innovation in yachts, materials, and racing techniques improves the speed of yachts over the long term.
This innovation drives the average elapsed time down over time for any ocean yacht race.
Innovation can be seen in yacht design, materials (used for hulls, rigs, and sails), navigation systems, routing systems and weather technologies. Also innovation enhances sailor’s abilities.
The other issue around average elapsed time is the weather conditions.
A race where kites are used for the whole race is very different to one which is mainly reaching, and these two conditions are very different to a race that is on the nose the whole way. One where the sea state is favourable, following seas, to one where the sea looks like a washing machine.
We have devised a formula that isolates these two components – the impact of innovation and conditions. The SOR formula provides an estimate of what the average elapsed time for a yacht should be in any year adjusting for innovation. This provides a baseline of average speed.
The formula for the baseline is
BASELINE ELAPSED TIME (BET) = 545384*EXP(-0.008*( RACEYEAR - 1944 ) )
The baseline average speed allows us to work out if a race was predominately fast because of weather and sea state conditions.
For example
in the 2016 S2H the average elapsed time (AET) of the fleet was 260,568 seconds and the BET was 306,583 seconds this means that the conditions factor was 0.8499 ( AET/BET ).
in the 2015 S2H the average elapsed time (AET) of the fleet was 366,182 seconds and the BET was 309,046 seconds this means that the conditions factor was 1.1849 ( AET/BET ).
BRUTALITY OF CONDITIONS – KNOCKOUT FACTOR
Some races are wickedly brutal others pleasantly kind.
The next consideration in SOR is working out the brutality of a race. It is important when "normalizing" to considering the number of retirements. SOR calls this the Knockout Factor.
A race that has no retirements indicates that the environmental conditions were favourable. A race that has many retirements indicates that conditions were wickedly brutal.
The brutality of a race can be short-lived like a storm front passing through the fleet or a weather system that lasts for days. We did consider the lack of preparation of yachts for the race, but concluded that the very small minority of owners would undertake a race like Sydney to Hobart without serious planning and preparation. While every race may have naive or ill prepared yachts, these are few and far between. Organisers like the CYCA work hard to ensure a high standard of preparation of entrants.
We measure the actual retirements and use this to come up with a factor that represents the brutality of a race. For example 1984, 1993, 1998 and 2004 were all very brutal races, or Gruelling races.
To work out the Knockout Factor we SOR uses the following formula.
DNF = ( ( STARTERS – FINISHED ) / STARTERS )
KNOCKOUT FACTOR = EXP ( DNF )
Examples
in 2015 S2H there were 108 starters and 77 finished giving a DNF of 28.7%. The Knockout Factor was 1.185;
in 2016 S2H there were 87 starters and 82 finished giving a DNF of 5.75%. The Knockout Factor was 0.85;
NUMBER OF PARTICIPANTS IN A RACE – FLEET FACTOR
The number of participating yachts in a race also affects the rating for sailors.
Winning a Sydney to Hobart yacht race with 9 yachts (1945) is not the same as winning with 331 yachts (1994) in a race.
Obviously the lower the number of entrants the less weight a result has. We came up with a measure that takes the number of entrants into account to "normalize" a race across time.
As a rule of thumb most races with around 100 participants will have Fleet Factor of .98
The formula for Fleet Factor is
PART A = ( STANDARD DEVIATION FLEET ELAPSED TIME / SQRT (STARTERS) )
FLEET FACTOR = 1 - ( PART A / FLEET AVERAGE ELAPSED TIME )
Examples
In 1945 S2H the FLEET FACTOR was 0.946584. The number of starters was 9, fleet standard deviation was 116,351 seconds, fleet average elapsed time was 726,060 seconds;
In 2015 S2H the FLEET FACTOR was 0.987326 and 2016 the FLEET FACTOR was 0.980404
THE DIFFICULTY INDEX
SOR combines these three factors (AVERAGE SPEED, BRUTALITY, PARTICIPANTS) to determine the overall normalisation factor. SOR calls this the "difficulty index" and it is computed for every major race run.
The difficulty index, when used in the right way, allows us to compare results across time for the same race.
The formula for the Difficulty Index is:
DIFFICULTY INDEX = CONDITIONS FACTOR * KNOCKOUT FACTOR * FLEET FACTOR
The SOR computes a unique "difficulty index" for every major yacht race that is rated by SOR. Every major race under SOR has its own unique normalisation model, due the variation in races and conditions.
The table lists out the most difficult races in the Sydney to Hobart using its unique difficulty index.
We have spent considerable time researching archives and speaking to sailors about their personal experiences to calibrate our results with their personal experiences. Many sailors views are biased based on their unique experience on a very vast race track.
A difficult race for a yacht may be the result of their choice of route or just being unfortunate to experience conditions limited to their patch of the ocean race track at that time.
In SOR Difficulty Index is categorised into 5 categories. These are:
KIND: Up to and including 0.93
NORMAL: Beyond 0.93 and up to and including 1.16
HARD: Beyond 1.16 and up to and including 1.39
TOUGH: Beyond 1.39 and up to and including 1.86
GRUELING: Beyond 1.86
NORMALISING USING THE DIFFICULTY INDEX
SOR rates the performance of each yacht in a race using a normal distribution approach based on a yachts elapsed time. SOR uses two elapsed times: (a) raw, being its actual elapsed time; and (b) handicap, being the handicap adjusted time using the primary handicapping system of the race.
Using the distribution curve, we assign a number between 1 and 0, where closer to zero means that a yacht has better performance. This number reflects which percentile the yacht fell into based on the whole fleet.
A yacht with a number of .5 means that finished in at an average time.
SOR takes these percentages and convert them into what is known in statistics as t-scores. To standardise the scores across other like yacht races.
We use the t-scores and multiply the difficulty index to normalize these t-scores.
Multiplying the t-score with a difficulty index has an effect of stretching or shrinking the t-scores around the average. This means that yachts with high scores in difficult races will exceed those yachts with similar normal scores in easy fast conditions.
Those with poor scores in difficult conditions will be rated lower than those in ideal conditions.
SOR does this because difficult conditions are a truer reflection of the reality of sailors rating than ideal conditions. SOR view is that ideal conditions create an invisibility cloak for both good and poor sailors. Brutal conditions reveal what is under this invisibility cloak. They test the mettle of sailors and their yachts.
In SOR if you do well in a difficult race you get rewarded, do badly you get penalised. Do well in an easy race you get rewarded, but not as much.
The number that emerges is a SOR rating for a yacht in a race.
The Sailor Rating is averaged over the all races that a sailor is involved with for which we have ratings. Over time and with many races a sailor will get closer to their true rating, luck plays a smaller role, but is always there.
This is what makes ocean racing exciting and interesting.
SAILOR.COM.AU OCEAN RATING
A SOR rating is a good starting point for owners and sailors to screen yachts and other sailors.
Sailor.com.au also provides list of races that a sailor has undertaken, this will give a good context on what they have done and who they have sailed with. Sailor.com.au is like a LinkedIn for sailing.
It connects owners with sailors and provides an objective way to work out someone’s past performance in substantial yacht races.
We have started with the Sydney to Hobart yacht race and plan to incorporate other major yacht races.
We work with the race organizers to obtain their race data and also undertake our own research to obtain facts available in the public sphere.
Comments