CSR Elite wheel benchmarked

Remember when I sad I will have something very special regarding Fanatec CSR Elite wheel for you, but then did not post it on Saturday, when embargo was lifted? I wanted to take this wheel review to the next level and thought this would be the way to accomplish that. We never see any real benchmarks in wheel reviews, so why not change that practice? Numbers do not lie.

This is not the full review, I am only posting benchmark results for now. The reason these benchmarks were postponed is because I decided to wait for a firmware version that would behave similarly to the firmware that will be available, when the first customers receive their wheels and I was told the one we have now (rev. 717) is indeed very close. I think the results are quite interesting and will not only inform you about the new wheel, but also help you understand how Fanatec's tuning options work and why changing force feedback strength in the game is different from changing it on the wheel.

Testing procedure

The only tool I used for testing the CSR Elite wheel is a program called "iRacing Force Feedback Test 1.53", which is a program created by David Tucker, a member of iRacing development team. The program does not really have anything to do with the iRacing simulation and was designed to quantitatively measure different wheels to see how they perform. This way they can understand different wheels a bit better and ultimately make them behave better with their simulation. Every iRacing member can download it in the thread David started on iRacing forums, test his wheel and compare it to a ton of different wheels that have already been tested and results stored in David's wheel database. For this article I decided to use two different tests - one to try to measure power (Test 1) and one to measure linearity of force feedback response (test 2). I will mainly concentrate on performance of CSR Elite, but feel free to discuss all the other wheel's results in the comments.

Because I do not have enough different wheels around me to make this benchmark interesting, I asked David if he would let me use his results on my website and he very kindly allowed me to use both his software and everything from his database. This means that I did not perform all wheel tests and can not give you precise information about the driver and firmware versions that were used. Thus benchmarks below may not be 100% representative of the current state, because I can not guarantee they were all conducted with the latest drivers and firmware versions. What I can tell you is that CSR Elite wheel was running firmware version 717 and GT2 wheel was running firmware version 681. I used the latest official drivers by Fanatec, version 115. Because these tests were performed on a PC, I can not guarantee they are 100% valid for comparison of  performance, when connected to consoles.

Test 1.1: Measuring force feedback strength

The first test I ran is designed to give an idea about wheels force feedback strength. It first centers the wheel and after that gives it FULL LEFT, FULL RIGHT and again FULL LEFT commands with 300 ms of delay. The program tracks wheel's position and by looking at how far wheel moves during each 300 ms period, we get a rough idea about its strength and speed. Correlation between response in this test and actual wheel strength is not perfect, because not all wheels weigh the same and don't have same internal friction (Frex SimWHEEL has built in hardware damping, so its test results should not be compared to other wheels in this tests). Because measurements are recorded each millisecond of the test, it is also very easy to figure out the frequency at which wheel is refreshing its position and see how much delay there is between the time when wheel receives a force feedback command to move and the time when first movement is detected by the PC.

Chart 1: Movement during full left, full right, full left force feedback commands with 300ms delay (click for larger image)

As you can see, CSR Elite wheel outperforms every other wheel in in this test, even Thrustmaster's T500 RS, that was until now much faster than all the other mass production force feedback wheels. It accelerates very fast at the beginning of the test and only increases its "lead" when it comes to changing direction, reaching much higher rotation speeds in this short test. I am not sure which way the results would go, if delay between opposing force feedback commands was higher than 300 ms (such test is not available in the program), but at this conditions CSR Elite is a clear winner. You can also see how little internal friction this wheel has, when it continues to rotate at almost constant speed after force feedback turns off 1200 ms into the test. Same applies for Logitech's gear driven G25 and G27 wheels. When it comes to update rate, Logitechs win the test, updating their position at 500Hz. Thrustmaster T500 RS updates its position 120 times a second, Fanatec wheels communicate with the PC at 100Hz and Frex SimWHEEL communicates at 50Hz. High update rate alone does not mean much, because to my knowledge physics engines in games do not reach 500Hz, but in the end Logitech wheels have the shortest delay between when they receive a FFB command and when the first movement is recorded by the sensor, generally reacting about 10ms quicker than Fanatec and Thrustmaster wheels. Frex responds more slowly, but due to lack of tests I would rather not make any conclusions.
It needs to be pointed out, that CSR Elite's sensor is not mounted directly to the motor like on other wheels, so in real world delay between when you move the wheel with your hand and the time when PC registers movement may be shorter than these tests show, but I do not have the tools necessary to measure that. If sensor is mounted directly to the motor and motor starts to move because it received a force feedback command, first movement in these tests may be recorded even before the rim actually moves due to slack in force feedback transmission mechanism.

EDIT on 15.12.2011:  After further observation, I discovered that with firmware 717 CSR Elite actually started communicating at 500Hz, as I PC receives new wheel position readout every 2ms. I am terribly sorry for the mistake I made. I ran a ton of tests with different beta firmware versions and unfortunately looked at the wrong data when researching update rates. This higher refresh rate also looks to have decreased wheel's latency by about 5ms.

Test 1.2: What about spring on CSR Elite?

I was interested in how turning on the spring effect would affect the test results you saw above. I turned it on in wheel's drivers and after that controlled spring strength directly through on-the-wheel menu. I managed to test response at spring settings from -3 to +2, but failed to test it at +3 because of a bug in the beta firmware I was using.

Chart 2: Movement during full left, full right, full left force feedback commands with 300ms delay with different spring settings (click for larger image)

Test results show exactly what you would expect, spring effect works exactly as it should and you can see how spring effect counters force feedback command, making it weaker. Another thing you can notice here is damped, high-frequency oscillation that occurs at the end of the test, when spring effect is set to a high setting. That does not feel very natural, so lower spring values are recommended, at least until there is no damper support in the drivers (I was told it is coming soon). Also spring effect gets very strong, when set to +2, so you would not want to use such high spring setting in normal circumstances. Most sim racers play with spring effect completely disabled in the PC drivers and personally I like to keep it that way too, as PC simulations do a very good job with FFB, without having the spring effect enabled.

Test 1.3: What about drift mode?

I also decided to run the same test with different "drift mode" settings. Just like spring effect strength, drift mode is controlled via the on-wheel tuning menu. It got added to Fanatec's Porsche line of wheels for faster wheel movement while drifting. What exactly does it do? Is it necessary? Let's look at the chart.

Chart 3: Movement during full left, full right, full left force feedback commands with 300ms delay with different drift mode settings (click for larger image)

Not much of a difference you say, huh? I think the part that explains how exactly wheel behaves actually comes after the test is over. If you watch what happens after 1200ms time, you will see that wheel was carrying almost exactly the same speed at all drift mode settings, including "off", but behaved quite differently from there on, especially after it "bumped" off the 10000 mark. That suggests that in drift mode force feedback motor "helps" to keep the wheel moving in the direction it is moved, reducing drag. It did the job very well in the Porsche line of Fanatec wheels, which were heavier to move, but I think most of you will not need it with the CSR Elite wheel, simply because it is very light to move and counter-steers very quickly, it is always in "drift mode".

Test 2.1: Linearity benchmark

Another quality measure that can be tested with David's program is wheel's linearity. We want to know, how linearly wheel responds to variously strong FFB inputs from the game/software (in this particular test we measure how much it moves in 300ms). A perfect wheel should be completely linear. For example: if it gets a command to deliver a force feedback effect at 40% strength, it should actually deliver the effect at 40% of it's maximum strength. Same applies for 10%, 20%, 30%, 50% and so on.
The second problem is that usually we get two ways to control the strength of force feedback - one in the game's controller menu and one in wheel's driver or on the wheel. In an ideal scenario both should work exactly the same and the end result should be a product of both settings. Let us look at what the test results tell us. I decided to run two test  for each wheel, one at 100% FFB strength setting in the driver/(on the wheel) and one at 60% FFB strength setting in the driver/(on the wheel). (I used 100% and 50% for G25, because I did not have test results for 60% FBB strength).

Chart 4: Linearity testing - wheel's response to variously strong FFB effects at two different strength settings in the driver (click for larger image)

The first thing I noticed is how linear CSR Elite wheel is at 100% FFB strength set on the wheel. It almost a perfect example of linearity. G25 and GT2 wheels are also quite linear, while the T500 RS is a bit less so, but not too bad.
There is one problem though - and it affects both Fanatecs. See how their result changes, when we set FFB strength on the wheel to 60%? Instead of wheel multiplying all FFB effects by 0.6 and getting weaker, but still linear response, it does something very different. All effects below 60% strength are exactly the same as before, but after that the line is flat. Instead of adjusting all effects, wheel simply limits the maximum force it will output. Is that true? Let's investigate this deeper in the next test.

Test 2.2: Investigating CSR Elite's FFB strength function

To find out if I am right about what is going on, I decided to conduct a series of tests at various FFB strength settings in wheel's tuning menu. Here are the results.

Chart 5: Linearity testing - wheel's response to variously strong FFB effects at various strength settings on the wheel (click for larger image)

As you can see, results confirm my suspicion. FFB strength function on CSR Elite does not work the same way as force feedback strength setting in drivers of other wheel does. It only limits the maximum force wheel will output, without accordingly decreasing the strength of weaker effects. It basically acts as a cut-off filter and ruins that beautiful, class leading linearity that you get from CSR Elite at 100% FFB setting. If you do not completely understand this, please have a look at the "perfect world test results", that show how the chart above should look like. Is this a deal breaker?
While the wheel does not behave how you would expect it to, that in my opinion is not a big drawback, but you should be aware of how this works. You should know, that decreasing FFB strength on the wheel will only limit the maximum force your wheel will output and that if you want perfectly linear force feedback response at a lower strength setting, you should decrease FFB strength in the game instead of changing it on the wheel. OK? OK ;)

Does FFB strength function on other Fanatec wheels work differently?

After finding how FFB strength on CSR Elite works, I decided to test my good old GT2 wheel out and surprisingly, got very similar results. It looks like this is how FFB strength function works on all Fanatecs, so keep in mind: Lowering FFB strength on the wheel under 100% only cuts off forces stronger than the percentage you set. If you want to keep linear force feedback response, decrease FFB strength in the game instead of decreasing it on the wheel.

Conclusion

I think that test results show quite clearly that CSR Elite is the wheel to beat at the moment. It is exceptionally strong, fast and has almost perfectly linear force feedback response (as long as you keep it at 100% FFB). More about its design, quality, real world performance, etc. in the final review. I hope you enjoyed the reading and please comment. I am open to any critics you may have, as this is something new for me and I want to be corrected, if you think I made any mistakes. Thank you for reading!

Special thanks to David Tucker from iRacing for allowing me to use both his program and his data to make this article.

Sean Cebron - aka sonac