An open Communication on Muc-Off's Claims on Testing
This article was originally published back in October 2017 as a correspondence to Muc-Off’s claims on testing and 10-watt savings over competitor chain performance.
Following Muc-Off’s recent release of their Nanotube Chain, CeramicSpeed have chosen to provide a detailed response aiming to address Muc-Off’s claims on testing.
Muc-Off recently released a brochure in connection to the launch of their Nanotube Chain which included data regarding the long-term performance of CeramicSpeed UFO Chains.
The following communication addresses the misleading nature of Muc-Off’s data. Based on what we know of Muc-Off’s testing procedure, we assume this misleading data is caused by incorrect methods of acquiring ‘endurance test’ data. We describe the correct test methods, and show the results of a correct test as well the results of a test recreating the misleading data.
In the following paragraph, we will discuss how and why using the wrong test machines can provide misleading data. This is followed by a thorough discussion of how we assume Muc-Off has performed their testing which is supported by a replication of their assumed testing method compared to the correct method. As the results will show, the two different ways of performing the testing, leads to two very different data results.
1.1 How can misleading data be created from an Endurance Test?
Based on the test results Muc-Off presented, we must assume that they performed the ‘endurance tests’ with their tester in “Full Tension” mode. While this mode provides the most precise measurement of friction in a chain, it is designed to be used only for short-term friction measurements. The “Full Tension” mode should never be used for long-term endurance testing. For long term testing, an Endurance Machine should be used to more accurately simulate an actual bike drivetrain. Alternatively, for the purest ‘endurance test’ possible, the chain should be ridden on an actual bike for several hours with pre-ride and post-ride friction measurements taken using the Full Tension Tester.
In the following we will discuss how a Full Tension Tester used for long-term testing will cause misleading data.
1.2 Why should the Full Tension Tester not be used for long-term testing?
When attempting to accurately determine the “Friction vs. Time” graph for endurance testing purposes, both the Full Tension Tester and an Endurance Machine must be used. The Full Tension Tester is used solely for obtaining friction measurements and the Endurance Machine is used for subjecting the chain to long-term run times under load.
The Full Tension Tester measures chain friction by applying high levels of tension symmetrically on two chain spans. Rear derailleur chain spans are not present on a Full Tension Tester. Granted, this type of tester produces the most precise chain friction measurements of any test method. Yet the constant tension loading conditions do not mimic a true bicycle drivetrain, and inaccurate results for long-term chain endurance measurements can occur.
Simply put, the Full Tension Tester was designed for short-term, high-precision friction measurements. To perform proper endurance testing of a chain, the chain should be tested on Full Tension Tester for an initial friction measurement, then swapped and long-term tested on a rig that is set up to represent a bicycle drivetrain, with true drive power at the ring, load on the cog, and a rear derailleur setup, such as an Endurance Machine, or even an actual bicycle. After the chain is run under load for a given amount of time, the chain should be transferred back to the Full Tension Tester to get a subsequent friction measurement.
It is irrelevant to test whether a chain remains fast (retains low friction) for long periods of time on the Full Tension Tester, as no bike chain is ever at symmetrical full tension for several hours. Put differently, the Full Tension Tester does not reflect a real-use scenario.
The ‘Slacking Effect’
The issues with using the Full Tension Tester for long term testing arise from the fact that the chain is always under heavy tension, on both upper and lower spans, and the chain is never allowed to slack. In a true bicycle drivetrain, the chain slacks as it snakes through the rear derailleur. When the chain slacks, the lubricants (solids or liquids), on a microscopic level, are redistributed, and the contact lines of the sliding surfaces are constantly shifting, continuously creating new contact lines of lubricant. This slacking refreshes the chain and allows the lubricant to perform for long durations.
This ‘slacking effect’ was discovered in 2015 by Friction Facts. It was recognized that during longer testing periods on the Full Tension Tester, some lubricants would show abnormal increases in friction. During testing of a chain experiencing this abnormal increase in friction on the Full Tension Tester, the chain friction would instantly drop back down to a normal level, yet slowly creep up again over time, if the load was removed for a few seconds, with the equipment still turning (ie, the chain was slacked), and the load then re-applied. This phenomenon could easily be repeated many times during a single long-term test run, quickly lifting and re-applying the load to slack the chain and the friction levels would return to normal.
This behaviour of rising friction levels over longer periods due to a fully-tensed chain is obviously not conducive to accurate long-term testing. To confirm the Full Tension Tester was not suitable for long term measurements based on the theory that the Full Tension Tester did not allow slacking, endurance testing was performed on multiple chain samples to expose the differences between using only the Full Tension Tester versus the Full Tension/Endurance Machine combination. The Full Tension/Endurance Machine combination produced the repeatable and realistic results expected in this type of a long-term test situation.
After confirmation testing proved this theory, the findings led to the protocol of using 1) a full-fledged replica of a bicycle drivetrain for longevity loading (the Endurance Machine), and 2) using the Full Tension Tester at pre-described intervals for obtaining precision friction measurements. Undoubtedly, the findings proved the Full Tension Tester cannot be used alone for long term endurance tests.
Because of this discovery, Friction Facts developed a new Endurance Test protocol, which incorporated the use of an Endurance Machine in conjunction with the Full Tension Tester for accurate, fair, and repeatable “friction vs. time” endurance testing. In the following paragraphs, we will discuss why it appears that Muc-Off are basing their test results on incorrect testing methods and as a consequence of that, has published misleading data. The discussion is based on assumptions since we were not present during the testing, but these assumptions is supported by a thorough replication of the assumed testing method compared to the correct testing methods as being discussed in the previous paragraphs.
To get the full understanding of how and why we assume Muc-Off has come up with this data, please read the below discussion.
2.1 Why is it assumed that Muc-Off performed the endurance testing with only the Full Tension Tester?
The ‘slacking effect’ abnormality is easily detectable on any “chain friction vs. time” graph. The slacking effect presents itself as a relatively large rate-of-increase in friction over time, with a drastic decrease in friction when the full tension is removed and re-applied, followed by a subsequent increase in friction similar to the initial increase in friction before the chain was slacked.
For this document, Friction Facts performed an endurance test using the incorrect test procedures, effortlessly recreating similar (and incorrect) data to the data that Muc-Off published. A test using the correct procedures was also performed, highlighting the procedural errors, and proving the UFO Chain is a 4-watt chain, not a 14-watt chain. This replication of data can be seen in a later section.
Muc-Off originally produced the “UFO chain friction vs time” graph with two days of data, Day 1 and Day 2, using the same UFO Chain for both days (see Graph 1). Note how the UFO Chain’s friction levels dropped from 14+ watts down to 6 watts when the chain was removed from the Full Tension Tester after Day 1 and re-tested for the subsequent Day 2 test run.
This drastic drop in the UFO Chain friction obviously begs the question; How could a chain get 8 watts faster by sitting overnight? The answer is it didn’t get 8 watts faster. The chain was never a “14-watt chain” to begin with, even though the Full Tension Tester was putting out this data. The Full Tension Tester was producing incorrect and excessively high friction readings due to the long duration test. This is undoubtedly due to the ‘slacking effect”, and is a tell-tale sign that the Full Tension Tester was employed incorrectly for long term testing in this case.
Graph 1: Muc-Off’s original graph showing a UFO Chain decreasing 8 watts overnight, between Day 1 and Day 2 testing.
Muc-Off’s original graph indicates something is potentially flawed with the test due to the instantaneous drop in friction on the same UFO Chain between test runs. When the recent NTC Chain brochure was launched, the “Day 2” data containing the anomaly was removed from the graph (see Graph 2).
Graph 2: The “revised” graph as seen in the NTC Brochure. The Day 2 data was deleted for the publication of the NTC brochure.
CeramicSpeed has no way of knowing if Muc-Off is aware of the consequences of the ‘slacking effect’ phenomena or indeed why the graph was revised.
Regardless of that, an 8-watt immediate decrease in chain friction between test runs, on the same chain, should quickly raise eyebrows in any situation, and Muc-Off ought to have revisited their test.
2.2 Replication of Muc-Off’s Endurance Test and Subsequent Data Collected
CeramicSpeed has carried out an endurance test to illustrate the difference between a correct and incorrect test protocol.
A single CeramicSpeed UFO Racing Chain (KMC version for the sake of testing) was run continuously for a 4-hour endurance test. The first 3 hours of the test were performed properly. The 4th hour was performed incorrectly. The graph below is an actual “print screen” from the tester. It has not been altered or manipulated in any way except for the added text.
For the first three hours, the chain was run under 250W load on the Endurance Machine. At 15-minute intervals, the chain was removed from the Endurance Machine, placed onto the Full Tension Tester for a 1-minute friction measurement, and then placed back on the Endurance Machine. Note the stable, realistic, and expected behaviour of the UFO Chain. The friction levels maintain a mid 4-watt range for the first three hours of the test.
After the 3-hour correct test, the incorrect test procedure was initiated. The chain was removed from the Endurance Machine and placed on the Full Tension Tester. However, the chain was allowed to remain on the Tension Tester for one hour. Note the relatively fast increase in friction levels when the chain is on the Full Tension Tester for this long period of time. This steady increase in friction during the 4th hour is very similar to the UFO Chain friction increase seen in Muc-Off’s graph.
After an hour of the incorrect procedure, the chain was removed from the Full Tension Tester and placed back on the Endurance Machine for 10 minutes, to allow the chain to slack through the rear derailleur spans, and then a final friction reading was taken on the Full Tension Tester. The final friction reading was 4.75W.
This test shows how friction measurements can incorrectly climb when a chain is run long-term on a Full Tension Tester, and how the friction almost immediately drops to original levels when the chain is taken off the tester and allowed to slack. The data from this incorrect test is similar to Muc-Off’s original graph. The chain friction increased significantly during Day 1 under Full Tension, then suddenly dropped due to slacking at the end of Day 1, then the friction increased again during Day 2 under Full Tension Tester.
This test also proves that a UFO Chain maintains sub 5-watt friction level at 250W over 4 hours.
As mentioned earlier, the Full Tension Tester is the only known way to measure chain friction precisely, but must be used for short durations to obtain accurate friction readings.
Both CeramicSpeed’s Friction Facts lab in the US, and their test lab at the Denmark headquarters have matching Full Tension Testers and Endurance Machines. Any and all endurance testing is performed with the proper protocol, by using these two machines together.
Several months ago, CeramicSpeed tested various factory-treated chains for comparison purposes. The graph below shows the results a properly performed, realistic endurance test, using the two machines.
Please note, that the Muc-Off Nanotube Chain wasn’t released at the time this testing was performed. The Muc-Off chain in this graph is therefore not the Nanotube Chain but the Muc-Off NTOC Chain (Nanotube Optimised Chain).
CeramicSpeed has publicly posted the “Endurance Test Protocol” for purposes of transparency and integrity in test methodology.
The protocol can be found online HERE.