Lookback Extension
By using the long term returns from miners, we are betting that miners who have historically been able to yield returns have a competitive model on market movement. This assumption becomes more powerful when the miner has demonstrated a greater level of historical performance. We currently use a 30 day lookback period with risk adjusted returns to evaluate the quality of a miner, with a hard limit on 5% max drawdown before the miner receives no incentive at all. This has served us well as a preliminary metric for evaluating miner risk, but may be too rigid as we increase the lookback period.
Problem Statement
With a longer lookback window, miners have a higher probability of dropping into deeper drawdowns. While this is undesirable, what may be more important to consider is the duration of risk exposure and the frequency with which the miners drop into these deeper drawdowns. Max drawdown does not capture this nuance, and is perhaps too prescriptive of a threshold for risk normalization.
Proposed Changes
There are three major changes that will come with an increased lookback window:
- Scoring
- Effective Max Drawdown
- Consistency
Scoring
We will continue to evaluate the miner’s performance based on their realized returns, but we will increase the lookback window to 90 days from 30 days. In addition, to more appropriately benefit sustainable per position returns, we will incorporate the Sharpe ratio and Omega ratio into the scoring mechanism. The scoring mechanism will still be dominated by long term realized returns, with short term realized returns and sharpe ratio likely playing more minor roles.
| Metric | Scoring Weight |
|---|---|
| Long Term Realized Returns | 100% |
| Short Term Realized Returns | 25% |
| Sharpe Ratio | 5%-35% |
| Omega Ratio | 5%-35% |
Sharpe Ratio
Sharpe ratio will be defined as the expected value of the miner’s fee adjusted returns, divided by the standard deviation. We don’t expect any miners to have a perfect strategy with zero variance. To avoid gaming on the bottom, we are going to have a minimum standard deviation.
Omega Ratio
The omega ratio is the miner’s wins over the losses. This gives us an estimation of the risk to reward ratio taken by each position. We will use the same logic as the Sharpe ratio to set a lower threshold on the loss, if the miner doesn’t take at least this base loss then they will get the default value.
Effective Max Drawdown
There will be two derived values used in conjunction to determine the overall penalty: Effective Max Drawdown and Max Drawdown.
In our existing system, we use the max drawdown seen on the portfolio to normalize the miner’s returns. One of the primary benefits of this is that we are able to prune away miners as their entire portfolio spirals into uncontrolled drawdown. We would like to keep this feature as we transition to per position drawdown, as an awareness of the overall portfolio drawdown is useful. Here is the current definition of the penalty curve associated with the current max drawdown:
The first term, Effective Max Drawdown, will use similar logic as the Max Drawdown and will be defined against the overall portfolio. Max Drawdown will have the same definition as before, but instead of using the entire lookback period, we will only look at the prior week. Effective Max Drawdown will determine the 90th percentile of the miner’s drawdowns using the entire duration of the lookback window. We will always use the largest of the Effective Max Drawdown and Recent Max Drawdown to determine the miner’s penalty.. This will allow us to capture the miner’s overall risk exposure, while also protecting the system from runaway drawdown.
The existing drawdown window is too prescriptive, with many successful miners regularly reaching the limits on the boundaries. The following graphic shows the current penalty curve and the effective additional penalties imposed on excess drawdown values:
We will expand the drawdown window from a 5% max to 10% max to allow miners more flexibility in their trading strategies. The upper boundary penalty will now linearly taper from 5% effective drawdown to 10% effective drawdown instead of 3.5% to 5%. Additionally, we propose an increase to the base drawdown from 0.2% to 0.5%, which is both more in line with the max drawdown and more appropriate for pairing with Sharpe ratio as a scoring consideration:
Consistency
Consistency metrics will be updated to reflect this larger lookback window and simplify behavior. In line with proposal 8, we will use ratios of historical returns to determine the miner’s consistency of growth. We will look to implement the following penalties:
- Max Positional Return: A single position should not represent more than 15% of total realized return.
- Realized Return Distribution: No more than 30% of the miner’s realized returns should be from positions all closed in a single week.
- Max Portfolio Value Change - Daily: A single day of trading should not represent more than 20% of the total unrealized return.
- Max Portfolio Value Change - Biweekly: A single two-week period should not account for more than 35% of total unrealized return.
Each of these penalties will be applied using a sigmoid curve, with minimal penalty if the miners hit these numbers.
Some logic to explain the choice for each of these percentages:
- We want to see sustained success from the miner, over a variety of conditions as a way to pressure test their strategy prior to eligibility for top miner status. Having no one position represent more than 15% of the total return ensures that the miner has at least 7 positions of similar size to be considered for top miner status. More likely, the miner will have at least 15 positions over 90 days of roughly the same size to effectively reduce their risk exposure. This is equivalent to at least one good run per week over 90 days.
- Realized Return Distribution: This is meant to ensure that the miner is maintaining an exit strategy for all of their returns by reliably closing them over time.
- Max Portfolio Value Change: Daily - If a miner doesn’t have at least five equally good days of trading from the prior 90 days, they are too inconsistent to be a strong consideration for the top position. Additionally, this protects from the scenario where a miner might open a number of trades on a single day, ride the same market wave with all of them, and mask their strategy as consistent by closing them over time.
- Max Portfolio Value Change: Biweekly - This is meant to capture the miner’s ability to sustain growth over a longer period of time. We want the miner to demonstrate their capacity to continue trading well in a variety of market conditions.
Timeline
Proposal Delivery: August 8, 2024
PR: August 12, 2024
Implementation: August 23, 2024