Dynamic Challenge Period
The current challenge period uses some of the metrics of the main competition and sets up miners to have low penalties once they pass. We designed a challenge period to prepare miners for the main competition and provide confirmation that miners have a competitive strategy that is consistently profitable with reasonable risk. Most recently in proposal 12, the challenge period has undergone improvements and continued development to better align with these goals.
Problem Statement
Further improvements can be made in the following areas:
- There are differences between the scoring in the main competition and scoring in the challenge period.
- Further development on the incentive mechanism or challenge period can result in disharmony since they are similar but ultimately different systems. An update to one system may not be directly reflected in the other.
- Challenge period should be effectively measuring the performance and risk of miners to ensure their success as they transition into the main competition.
- We should be clearly communicating to challenge period miners with clear guidelines as to how they might perform in the main competition should they pass the challenge period.
Proposed Changes
Although the current challenge period ensures that miners are performing well in key areas relevant to the main competition before passing, it requires continued maintenance to address the problems listed above. In direct response to these issues, we propose to replace the existing system with a dynamic challenge period that adjusts for changes in the incentive mechanism and the overall competitive state of the main competition.
Challenge period will use the exact same metrics and penalties as the main competition and score challenge period miners relative to miners that have already passed. Any modifications or improvements to the incentive mechanism will automatically propagate to the challenge period. For reference, the major proposed changes are as follows:
- Miners with an overall score at or above the 75th percentile relative to the main competition will pass the challenge period.
- Elimination criteria will be the same as the main competition.
Scoring Process
To understand the challenge period calculations, we will go through the steps of the challenge period scoring process with a toy example. We will be using a simplified scoring mechanism that only uses a sharpe ratio, an omega ratio and no penalties. For every miner in challenge period, we will compute their scores relative to the miners in the main competition:
We calculate individual metric scores and percentiles for all main competition miners and the testing miner being evaluated. These metric percentiles are then aggregated into final scores, which are also converted into percentiles. This process is repeated for each challenge period miner, ensuring their evaluation is solely against the main competition rather than other testing miners.
Challenge period miners in the 75th percentile or higher will be admitted into the main competition. Requiring 75th percentile or higher ensures that challenge period miners have a competitive strategy that will likely perform well in the main competition. As of November 22, 2024, this miner at the 75th percentile would capture about 1% of emissions on our network while the top miner receives about 7% of emissions. This clarity and transparency on the performance of miners in the challenge period should make the transition to the main competition more seamless with fewer unknowns. Setting the threshold at the 75th percentile also provides confidence that miners have a robust strategy that will remain valuable in the overall network even in the event of potential fluctuations in performance.
Information for Challenge Period Miners
The overall percentile score of miners will be a key indicator of performance in the network. In addition to this percentile, miners will receive detailed performance insights, including percentile scores for each metric, offering a clear picture of how they would likely fare in the main competition.
Percentiles for the individual metrics will allow miners to view the strengths and weaknesses of their strategies. For example, miners will be able to view their percentile for their sharpe ratio and their percentile for their omega ratio. If they are in the 60th percentile for their omega ratio but the 20th percentile for their sharpe ratio, they may want to focus on increasing their sharpe ratio. Although statistics on miner performance are already calculated relative to all miners (those in the main competition and challenge period), providing percentile scores for each challenge period miner relative only to miners in the main competition provides miners with a more accurate indication of their performance should they pass.
Elimination Criteria
Miners in the challenge period will be subject to the same elimination criteria as those in the main competition, namely elimination for a maximum drawdown of 10% or greater. This ensures consistent risk management across both phases. Drawdowns will be monitored frequently to promptly identify any miners exceeding this threshold.
Transitioning Process For Rollout
To facilitate a smooth transition to the new scoring system, miners in the challenge period will have two choices:
- Continue the challenge period under the new scoring system, retaining performance history and remaining time.
- Restart the challenge period with a clean slate, resetting trading data and starting with the full challenge period duration.
We will release more details on this process closer to launch.
Timeline
Proposal Delivery: Nov 23, 2024
PR: Nov 25, 2024
Integration: Nov 27, 2024