Skip to content
Taoshi Documentation

Challenge Period

Incentive on the system currently looks at the ratio of max drawdown to returns by each miner, evaluating the quality of their trades relative to the maximum risk they absorbed. The current rules of the challenge period continue to use risk adjusted metrics like sortino, meaning that even if a miner is rewarded by passing the challenge period they will be evaluated against a new set of criteria for the main competition.

Current Benefits of Challenge:

  1. Challenge period provides miners with an opportunity to transition into the main competition, provided with grace period value until they pass / fail challenge
  2. We need to ensure that the signals procured by top miners have evidence of repeatability. Challenge period requires a level of sustained interaction with the system.

Problem Statement

Current Issues with Challenge

  1. Challenge period is simply too hard, we need to better regulate the thresholds to allow miners to competitively enter the system.
  2. While the challenge period requires sustained interaction, the signals produced by top miners in the system will be directly translated into trading infrastructure. We need to ensure that the signals produced by these top miners show evidence of reproducibility and long term value.
  3. Challenge period should set the miners up for success after they pass challenge, not immediately penalize them with an incongruent system.

Proposed Phases / Features

We will still look to transition the challenge period away entirely after we introduce more accurate fees and cost of transactions. Until then, we need to make changes to allow miners to successfully pass through.

We propose the following:

  1. We look at the relationship between sortino and max drawdown among all miners on our network, and try to determine the equivalent sortino value which would correspond to a roughly 2% MDD. Almost all successful miners with MDD below 2% MDD have a sortino of less than 1.1e-8, although one successful miner has a sortino between 2e-8 and 3e-8. We propose to set the sortino limit to 3e-8, making it substantially easier to clear.
  2. We then look at the ratios of returns to mdd for our top miners. We use the ratio of the miner around the 20th place as the criteria, adjusted for a 2% MDD rate. The ratios of the top miners in our system are above 2.5, meaning they have 2.5% returns for each 1% MDD. Miners around the 20th place have a ratio of 1.0, meaning they have at least 1% return per 1% MDD. If we use this ratio for our 2% MDD target, then we would want to see a return of 2% from the miners.
  3. Omega will be removed entirely from the challenge criteria, as it plays a small role in the overall score.

A rocketship in space

Note that this will be a forward looking update, meaning it will only impact the current state of miners and the state of miners moving forward. In summary, we propose the following modifications to challenge to make it more reasonable for miners to enter the system:

  • Increase of sortino from 1.5e-8 maximum to 3.0e-8 maximum (easier)
  • Reduction of returns from 3.25% minimum to 2% minimum (easier)

We also encourage our miners to be mindful of their max drawdown, as it will be used for your scoring if you transition into the main competition.

Timeline

Proposal Delivery: June 7, 2024

PR: Jun 7, 2024

Integration: Jun 10, 2024

Pull Request