Updating Primary Scoring Mechanism - Daily Returns
As of November 25, 2024, we have been using a scoring system which prioritizes a certain style of trader. This system has been successful in identifying top miners who make frequent, balanced trades. We prioritize the trading through our metrics, which are typical trading evaluations, and penalties, which help us prioritize strategies we believe are most repeatable and successful.
While this system has thus far been able to curate a number of useful strategies, it has also limited the breadth of trading styles available on our platform. In addition, we want to make our system more familiar to traditional traders, to ease them in by using the same metrics, the same annualized values, and a straightforward, simple penalty system.
Problem Statement
We look to remedy the following issues which currently exist on PTN’s scoring system:
- Penalties are challenging to navigate and interpret.
- We do not annualize values, making comparison between miners inconsistent.
- We do not impose a statistical confidence threshold or method to compete on statistical confidence.
Proposed Adjustments
We propose the following changes, which should improve interpretability on our platform and address the issues outlined above:
Position-based Returns to Daily Value Change
By measuring daily value change, we can normalize the scoring system to be more in line with traditional prop trading systems. This will reduce the complexity for new traders and permit compatible comparisons between strategies with larger and shorter trading windows. If a buy/hold trader doesn’t effectively manage their day-to-day longitudinal risk, this will be measured as part of their performance.
All metrics will now use these daily returns, which will be calculated solely for days with a full 24-hour observation period. If we do not have enough information to witness the entire day of trading, we will not calculate the daily return for that day. As such, the first day of trading will typically not have a daily return used in calculations.
Annualization of Metrics
Annualizing metrics will allow us to compare miners on a more consistent basis. This will also allow us to compare our metrics to traditional trading metrics, which are typically annualized. Sharpe ratio, return, and sortino ratio will all be annualized.
Normalization of Metrics
We will normalize each metric using the annualized risk-free rate of T-bills. This will allow us to compare our metrics to traditional trading metrics, which are typically normalized using the risk-free rate. This will also allow us to compare miners on a more consistent basis. We will start by using a static value for the risk-free rate, as a bridge while we build out an integration to fetch real-time daily rates.
Introduction of Statistical Confidence Metric
We will introduce a statistical confidence metric based on a t-statistic for miners to compete on. This will measure the similarity between a miner’s daily returns and that of a normal distribution with zero mean. The greater the discrepancy in the means, the higher the confidence that we have a statistically significant strategy that differs from a random distribution. The confidence of this metric will also be increased by the number of days of trading, as more days of trading will increase the confidence that the strategy is repeatable.
Metrics
The following section outlines in detail the metrics we will use to evaluate miners on our platform.
Risk-adjusted Return
This will operate in a similar manner, by comparing the average daily return to the average daily max drawdown. Please note that the max drawdown will operate as before, such that the drawdown will be a cumulative metric describing your overall portfolio value relative to the highest value it has reached. There will be a short-term return metric and a long-term return metric. Short term return will simply look at the prior 7 days of activity. The formula for the risk-adjusted return will be as follows:
Sharpe Ratio
The sharpe ratio will use the same logic as before, but will now be annualized and corrected for population sample sizes instead of using the population standard deviation. The following formula will be used to calculate the sharpe ratio:
Omega Ratio
The omega ratio is the ratio between the sum of positive daily returns and negative daily returns.
Sortino Ratio
The sortino ratio is the ratio of annualized excess returns to the annualized volatility of negative returns:
T-Statistic Confidence
The t-statistic provides an indicator of the statistical confidence that the daily distribution of returns is unique from a normal distribution with zero mean. The formula for the t-statistic is as follows:
Updated Penalties
Our penalties generally add a layer of complexity to our system in an attempt to curate a specific type of trader. However, we find in practice that the barrier they create is an added layer of complexity and confusion for new traders. In addition, we risk alienating traders who may have a successful strategy but are penalized for not fitting our mold. We will remove all penalties except for the max drawdown stipulation, the returns ratio penalty, and a new abnormality measure, as we still feel these are necessary to maintain some level of risk management with trading decision-making.
Max Drawdown Penalty
If the max drawdown seen over the 90 day interval is greater than 10%, the miner meets elimination criteria.
Abnormality Penalty
The abnormality penalty will be a new penalty that captures some of the benefits from the prior risk-adjusted return factor into a new penalty. This will be calculated as the current difference between the current max drawdown and the historically typical max drawdown. If a miner is currently in uncharted territory with their max drawdown, we will witness their behavior as abnormal and penalize them accordingly.
Integration Details
This update will mechanically require no modification of behavior from the miners, as all values will be passively captured and measured by the system. Our dashboard will be updated to reflect these changes, and to communicate the new scoring system to miners. We will also update our documentation to reflect these changes.
Timeline
Proposal Delivery: Nov 25, 2024
PR: Nov 27, 2024
Integration: Dec 3-6, 2024