Robert Whittaker aka Bobby Knuckles aka The Biracial Angel, my favorite active UFC fighter returns this weekend, and it is as good of a reason as any to launch the upgrades to all 3 existing WolfTicketsAI models! This blog explains the reason for the changes, what’s new, and how to use their insights effectively.

Why Create New Models?

Simply put, the library that is used to create the predictions hit their 1.0 release and with that, a ton of new features for making better models became available, however the old models were not compatible with newer library. Also it was a chance to leverage more things I learned about the UFC, the data, and ML to make hopefully better models all around. Lastly, I wanted to bring back explainability charts for each prediction, these help me a lot, and with a few tweaks to my process, they are back!

How Are The New Models Different?

The new models mirror the 3 previous models in terms of the data each has available, and the goals for training each of them, none of that changed. With one exception, the Plain model, no longer takes into account a fight’s weight class(this was one of the main breaking points for the charts previously).

AutoGluon has released a lot of improvements for training models, this allowed for more diverse experiments, and faster exploration using a GPU as well. In short, the models should be more resilient but behave pretty consistently in terms of prediction accuracy.

Most importantly, all the data was captured during training to unlock the ability to provide SHAP forces plots aka explainability charts again.

Explainability Charts… Explained

Explaining the predictions of a machine learning model, especially a complex model is a highly active topic in academic research still, there are many methods, all have pros and cons, but the idea is to give some form of guidance as to how the model made its prediction.

The approach used by me is to create what is called a SHAP forces plot, a great talk of them is here . The short explanation though is that you sample a number of predictions that were made by the model to get a base number, think of it as an average prediction, then you look at a specific prediction and see how much each feature(statistic or attribute) impacted that model.

To better understand this system, take a look at the WTAI model’s prediction of Volkanovski vs Topuria:

In all of our models a prediction above .50 is scored for the red corner(ignore the line that says base value around .449, I am working to figure out how to render it better… also how to fix the white background, and how to make it more readable.).

In this chart however we see that a few stats are driving the prediction towards the red corner(Volkanovski) and a few are driving it towards the blue corner(Topuria).

Red Features:

• Striking impact differential (The difference between a fighter’s strikes landed statistic at the time of their fight and their opponent, over how many fights a fighter has had.)
• Reach: 71” vs 69”
• Odds: -110 vs -107

Blue Features:

• Recent win percentage: 33% vs 100%

Interestingly if we look at their history we see that Volk has 2 losses to Islam, who is arguably one of the greatest fighters in the UFC’s roster. While Islam is not undefeated like Khabib, his talent is unquestioned. Volk is the only fighter to take him to a decision in 5 years, though he did take a short notice fight for their rematch and was subsequently KO’ed. We do see here that it has taken the absolute best to beat Volk.

This leaves us with two specific questions for this fight:

1. Has Volk spent enough time after his KO loss( October 21, 2023) to fully recover and prepare for this fight?
2. Is Ilia Topuria good enough to best Volk?

Bonus unanswered question, can Volk overcome greatest threat in all of combat sports, Father Time?

A good breakdown on this: MMA On Point Digs Into The Specific Age of Decline

These charts are available now on the prediction details page for each upcoming prediction and can be found on the prediction like here

Changes In The Models

Specifically the following things were done differently for these models:

1. Each model is technically an ensemble of many smaller models, several new algorithms were enabled as options for the choices in the ensemble.
2. A GPU was used for all training jobs to enable more work in the same training time(30 minutes max per model).
3. Weight class was removed from the Plain model, it had a negative impact on accuracy anyway.

The specific features that go into each model did not change, the training and test data also remained the same.

With that it is now possible to compare the performance of each vs its older counterpart, the good news is that overall accuracy stayed nearly the same, just the scores for each model shifted in range.

WTAI Model

Note: The new model charts are against 2023 predictions and the old model charts are against 2022, both years had similar numbers of fights with similar breakdowns so they should be seen as close.

The new model looks like this:

Vs the old Model:

The newer model shows a much healthier state where the model starts to get significantly more accurate as the confidence scores rise, and we see a nice balance of the number of predictions at each, rather than the previous peak we had for the WTAI model around a score of 25. This tells you that anything beyond a 20 is pretty solid, but lets you see the nearly even performance in the lower confidence intervals.

Accuracy of New Model: 67.31% for 2023 Accuracy of Old Model: 67.26% for 2022

Profit Model

The new model looks like this:

And the old model looks like this:

The newer model seems to struggle with the lower confidence intervals, and it does not rank things beyond 25 in a score, but the accuracy really sharpens up past 10 and yields plenty of area for accurate performances.

Accuracy of New Model: 65.02% for 2023 Accuracy of Old Model: 64.81% for 2022

Plain Model

Lastly the new plain model looks like this:

The old looks like:

Overall we see the least change here in the shape of their responses, though the peak score is again lower in the new model. Overall an OK model but the problem is that it does not increase in performance with its confidence unlike the other models.

Accuracy of New Model: 58.9% for 2023 Accuracy of Old Model: 59.78% for 2022

Insights

With the performance being similar, and even seeing a tiny gain for WTAI, continue with any existing betting strategies you were using!

The biggest impact should now come from better leveraging the writeups and alerts to potential gaps in model knowledge that come from each fight, allowing you to spot areas quickly where the model may be overconfident.

The strategy of parlays that allow for a number of predictions to fail, also seems to work well, yielding a +10% return on a night of only 60% prediction accuracy in the last UFC Fight Night, so stay tuned to the official emails for those.

Good luck and I’m stoked for the fight this weekend and for the future of WolfTickets.AI!

-Chris King