The overwatch system in CS:GO shows gameplay clips to players, and lets them determine whether
or not the suspect is cheating, or griefing in some way. Over time, this helps train the
system to get better at detecting potential cheaters, which can then present players with
cases which are more likely to show cheaters, in a never-ending, continually-improving feedback loop.
But one player found a way to automate the process. 2EGGS built his own AI tool to
watch overwatch cases and to decide whether or not they were guilty! Named Hestianet,
it had a high success rate catching
15,000 cheaters, helping in his goal to rid the game of as many cheaters as possible.
Then he too got a VAC ban.
…but did his system actually help the game?
The overwatch system has had troubles. How many, it’s difficult to tell. Sometimes, it doesn’t even
seem like it’s working at all- likely by design! But I think it’s safe to say that it will have
faced heavy sabotage from cheat-makers, and from other people trying to bring the system down and
to break it in one way or another. If you want a good anti-cheat,
the most effective thing you can do is to not reveal how the anti-cheat works. And that must
be infuriating for Valve employees, who see their beloved overwatch system being attacked
by cheaters on one side, and their efforts to keep it operating criticised by everybody else.
Yet they must grit their teeth, and keep SCHTUM. For that is how they keep the whole thing safe.
It’s one of the reasons why Valve doesn’t want people to get their hands on CS:GO’s source code,
and anything else that would let them get a better look at the game’s inner workings- which would
give them a better idea of how to exploit it! So it’s understandable why we don’t know specifics
about overwatch- in fact, I was surprised they revealed as much about it as they did!
Say you’re a cheat maker. Say you want to bypass the overwatch system. One strategy
you’ve probably heard cheaters using is to crash the server, presumably to stop it from
ever being able to send a demo of that match over to the overwatch system. Others have spoken about
broken or corrupted demos where it just… doesn’t look right, or where it immediately fast-forwards
all the way to the end. I have no inside knowledge about such measures, but I strongly suspect that
these are things that cheaters have used to attempt to hide from the overwatch system,
and Valve are probably aware of this and have employed countermeasures of some sort.
When they introduced that demo skipping feature to instantly jump to each round,
it was great for people watching demos back! But I also wonder if it was done to split
demos up into smaller segments for overwatch purposes, potentially making downloads smaller
for people doing the cases, or simply to make the recordings less susceptible to corruption.
So yes. One way you can try to bypass overwatch is to sabotage the demos. But the another,
and more insidious you could damage it, is to break the overwatch system itself.
You see, there’s more to overwatch than just the human element- there’s also VACNET,
which is Valve’s own AI-driven program, kind of like Hestianet, but BIGGER. And
this system scans through every match of CS:GO in search of stuff that isn’t right,
and can automatically submit suspicious cases to overwatch itself! And this program is being
trained on overwatch, meaning that users’ verdicts will -over time- shape how VACNET behaves,
and which sort of things that, in the future, it will detect as being rather sus. Clearly,
VACNET is the thing to sabotage if cheat makers want to break the overwatch system!
And you’d do this by training it on incorrect data. Say, for instance,
somebody reports non-cheaters as cheaters, and the cheaters as non-cheaters. Overwatch
has measures to address this: it gives your profile a rating based on how accurate your
past scores were. But imagine MILLIONS of people started reporting the wrong things. So many that
those who were actually reporting the cheaters are suddenly the minority! They’ll become the
ones that overwatch would consider to be wrong. Now I’m SURE the overwatch system has faced this
kind of attack. And I’m sure that Valve has measures in place to combat this. Likely by
getting people to review cases of known cheaters and already-VAC-banned players, since then the
system already knows whether or not that player is cheating… and it would use these demos simply
to ensure that whoever was doing that overwatch case was reaching the right verdict about it.
Let’s go back to 2egg. Let’s assume for now that his automated overwatch system had the best of
intentions… it may still have unintentionally been part of the problem. The very reason that
real people are needed in the overwatch system is to TRAIN the AI to get better at detecting
cheaters! If you try training the AI with AI, it becomes an uncontrolled feedback loop with
no way of learning and detecting new ways of cheating! A few years ago we were informed that
we should see fewer obvious spinbot cases in the overwatch system- which signals to me that
Valve’s own Overwatch AI systems have reached a level where they can quite consistently detect
and ban spinbot cases automatically. Which frees humans up for the more difficult cases instead.
But look at how many years that took,
before Valve themselves were confident enough to make their own systems to ban
stuff as obvious as SPINBOTTERS! That’s not a weakness of Valve’s- merely caution,
built upon staggering amounts of data and feedback. Something that a third party program
like Hestianet will never be able to match. An AI system like VACNET cannot stand still,
and it will have to continually adapt to spot new cheats, and to overcome new
sabotage attempts. And for that reason, it will always need a human helping hand, just to keep it
on the right track and so trying to teach it with another AI system is missing the point.
So it could be that one of Overwatch’s biggest problems IS from well-intentioned players,
building their own automated cheat-detection systems. Even if Hestianet currently has a high
success rate at spotting cheats, any discrepancies in its behaviour will become greater over time.
And so if Hestianet is operated by enough people, it could threaten to destroy VACNET
in a never-ending, AI-driven feedback loop! Anti-cheat is a game of cat and mouse. One
side cannot beat the other- it is a constant back and forth, with each side responding to
what the other side has just done. In the words of Jim Gordon: escalation.
We start carrying semi-automatics, they buy automatics. We start wearing Kevlar, they buy
aimbots. Or in this case…. AI-systems training the AI systems training the AI systems that Valve is
using to try and detect cheaters. Valve, pls fix.