Rensch vs Regan

"Rensch said his platform had developed an industry-leading anti-cheating model trained on a staggering trove of real-world game data from games played on its platform. “What we did that really is different than any others do – and it’s because we were a private company that was making money and were able to invest – is we went out and built what I would call DNA crime scene analysis for every chess player in the world,” Rensch said. That means Chess.com has a highly detailed model of what legitimate behavior looks like for millions of users over hundreds of millions of games, which it can use to detect discrepancies.......

....Rensch declined to elaborate on Niemann. “I’m not going on the record on anything that I think about the over-the-board scandal with Hans or Magnus, but you can imply what you want based on what I’m saying,” Rensch said. In forum posts this week, the Chess.com CEO, Erik Allebest, has hinted his company might soon release more information......

....Because the software analyzes the moves of the game itself, it works on over-the-board games as well as online, where the cheating rate is “100 to 200 times” higher, Regan said. Sinquefield Cup officials asked Regan to run the program on Carlsen and Niemann’s game and the results were unambiguous: “I found nothing,” he said. Regan’s model showed Neimann’s performance “was one standard deviation up” on some metrics, “but by definition the standard deviation standardly happens”.

But that’s led to an apparent disagreement between believers in Regan’s model and those of Chess.com’s model, which it doesn’t seem can be resolved without more evidence being made public. “It’s Chess.com’s move,” Regan said. The platform, he suggested, needs to “divulge or explain the reasons for their further action against Niemann”."

www.theguardian.com/sport/2022/sep/23/the-people-who-police-chess-cheats-we-built-a-scene-analysis-for-every-player-in-the-world

Will Chess . com reveal enough about it's system so that Regan can coherently critique the system that Danny Rensch and his followers claim is near perfection in its ability to spot cheating? I doubt that Rensch will reveal enough of his system for a statistics expert of Regan's caliber to assess it. But, we'll see.

@VTWood said in #1:
> Will Chess . com reveal enough about it's system so that Regan can coherently critique the system that Danny Rensch and his followers claim is near perfection in its ability to spot cheating? I doubt that Rensch will reveal enough of his system for a statistics expert of Regan's caliber to assess it. But, we'll see.

Chess servers never discuss methods of catching cheaters, for very obvious reasons.

That said: chess com isn't all that special. Methods of catching cheaters have been near perfect since the days of ICC. Which is to say: several decades now.

yet another tempest in a chesspot

@Molurus said in #2:
> Chess servers never discuss methods of catching cheaters, for very obvious reasons.
>
> That said: chess com isn't all that special. Methods of catching cheaters have been near perfect since the days of ICC. Which is to say: several decades now.
Chess . com's CEO indicates that they may release more information soon. How much they will release is the question. Likely not enough to allow Regan to meaningfully analyze their method. Thus it is unlikely they will present anything meaningful as to Niemann other than the bald assertions of additional cheating by Niemann that they've already made.

ICC's Speed Trap is very ineffective. There's a player there who has likely cheated in simuls against Voja Milanovik for years. ICC never flagged him. The methods for catching cheaters are far from perfect and cheating remains rampant online along with possible false accusations against some players. Sites like Chess. com will avoid court cases where their methods will be subject to discovery and analysis by a plaintiff's expert. That's why a few years ago Chess. com quickly settled a defamation suit brought in Federal District Court for the Eastern District of NY. Any allegation of cheating by any site is no more than an allegation unless and until it is subjected to analysis by an independent third party. They can and certainly do ban players, but that's far from a transparent finding that a player actually cheated.

The battle is Rensch(et al) v FIDE. The real attack is on FIDE, Neimann is just a 'pawn in the game'.

Chess.com biggest weakest is cheating, it's been going for 15 years, and apparently has "World Class" anti-cheating system - yet is still banning 20K accounts a month. This might sound efficient, but if the system was any good it would be a deterrent., which it clearly isn't. Who just invested in chess.com/?

@AlexiHarvey said in #5:
> The battle is Rensch(et al) v FIDE. The real attack is on FIDE, Neimann is just a 'pawn in the game'.
>
> Chess.com biggest weakest is cheating, it's been going for 15 years, and apparently has "World Class" anti-cheating system - yet is still banning 20K accounts a month. This might sound efficient, but if the system was any good it would be a deterrent., which it clearly isn't. Who just invested in chess.com/?

It pretty much does come down to Rensch vs FIDE. IIRC, Regan helped Chess.com develop their initial algorithm, but my non statistician understanding of what Rensch claims is that they have incorporated into their system literally millions of games played on their site. Given the abysmal quality of many of the games played there, it seems that their algorithm may have created a very low bar for cheat detection that actually catches more cheaters than are actually cheating. I'd like to see Regan address this.

That said, I have a friend who was on chess. com with a buddy "playing" an unrated game actually teaching and exploring an opening by chatting on FaceTime while making moves and taking back moves as they did analysis with a Fritz program as they tried and took back different variations. After perhaps 20 or so moves, they declared the "game" a draw, closed the board and did the same thing looking at a different opening. Again, these were unrated, takeback non-games looking at variations and checking them against Fritz to learn why their human choices were (very often) terrible. The next day, chess. com suspended ONE of them for cheating. He explained to one of the underlings at chess. com what they had been doing but this idiot would only reinstate his account if he admitted to cheating. He and most of his friends now play here on lichess.

Few comments

1) it is extremely unlikely that c.com people would expose themselves in blaming someone of cheating on their platform unless they have strong evidence. What they say about their own platform is most likely accurate, and I find it hard to believe they would put their reputation at risk by accusing Niemann unless they had good reasons to do so.

2) it is often rather hard to determine from a single classical game among top players whether there was cheating. The fact is that at those levels it might be sufficient to use help in even just 2 or three moves, making anomalies hard to spot.

@esmiro said in #7:
> Few comments
>
> 1) it is extremely unlikely that c.com people would expose themselves in blaming someone of cheating on their platform unless they have strong evidence. What they say about their own platform is most likely accurate, and I find it hard to believe they would put their reputation at risk by accusing Niemann unless they had good reasons to do so.
>
> 2) it is often rather hard to determine from a single classical game among top players whether there was cheating. The fact is that at those levels it might be sufficient to use help in even just 2 or three moves, making anomalies hard to spot.

I don't expect chess. com to make any direct accusation of cheating by Niemann other than the two instances Niemann has admitted to at ages 12 and 16. A direct accusation of cheating could lead to a court case in which chess. com would have to produce factual data as to its algorithm(s) and their implementation. I'm aware of only one legal action commenced against chess. com for defamation. That case was settled very quickly before discovery had taken place.

Yes, very difficult to determine cheating from a single game or small sampling of games. At this point only Ken Regan has released his analysis of all of Niemann's games available for, IIRC, the past two or three years finding nothing unusual. If Carlsen and chess. com have anything factually to add, they should produce it as FIDE has just called on all parties to do.

I don't believe statistical anti-cheating systems are workable for Elite GMs, for technical reasons.

But a far more understandable reason is false positives. Statistics are a set of objective formal methods of making decisions when certainly is not possible or impractical. You will always make some errors and that is acceptable giving the 'cost' of making no decision at all. Now whereas wrongly banning an ordinary player is of no real consequence - an acceptable cost giving the benefits - wrongly banning a player who earns their living from chess is clearly a much bigger consequence. Simple fact is that anything statistical will always risk wrongly banning an Elite GM in order to trap truly cheating Elite GMs. There is simply no way to avoid this.

Recently a number of Elite GMs have said they can tell when someone is cheating - even from a single move. I have no idea if this is true but clearly this would be a highly subjective assessment and therefore far more likely to produce false positives than objective statistical methods. It's just not possible for a human to apply consistently a subjective assessment with absolute rigor or certainty of outcome, nor is it likely the process would be repeatable in blind tests.

To my mind you can only ban Elite GMs if you have strong physical evidence, inference that such evidence exists but its lack of is irrelevant giving chosen moves is just not acceptable. You have to detail the method of cheating and thus reduce the likelihood of Elite GMs cheating in the future. You can't have a Jury of Elite GMs deciding the future of another Elite GM by inference - there are other reasons this would be unwise as well.

@VTWood said in #8:
> I don't expect chess. com to make any direct accusation of cheating by Niemann other than the two instances Niemann has admitted to at ages 12 and 16.

They did already, they said in the statement that the amount of cheating by Niemann was superior to what he admitted.