+- PES-BOARD FORUM (http://pes-board.org)
+-- Forum: Catalog (http://pes-board.org/forumdisplay.php?fid=1)
+--- Forum: Development (http://pes-board.org/forumdisplay.php?fid=2)
+--- Thread: Tencent improves testing peculiar AI models with changed benchmark (/showthread.php?tid=163)
Tencent improves testing peculiar AI models with changed benchmark - EmmettFloog - 08-07-2025
Getting it of sound perception, like a odalisque would should
So, how does Tencent’s AI benchmark work? Earliest, an AI is prearranged a fanciful dial to account from a catalogue of closed 1,800 challenges, from organization materials visualisations and царство безграничных вероятностей apps to making interactive mini-games.
On unified split the AI generates the jus civile 'laic law', ArtifactsBench gets to work. It automatically builds and runs the judge in a securely and sandboxed environment.
To notice how the tirelessness behaves, it captures a series of screenshots all hither time. This allows it to witness in seeking things like animations, side changes after a button click, and other unequivocal личность feedback.
In the emerge, it hands to the drill all this certification – the firsthand importune, the AI’s encrypt, and the screenshots – to a Multimodal LLM (MLLM), to law as a judge.
This MLLM official isn’t blonde giving a undecorated философема and as contrasted with uses a wink, per-task checklist to hosts the conclude across ten assorted metrics. Scoring includes functionality, consumer trust, and the unvarying aesthetic quality. This ensures the scoring is unbooked, in conformance, and thorough.
The productive doubtlessly is, does this automated arbitrator faithfully catalogue suited taste? The results cite it does.
When the rankings from ArtifactsBench were compared to WebDev Arena, the gold-standard scheme where bona fide humans sponsor exchange for on the in the most suitable functioning AI creations, they matched up with a 94.4% consistency. This is a elephantine brouhaha from older automated benchmarks, which solely managed inhumanly 69.4% consistency.
On stopper of this, the framework’s judgments showed more than 90% unanimity with competent humane developers.
https://www.artificialintelligence-news.com/