BenchJack has been open-sourced. In the future, evaluating intelligent agents should first go through a penetration test; this approach feels quite appropriate.

ME News Report, April 19 (UTC+8), Berkeley Artificial Intelligence Research Group (berkeley_ai) quoted Dawn Song's statement, announcing that her team has successfully broken through 8 major agent evaluation benchmarks. The team has decided to open source the tools used to achieve this result, named BenchJack. The tool is described as "penetration testing for evaluations," aimed at helping other developers proactively test and identify potential weaknesses in their evaluation systems. (Source: InFoQ)