Methodology for selecting effectiveness criteria of information Security Systems during Red Team simulated attacks

Objective. A methodology is proposed for selecting criteria for assessing the effectiveness of an organization's information security system based on conducting simulated Red Team attacks. This urgency is driven by the growing sophistication of cyberattacks and the need to test the readiness of organizations of various sizes from critical information infrastructure facilities to the financial and government sectors to withstand targeted attacks.

Method. The methodology combines a comparative analysis of existing approaches, case studies of real cyber exercises, threat modeling (based on the MITRE ATT&CK matrix), and expert interviews with security specialists.

Result. A review of regulatory documents (Russian GOST standards and federal laws, FSTEK guidelines, international standards ISO/IEC 27001 and NIST SP 800-53) and modern Red Team/Blue Team practices, including the use of SIEM, SOAR, and XDR systems, is conducted. A classification of security performance indicators (incident detection time, response speed, attack detection rate, etc.) is provided, illustrated with practical examples and architecture diagrams of security monitoring centers with SIEM/SOAR integration.

Conclusion. Alternative approaches to assessment (audit without active attacks, pentests), limitations and risks of Red Team methods and recommendations for taking into account the results of simulated attacks in regulatory frameworks and corporate audits are provided.

1. Federal Law of July 27, 2006 No.152-FZ (as amended on December 29, 2022)"On Personal Data" [Electronic resource]. Access mode:http://www.consultant.ru/document/cons_doc_LAW_61801/free. Date of access:March 28, 2025.

2. Federal Law of July 26, 2017 No. 187-FZ (as amended on July 14, 2022) "On the Security of Critical Information Infrastructure of the Russian Federation" [Electronic resource]. Access mode: http://www.consultant.ru/document/cons_doc_LAW_221160/, free. Date of access: March 28, 2025.

3. GOST R 57580.1-2017. Security of financial (banking) organizations. Information protection. Part 1. General provisions [Text]. – M.: Standartinform, 2017. 34 p.

4. ISO/IEC 27001:2013. Information technology – Security techniques – Information security management systems – Requirements. – Geneva: ISO, 2013. 39 p.

5. ISO/IEC 27004:2016. Information technology – Security techniques – Information security management – Monitoring, measurement, analysis and evaluation. – Geneva: ISO, 2016. – 55 p.

6. NIST SP 800-53 Rev. 5. Security and Privacy Controls for Information Systems and Organizations [Electronic resource]National Institute of Standards and Technology, 2020. –https://csrc.nist.gov/publications/detail/sp/800-53/rev5/final, свободный. – Date of access: 28.03.2025.

7. Hollis R. Red team testing: essential KPIs and metrics. Cyber Security: A Peer-Reviewed Journal. 2024;7(4):323–332.

8. CISA. Red Team Shares Key Findings to Improve Monitoring and Hardening of Networks [Electronic resource]. – Cybersecurity Advisory AA23-059A. – 2023. https://www.cisa.gov/sites/default/files/2023-03/cisa-red-team-advisoryaa23-059a.pdf, свободный. – Date of access:: 28.03.2025.

9. TechTarget. SIEM vs. SOAR vs. XDR: Evaluate the differences [Electronic resource]. 2024. https://www.techtarget.com, свободный. – Date of access: 28.03.2025.

10. Secureframe. 110+ Latest Data Breach Statistics [Electronic resource]. 2025. https://secureframe.com/blog/data-breachstatistics. Date of access:28.03.2025.

11. SentinelOne. What is SIEM Architecture? Components & Best Practices [Electronic resource]. 2024.: https://www.sentinelone.com/Date of access: 28.03.2025.

12. Solar Security. Red Teaming: описание услуги [Electronic resource]. https://www.solar.ru/services/red-teaming/, свободный. – Date of access: 28.03.2025.

13. Bank for International Settlements (BIS). Varying shades of red: red team testing frameworks. FSI Insights on Policy Implementation. – 2022:21. https://www.bis.org/fsi/publ/insights21.pdf. Date of access: 28.03.2025.