The winner of the $100,000 first place prize went to FireEye, the intelligence-led security company. FireEye Endpoint Security had the best performance based on the criteria of the challenge and the cost-benefit framework. This includes accuracy metrics, time to
detection in terms of costs lost due to malware and the monetary cost for setup, triage and incident response.
“In this era of great power competition, we have to think differently about the way we design, develop and field capabilities to keep our Navy secure,” said NAVWAR Commander, Rear Adm. Christian Becker. “Prize challenges are a unique way to harness agility and innovation to cast a wider net to get the most advanced technology to the fleet faster.”
Managed by Program Executive Office (PEO) for Command, Control, Communications, Computers and Intelligence (C4I) and Space System’s Cyber Security Program Office (PMW 130), AI ATAC explored the capability for endpoint security products to incorporate machine learning (ML) and artificial intelligence (AI) models to detect and defeat indicators of compromise from various advanced malware strains. The program office partnered with Oak Ridge National Laboratory, a Department of Energy laboratory in Tennessee, which provided facilities and expertise for evaluating the technical solutions.
“With our unique product line of innovative security technologies, especially around machine learning, we knew that FireEye would be a great fit for this challenge,” said Seth Summersett, chief scientist at FireEye. “We are honored to support the Navy as they continue to hone cybersecurity solutions that will keep the fleet safe today and into the future.”
This challenge was the first of its kind for several reasons. First, the amount of malware that the endpoint solutions had to detect was larger than any vendor or third party testing organization has claimed to have tested. The collection of malware used also spanned categories likely to be seen in the real-world, as opposed to testing only the types of malware a company claims to protect.
Additionally, the malware was downloaded to the test machine as if it was real malware and tested each tool independently, realistically, and with a set of files that had malware that entrants had never seen, including custom strains created by the prize challenge team.
Sponsoring AI ATAC allowed the command to gather insight and participation from nontraditional defense vendors, teams and individuals.
“Overall our first prize challenge, by all definitions, was a success,” said John T. Armantrout, PMW 130 program manager. “The valuable insights we gained in the areas of AI and ML applied to cybersecurity will help inform solutions and investments that will be a valuable asset in protecting maritime networks. We look forward to significantly increasing the use of AI and ML to assist the warfighter in the defense of our networks.”
The second place winner of the $50,000 prize was Elastic, Inc. and their submission Elastic Endpoint Security.
The next prize challenge in the AI ATAC series will focus on adversarial campaign detection through network observable behavior analysis, with a $500,000 prize for the winner. For more information, go to https://www.challenge.gov/challenge/network-detection-of-adversarial-campaigns/. Submissions are due May 29, 2020.