E27: Security Operations at Scale with Panther (And, from Open to Closed Source)

Traditional Limits

Traditional security information and event management systems (SIMs) face significant limitations in handling large-scale data operations. Jack Naglieri explains that these systems were not designed for the massive data intake and analysis required today, leading to inefficiencies and false positives 1. He notes that many in the industry have turned to more flexible tools like Splunk and Elastic, but these too have their scale limits. Panther, however, has achieved remarkable scalability, processing over 50 terabytes of data daily, thanks to its serverless architecture and innovative design 2.

The scales we've been able to achieve in Panther are mind blowing to me.

--- Jack Naglieri

This performance edge is further enhanced by Panther's use of Snowflake, providing significant gains over previous systems like AWS Athena.

   

Serverless Benefits

Serverless technology offers substantial benefits for security operations, particularly in terms of scalability and performance. Jack Naglieri highlights how serverless architecture allows Panther to handle vast amounts of data with minimal operational effort, unlike traditional systems that require extensive DevOps resources 1. This architecture supports real-time data analysis and is backed by Snowflake, which enhances performance significantly 2.

Serverless enabled that fully because we were able to just feed more data into it and the Amazon service would just elastically handle it.

--- Jack Naglieri

Naglieri also discusses the evolution of cloud security, noting that Panther builds on the serverless framework initially developed at Airbnb, leading to a more elastic and scalable platform 3.