Real-World Benchmarking of Runtime Protection

As the software development lifecycle (SDLC) and the threat landscape evolve, so too must the way we think about testing runtime security products. Yet, benchmarks that exist neither accurately reflect the modern SDLC, nor do they simulate real-world attack scenarios well.

DevOps automation, agile methods, cloud computing, virtual machines and containers have dramatically sped up the pace of releases, while application security and runtime protection have barely evolved. Therefore, to mimic real-world constraints, products must be tested for their ability to identify and protect against vulnerabilities, without slowing down new feature releases.

As such, ShiftLeft submitted its continuous security solution to expert penetration testing performed by Cobalt.io. In order to benchmark ShiftLeft's real-world protection capabilities, two identical instances of an application were tested by Cobalt.io, one unprotected and the other instrumented with ShiftLeft.

This type of testing fills a gap in our understanding of how security products perform in the real world because security teams naturally do not share their own results. While standardized laboratory testing is important, and will always have a place in product evaluation, penetration testing can leverage human expertise that goes beyond tools based probing. Furthermore, simulating real-world security scenarios increasingly must account for shrinking software development lifecycles (SDLC) and the pressure to get new software releases out on time.

This webinar will cover:

  • Why Real-World Benchmarking of Runtime Solutions is Unique and Necessary
  • The Test Methodology
  • The Test Application
  • Penetration Testing Methodology
  • Penetration Testing Results
  • Continuous Security with ShiftLeft
  • Pen Testing as a Service with Cobalt.io
  • Q&A

Presenters of the Webinar

Chetan Conikee
Chetan Conikee
CTO & Co-Founder | ShiftLeft
Joe Sechman
Joe Sechman
VP of PenTest Delivery | Cobalt.io