← Back to Publications List

A Comparative Analysis of Sandboxing and Nested Virtualization for Malware Analysis and Threat Detection

Students & Supervisors

Student Authors
Tamim Hasan Apurbo
Bachelor of Science in Computer Science & Engineering, FST
Mahdi Hassan Noor Asif
Bachelor of Science in Computer Science & Engineering, FST
Koushik Biswas Arko
Bachelor of Science in Computer Science & Engineering, FST
Koushik Biswas Arko
Bachelor of Science in Computer Science & Engineering, FST
Supervisors
Md. Mortuza Ahmmed
Associate Professor, Faculty, FST

Abstract

"We compare two isolation strategies for malware analysis: sandboxing and nested virtualization. Detection performance for sand- boxing is grounded in the empirical baseline of Juwono et al. (CON- MEDIA2015),whoreport=97%F1onbehavior-analysistraces(Cuckoo/Anubis). Tostudylayeredcontainment,webuildadiscrete-eventsimulation(DES) of nested virtualization that models L2→L1 escapes, evasion checks, time-to-decision, and the cascading cost of containment failure (lateral spread). Results show a clear trade-off: sandboxing provides near-ceiling detection on the referenced dataset, while nested virtualization yields strongercontainment (lowerhost-breachprobability)atthecostofhigher latency/overhead and rare-but-severe failure modes when L1 contain- ment breaks. We outline where each technique is preferable and moti- vate a hybrid design: sandboxing for high-throughput triage and nested virtualization for high-value assets requiring defense-in-depth. Keywords: Malware analysis, sandboxing, nested virtualization, con- tainment, evasion, discrete-event simulation"

Keywords

"Keywords: Malware analysis sandboxing nested virtualization con- tainment evasion discrete-event simulation"

Publication Details

  • Type of Publication:
  • Conference Name: 3rd International Conference on BIG DATA,IOT and MACHINE LEARNING (BIM2025)
  • Date of Conference: 25/09/2025 - 25/09/2025
  • Venue: Dhaka International University,Bangladesh
  • Organizer: Dhaka International University,Bangladesh