Malware analysis is the study or process of determining the functionality, origin and potential impact of a given malware sample, such as a virus, worm, Trojan horse, rootkit or backdoor. Malware, or malicious software, is any computer software intended to harm the host operating system or to steal sensitive data from users, organizations or companies. Malware may include software that gathers user information without permission. It supports incident response by helping investigators identify infections, assess their consequences and plan containment and recovery.
Two principal approaches are static analysis, which examines a program without executing it, and dynamic analysis, which observes it during execution. These approaches provide different kinds of evidence and have different limitations.
Analysis may also include memory forensics and manual reverse engineering. Findings can describe a sample's capabilities, its dependencies and the evidence used to identify it, together with indicators and detection rules that support further investigation.
Applications
During an incident, malware analysis helps establish how malicious software affects a host, how it spreads and which other systems may be compromised. Understanding these characteristics informs decisions about incident severity, containment, eradication and recovery. Direct examination can be particularly important for customised malware for which existing descriptions or detection signatures are unavailable.
Analysis also contributes to the development of indicators of compromise (IoCs). These can include file hashes, domain names, IP addresses and other artefacts associated with malicious activity. Defenders use such information to search for related activity, identify previously unnoticed attacks and deploy detection or blocking controls.
Use cases
There are three typical use cases that drive the need for malware analysis:
Computer security incident management: If an organization discovers or suspects that some malware may have gotten into its systems, a response team may wish to perform malware analysis on any potential samples that are discovered during the investigation process to determine if they are malware and, if so, what impact that malware might have on the systems within the target organizations' environment.
Malware research: Academic or industry malware researchers may perform malware analysis simply to understand how malware behaves and the latest techniques used in its construction.
Indicator of compromise extraction: Vendors of software products and solutions may perform bulk malware analysis in order to determine potential new indicators of compromise; this information may then feed the security product or solution to help organizations better defend themselves against attack by malware.
Methods
The method by which malware analysis is performed typically falls under one of two types:
Static analysis
Static or Code Analysis is usually performed by dissecting the different resources of the binary file without executing it and studying each component. The binary file can also be disassembled (or reverse engineered) using a disassembler such as IDA, Ghidra or Binary Ninja. The machine code will oftentimes be translated into assembly, which is the readable format of machine code for humans: the malware analyst can then read the assembly as it is correlated with specific functions and actions inside the program, then make sense of the assembly instructions and have a better visualization of what the program is doing and how it was originally designed. Viewing the assembly allows the malware analyst/reverse engineer to get a better understanding of what is supposed to happen versus what is really happening and start to map out hidden actions or unintended functionality. Some modern malware is authored using evasive techniques to defeat this type of analysis, for example by embedding syntactic code errors that will confuse disassemblers but that will still function during actual execution.
Dynamic analysis
Dynamic or behavioural analysis is performed by observing the behavior of the malware while it is actually running on a host system. This form of analysis is often performed in a sandbox environment to prevent the malware from actually infecting production systems; many such sandboxes are virtual systems that can easily be rolled back to a clean state after the analysis is complete. The malware may also be debugged while running using a debugger such as GDB or x64dbg to watch the behavior and effects on the host system of the malware step by step while its instructions are being processed. Malware that has a hidden, slightly incomprehensible execution path (for protection) can also be mitigated using tracing, emulation, instrumentation or symbolic execution. Modern malware can exhibit a wide variety of evasive techniques designed to defeat dynamic analysis including testing for virtual environments or active debuggers, delaying execution of malicious payloads, or requiring some form of interactive user input.
Memory analysis
Memory analysis examines volatile memory from a running or potentially compromised system for evidence of malicious activity. Because running processes pass through memory, memory analysis can reveal malware while it is executing, including memory-only malware that does not install files or otherwise interact directly with persistent storage. Because memory is volatile, this evidence may be lost when a system is shut down or restarted.
Frameworks such as Volatility support the examination of memory captures and the extraction of suspicious content for further analysis. Memory examination can therefore complement file inspection, particularly when a sample's on-disk representation conceals its executable code.
Combined analysis
Analysts frequently combine automated inspection, static examination, behavioural experiments and manual code analysis. The process is iterative: an observation made during execution may direct attention to a particular routine, while code examination may explain behaviour that was difficult to reproduce. Reverse engineering can also reveal capabilities that did not appear during behavioural testing.
Stages
Examining malicious software involves several stages, including, but not limited to the following:
Manual Code Reversing
Interactive Behavior Analysis
Static Properties Analysis
Fully-Automated Analysis
Standardized evaluation of sandbox-based analysis products has also emerged. In 2025, the Anti-Malware Testing Standards Organization (AMTSO) introduced the first Sandbox Evaluation Framework, aimed at providing consistent, use-case-driven testing criteria for sandbox malware analysis tools.
Artificial Intelligence vs. Humans
The use of artificial intelligence (AI) in malware detection has been an active area of research within the field of cybersecurity. One study compared the behavior of AI against human behavior to understand the similarities and differences in how these two parties performed. The results included:
AI and humans have similar detection rates
AI focused more on static analysis
Humans focused more on dynamic analysis
AI does not perform well with only dynamic analysis
Antivirus softwares have used these studies to come to a conclusion that a combination of both practices would be most ideal. Since AI can use static analysis at a much faster rate than humans in malware detection, the antivirus software will send the samples to the AI for an initial screening. If the sample is more complex and the AI cannot make a definitive conclusion, then the sample will be sent to a human to observe the sample in more detail. In this way, more samples can be correctly identified as malware or benign. In practical threat-detection systems, machine-learning models may also operate alongside rule-based systems, threat-intelligence feeds, and human review rather than functioning as a standalone detection method.
