Before I trained a single model for this assignment, the dataset already offered a perfect score.
The task was static malware detection. Each Windows executable had to be classified as malicious or benign from its Portable Executable (PE) header fields alone, without running it. The dataset had 19,611 files, 77 numeric header features, and one label.
The column that gave away the answer
The dataset included a filename column. Every benign file had a real Windows name, like ACCTRES.dll. Every malicious file was named VirusShare_ followed by a hash.
A one-line string check would have scored 100 percent. It would also have learned nothing about executables. Point it at a file from anywhere other than a malware repository and it fails.
So that column went first, before any modeling.
The duplicates that only appeared afterward
Dropping the filename exposed a second problem. With the names gone, 2,980 rows turned out to be exact duplicates. Of those, 98.5 percent were malware.
That imbalance tells a story. These were different files with identical headers. Repacking a sample changes its hash but not the header its builder produced. That’s polymorphism showing up in the data.
Left in, those copies would land on both sides of the train/test split. The model would be graded partly on files it had already memorized.
What this changed about how I evaluate models
Once the data was clean, k-nearest neighbors beat a decision tree on every metric. The F1 gap was 0.0065.
Either preprocessing mistake would have moved the numbers far more than that. Choosing a model mattered less than making sure the results meant anything.
I also made one mistake of my own. I picked hyperparameters by comparing scores on the test set, which lets the evaluation data shape the model. The fix is cross-validation on the training data, and I used it in the next assignment.
The takeaway
A high score is a claim, not a result. Before trusting one, I now ask what the model could have cheated with.
The full report, including the PCA analysis, decision boundaries, and error breakdown, is on my CYSE 420 course page.