One of my models scored 0.350 ROC-AUC. That\u2019s worse than random guessing, and I left it in the report.<\/p>\n\n\n\n
This assignment used K-Means and DBSCAN to detect malware without labels. The models never saw which files were malicious. Labels came in only afterward, to grade the results.<\/p>\n\n\n\n
Clustering-based anomaly detection assumes anomalies are rare. They should sit apart from the dense crowd of normal data.<\/p>\n\n\n\n
The instructions treated malware as the anomaly. But in this dataset, 70 percent of the files were malware. The \u201canomaly\u201d was the crowd.<\/p>\n\n\n\n
So K-Means did what it was built to do. It flagged the files farthest from the center of the data. Those turned out to be mostly benign, and the malware was scored as normal.<\/p>\n\n\n\n
Reversing the score would have produced 0.650. But I would only know to flip it by looking at the labels. That sneaks label knowledge into a method that\u2019s supposed to work without it.<\/p>\n\n\n\n
Instead, I ran a second set with malware cut to 5 percent. The pipeline and parameter rules stayed the same. K-Means rose to 0.794, and DBSCAN reached 0.910.<\/p>\n\n\n\n
Both algorithms independently found the same 290-file cluster, and every file in it was benign. Small clusters aren\u2019t automatically suspicious.<\/p>\n\n\n\n
DBSCAN also found a 100-file cluster that was entirely malware. To a density method, a tight group looks normal. Malware arriving in volume can hide in plain sight.<\/p>\n\n\n\n
The makeup of the data mattered more than the algorithm. Before choosing an unsupervised method, I now check whether its core assumption holds.<\/p>\n\n\n\n
The full report is on my CYSE 420 course page<\/a>.<\/p>\n\n\n\n