A spam filter that never catches a single spam message would still score 87.4 percent accuracy on my dataset. That one fact shaped this whole assignment.<\/p>\n\n\n\n
I built two classifiers for SMS spam: logistic regression and multinomial Naive Bayes. I trained each on two train\/test splits, 70\/30 and 90\/10, and compared them on five metrics.<\/p>\n\n\n\n
After removing 403 duplicate messages, only 12.6 percent of the data was spam. Labeling everything as legitimate gets you 87.4 percent for free.<\/p>\n\n\n\n
All four of my runs landed between 97.2 and 98.5 percent accuracy. By that measure, they were basically equal.<\/p>\n\n\n\n
They weren\u2019t equal. On the 70\/30 split, Naive Bayes caught 91 percent of spam and missed 18 messages. Logistic regression missed 42.<\/p>\n\n\n\n
Logistic regression was the cautious model. It flagged only one legitimate message as spam, but it let much more spam through. Naive Bayes struck the better balance, with the higher F1 score on both splits.<\/p>\n\n\n\n
Which one is \u201cbetter\u201d depends on the cost of each mistake. A false positive hides a real message. A false negative puts spam in the inbox.<\/p>\n\n\n\n
Both models scored 0.98 to 0.99 ROC-AUC. That means they ranked spam above legitimate messages almost equally well. Logistic regression\u2019s low recall came from the default 0.5 cutoff, not from weak learning. Lowering its threshold would recover much of the gap.<\/p>\n\n\n\n
On imbalanced data, the metric you pick decides what you see. Accuracy said these filters were the same. Recall said they weren\u2019t.<\/p>\n\n\n\n
Other related coursework CYSE 420 course page<\/a>.<\/p>\n\n\n\n