KNN vs Logistic Regression Boundary Brawl

Draw 2D points and compare nonparametric k-NN boundaries with logistic regression trained by gradient descent.

TriWei AI Lab

KNN vs Logistic Regression: Boundary Brawl

Draw your own classification problem and compare local k-NN behavior with a global logistic boundary.

Non-parametric k-NN Logistic classifier

How to play + what to look for

• Goal: draw points and compare kNN (non‑parametric) vs logistic regression (parametric).
• Tap/click the canvas to add points for the current class. Toggle mode to switch labels.
• Train logistic regression and watch the decision boundary change.
• Keyboard: C=Clear, T=Train logistic, G=Gradient check.

Learning objectives

• Concept focus: contrast instance-based methods (k‑NN) with parametric classifiers (logistic regression).
• Core definition: k‑NN predicts by majority vote among neighbours, while logistic regression fits a linear decision boundary using cross-entropy loss.
• Common mistake: choosing too small a k leads to noisy boundaries; forgetting to normalize coordinates when computing probabilities.
• Why it matters: seeing these models side by side helps learners appreciate the trade-offs between flexibility and interpretability.
• Toy disclaimer: this demo is limited to two dimensions and a linear logistic classifier; real datasets may require higher dimensions and regularization.

Click/tap to place points for two classes. Then compare: k‑Nearest Neighbors (local, flexible) vs Logistic Regression (global linear separator trained by gradient descent).

Class to place: Class 0 Class 1 Model: k‑NN Logistic regression k (for k‑NN): 5 LR steps: η: 0.100 Train/Test split: 0.80

Playfield

# points: 0

Train accuracy: —

Test accuracy: —

Notes

• k‑NN predicts based on the labels of the k nearest stored points (no “training”, just searching).
• Logistic regression fits a linear decision boundary and minimizes cross‑entropy via gradient descent.
• Try k=1 (very wiggly) vs larger k (smoother). Logistic regression can’t represent non‑linear boundaries.

Logistic regression and gradient descent are covered in many ML courses (e.g., Stanford CS229 notes). A common reference: cs229-notes1.pdf.

Collaboration Credits

These interactive labs are the result of a close collaboration between a human author and an AI assistant (ChatGPT). The AI contributed algorithmic refinements, numerical safeguards and visual improvements, while the human designed the pedagogical structure, reviewed all code, and ensured educational accuracy. Mathematical formulas and derivations are referenced to reputable course notes and textbooks. All code runs entirely in the browser; no data is sent to any server.