GWU_DNSC 6314 & 6315: Course Outline

Materials for an introduction to machine learning.

• Lecture 1: Preliminaries, Feature Engineering and Feature Selection
• Lecture 2: Contemporary Linear Model Approaches
• Lecture 3: Model Assessment and Selection
• Lecture 4: Decision Trees
• Lecture 5: Artificial Neural Networks
• Lecture 6: Other Estimators: Support Vector Machines (SVM) k-Nearest-Neighbors (kNN), etc.
• Lecture 7: Decision Tree Ensembles
• Lecture 8: Convolutional Neural Networks
• Lecture 9: Clustering
• Lecture 10: Dimension Reduction
• Lecture 11: Association Rules and Recommendation
• Lecture 12: Deployment

Corrections or suggestions? Please file a GitHub issue.

---

Preliminary Resources

• Basic Data Manipulation
• Primer on Technical Malpractice
• Syllabi:
  • DNSC 6314
  • DNSC 6315
• Whiteboard Notation

Lecture 1: Preliminaries, Feature Engineering and Feature Selection

_{^{Source: Lecture 1 feature extraction example.}}

Lecture 1 Class Materials

• Lecture Notes
• Lecture 1 Feature Engineering Table
• Assignment 1
• Software Examples:
  • Feature selection
  • Feature extraction
  • Over- and under-sampling
  • Simple Encoding
  • Target Encoding (Categorical)
  • Target Encoding (Numeric)
  • Discretization
  • Winsorizing
  • Imputation
  • Standardization

_{^{All notebooks also available in the notebook folder.}}

Lecture 1 Reading

• Label, Segment, Featurize: a cross domain framework for prediction engineering
• Introduction to Data Mining - Sections 2.2-2.3 (Chapter 2 notes)
• Introduction to the Foundations of Causal Discovery - Sections 1-4, and 7

Lecture 1 Links

• LiNGAM for causal discovery
• Causal Discovery Toolbox
• pandas-profiling

Lecture 2: Contemporary Linear Model Approaches

_{^{Source: From GLM to GBM: Building the Case For Complexity.}}

Lecture 2 Class Materials

• Lecture Notes
• Lecture 2 Software Example
• Assignment 2
  • Assignment 2 Notebook
  • Assignment 2 Data

_{^{Notebooks and data also available via GitHub.}}

Lecture 2 Reading

• Elements of Statistical Learning:
  • Sections 3.1 - 3.4
  • Section 4.4
• Regularization and variable selection via the elastic net

Lecture 2 Links

• h2o (Python or R download, requires Java)
• Generalized Linear Model (GLM) documentation
• Generalized Linear Modeling with H2O
• elasticnet (R)
• glmnet (R)

Lecture 3: Model Assessment and Selection

_{^{Source: From Lecture 3.}}

Lecture 3 Class Materials

• Lecture Notes
• Lecture 3 Software Example
• Assignment 3
  • Assignment 3 Notebook
  • Assignment 3 Data

_{^{Notebooks and data also available via GitHub.}}

Lecture 3 Reading

• Elements of Statistical Learning:
  • Sections 7.1 - 7.5
  • Section 7.10
• Introduction to Data Mining:
  • Sections 3.4 - 3.6
• KDD-Cup 2004: Results and Analysis

Lecture 4: Decision Trees

_{^{Source: Machine Learning for High-Risk Applications.}}

Lecture 4 Class Materials

• Lecture Notes
• Lecture 4 Software Example
• Assignment 4
  • Assignment 4 Notebook
  • Assignment 4 Data

_{^{Notebooks and data also available via GitHub.}}

Lecture 4 Reading

• Introduction to Data Mining:
  • Sections 3.1 - 3.3
• Elements of Statistical Learning:
  • Section 9.2

Lecture 5: Artificial Neural Networks

_{^{Source: Demystifying Deep Learning, SAS Institute.}}

Lecture 5 Class Materials

• Lecture Notes
• Lecture 5 Software Example
• Assignment 5
  • Assignment 5 Notebook
  • Assignment 5 Data

_{^{Notebooks and data also available via GitHub.}}

Lecture 5 Reading

• Introduction to Data Mining:
  • Section 4.7
• Elements of Statistical Learning:
  • Sections 11.3 - 11.7

Lecture 5 Links

• Neural Network Zoo
• Deep Learning with H2O
• Neural Additive Models: Interpretable Machine Learning with Neural Nets (How I would recommend training an ANN for structured data.)
• THE MNIST DATABASE

Lecture 6: Support Vector Machines and k-Nearest-Neighbors

_{^{Source: From Assignment 6.}}

Lecture 6 Class Materials

• Lecture Notes
• Lecture 6 Software Example
• Assignment 6

_{^{Notebooks and data also available via GitHub.}}

Lecture 6 Reading

• Introduction to Data Mining:
  • Section 4.9
• Elements of Statistical Learning:
  • Sections 12.1 - 12.3

Lecture 7: Decision Tree Ensembles

_{^{Source: From Lecture 7.}}

Lecture 7 Class Materials

• Lecture Notes
• Lecture 7 Software Example
• Assignment 7
  • Assignment 7 Notebook
  • Assignment 7 Data

_{^{Notebooks and data also available via GitHub.}}

Lecture 7 Reading

• Introduction to Data Mining:
  • Section 4.10
• Elements of Statistical Learning:
  • Chapter 10
  • Chapter 15

Lecture 7 Links

• Random Forests
• Greedy Function Approximation: A Gradient Boosting Machine
• Gradient Boosting Machine With H2o

Lecture 8: Convolutional Neural Networks

_{^{Source: From Lecture 8, with thanks to Wen Phan.}}

Lecture 8 Class Materials

• Lecture Notes
• Assignment 8
  • Assignment 8 Notebook

_{^{Notebooks are also available via GitHub.}}

Lecture 8 Reading

• Introduction to Data Mining:
  • Section 4.8
• Deep Learning:
  • Chapter 9

Lecture 8 Links

• Keras

Lecture 9: Clustering

_{^{Source: From Assignment 9 Notebook.}}

Lecture 9 Class Materials

• Lecture Notes
• Assignment 9
  • Assignment 9 Notebook

_{^{Notebooks and data are also available via GitHub.}}

Lecture 9 Reading

• Introduction to Data Mining:
  • Chapter 7, through Section 7.3
• Elements of Statistical Learning:
  • Section 14.3

Lecture 10: Dimension Reduction

_{^{Source: From Lecture 10 Code Example.}}

Lecture 10 Class Materials

• Lecture Notes
• Lecture 10 Software Example
• Assignment 10
  • Assignment 10 Notebook
  • Assignment 10 Data

_{^{Notebooks and data are also available via GitHub.}}

Lecture 10 Reading

• Elements of Statistical Learning:
  • Section 14.5 - 14.6
• Generalized Low Rank Models
• Near Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?

Lecture 11: Association Rules and Recommendation

Lecture 11 Class Materials

• Lecture Notes
• Assignment 11
  • Assignment 11 Notebook
  • Assignment 11 Data

_{^{Notebooks and data are also available via GitHub.}}

Lecture 11 Reading

• Introduction to Data Mining, Chapter 5
  • Sections 5.1 – 5.3, and 5.7
• Elements of Statistical Learning:
  • Section 14.5 - 14.6
• Introduction to Recommender Systems Handbook

Lecture 12: Deployment

_{^{Source: From Lecture 12.}}

Lecture 12 Class Materials

• Lecture Notes
• Assignment 12
  • Assignment 12 Notebook
  • Assignment 12 Data

_{^{Notebooks and data are also available via GitHub.}}

Lecture 12 Reading

• How to Deploy Machine Learning Models
• Machine Learning: The High-Interest Credit Card of Technical Debt
• SUPERVISORY GUIDANCE ON MODEL RISK MANAGEMENT