GWU_DNSC 6290: Course Outline

MSBA elective workshop on AI, from logistic regression to LLMs.

• Session 1: Penalized Regression
• Session 2: Artificial Neural Networks (ANNs) for Structured Data
• Session 3: ANNs for Computer Vision
• Session 4: Large Language Models

Corrections or suggestions? Please file a GitHub issue.

---

Preliminary Materials

• Syllabus
• Basic Data Manipulation
• Primer on Technical Malpractice
• Whiteboard Notation
• Code Instructions

Session 1: Penalized Regression

_{^{Trace plot for a simple elastic net model. Source: Assignment 1.}}

Session 1 Materials

• Lecture Notes
• Software Example
• Assignment 1
  • Worksheet
  • Notebook
• References:
  • Elements of Statistical Learning, Sections 3.1 - 3.4, Section 4.4
  • Elastic net: Regularization and variable selection via the elastic net
  • LASSO: Regression Shrinkage and Selection via the Lasso
  • Computation and glmnet: Regularization Paths for Generalized Linear Models via Coordinate Descent

---

Session 2: Artificial Neural Networks (ANNs) for Structured Data

_{^{A very simple autoencoder network. Source: Session 2 notes.}}

Session 2 Materials

• Lecture Notes
• Software Examples:
  • MLP with simple gradient descent on Sonar data
  • MLP with simple gradient descent on Iris data
  • MNIST Autoencoder Usage Example
• Assignment 2
  • Worksheet
  • Notebook
• References:
  • Elements of Statistical Learning, Sections 11.3 - 11.7
  • Deep Learning, Chapter 6
  • History:
    • The perceptron: A probabilistic model for information storage and organization in the brain
    • Perceptrons: An Introduction to Computational Geometry
    • Learning representations by back-propagating errors
  • Autoencoders:
    • Deep Learning, Chapter 14
    • Reducing the Dimensionality of Data with Neural Networks
  • Generalized Additive Models (GAMs):
    • GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions
    • Neural Additive Models: Interpretable Machine Learning with Neural Nets
  • Feature Encoding: Entity Embeddings of Categorical Variables

---

Session 3: ANNs for Computer Vision

_{^{An illustration of image convolution. Source: Session 3 notes, with thanks to Wen Phan.}}

Session 3 Materials

• Lecture Notes
• Software Example
• Assignment 3
  • Worksheet
  • Notebook
• References:
  • Deep Learning, Chapter 9
  • LeNet: Gradient-Based Learning Applied to Document Recognition
  • AlexNet: ImageNet Classification with Deep Convolutional Neural Networks
  • Inception/Google LeNet: Going Deeper with Convolutions
  • ImageNet (competition): ImageNet Large Scale Visual Recognition Challenge
  • VGGNet: Very deep convolutional networks for large-scale image recognition
  • ResNet: Deep residual learning for image recognition
  • ViT: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

---

Session 4: Large Language Models

_{^{An illustration of retrieval augmented generation (RAG). Source: Session 4 notes.}}

Session 4 Materials

• Lecture Notes
• Software Example
• Assignment 4
  • Worksheet
  • Notebook
• References:
  • Term embedding: Efficient Estimation of Word Representations in Vector Space
  • Long-short term memory (LSTM): Sequence to Sequence Learning with Neural Networks
  • Transformers and attention: Attention is All You Need
  • Masked language modeling and bidirectional context: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  • GPT-2: Language Models Are Unsupervised Multitask Learners
  • GPT-3 and in-context learning: Language Models are Few-Shot Learners
  • Retrieval augmented generation: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

---

Code Instructions

• Download the entire class materials folder from the following URL: https://drive.google.com/drive/folders/17aP3EhWWSho7p1CH7NCrwCIhh7GYOHbR?usp=sharing.
  • Beside the name of the AI Workshop folder click the small triangle.
  • Click Download.
• This should download a zipped folder with a name like AI Workshop-20250220T171842Z-001.zip.
  • Extract the zip file into a folder with a name like AI Workshop-20250220T171842Z-001.
  • Inside the AI Workshop-20250220T171842Z-001 folder there should be a subfolder named AI Workshop.
  • Upload the subfolder (AI Workshop-20250220T171842Z-001/AI Workshop) to your own Google drive.
• The structure of the folder should look like:

AI Workshop
├── 01_Regression_Materials
├── 02_ANN_Materials
├── 03_CNN_Materials
└── 04_LLM_Materials

• If the Google Colab (Collaboratory) app is not installed, install it by clicking Open With and Connect more apps.

• In the top cells of most notebooks, you should notice a command like: %cd /content/drive/MyDrive/AI_Workshop/01_Regression_Materials/
  • If that is not the right folder for your Drive, use %ls and %cd to find the right path.
  • Generally, if this step fails, go to the Runtime menu and Restart Session.
• Run code examples by running each cell in order or by going to the Runtime menu and choosing Run All.
  • All the Colab example notebooks should run without errors after updating the path in the first cell.
  • When prompted, allow Colab to connect to your Google drive.
• To run the last session notebook (RAG_query.ipynb), you will need an access token from HuggingFace.co.
  • Create a free Hugging Face account (https://huggingface.co/).
  • Generate an access token (Profile -> Access Tokens).
  • Preserve the text of the token somewhere secure, as you may not be able to see it again after it is granted.
  • In the RAG_query.ipynb create new secret (by clicking on the key icon on left) named HF_TOKEN and populated with the Hugging Face access token.