Lecture materials for Senior Project II will be posted here as they become available. Students should check this page regularly for updates and supporting materials related to the current course.

  • Lecture 1: Introduction
    tl;dr: Overview of the course, AI agents for decision-making, and project structure.
    [PDF]
  • Lecture 2: Tasks and Problem Formulation
    tl;dr: Formulating decision-making tasks, defining objectives, and constraints for AI agents.
    [PDF]
  • Lecture 3: Scaling to Real-World Deployment
    tl;dr: Four-tier autonomous architecture, Sim2Real alignment, real-time control loops, and fluid dynamics in automated experimentation.
    [PDF]
  • Lecture 4: System-Level Parameters: Optimizing Latency, Memory Inflation, and Network Bandwidth
    tl;dr: High-throughput agent orchestration, reward hacking mitigation, and network link latency modeling.
    [PDF]
  • Lecture 5: Interconnection Topologies, Latency Bounds, and Grid Energy Infrastructure
    tl;dr: Topological bounds, collective communications, cluster sizing derivation, and RL-based smart grid control.
    [PDF]
  • Lecture 6: Counterfactual Design, Active Learning, and Autonomous Exploration
    tl;dr: Multi-agent credit assignment, Gaussian process surrogates, batch expected improvement, and intrinsic exploration with random network distillation.
    [PDF]

    In-Class Classwork 1: Sizing Processor Clusters.

  • Lecture 7: Scientific Agents: Design, Synthesis, and Autonomous Discovery
    tl;dr: Autonomous closed-loop discovery, generative inverse molecular design with GFlowNets, 3N-MCTS retrosynthesis planning, ChemOS orchestration, and Coscientist robotic execution.
    [Interactive Slides]

    Reference Readings:

    • Bengio, E., et al. (2021). Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation and Molecule Discovery. arXiv:2106.04399.
    • Segler, M. H. S., Preuss, M., & Waller, M. P. (2018). Planning chemical syntheses with deep neural networks and symbolic AI. Nature, 555(7698), 604–610.
    • Roch, L. M., et al. (2020). ChemOS: An orchestration software to democratize autonomous discovery. PLOS ONE, 15(4), e0229862.
    • Boiko, D. A., et al. (2023). Autonomous chemical research with large language models. Nature, 624(7992), 570–578.
  • Lecture 8: AI Agents for Biology: Protein Design, Cell Simulation, and Disease Modeling
    tl;dr: Foundation models for biology, agentic workflows for protein design, virtual-cell simulation, and closed-loop disease modeling across wet-lab and computational pipelines.
    [PDF]
  • Lecture 9: Financial Agents, Stock Portfolio Orchestration, and Trading
    tl;dr: Deep reinforcement learning for portfolio allocation, transaction costs and portfolio-vector memory, limit order books and ABIDES multi-agent simulation, and optimal trade execution with the Almgren-Chriss framework.
    [PDF]

    In-Class Problem: Optimal Sizing of Trade Execution.

  • Lecture 10: Supply Chain Logistics and Orchestration
    tl;dr: Stochastic inventory control with lead times and state augmentation, multi-agent rollout, warehouse coordination as a partially observable stochastic game, invalid action masking, and hierarchical multi-agent reinforcement learning for scalable fleet orchestration.
    [PDF]
  • Lecture 11: Drone Swarms, Autonomous Vehicles, and Transportation; Advanced Topics: Augmented Reality (AR/VR/XR) and Hardware Agents
    tl;dr: Distributed flocking control, dynamic graph connectivity and communication latency, multi-agent rollout for fleet trajectories, cooperative intersections and highway platooning, multimodal perception, and fleet orchestration as a Dec-POMDP or partially observable stochastic game.
    [PDF]
  • Lecture 12: Quantum Control Mechanisms and Non-Deterministic AI
    tl;dr: Control barrier functions and safety filtering, hierarchical fleet coordination, cooperative training and sim-to-real transfer, spatial XR and edge hardware agents, and quantum approaches to task allocation and reinforcement learning.
    [PDF]
  • Lecture 13: Navigating Open-Ended Problems, Establishing New Metrics, and Reliability
    tl;dr: Open-ended problem formulation and curricula, counterfactual baselines and k-step rollout, balancing decision quality against latency and resource costs, and agent reliability through multi-seed evaluation, robustness checks, and safeguards against metric gaming.
    [PDF]

    In-Class Problem: Computing the Optimal Rollout Horizon.

  • Lecture 14: Scaling Multi-Agent AI Problems with Distributed Learning
    tl;dr: Distributed multi-agent reinforcement learning, system-level scaling challenges, gradient aggregation, parameter-server architectures, synchronous and asynchronous training, and network bandwidth bottlenecks.
    [PDF]
  • Lecture 15: Scaling Multi-Agent AI Problems: Modern Distributed Learning and JAX
    tl;dr: Ring all-reduce with reduce-scatter and all-gather, bandwidth-efficient gradient aggregation, and accelerated multi-agent reinforcement learning with JaxMARL, vectorized environments, and JIT compilation.
    [PDF]