Schedule

Introduction to Senior Project I.

Overview of course expectations, deliverables, and grading criteria.

Suggested Readings:

• How to Read a Paper (S. Keshav)

Overview of research methods, how to read research papers, and an introduction to AI evaluation metrics (Precision, Recall, F-Measure).

Introduction to Single-Agent vs. Multi-Agent systems, GridWorld examples, Markov Decision Processes (MDP), and the curse of dimensionality in multi-agent environments.

Overview of Generalizability in AI Agents versus simple memorization. The lecture covers Reward Modeling strategies, including the trade-offs between Dense (hackable) and Sparse (slow) rewards, as well as advanced techniques like Curriculum Learning and Inverse Reinforcement Learning (IRL).

Overview of the hierarchy of decision-making models, moving from simple MDPs to Partially Observable Stochastic Games (POSG). The lecture also explores the necessity of distributed processing for AI agents, covering Data versus Model Parallelism and the differences between Synchronous and Asynchronous execution.

Exploration of the physical limits of AI agents regarding latency, bandwidth, and throughput. The lecture contrasts Cloud versus Edge architectures and discusses engineering solutions for system constraints, such as model compression, quantization, and activation offloading, particularly in latency-critical scenarios like autonomous braking.

The Transformer architecture and its applications in Deep Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL). The lecture covers training and finetuning AI models, the Bellman Equation, Q Learning, and Deep Q Networks, providing insights into their roles in decision-making processes.

Walkthroughs on DQL, Experience Buffers, and comparison of Q-Learning with REINFORCE.

Discussion on Sample Complexity, Generalizability, Independent Q Learning (IQL), Actor-Critic methods, Multi-Agent Deep Deterministic Policy Gradient (MADDPG), and Imitation Learning methods (ehavioral Cloning, DAgger) in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Decision Making Algorithms for AI Agents like AlphaGo (RL + MCTS), TRPO, PPO, DPO, GRPO, StrateGo, and Intelligent AI Delegation

Discussion on Robustness, Consistency and Risk Mitigation in AI Agents, Offline Reinforcement Learning, and Inverse Reinforcement Learning (IRL) in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on Model Predictive Control (MPC): Real-time constraints, mistake correction and daily control tasks by AI Agents in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on Game Theory for AI Agents in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on Evolutionary Learning, Mean-Field Learning, Self-play for self-improvement and Continual Learning: Communication paradigms for AI Agents to Adapt in Real-Time Decentralized AI: When and How to Scale AI Agents in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on implementing Multi-Agent Deep Reinforcement Learning algorithms.

Discussion on optimizing agentic systems for efficiency in terms of energy, network bandwidth, memory, chip utilization, and power.

Discussion on evaluating multimodal agentic systems post-training, including inference, qualitative and quantitative decision-making assessment, with a focus on trustworthy and explainable agents.

Exploration of the role of AI agents in the context of 6G and 7G networks, discussing the opportunities they present as well as the challenges that need to be addressed for successful integration and deployment.

Exploration of AI agents applied to scientific discovery, detailing generative diffusion models for drug discovery, Bayesian Optimization for molecular search, and the automation of materials discovery through robotic Self-Driving Laboratories.

Discussion on the transition to agentic smart grids using decentralized multi-agent frameworks, focusing on balancing economic dispatch constraints, utilizing the Model Context Protocol (MCP), and ensuring safe reinforcement learning via Digital Twins.

Examination of agents operating in the physical and spatial world, covering robotic imitation learning and Vision-Language-Action (VLA) models, generative world models and causal reasoning for autonomous driving, and proactive interventions in AR/VR environments.

Overview of agents navigating digital spaces, detailing Agentic RAG to reduce hallucinations, the Reflexion framework for software coding, digital embodiment for web interactions, and grounding multimodal models in real-world physical affordances.

Analysis of applying distributed multi-agent systems to real-world logistics and volatile stock portfolio orchestration, alongside an overview of foundation models for physical robotics and the challenge of closing the sim-to-real gap.

Exploration of the scaling laws and architectures of multi-agent coordination, mixed-motive dilemmas where agents cooperate to compete, and human-agent collaboration through zero-shot adaptation and generative partner modeling.

In-depth review of agent evaluation of non-binding deals using expected utility, the GAMMA framework for simulating diverse human behavior, and enhancing human-agent alignment via iterative simulation sandboxes and explicit goal abstractions.

Discussion on overcoming limited context windows using episodic and semantic dynamic memory, featuring the Zettelkasten-style A-Mem architecture, and utilizing Elastic Weight Consolidation to prevent catastrophic forgetting during agentic adaptation.

Examination of interoperability in multi-agent systems via the Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocol, concluding with the strict industrial governance laws required for deploying trustworthy autonomous operations.