Schedule

Introduction to Senior Project I.

Overview of course expectations, deliverables, and grading criteria.

Suggested Readings:

• How to Read a Paper (S. Keshav)

Overview of research methods, how to read research papers, and an introduction to AI evaluation metrics (Precision, Recall, F-Measure).

Introduction to Single-Agent vs. Multi-Agent systems, GridWorld examples, Markov Decision Processes (MDP), and the curse of dimensionality in multi-agent environments.

Overview of Generalizability in AI Agents versus simple memorization. The lecture covers Reward Modeling strategies, including the trade-offs between Dense (hackable) and Sparse (slow) rewards, as well as advanced techniques like Curriculum Learning and Inverse Reinforcement Learning (IRL).

Overview of the hierarchy of decision-making models, moving from simple MDPs to Partially Observable Stochastic Games (POSG). The lecture also explores the necessity of distributed processing for AI agents, covering Data versus Model Parallelism and the differences between Synchronous and Asynchronous execution.

Exploration of the physical limits of AI agents regarding latency, bandwidth, and throughput. The lecture contrasts Cloud versus Edge architectures and discusses engineering solutions for system constraints, such as model compression, quantization, and activation offloading, particularly in latency-critical scenarios like autonomous braking.

The Transformer architecture and its applications in Deep Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL). The lecture covers training and finetuning AI models, the Bellman Equation, Q Learning, and Deep Q Networks, providing insights into their roles in decision-making processes.

Walkthroughs on DQL, Experience Buffers, and comparison of Q-Learning with REINFORCE.

Discussion on Sample Complexity, Generalizability, Independent Q Learning (IQL), Actor-Critic methods, Multi-Agent Deep Deterministic Policy Gradient (MADDPG), and Imitation Learning methods (ehavioral Cloning, DAgger) in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Decision Making Algorithms for AI Agents like AlphaGo (RL + MCTS), TRPO, PPO, DPO, GRPO, StrateGo, and Intelligent AI Delegation

Discussion on Robustness, Consistency and Risk Mitigation in AI Agents, Offline Reinforcement Learning, and Inverse Reinforcement Learning (IRL) in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on Model Predictive Control (MPC): Real-time constraints, mistake correction and daily control tasks by AI Agents in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on Game Theory for AI Agents in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on Evolutionary Learning, Mean-Field Learning, Self-play for self-improvement and Continual Learning: Communication paradigms for AI Agents to Adapt in Real-Time Decentralized AI: When and How to Scale AI Agents in the context of Reinforcement Learning (RL) and Multi-Agent Deep Reinforcement Learning (MARL).

Discussion on implementing Multi-Agent Deep Reinforcement Learning algorithms.

Discussion on optimizing agentic systems for efficiency in terms of energy, network bandwidth, memory, chip utilization, and power.

Discussion on evaluating multimodal agentic systems post-training, including inference, qualitative and quantitative decision-making assessment, with a focus on trustworthy and explainable agents.

Exploration of the role of AI agents in the context of 6G and 7G networks, discussing the opportunities they present as well as the challenges that need to be addressed for successful integration and deployment.