CAP 5636 - Advanced Artificial Intelligence

Fall 2025

Course description:	Principles of artificial intelligence. Uninformed and informed search. Constraint satisfaction. AI for game playing. Probabilistic reasoning, Markov decision processes, hidden Markov models, Bayes nets. Neural networks and deep learning.
Student learning outcomes:	By the end of the semester the students will be able to: understand the search and decision making techniques used in modern artificial intelligence apply artificial intelligence techniques in their own code understand the societal and ethical implications of artificial intelligence
Instructor:	Dr. Lotzi Bölöni
Office Location:	HEC - 319
E-mail:	Ladislau.Boloni@ucf.edu (preferred means of communication)
Team:	TA: Anthony Bilic anthony.bilic@ucf.edu Grader: Rafael Montalvo rafael.montalvo2@ucf.edu
Web Site:	http://www.cs.ucf.edu/~lboloni/Teaching/CAP5636_Fall2025/index.html The assignments and the other announcements will be posted on the course web site
Classroom:	CB-0307
Class hours:	Tue, Th 12:00pm - 1:15pm
Office hours:	Tue, Th 1:30pm - 3:00pm (in HEC 319)
Enrollment requirements:	CAP 4630, or consent of instructor.
Required texts:	There is no required textbook.
Recommended readings:	Stuart Russel and Peter Norvig, Artificial Intelligence - A Modern Approach, 4th edition
Verification of engagement:	As of Fall 2014, all faculty members are required to document students' academic activity at the beginning of each course. In order to document that you began this course, please complete the following academic activity by the end of the first week of classes, or as soon as possible after adding the course. Failure to do so will result in a delay in the disbursement of your financial aid. To satisfy this requirement, you must finish the first quiz posted online. Log in to Webcourses, choose CAP 5636, and submit your answers online.

Syllabus

Date	Topic	Lecture Notes, Readings, Homeworks
Tue, Aug. 19	Introduction and content of the class The agent view of AI. Topics covered by the AI and ML classes. History and positioning of AI Motivating AI. Dangers of AI and AGI. Early history Expert systems	[slides] Content of the class [slides] History and positioning of AI [homework] HW1: AI History and Future - Due Aug. 24, 2025 [reading] Perceptrons (New York Times, July 13, 1958) [reading] Nick Bostrom - How long before superintelligence? (1998)
Thu, Aug. 21	History and positioning of AI Neural networks The two intellectual traditions: logic vs neural networks A melting pot of other ideas The agent view of AI	[homework] HW2: Intro to deep learning - Due Sept. 4, 2025
Tue, Aug. 26	Probability Random variables Joint and marginal distributions, conditional distribution	[slides] Probabilities - Introduction
Thu, Aug. 28	Independent variables and Bayes' nets Product rule, chain rule, Bayes' rule Inference Independence Independent random variables Conditional independence Bayes' nets	[slides] Independence
Tue, Sept. 2	Planning with uninformed search Reflex agents Search problems	[slides] Uninformed search
Thu, Sep. 4	Planning with uninformed search (cont'd) Depth first and breadth first search Uniform cost search
Tue, Sep. 9	*Planning with informed search: A search and heuristics** Informed search methods Heuristics Greedy search A* search Graph search	[slides] Informed search
Thu, Sep. 11	Game playing and adversarial search Types of games Adversarial search, minimax Alpha Beta pruning	[homework] HW3 - Planning with a goat - Due September 25th, 2025 [slides] Adversarial search
Tue, Sep. 16	*Midterm 1 - Introduction to A (not including game play)**
Thu, Sept. 18	Game playing and adversarial search (cont'd) The problem of depth Evaluation functions
Tue, Sep. 23	Expectimax search Expectimax search	[slides] Expectimax search
Thu, Sept. 25	State of the art in game play Monte Carlo tree search AlphaGo AlphaGo Zero Real life games	[slides] Game play state of the art
Tue, Sept 30	Utilities and rationality Utilities Axioms of rationality Rewards	[slides] Utilities and rationality
Thu, Oct. 2	Markov decision processes Defining MDPs: policies and utilities Optimal policy, value of state, value of Q-state	[slides] Markov Decision Processes
Tue, Oct. 7	Markov decision processes 2 Value iteration	[homework] HW4 - MDP and Q-Learning - Due October 24
Thu, Oct. 9	Markov decision processes 3 Policy extraction Policy iteration
Tue, Oct. 14	Reinforcement learning Reinforcement learning as a twist on MDPs Model-based and model-free learning Temporal difference learning	[slides] Reinforcement learning
Thu, Oct 16	Reinforcement learning (cont'd) Exploration vs. exploitation, regret Generalization across states	[homework] HW5 - DQN - Due November 6th
Tue, Oct. 21	Midterm 2: from Game Play to MDPs (not including Reinforcement Learning)
Thu, Oct. 23	Deep reinforcement learning Challenges with Q estimation The "deadly triad" DQN Double Q-learning
Tue, Oct. 28	Policy gradient RL Policy gradient theorem Using baselines	[slides] Policy gradient RL
Thu, Oct. 30	Reinforcement learning state of the art Soft actor-critic (SAC) Proximal policy optimization (PPO) Application: reinforcement learning from human feedback (RLHF)	[slides] RL State of the Art
Tue, Nov. 4	Imitation learning Behavior cloning Inverse reinforcement learning	[slides] Imitation learning
Thu, Nov. 6	Imitation learning (cont'd)
Tue, Nov. 11	Veteran's day, no class
Thu, Nov. 13	Hidden Markov models Hidden Markov models Example: robot localization	[slides] Hidden Markov Models
Tue, Nov. 18	Particle filters and applications of HMMs Particle filters Robot localization with particle filters	[slides] Particle filters and Applications of HMMs
Thu, Nov. 20	Artificial General Intelligence - Definitions and Tests Intelligence tests (Turing, Loebner prize, Winograd schema) Artificial Superintelligence Alignment problems LLMs	[slides] AGI definitions and tests
Tue, Nov. 25	Societal implications of AI Economic impact Societal changes Existential threat Ethical issues	[slides] Societal implications of AI
Thu, Dec. 5	Final exam: Thursday December 5, 2024, 10:00am - 12:50pm