CAP 5636 - Advanced Artificial Intelligence

Fall 2026

Course description:	Principles of artificial intelligence. Uninformed and informed search. Constraint satisfaction. AI for game playing. Probabilistic reasoning, Markov decision processes, hidden Markov models, Bayes nets. Neural networks and deep learning.
Student learning outcomes:	By the end of the semester the students will be able to: understand the search and decision making techniques used in modern artificial intelligence apply artificial intelligence techniques in their own code understand the societal and ethical implications of artificial intelligence
Instructor:	Dr. Lotzi Bölöni
Office Location:	HEC - 319
E-mail:	Ladislau.Boloni@ucf.edu (preferred means of communication)
Team:
Web Site:	http://www.cs.ucf.edu/~lboloni/Teaching/CAP5636_Fall2026/index.html The assignments and the other announcements will be posted on the course web site
Classroom:	NSC O110
Class hours:	Tue, Th 12:00pm - 1:15pm
Office hours:	Tue, Th 1:30pm - 3:00pm (in HEC 304)
Enrollment requirements:	CAP 4630, or consent of instructor.
Required texts:	There is no required textbook.
Recommended readings:	Stuart Russell and Peter Norvig, Artificial Intelligence - A Modern Approach, 4th edition
Verification of engagement:	As of Fall 2014, all faculty members are required to document students' academic activity at the beginning of each course. In order to document that you began this course, please complete the following academic activity by the end of the first week of classes, or as soon as possible after adding the course. Failure to do so will result in a delay in the disbursement of your financial aid. To satisfy this requirement, you must finish the first quiz posted online. Log in to Webcourses, choose CAP 5636, and submit your answers online.

Syllabus

Date	Topic	Lecture Notes, Readings, Homeworks
Tue, Aug. 25	Introduction and content of the class The agent view of AI. Topics covered by the AI and ML classes. History and positioning of AI Motivating AI. Dangers of AI and AGI. Early history Expert systems	[slides] Content of the class [slides] History and positioning of AI [homework] HW0: AI Near Future - Due Aug. 30, 2026 [reading] Perceptrons (New York Times, July 13, 1958) [reading] Nick Bostrom - How long before superintelligence? (1998) [homework] HW1: Background - Due Sept. 14, 2026
Thu, Aug. 27	History and positioning of AI Neural networks The two intellectual traditions: logic vs neural networks A melting pot of other ideas The agent view of AI
Tue, Sept. 1	Probability Random variables Joint and marginal distributions, conditional distribution	[slides] Probabilities - Introduction
Thu, Sept. 3	Independent variables and Bayes' nets Product rule, chain rule, Bayes' rule Inference Independence Independent random variables Conditional independence Bayes' nets	[slides] Independence
Tue, Sept. 8	Planning with uninformed search Reflex agents Search problems	[slides] Uninformed search
Thu, Sept. 10	Planning with uninformed search (cont'd) Depth first and breadth first search Uniform cost search
Tue, Sept. 15	*Planning with informed search: A search and heuristics** Informed search methods Heuristics Greedy search A* search Graph search	[slides] Informed search
Thu, Sept. 17	Game playing and adversarial search Types of games Adversarial search, minimax Alpha Beta pruning	[slides] Adversarial search
Tue, Sept. 22	Game playing and adversarial search (cont'd)
Thu, Sept. 24	Game playing and adversarial search (cont'd) The problem of depth Evaluation functions
Tue, Sept. 29	Expectimax search Expectimax search	[slides] Expectimax search
Thu, Oct. 1	State of the art in game play Monte Carlo tree search AlphaGo AlphaGo Zero Real life games	[slides] Game play state of the art
Tue, Oct. 6	Utilities and rationality Utilities Axioms of rationality Rewards	[slides] Utilities and rationality
Thu, Oct. 8	Markov decision processes Defining MDPs: policies and utilities Optimal policy, value of state, value of Q-state	[slides] Markov Decision Processes
Tue, Oct. 13	Markov decision processes 2 Value iteration
Thu, Oct. 15	Midterm - Introduction through Rationality (not including MDPs)
Tue, Oct. 20	Markov decision processes 3 Policy extraction Policy iteration
Thu, Oct. 22	Reinforcement learning (cont'd) Exploration vs. exploitation, regret Generalization across states
Tue, Oct. 27	Reinforcement learning Reinforcement learning as a twist on MDPs Model-based and model-free learning Temporal difference learning	[slides] Reinforcement learning
Thu, Oct. 29	Deep reinforcement learning Challenges with Q estimation The "deadly triad" DQN Double Q-learning
Tue, Nov. 3	Policy gradient RL Policy gradient theorem Using baselines	[slides] Policy gradient RL
Thu, Nov. 5	Reinforcement learning state of the art Soft actor-critic (SAC) Proximal policy optimization (PPO) Application: reinforcement learning from human feedback (RLHF)	[slides] RL State of the Art
Tue, Nov. 10	Imitation learning Behavior cloning Inverse reinforcement learning	[slides] Imitation learning
Thu, Nov. 12	Imitation learning (cont'd)
Tue, Nov. 17	Hidden Markov models and applications Hidden Markov models Example: robot localization Particle filters Robot localization with particle filters	[slides] Hidden Markov Models
Thu, Nov. 19	Artificial General Intelligence - Definitions and Tests Intelligence tests (Turing, Loebner prize, Winograd schema) Artificial Superintelligence Alignment problems LLMs	[slides] AGI definitions and tests
Tue, Nov. 24	Thanksgiving break (Nov. 23–27) - no class
Thu, Nov. 26	Thanksgiving break (Nov. 23–27) - no class
Tue, Dec. 1	Societal implications of AI Economic impact Societal changes Existential threat Ethical issues	[slides] Societal implications of AI
Thu, Dec. 3	Societal implications of AI (cont'd)
Thu, Dec. 10 10:00 AM – 12:50 PM	Final exam