FCHomeProjects
dark mode
*
click to change color
← Back to Projects

Flappy Bird DQN

Deep Q-Learning Neural Network

View on GitHub

A reinforcement learning agent that masters Flappy Bird using Deep Q-Learning (DQN), an off-policy algorithm that combines Q-learning with deep neural networks.

The agent learns through trial and error, receiving rewards for surviving and passing pipes, gradually improving its policy over thousands of training episodes.

Neural Network Architecture

Algorithm:
Deep Q-Network (DQN)
Input (State Vector):
6 continuous features:
  • Bird vertical position
  • Velocity
  • Distance to next pipe
  • Pipe gaps
  • Additional environmental features
Output:
2 Q-values for actions: flap or no flap
Structure:
  • Input layer: 6 neurons
  • Hidden layer 1: 128 neurons, ReLU activation
  • Hidden layer 2: 128 neurons, ReLU activation
  • Output layer: 2 neurons (linear activation for Q-values)
Total Parameters:
20,000 to 30,000

Training Mechanism

Method:
Deep Q-Learning (off-policy reinforcement learning)
Key Components:
  • Replay buffer: Stores experience tuples (state, action, reward, next_state, done)
  • Sampling: Mini-batches randomly drawn from buffer to decorrelate samples
  • Target network: Updated periodically to stabilize learning
  • Loss function: Mean squared error between predicted and target Q-values

Loss Function

L = (r + γ × max_a' Q_target(s', a') − Q_policy(s, a))²
rReward at the current step
γDiscount factor (typically 0.99)
Q_targetTarget network's estimate of the next state's Q-values
Q_policyMain network's predicted Q-value for the current state-action pair

Training Parameters

Optimizer:
Adam
Discount Factor (γ):
0.99
Exploration Policy:
Epsilon-greedy
Epsilon Range:
1.0 → 0.01

Training Loop

  1. Interact with the environment to collect transitions
  2. Store each transition in the replay buffer
  3. Sample a mini-batch from the buffer
  4. Compute the target Q-values and the loss using the formula above
  5. Backpropagate the loss to update network weights
  6. Update the target network every fixed number of steps
  7. Repeat this process for thousands of episodes

Learning Objective

The goal is to maximize expected cumulative rewards. The agent receives small rewards for surviving each frame and larger rewards for passing pipes successfully. Through thousands of training episodes, the network learns an optimal policy that balances short-term survival with long-term score maximization.