ExploreRL
A SwiftUI app to help learn about and understand reinforcement learning. Experiment with environments, visualize learning signals, and explore different algorithms. ExploreRL runs on Gymnazo, a native Swift library that brings Gymnasium-style environments and RL utilities to Apple Silicon devices with MLX.
Interactive Training
Train agents on device using robust algorithm implementations. Adjust hyperparameters like learning rate, gamma, epsilon, and batch sizes to see how they directly affect learning performance.
Live Visualization
Watch your agent learn in real-time. ExploreRL can render the environment state alongside live charts displaying episode rewards, value estimates, and policy metrics.
Save Session
Stop training at any time. Saved sessions archive the environment ID, algorithm state, hyperparameters, and checkpoints (network weights/Q-tables). Export and share them anywhere via .xrlsession files.
Explore
A dedicated in-app educational Hub breaking down the math and logic behind the RL concepts used within the app.
How It Works
Configure the Environment
Choose an environment from the library and choose from available RL algorithms. Customize environment settings and tweak hyperparameters.
Train & Watch
Initiate the loop. The agent will start training with live rendering and provide charts mapping performance over time.
Save Session
Save your run to the Library. This safely archives the neural network weights (or Q-Table) and the environment configurations and can be shared with others.
Evaluate Policy
Reload saved sessions into the Evaluate tab to let the agent act purely on its learned policy.
Changelog
- Fixed Import button on macOS not showing list
- Setting added for training multiple environments
- Added import/export progress
- Add training indicators
- Fix styling in Settings page
- Rename saved agents
- Open .xrlsession files automatically
- Better benchmarking, and useful system info
- Fixed issue where exporting agents was not working
- Performance fixes in some environments
- Minor UI fixes
- Initial application supporting iOS 18+ and macOS 15+.
- Trainable and configurable environments for RL experiments.
- Integration with Gymnazo for Classic Control, Toy Text, and Box2D environments.
- Implementation of Q-Learning, SARSA, DQN, and SAC using MLX Swift.
- Real-time rendering, charting, and agent management.
- Learning Hub with learning resources.
- Import and Export agent session files.
Support & Feedback
Have questions, found a bug, or want to request a new environment or algorithm?
Supported Devices
ExploreRL leverages MLX Swift. It currently requires an Apple Silicon device running macOS 15+ or iOS 18+. Devices with newer hardware and more memory will have an easier time training more demanding environments and algorithms.
Tested on MacBook Pro M4 Pro, iPad Pro (M2), and iPhone 17 Pro (A19 Pro).