Welcome to my blog! I’m using this space to explore one of the most nebulous questions in reinforcement learning: Why is RL so hard?
There’s a lot of very broad answers to this question, but more specific clarity is often hard to find, and a lot of the challenges are application-specific. My goal in writing this blog is to explore some of the fundamental challenges that come up in the practice of RL, and to look broadly at ways of addressing them (above the resolution of a specific research paper).
I’m aiming for about 1 post per month (knock on wood), looking at different angles of that question. In writing I’m trying to keep the content accessible to anyone who is broadly familiar with the high-level concepts of RL (policies, value functions, learning from experiences, etc), which means I’m avoiding heavy equations or some of the denser jargon from the field.
If this sounds interesting to you, take a look below- I hope you find it interesting!
Latest Posts
Why is Reinforcement Learning Hard: Multitask Learning
While a great deal of focus in modern machine learning is on improving performance on well-defined single tasks such as Imagenet classification, even the simplest animal must learn to perform multiple “tasks.” While some machine learning tasks can be learned and applied in isolation, most of the applications we imagine using reinforcement learning for (robotic…
Why is Reinforcement Learning Hard: Generalization
Anyone who is passingly familiar with reinforcement learning knows that getting an RL agent to work for a task, whether a research benchmark or a real-world application, is difficult. Further, there’s no one reason for this, and the causes span both the practical and theoretical. In this post, I’m gonna go into detail on one…
Detecting Bottlenecks in Deep RL, Part 2: Sampling
This blog post is about methods for detecting bottlenecks in RL agents, a topic which (to my knowledge) hasn’t been explicitly described anywhere before. The goal is to define metrics of an agent’s performance which can be used by a human researcher to diagnose which factors are performance-limiting so they can be improved upon via…
Detecting Bottlenecks in Deep Reinforcement Learning, Part 1
Today I’m starting a series of posts on learning bottlenecks in deep reinforcement learning. In this post I’ll describe the problem and some examples of how it expresses itself, with follow-on posts that go into more details on bottlenecks in specific aspects of an RL agent and how they can be resolved.
Follow My Blog
Get new content delivered directly to your inbox.
Riley Simmons-Edler 