In reinforcement learning, an agent must explore the action space on its own to learn which actions are good (yields a higher reward) and which aren’t.
Take a robot with one hour of battery, trying to clean a house, here reward is the amount of dirt it collects in 1hr. It can exploit a known action, like going to the kitchen, since the kitchen is always dirty and guarantees a decent reward. Or it can explore other parts of the house, where the reward might be lower, or it might find a corner of the bedroom dirtier than the kitchen and worth a much higher reward.
This is the essence of exploration versus exploitation.
As humans navigating life, we’re not all that different from an RL agent. We explore parts of life (action spaces), things like taking a music class or going to the gym, to find the action (or policy) that works best for us, then try to exploit it.
Often times, people only ever exploit what they know and never explore.
Try exploring more often.
Take something as simple as a restaurant. Instead of ordering the dish you always like, try something you’ve never had and see how it tastes.
The same applies to your career. Instead of sticking to a job you know and feel comfortable in, take one that’s messy and challenging, and explore it. There might be a reward waiting there greater than anything you’ve found in past jobs.
Explore because the goal post is moving.
This matters even more once you realize life is a non-stationary environment, one where the goalposts shift as you age.
For example, a job in your twenties might pay well but demand a lot from you and eat into your personal time. That same job might not work in your thirties, when you have a family and need time for yourself and the people you love, and work becomes your second priority. If you keep exploiting the same action instead of exploring, you’ll lose out on what that new season of life actually needs from you.
Exploration isn’t free.
Every time an agent explores, it risks a lower reward than it would’ve gotten by exploiting a known good action. The same is true in life. Not every new job, hobby, or dish will work out. Sometimes the reward is low, and the exploration simply doesn’t pay off. That’s a cost you have to be ready to bear.
This is also why exploring early in life is much easier than exploring later on. When you’re not yet deep inside an action space, with no one depending on you, quitting a job and trying something completely different is a fairly low-cost move. The deeper you go into one action, the more exploring away from it costs you.
Take smaller bets.
Instead of committing to a new career outright, give yourself a six-month or even a one-month trial, and properly explore that action space within that window. In other situations maybe much smaller bets work too: spending a weekend somewhere you don’t usually enjoy is also exploration, and it only costs you a day or two. If it doesn’t go well, you’ve lost very little, but learned something you didn’t know before.
Thinking about life through the lens of exploration versus exploitation gives you a basis to keep learning, and to keep flowing into action spaces that might turn out highly rewarding.