New concept: reward hacking, where optimizing a flawed reward function can hurt performance.
problem Optimizing imperfect reward functions leads to poor performance.
method Formal definition and analysis of reward hacking, examining conditions for unhackability.
result Reward functions are usually hackable, making it hard to align AI with human values.