The Bitter Lesson
General methods that leverage computation are ultimately the most effective, and by a large margin.
This essay argues that, across 70 years of AI research, the methods that ultimately succeeded were those that scaled computation - via search and learning - rather than those that encoded human knowledge about a domain. Time after time, from computer chess to speech recognition to computer vision, researchers initially tried to build their own understanding into their systems, only to be outperformed once massive computation was applied through general-purpose methods. The bitter lesson is that, when addressing a problem, writing a solution that relies on domain-specific knowledge does not work in the long run. Instead, we should build in the meta-methods that can find and capture complexity on their own. Compare this law to Moore’s Law, which underpins the ever-increasing availability of computation that makes this approach viable.