Twin Prime Conjecture: Zhang’s Breakthrough and the 246 Gap Record

On April 17, 2013, the journal Annals of Mathematics received an email containing a 50-page proof related to one of mathematics' oldest unsolved problems, the twin prime conjecture. This proof came from an unknown individual, Yitang Zhang, who had previously worked at a Subway restaurant. The journal's referees, expecting to quickly find a mistake, instead realized they were witnessing a significant breakthrough.

The Twin Prime Conjecture

The twin prime conjecture posits that there are infinitely many pairs of prime numbers separated by just one number (e.g., 11 and 13, or 17 and 19). While primes and twin primes become rarer as numbers increase, the conjecture suggests they never run out.

One way to approach this problem is by examining the gaps between consecutive primes. Although these gaps appear chaotic, their average size grows roughly as the natural logarithm of the number N. For instance, the average gap around 100 is about 4.6, and around 1000, it's 6.9. This increasing average gap is not encouraging for finding primes that are always two apart. However, twin primes have been found at extremely large scales, such as a pair each 388,342 digits long. While these discoveries are impressive, they don't prove the conjecture, as one cannot physically check numbers up to infinity.

Early Attempts: Hardy and Littlewood's Heuristic

Around a century ago, mathematicians began developing more sophisticated methods. In 1923, English mathematicians Hardy and Littlewood devised a way to estimate the number of twin primes. They started with the prime number theorem, which states that the probability of a large number N being prime is approximately 1/ln(N).

To estimate the probability of both N and N+2 being prime, they multiplied their individual probabilities: (1/ln(N)) * (1/ln(N+2)). For large N, N+2 is insignificant, so the probability simplifies to approximately 1/ln(N)^2. To count all twin primes up to N, they integrated this expression. Hardy and Littlewood refined this by adding a correction factor, resulting in an estimate that closely matches the actual count of twin primes up to 1 trillion, with an error of only 0.001%.

However, this was merely an estimate, or a heuristic. It couldn't guarantee an infinite supply of twin primes. As mathematician Terry Tao noted, there could be a "conspiracy" where a prime N prevents N+2 from also being prime. A rigorous, airtight mathematical proof was needed.

Viggo Brun's Sieve and the Error Term Problem

One of the first mathematicians to seriously tackle this was Viggo Brun in the early 20th century. Brun aimed to prove that the count of twin primes always increases. He adapted the 2000-year-old Sieve of Eratosthenes for twin primes.

The Sieve of Eratosthenes identifies primes by iteratively removing multiples of primes. For example, to find primes up to 100, one removes 1, then multiples of 2, then multiples of 3, and so on. Brun's innovation was to count how many numbers were not crossed out. This involves an "inclusion-exclusion" principle, where numbers crossed out multiple times are added back in.

For twin primes, Brun's sieve had to remove numbers where N is divisible by a prime P, and also where N+2 is divisible by P. This meant that for each prime P, the twin prime sieve removed about two numbers, compared to one for the ordinary sieve. This change led to a predicted count of twin primes that also grew roughly as N/ln(N)^2, similar to the heuristic.

The problem, however, lay in controlling the "error terms" introduced by rounding in the inclusion-exclusion process. For the normal sieve, the error grows roughly as 2^K (where K is the number of primes sieved). For the twin prime sieve, it grows even faster, approximately 4^K. These error terms quickly dominated the main term, making it impossible to prove that the lower bound for twin primes remained positive and growing.

Brun's eventual realization was that by weakening the sieve—not sieving all the way up to the square root of N, but to a smaller limit like N^(1/10)—he could gain control over the error terms. This came at a cost: the sieve would leave "survivors" that weren't prime but had multiple prime factors. Brun proved there are infinitely many pairs of numbers two apart, where each number has at most nine prime factors.

Progress Towards Bounded Gaps

Brun's techniques were refined over decades. By 1973, Chinese mathematician Chen Jingrun proved there are infinitely many primes P where P+2 has at most two prime factors. This was the closest anyone had come to the twin prime conjecture without fully proving it.

Another approach focused on the smallest gap between consecutive primes. The average gap is ln(N). Mathematicians sought to prove that primes sometimes come closer than this average. By 1988, the gap had been reduced to roughly a quarter of the average.

In 2005, Goldston, Pintz, and Yildirim (GPY) made a shocking breakthrough, proving that the gap between primes could be an arbitrarily small percentage of the average gap. This meant the gap could be 1/10th, 1/100th, or even 1/1,000,000th of the average, infinitely often. Their method also came close to proving an absolute bounded gap between two primes.

The GPY Method and the "Wall"

The GPY method involved a "stencil" with holes, representing potential prime positions. By sliding this stencil along the number line and counting primes, they aimed to show that infinitely often, two primes would fall within the stencil's bounded diameter. Since they couldn't check every number, they used an "averaging machine" based on the average distribution of primes.

The key was to get the average number of primes caught by the stencil to be greater than one. If the average was above one, it guaranteed that at least one position caught two primes. To improve the average, GPY introduced a "weighted averaging machine," assigning weights to starting positions based on their likelihood of containing primes.

With this, they could get the weighted average arbitrarily close to one, but not above it. The "wall" they hit was due to their reliance on a theorem about prime distribution in arithmetic progressions, which only worked for step sizes up to X^(1/2). This limit, known as the "level of distribution" or theta, meant their maximum weighted average was two times theta, which was always less than one. They showed that if they could push past this 1/2 barrier, even by a tiny fraction (e.g., 0.5000001), they could achieve bounded gaps.

In 2005, a meeting of leading experts, including GPY, concluded that pushing past the 1/2 barrier was impossible.

Yitang Zhang's Breakthrough

Yitang Zhang, an unknown mathematician who had struggled to find academic employment and worked odd jobs, spent years in isolation studying number theory. In 2010, he focused on bounded gaps between primes.

Zhang spent two years internalizing the GPY argument. In the summer of 2012, while on a break, he had a sudden insight. He realized that GPY had to count primes in many different arithmetic progressions, but by focusing on a special class of step sizes (those built from small prime factors), he could reorganize the error terms such that most of them canceled out. This allowed him to push past the 1/2 barrier by a tiny fraction: 1/584.

Zhang finalized his proof and sent it to the Annals of Mathematics. The referees, initially skeptical, were astonished to find his proof was correct. Zhang's stencil had 3.5 million slots across a span of 70 million, and by proving two of those slots always catch primes, he proved a bounded gap of 70 million. This news shocked the mathematical community, as Zhang was an unknown figure. He was later awarded the MacArthur Genius Grant.

Further Refinements and James Maynard's Contribution

After Zhang's breakthrough, the mathematical community, led by Terence Tao's Polymath group, optimized Zhang's method. The upper bound for the gap rapidly decreased, eventually reaching 4,680.

Meanwhile, James Maynard, a young postdoc, developed a completely independent approach to the problem. Despite his advisor's skepticism, Maynard's method proved successful, further reducing the gap to 600. Crucially, Maynard's method also showed that the 1/2 exponent limit was a "mirage" and not a fundamental barrier. His average grew roughly as theta/2 * ln(K) (where K is the number of slots), meaning he only needed enough slots to make it work, regardless of the specific value of theta.

Tao and Maynard discovered they had similar ideas. Tao, a Fields medalist, encouraged Maynard to take credit for the work. By early 2014, Maynard joined the Polymath group, and together they further refined the methods. The current world record for the smallest proven bounded gap between primes is 246.

In 2022, James Maynard was awarded the Fields Medal for his work on prime gaps.

The Future of the Twin Prime Conjecture

The story of Zhang and Maynard highlights how a problem once deemed impossible can be cracked. It's akin to the four-minute mile, where once one person achieved it, many others followed.

Mathematicians have found ways to reduce the gap even further, but these results are conditional. For example, if the Elliot-Halberstam conjecture (which assumes primes are evenly spread across arithmetic progressions, or that the level of distribution can be taken as large as one) is true, Maynard showed the gap plummets to 12. A stronger version of this conjecture would reduce the gap to six. However, without assuming any conjectures, the record remains 246.

The question of whether humanity will eventually solve the twin prime conjecture remains open. While some believe it will be solved, the uncertainty has driven significant innovation and new mathematical methods over the past century.