Finding primes: Difference between revisions

From Polymath Wiki
Jump to navigationJump to search
 
(123 intermediate revisions by 6 users not shown)
Line 1: Line 1:
This is the main blog page for the "[http://en.wordpress.com/tag/finding-primes/ Deterministic way to find primes]" project, which is currently fairly active already, and should be formally launched within a few weeks.
This is the main wiki page for the Polymath4 project "[http://en.wordpress.com/tag/finding-primes/ Finding primes]".


The main aim of the project is to resolve the following conjecture:
The main aim of the project is to resolve the following conjecture:


:'''Conjecture'''. There exists ''deterministic'' algorithm which, when given an integer k, is guaranteed to find a prime of at least k digits in length of time polynomial in k.  You may assume as many standard conjectures in number theory (e.g. the generalised Riemann hypothesis) as necessary, but avoid powerful conjectures in complexity theory (e.g. P=BPP) if possible.
:'''(Strong) conjecture'''. There exists ''deterministic'' algorithm which, when given an integer k, is guaranteed to find a prime of at least k digits in length of time polynomial in k.  You may assume as many standard conjectures in number theory (e.g. the generalised Riemann hypothesis) as necessary, but avoid powerful conjectures in complexity theory (e.g. P=BPP) if possible.


Since primality is known to be in P by the AKS algorithm, we may assume a primality oracle that decides whether any given number is prime in unit time.
Since primality is known to be in [[The complexity class P|P]] by the [[wikipedia:AKS_primality_test|AKS]] algorithm, we may assume a primality oracle that decides whether any given number is prime in unit time.  Weaker versions of the strong conjecture are proposed below.


Here is the [http://polymathprojects.org/2009/07/27/proposal-deterministic-way-to-find-primes/ proposal for the project], which is also the ''de facto'' research thread for that project.  Here is the [http://polymathprojects.org/2009/07/28/deterministic-way-to-find-primes-discussion-thread/ discussion thread for the project].
Contributions and corrections to this wiki page (and its subpages) are very welcome.


Please add and develop this wiki; in order to maximise participation in the project when it launches, we will need as much expository material on this project as we can manage.
== Threads ==
 
# [http://polymathprojects.org/2009/07/27/proposal-deterministic-way-to-find-primes/ Proposal for the project and initial research thread] Opened July 27. Inactive.
# [http://polymathprojects.org/2009/08/09/research-thread-ii-deterministic-way-to-find-primes/ Research thread II] Opened Aug 9.  Inactive.
# [http://polymathprojects.org/2009/08/13/research-thread-iii-determinstic-way-to-find-primes/ Research thread III] Opened Aug 13.  Inactive.
# [http://polymathprojects.org/2009/08/28/research-thread-iv-determinstic-way-to-find-primes/ Research thread IV] Opened Aug 28.  Inactive.
# [http://polymathprojects.org/2009/10/27/research-thread-v-determinstic-way-to-find-primes/ Research thread V] Opened Oct 27.  '''Active'''.
# [http://polymathprojects.org/2009/07/28/deterministic-way-to-find-primes-discussion-thread/ Discussion thread I] Opened July 28.  Inactive.
# [http://polymathprojects.org/2010/06/29/draft-version-of-polymath4-paper/ Discussion thread II] Opened Jun 29.  '''Active'''.


== Easier versions of the problem ==
== Easier versions of the problem ==
Line 19: Line 27:
:'''Weak conjecture''': There exists a deterministic algorithm, which, when given an integer k, is guaranteed to find a prime of at least <math>\omega(\log k)</math> digits in length in time polynomial in k, where <math>\omega(\log k)</math> grows faster than <math>\log k</math> as <math>k \to \infty</math>.  (Equivalently, find a deterministic algorithm which, when given k, is guaranteed to find a prime of at least k digits in time <math>\exp(o(k))</math>.  Note that the semi-weak conjecture implies the weak conjecture).
:'''Weak conjecture''': There exists a deterministic algorithm, which, when given an integer k, is guaranteed to find a prime of at least <math>\omega(\log k)</math> digits in length in time polynomial in k, where <math>\omega(\log k)</math> grows faster than <math>\log k</math> as <math>k \to \infty</math>.  (Equivalently, find a deterministic algorithm which, when given k, is guaranteed to find a prime of at least k digits in time <math>\exp(o(k))</math>.  Note that the semi-weak conjecture implies the weak conjecture).


:'''Conjecture with factoring''': Same as the original conjecture, but assume that factoring is in P (i.e. given a k-digit number, one can compute all of its factors in time polynomial in k).  This is more or less equivalent to assuming one has a ''factoring oracle'' that will return the factors of any number given to it (as a stream of bits) in unit time.   
:'''Strong conjecture with factoring''': Same as the original conjecture, but assume that one has a ''factoring oracle'' will return the factors of any number given to it (as a stream of bits) in unit time.  (This oracle can be given "for free" if factoring is in P, but we can also assume the existence of this oracle even if factoring is not in P.)


:'''Semi-weak conjecture with factoring''': Same as the semi-weak conjecture, but with factoring in P.
:'''Semi-weak conjecture with factoring''': Same as the semi-weak conjecture, but with a factoring oracle.


:'''Weak conjecture with factoring''': Same as the weak conjecture, but with factoring in P.
:'''Weak conjecture with factoring''': Same as the weak conjecture, but with a factoring oracle. (Note that there are deterministic sieve algorithms, such as the quadratic sieve, which are believed to run in subexponential time <math>\exp(o(k))</math>, so this oracle is in some sense available "for free" for this problem assuming this belief.)


The weak conjecture with factoring is the easiest of the six, and so perhaps one should focus on that one first.
The weak conjecture with factoring is the easiest of the six, and so perhaps one should focus on that one first.


# Assume complexity conjectures such as P=BPP, P=BQP, etc.
It is also of interest to see whether the conjecture becomes easier to solve assuming that P=BPP or P=BQP.
# Can one find a degree k irreducible polynomial over the rationals deterministically and quickly?


== Partial results ==
== Partial results ==
Line 34: Line 41:
# [[P=NP implies a deterministic algorithm to find primes]]
# [[P=NP implies a deterministic algorithm to find primes]]
# [[An efficient algorithm exists if Cramer's conjecture holds]]
# [[An efficient algorithm exists if Cramer's conjecture holds]]
# Problem is true if strong pseudorandom number generators exist (explain!)
# [[Finding primes with O(k) random bits|k-digit primes can be found with high probability using O(k) random bits]]
# k-digit primes can be found with high probability using O(k) random bits (explain!)
## If [[Pseudo-random generators (PRG)|pseudorandom number generators]] exist, the conjecture holds, as one can derandomise the above algorithm by substituting the pseudorandom number generator for the random string of bits.
# The function field analogue (i.e. finding degree k irreducible polynomials over a finite field) is known (see Adelman-Lenstra)
# The function field analogue (i.e. finding degree k irreducible polynomials over a finite field) is known [AL1986]
# It's easy to deterministically find large square-free numbers; just multiply lots of small primes together
## The analogue over the rationals is even easier: Eisenstein's criterion provides plenty of irreducible polynomials of degree k, e.g. <math>x^k-2</math>.
# Assuming a suitably quantitative version of [[Schinzel's hypothesis H]] (namely, the Bateman-Horn conjecture), the weak conjecture is true, as one can simply search over all k-digit numbers of the form <math>n^d-2</math> (say) for any fixed d to give an algorithm that works in time <math>O(10^{k/d})</math>.  By randomly sampling such integers, one also establishes the semi-weak conjecture (assuming that one has the expected density for primes in hypothesis H).
# It's easy to deterministically find large square-free numbers; just multiply lots of small primes together.  However, it is not known how to test a number for square-freeness deterministically and quickly (unless one has a factoring oracle, in which case it is trivial).
## However, it's not so obvious how to find large pairs <math>n,n+1</math> of consecutive square-free numbers, though such pairs must exist by counting arguments (the density of square-free numbers is <math>6/\pi^2 \approx 60\%</math>).  This could be a good variant problem.
### It is conjectured that every element of [[Sylvester's sequence]] is square-free.  If so, this solves the above problem.
# If a fast algorithm to find primes exists, then an ''explicit'' fast algorithm to find primes exists: simply run all algorithms of size at most A simultaneously for time <math>k^A</math>, then increment A from 1 onward, until one gets a hit.
# If a fast algorithm to find primes exists, then an ''explicit'' fast algorithm to find primes exists: simply run all algorithms of size at most A simultaneously for time <math>k^A</math>, then increment A from 1 onward, until one gets a hit.
Here are the current "world records" for the fastest way to deterministically (and provably) obtain a k-digit prime (ignoring errors of <math>\exp(o(k))</math> in the run time):
# <math>O(10^k)</math>: The trivial algorithm of testing each of the k-digit numbers in turn until one gets a hit.
# <math>O(10^k)^{3/4}</math>: Test the k-digit numbers of the form <math>a^2+b^4</math> for primality (the [[Friedlander-Iwaniec theorem]] guarantees a hit for k large enough).
# <math>O(10^k)^{2/3}</math>: Test the k-digit numbers of the form <math>a^3+2b^3</math> for primality (a hit is guaranteed for large k by a result of Heath-Brown).
# <math>O(10^k)^{2/3}</math>: Use the [[Meissel-Lehmer method]] for computing <math>\pi(x)</math>
# <math>O(10^{k})^{6/11+\epsilon} (\approx O(10^{k})^{.5454})</math>assuming factoring oracle: Factor each number in the interval <math>[x^{12/11},x^{12/11}+x^{6/11+\epsilon}]</math>, one will find a prime factor of order x by [HB1995].
# <math>O(10^k)^{0.525}</math>: Test the k-digit numbers from, say, <math>10^k</math> onwards until one gets a hit (here we use the [BHP2001] bound on [[prime gaps]]).
# <math>O(10^k)^{1/2}</math> assuming RH: Test the k-digit numbers from, say, <math>10^k</math> onwards until one gets a hit (here we use the RH bound on [[prime gaps]]).
# <math>O(10^k)^{1/2}</math> using [[Odlyzko's method]].
Note that if one can get <math>O(10^k)^{\varepsilon}</math> for each fixed <math>\varepsilon > 0</math>, then one has established the weak conjecture.
A slight variant of the problem: assuming a factoring oracle, given an algorithm that runs in k steps, how large a prime is the algorithm ''guaranteed'' to produce in the ''worst-case'' scenario?  (Note that this is very different from what the algorithm is ''heuristically'' predicted to produce in the ''average-case'' scenario.)
Here is a simple algorithm that produces a prime larger than about <math>k \log k</math> in k steps, based on Euclid's proof of the infinitude of primes:
* Initialise <math>p_1=2</math>.
* Once <math>p_1,\ldots,p_{k-1}</math> are computed, define <math>p_k</math> to be the largest prime factor of <math>p_1 \ldots p_{k-1}+1</math>.
This generates k distinct primes; the prime number theorem tells us that the k^th prime has size about <math>k \log k</math>, so at least one of the primes produced here has size at least <math>k \log k</math>.
If one instead works with [[Sylvester's sequence]] <math>a_k = a_1 \ldots a_{k-1}+1</math>, then it is known [O1985] that the number of primes less than n that can divide any one of the <math>a_k</math> is <math>O( n / \log n \log \log \log n )</math> rather than <math>O(n/\log n)</math> (the prime number theorem bound).  If we then factor the first k elements of this sequence, we must get a prime of size at least <math>k \log k \log \log \log k</math> or so.
One can do better still by working with the Fermat numbers, <math>F_n = 2^{2^n}+1</math>. It is known [KLS2002] that the number of primes dividing any of the <math>F_n</math> is <math>O(\sqrt{n}/\log n)</math>, so in particular if we factor the first k Fermat numbers, we get a prime almost of size <math>k^2</math>. In fact, it's a classical result of Euler that the prime divisors of <math>F_n</math> are all at least <math>2^{n+2}+1</math>, so we can obtain a large prime simply by factoring a Fermat number.
Assuming RH, one can find a prime larger than <math>k^2</math> in about k steps by the method indicated earlier (start at <math>k^2</math> and increment until one hits a prime).  Unconditionally, one gets a prime larger than <math>k^{1/0.525}</math> by this method.


== Variants of the problem ==
== Variants of the problem ==


# Can one derandomise factoring of polynomials over finite fields?  Note that if one could do this even in the quadratic case, this would allow for efficient location of quadratic non-residues, which is not known.
# Can one derandomise factoring of polynomials over finite fields?  Note that if one could do this even in the quadratic case, this would allow for efficient location of quadratic non-residues, which is not known.
=== Finding consecutive (or nearby) square-free integers ===
:'''Toy problem''' Is there an efficient algorithm to find consecutive square-free integers n, n+1 of at least k digits in length assuming an oracle to determine whether a number is square free (a special case of the factoring oracle)?  Or just square-free integers that are close together (e.g. polylog(n)).
Note that finding one large square-free number is easy: just multiply lots of distinct small primes together. Also, as the density of square-free numbers is <math>6/\pi^2 \approx 0.6</math>, a counting argument shows that pairs of consecutive square-free numbers exist in abundance, and are thus easy to find probabilistically. The density can be increased to arbitrarily close to 1 using the [[W-trick]], so one could also ask for, say, long arithmetic progressions of square-free numbers also.
If [[Sylvester's sequence]] is square-free, then this solves the problem.
Testing a number for square-freeness is trivial with a factoring oracle. No unconditional polynomial-time deterministic algorithm for square-freeness seems to be known.  (It is listed as an open problem in [http://portal.acm.org/citation.cfm?id=749544 this paper from 1994].)
Unconditionally, there are asymptotics for the number of square-free numbers in [1,N] with an error term of <math>O(N^{1/2})</math>, which leads to a <math>O(10^k)^{1/2}</math> algorithm (just search an interval of this length for square-freeness). This is the current unconditional world record. Assuming RH, this can be improved to <math>O(10^k)^{17/54+\varepsilon}</math>. 
There is an asymptotic [HB1984] for the number of consecutive square free numbers with an error term of <math>O(N^{7/11})</math>, leading to a <math>O(10^k)^{7/11+\varepsilon}</math>, though this is inferior to the above bounds.
It is known [FT1992] that the interval <math>[N, N+N^{1/5+\epsilon}]</math> contains at least one square-free number; Granville [G1998] improved this to <math>[N,N+N^{\varepsilon}]</math> assuming the [[ABC conjecture]].  If the density in such intervals can be made to exceed 50% (perhaps after first applying the [[W-trick]]) then we would have solved the weak version of this toy problem assuming ABC.
=== Finding pseudoprimes ===
:'''Problem:''' How quickly can one find a solution to the congruence <math>2^{n-1}=1 \hbox{ mod } n</math> with n at least k digits in length? (a pseudoprime)?
It is conjectured that the largest gap between k-digit pseudoprimes is poly(k) (this is weaker than [[Cramer's conjecture]], and would imply a polynomial time solution to the above problem).
It is conjectured that the number <math>\mathcal{P}_a(x)</math> of pseudoprimes to any base <math>a \neq \pm1</math> satisfies
:<math>\displaystyle \log\mathcal{P}_a(x) = \log(x) - \log(x)\frac{\log\log\log(x)}{\log\log(x)}(1+o(1)).</math>
However, the best known bounds seem to be
:<math>\displaystyle \log\mathcal{P}_a(x) \leq \log(x) - \frac{1}{2}\log(x)\frac{\log\log\log(x)}{\log\log(x)}(1+o(1)),</math>
and
:<math>\displaystyle \log\mathcal{P}_a(x) \geq (\frac{\log(x)}{\log a})^{E/(E+1)-o(1)},</math>
where E is a constant which is conjectured to equal 1, but is only known to have value at least 1-1/(2\sqrt{e}). These results can be found in [P1981], [P1982], [P1989].  Better bounds are known to hold on average [EP1986].
If n is a pseudoprime, then <math>2^n-1</math> is also a pseudoprime (because <math>2^n = 1 \hbox{ mod } 2^n - 1 </math>, and <math>2^n-2</math> is a multiple of n).  This, technically, solves the problem, but can we find denser sets of pseudoprimes?  What if we want <math>2^{n-1}=1 \hbox{ mod } n</math> and <math>3^{n-1} = 1 \hbox{ mod } n</math> simultaneously?
=== Finding Almost Primes===
An almost primes is a number that is the product of at most two primes (counted with multiplicity).
:'''problem''' How fast can we deterministically write down a <math>k</math> digit almost prime?
Clearly this is a relaxation of the prime problem, so we expect to get better results.  In [L1996] it is shown that the interval <math>[x,x+x^{.436}]</math> contains an almost prime. This gives us a <math>O(10^{k})^{.436}</math> algorithm.  Assuming a factoring oracle, the problems of deterministically finding a <math>k</math> digit almost prime and a <math>k</math> digit prime in polynomial time are equivalent.  If we can find an almost prime in polynomial time we can factor it and get (a slightly smaller) prime.  Conversely, if we can find a prime in polynomial time we can square it to find (a slightly larger) almost prime.  Using our prime technology this should give a <math>O(10^{k})^{.262}</math> algorithm.


== Observations and strategies ==
== Observations and strategies ==


# For the conjecture with factoring, it would suffice to show that every polylog(N)-length interval in [N,2N] contains a number which is not <math>N^\varepsilon</math>-smooth (or <math>\log^{O(1)} N</math> smooth, if one only wants to solve the weak conjecture with factoring).  One way to start approaching this sort of problem is to take iterated product sets of sets of medium sized primes and show that they spread out to fill all short intervals in [N,2N].
# For the conjecture with factoring, it would suffice to show that every polylog(N)-length interval in [N,2N] contains a number which is not <math>N^\varepsilon</math>-smooth (or <math>\log^{O(1)} N</math> smooth, if one only wants to solve the weak conjecture with factoring).   
## One can combine this with the "W-trick".  For instance, if one lets W be the product of all the primes less than k, and if one can show that not every element of the progression <math>\{ Wn+1: 1 \leq n \leq k^{100} \}</math> (say) is <math>k^A</math>-smooth, then one has found a prime with at least <math>A \log k</math> digits in polynomial time.  If one can make A arbitrarily large, one has solved the baby problem with factoring.
## One can combine this with the "[[W-trick]]".  For instance, if one lets W be the product of all the primes less than k, and if one can show that not every element of the progression <math>\{ Wn+1: 1 \leq n \leq k^{100} \}</math> (say) is <math>k^A</math>-smooth, then one has found a prime with at least <math>A \log k</math> digits in polynomial time.  If one can make A arbitrarily large, one has solved the baby problem with factoring.
# If a primality test can be expressed as a constant-degree polynomial over a finite field from the digits of the number-under-test to {0,1}, then known [http://en.wikipedia.org/wiki/Pseudorandom_generators_for_polynomials pseudorandom generators for polynomials] can be used to derandomize finding primes the same way as under the full P=BPP assumption. So, instead of requiring a general PRG, the proposal is trying to put the primality test into a fully derandomizable subclass of P.
## One way to proceed is to analyse [[iterated sumsets of log-primes]].
## Another is to analyse [[signed sums of prime reciprocals]].
## Probably the best known bound along these lines is given in [HB1995] and states that the interval <math>[x,x+x^{.5+\epsilon}]</math> contains an integer with a prime factor at least <math>x^{11/12 -\epsilon}</math>.  This only leads to a <math>O(10^{k})^{.5455}</math> algorithm.
# Another approach is to accurately compute the [[prime counting function]].
## A variant of this strategy is the [[polynomial strategy]].
# If one can find explicitly enumerable sets of k-digit numbers of size <math>O(10^{\varepsilon k})</math> for any fixed <math>\varepsilon > 0</math> that are guaranteed to contain a prime, then one has solved the weak conjecture.  If instead one can find randomly samplable sets of k-digit numbers of size  <math>O(10^{\varepsilon k})</math> that contain a large fraction (e.g. <math>\exp(-o(k))</math> of primes), one has solved the semi-weak conjecture.  If one can find explicit sets of poly(k) size that are guaranteed to contain a prime, one has solved the strong conjecture.
# If a primality test can be expressed as a constant-degree polynomial over a finite field from the digits of the number-under-test to {0,1}, then known [[wikipedia:Pseudorandom_generators_for_polynomials|pseudorandom generators for polynomials]] can be used to derandomize finding primes the same way as under the full P=BPP assumption. So, instead of requiring a general PRG, the proposal is trying to put the primality test into a fully derandomizable subclass of P.
# If one can find a sequence of easily constructible, mutually coprime integers whose prime factors are contained in a sparse subset of the primes, this generates (with the aid of a factoring oracle) a sparse set of primes.  So far the sparsest set of this type has come from factoring the Fermat numbers.
 
== Negative results and other obstructions ==
 
# Using sieve theory to find a non-smooth number in a short interval runs into the difficulty that most sieve theory methods are insensitive to the starting point of an interval.  Given that every integer in [1,n] is n-smooth, this implies that sieve theory is unlikely to establish the existence of a non-n-smooth number in any interval of length n, and in particular to find a non-<math>O(k^O(1))</math> smooth integer in any interval that can be searched in polynomial time. Indeed, there are heuristic arguments that suggest that even augmented with basic Fourier analysis and assuming RH, this method is unlikely to work, although they haven't yet been fully formalized.
# Average-case results for the primes cannot be used directly, because one could delete all constructible integers from the set of primes (a negligible set) without significantly impacting the average case results, while causing the conjecture to fail for this modified set of primes.
## In a similar spirit, one cannot hope to find non-smooth numbers in a short interval by analysing the iterated sumset of the log-primes and using average-case information about that set), because if one deletes all the primes arising from factors of that interval, one still captures most of the primes (and thus most of the average-case behaviour of the log-primes) while losing the information about that interval.
# It is not possible to prove the conjecture purely by "soft" (i.e. relativisable) complexity theory methods which do not use any additional information on the primes rather than their density.  Details are at [[Oracle counterexample to finding pseudoprimes]].


== How could the conjecture be false? ==
== How could the conjecture be false? ==
Line 57: Line 156:
# In fact, all constructible numbers are not only composite/smooth, but they sit inside intervals of length <math>k^{100}</math>, all of whose elements are composite/smooth.  (One can replace 100 by any other fixed number.)
# In fact, all constructible numbers are not only composite/smooth, but they sit inside intervals of length <math>k^{100}</math>, all of whose elements are composite/smooth.  (One can replace 100 by any other fixed number.)
## In particular, Cramer's conjecture fails; the gaps between adjacent k-digit primes exceeds <math>k^{100}</math> around every constructible number.
## In particular, Cramer's conjecture fails; the gaps between adjacent k-digit primes exceeds <math>k^{100}</math> around every constructible number.
# No pseudorandom number generators exist.
# No pseudorandom number generators exist.  In particular, this implies that one-way functions do not exist.
# <math>P \neq NP</math>.  More specifically, the decision problem "Does there exist a prime in a given interval?" is in NP, but cannot lie in P if the conjecture fails.
# <math>P \neq NP</math>.  More specifically, the decision problem "Does there exist a prime in a given interval?" is in [[The complexity class NP|NP]], but cannot lie in [[The complexity class P|P]] if the conjecture fails.


== Relevant concepts ==
== Relevant concepts ==
Complexity theory concepts:


# Complexity classes
# Complexity classes
Line 70: Line 171:
## [[The complexity class DTIME|DTIME(f(n))]]
## [[The complexity class DTIME|DTIME(f(n))]]
# [[Pseudo-random generators (PRG)]]
# [[Pseudo-random generators (PRG)]]
# Impagliazzo's five worlds: Algorithmica, Heuristica, Pessiland, Minicrypt, Cryptomania
# [http://blog.computationalcomplexity.org/2006/07/full-derandomization.html Full derandomization]
# Oracles
# [[wikipedia:One-way_function|one-way function]]
# Expander graphs
# [http://cseweb.ucsd.edu/~russell/average.ps Impagliazzo's five worlds]: Algorithmica, Heuristica, Pessiland, Minicrypt, Cryptomania
# Cramer's random model for the primes
# [[Kolmogorov complexity]]
# Prime gaps
# [[wikipedia:Oracle_machine|Oracles]]
# Smooth number
 
# The W-trick
Number theory concepts:
# Asymptotic notation
 
# [[Cramer's random model for the primes]]
# [[Prime gaps]]
# [[Generic prime]]
# [[Smooth number]]
# The [[W-trick]]
# [[Sylvester's sequence]]
# [[wikipedia:Siegel_zero|Siegel zero]]
# [[wikipedia:Big_O_notation|Asymptotic notation]]
# [[wikipedia:Fermat_number|Fermat number]]
# [[Pseudoprimes]]
 
Other:
 
# [[wikipedia:Expander_graph|Expander graphs]]
 
Note: articles with external links should eventually be replaced by in-house pages that can provide information that is more dedicated to this project.


== Relevant conjectures ==
== Relevant conjectures ==


# P=NP
# [[The complexity class NP|P=NP]]
# P=BPP
# [[The complexity class BPP|P=BPP]]
# P=promise-BPP
# [[The complexity class promise-BPP|P=promise-BPP]]
# P=BQP
# [[The complexity class BQP|P=BQP]]
# existence of PRG
# [[Pseudo-random generators (PRG)|existence of PRG]]
# existence of one-way functions
# [[wikipedia:One-way_function|existence of one-way functions]]
# whether DTIME(2^n) has subexponential circuits
# whether [[The complexity class DTIME|DTIME(2^n)]] has subexponential circuits
# GRH
# [[wikipedia:Riemann_hypothesis|The Riemann Hypothesis (RH)]]
# the Hardy-Littlewood prime tuples conjecture
## [[wikipedia:Generalized_Riemann_hypothesis|The generalised Riemann Hypothesis (GRH)]]
# the ABC conjecture
# the [[Hardy-Littlewood prime tuples conjecture]]
# Cramer’s conjecture
# the [[ABC conjecture]]
# discrete log in P
# [[Cramer's conjecture]]
# factoring in P.
# [[Schinzel's hypothesis H]]
# [[discrete logarithm|discrete log in P]]
# [[factoring|factoring in P]]
 
== Relevant results ==
 
# [[Brun-Titchmarsh inequality]]
# [[Erdos-Kac theorem]]
# [[Bertrand's postulate]]
# [[Friedlander-Iwaniec theorem]]
# [[List of results implied by the Riemann Hypothesis|Conditional on the Riemann hypothesis]]
 
== Paper ==


(These conjectures should have their own links at some point.)
Here is the [[http://arxiv.org/abs/1009.3956 arXiv copy]] of the research paper, now submitted to Mathematics of Computation.


== Relevant results ==
Here is the [[http://www2.xp-dev.com/sc/browse/86755/ Subversion repository]] of the research paper.


# Brun-Titchmarsh inequality
Here are the [[Polymath4 grant acknowledgments]].
# Erdos-Kac theorem
# Bertrand's postulate ("For every positive integer N, there is a prime between N and 2N" : it gives determinism, but not a polynomial algorithm.)


== Relevant papers ==
== Relevant papers ==


# Adelman, Lenstra, "Finding irreducible polynomials over finite fields"
* [AMcC1994] L. Adelman, K. McCurley, [http://portal.acm.org/citation.cfm?id=749544 Open problems in number theoretic complexity, II], Lecture Notes In Computer Science; Vol. 877, Proceedings of the First International Symposium on Algorithmic Number Theory  291 - 322  (1994)
# R. C. Baker and G. Harman, “The difference between consecutive primes,” Proc. Lond. Math. Soc., series 3, 72 (1996) 261–280.  
* [AL1986] L. Adelman, H. Lenstra, [http://www.math.leidenuniv.nl/~hwl/PUBLICATIONS/1986a/art.pdf Finding irreducible polynomials over finite fields], Proc. 18th Annual ACM Symp. on Theory of Computing (STOC), Berkeley, May 28–30, 350–355.
# P. Dusart, [http://www.unilim.fr/laco/theses/1998/T1998_01.html Autour de la fonction qui compte le nombre de nombres premiers]
* [B1990] E. Bach, Explicit bounds for primality testing and related problems, Math. Comp. 55 (1990), 355-380],
# O. Goldreich, A. Wigderson, [http://www.math.ias.edu/~avi/PUBLICATIONS/MYPAPERS/GW02/gw02.pdf Derandomization that is rarely wrong from short advice that is typically good]
* [BH1996] R. C. Baker and G. Harman, “The difference between consecutive primes,” Proc. Lond. Math. Soc., series 3, 72 (1996) 261–280.  
#A. Granville, [http://www.dartmouth.edu/~chance/chance_news/for_chance_news/Riemann/cramer.pdf "Harald Cramér and the distribution of prime numbers"], Scandinavian Actuarial J. 1 (1995), 12—28.  
*[BH1998] R. C. Baker, G. Harman, Shifted primes without large prime factors, Acta Arith. 83 (1998), no. 4, 331–361
# R. Impagliazzo, A. Wigderson, P = BPP unless E has sub-exponential circuits. In Proceedings of the 29th ACM Symposium on Theory of Computing, pages 220–229, 1997.
* [BHP2001] R. C. Baker, G. Harman and J. Pintz, The difference between consecutive primes, II, Proceedings of the London Mathematical Society 83, (2001), 532–562.
# K. Soundararajan, [http://arxiv.org/abs/math/0606408 The distribution of the primes] (survey)
* [BH1998] A. Balog, T. Wooley, “[http://www.austms.org.au/Publ/JAustMS/V64P2/abs/c30/ On strings of consecutive integers with no large prime factors]”, J. Austral. Math. Soc. Ser. A 64 (1998), no. 2, 266–276
# L. Trevisan, [http://arxiv.org/abs/cs.CC/0601100 Pseudorandomness and Combinatorial Constructions]
* [C2006] T. H. Chan, [http://arxiv.org/abs/math/0502199 Finding almost squares]. Acta Arith. 121 (2006), no. 3, 221--232.
* [D1998] P. Dusart, [http://www.unilim.fr/laco/theses/1998/T1998_01.html Autour de la fonction qui compte le nombre de nombres premiers], 1998
* [E1949] P. Erdős, [http://www.math-inst.hu/~p_erdos/1949-07.pdf On The Converse of Fermat's Theorem]
* [EP1986] P. Erdős and C. Pomerance, On the number of false witnesses for a composite number, Math. Comp. 46 (1986), 259-279.
* [FT????] M. Filaseta, O. Trifonov, [http://books.google.com/books?id=G02moOmuOX4C&pg=PA235&dq=squarefree+numbers#v=onepage&q=squarefree%20numbers&f=false On gaps between squarefree numbers], Analytic number theory: Proceedings of a conference in honor of Paul Bateman
* [FT1992] M. Filaseta and O. Trifonov, On gaps between squarefree numbers. II, J. London Math. Soc. (2), 45 (1992), no. 2, 215-221.
* [GW2002] O. Goldreich, A. Wigderson, [http://www.math.ias.edu/~avi/PUBLICATIONS/MYPAPERS/GW02/gw02.pdf Derandomization that is rarely wrong from short advice that is typically good]
* [G1998] A. Granville, ABC allows us to count squarefrees, Internat. Math. Res. Notices (1998), no. 19, 991–1009.
* [G1995] A. Granville, [http://www.dartmouth.edu/~chance/chance_news/for_chance_news/Riemann/cramer.pdf "Harald Cramér and the distribution of prime numbers"], Scandinavian Actuarial J. 1 (1995), 12—28.  
* [HB1984] D.R. Heath-Brown, The square sieve and consecutive square-free numbers, Math. Ann. 266 (1984), 251-259
* [HB1995] D.R. Heath-Brown, The largest prime factor of integers in an interval, Science in China ???
* [IW1997] R. Impagliazzo, A. Wigderson, P = BPP unless E has sub-exponential circuits. In Proceedings of the 29th ACM Symposium on Theory of Computing, pages 220–229, 1997.
* [J2006] R. Jones, [http://arxiv.org/abs/math/0612415 The density of prime divisors in the arithmetic dynamics of quadratic polynomials]
* [KLS2002] Křížek, Luca and Somer, On the convergence of series of reciprocals of primes related to the Fermat numbers, J. Number Theory 97(2002), 95–112
* [LMO???] J. Lagarias, V. Miller, A. Odlyzko, [http://www.dtc.umn.edu/~odlyzko/doc/arch/meissel.lehmer.pdf Computing <math>\pi(x)</math>: the Meissel-Lehmer method]
* [L1979] H.W. Lenstra, [https://openaccess.leidenuniv.nl/bitstream/1887/3801/1/346_036.pdf Miller's Primality Test], Information Processing Letters, 1979
* [L1996] H. Liu, Almost primes in short intervals. J. Number Theory 57 (1996), no. 2, 303--322.
* [LW1999] H. Liu, J. Wu, Numbers with a large prime factor, Acta Arith. 89 (1999), no. 2, 163–187.
* [M1994] U. Maurer, [http://eprints.kfupm.edu.sa/40655/ Fast Generation of Prime Numbers and Secure Public-Key Cryptographic Parameters], Journal of Cryptology 8 (1994), 123-155
* [O1993] A. Odlyzko, Analytic computations in number theory, Mathematics of Computation 1943–1993: a half-century of computational mathematics (Vancouver, BC, 1993), 451–463.
* [O1985] Odoni, On the prime divisors of the sequence <math>w_n+1 = 1+ w_1 \ldots w_n</math>. J. London Math. Soc. (2), 32(1):1–11, 1985.
* [P????] F. Pappalardi, [http://www.mat.uniroma3.it/users/pappa/papers/allahabad2003.pdf A survey on k-freeness]
* [P1981] C. Pomerance, [http://math.dartmouth.edu/~carlp/PDF/paper32.pdf On the distribution of pseudoprimes, Math. Comp. 37 (1981), 587-593.]
* [P1982] C. Pomerance, [http://math.dartmouth.edu/~carlp/PDF/lower.pdf A new lower bound for the pseudoprime counting function, Illinois J. Math. 26 (1982), 4-9.]
* [P1989] C. Pomerance, Two methods in elementary analytic number theory, Number theory and applications, R. A. Mollin, ed., Kluwer Academic Publishers, Dordrecht, 1989, pp. 135-161.
 
* [S2006] K. Soundararajan, [http://arxiv.org/abs/math/0606408 The distribution of the primes] (survey)
* [S2006b] K. Soundararajan, [http://arxiv.org/abs/math/0605696 Small gaps between prime numbers: The work of Goldston-Pintz-Yildirim]
* [T2006] L. Trevisan, [http://arxiv.org/abs/cs.CC/0601100 Pseudorandomness and Combinatorial Constructions]
* [V1903] Voronoi, Sur un probleme du calcul des fonctions asymptotiques, J. reine angew. Math 126 (1903), 241-282
 
Please help out by adding links or completing references, or adding further references to the list above!


== External links ==
== External links ==


# [http://qwiki.stanford.edu/wiki/Complexity_Zoo Complexity Zoo]
# [http://qwiki.stanford.edu/wiki/Complexity_Zoo Complexity Zoo]
# "[http://blog.computationalcomplexity.org/2009/08/finding-primes.html Finding Primes]", Lance Fortnow, August 4 2009

Latest revision as of 03:18, 14 February 2011

This is the main wiki page for the Polymath4 project "Finding primes".

The main aim of the project is to resolve the following conjecture:

(Strong) conjecture. There exists deterministic algorithm which, when given an integer k, is guaranteed to find a prime of at least k digits in length of time polynomial in k. You may assume as many standard conjectures in number theory (e.g. the generalised Riemann hypothesis) as necessary, but avoid powerful conjectures in complexity theory (e.g. P=BPP) if possible.

Since primality is known to be in P by the AKS algorithm, we may assume a primality oracle that decides whether any given number is prime in unit time. Weaker versions of the strong conjecture are proposed below.

Contributions and corrections to this wiki page (and its subpages) are very welcome.

Threads

  1. Proposal for the project and initial research thread Opened July 27. Inactive.
  2. Research thread II Opened Aug 9. Inactive.
  3. Research thread III Opened Aug 13. Inactive.
  4. Research thread IV Opened Aug 28. Inactive.
  5. Research thread V Opened Oct 27. Active.
  6. Discussion thread I Opened July 28. Inactive.
  7. Discussion thread II Opened Jun 29. Active.

Easier versions of the problem

Semi-weak conjecture: There exists a probabilistic algorithm which, when given an integer k, has probability at least 1/2 (say) of finding a prime of at least k digits in length in time polynomial in k, using o(k) random bits. (Note that it is known that one can do this with O(k) random bits; and if one can use just [math]\displaystyle{ O(\log k) }[/math] random bits, one is done by exhausting over the random bits.)

One way to solve the semi-weak conjecture would be to find an "explorable" set of size [math]\displaystyle{ \exp(o(k)) }[/math] of integers, a significant fraction (at least [math]\displaystyle{ k^{-O(1)} }[/math]) of which are primes at least k digits in length. By explorable, we mean that one has a quick way to produce a reasonably uniformly distributed element of that set using o(k) random bits (e.g. the set might be an interval, arithmetic progression, or other explicitly parameterisable set, or one might have an explicit expander graph on this set).

Weak conjecture: There exists a deterministic algorithm, which, when given an integer k, is guaranteed to find a prime of at least [math]\displaystyle{ \omega(\log k) }[/math] digits in length in time polynomial in k, where [math]\displaystyle{ \omega(\log k) }[/math] grows faster than [math]\displaystyle{ \log k }[/math] as [math]\displaystyle{ k \to \infty }[/math]. (Equivalently, find a deterministic algorithm which, when given k, is guaranteed to find a prime of at least k digits in time [math]\displaystyle{ \exp(o(k)) }[/math]. Note that the semi-weak conjecture implies the weak conjecture).
Strong conjecture with factoring: Same as the original conjecture, but assume that one has a factoring oracle will return the factors of any number given to it (as a stream of bits) in unit time. (This oracle can be given "for free" if factoring is in P, but we can also assume the existence of this oracle even if factoring is not in P.)
Semi-weak conjecture with factoring: Same as the semi-weak conjecture, but with a factoring oracle.
Weak conjecture with factoring: Same as the weak conjecture, but with a factoring oracle. (Note that there are deterministic sieve algorithms, such as the quadratic sieve, which are believed to run in subexponential time [math]\displaystyle{ \exp(o(k)) }[/math], so this oracle is in some sense available "for free" for this problem assuming this belief.)

The weak conjecture with factoring is the easiest of the six, and so perhaps one should focus on that one first.

It is also of interest to see whether the conjecture becomes easier to solve assuming that P=BPP or P=BQP.

Partial results

  1. P=NP implies a deterministic algorithm to find primes
  2. An efficient algorithm exists if Cramer's conjecture holds
  3. k-digit primes can be found with high probability using O(k) random bits
    1. If pseudorandom number generators exist, the conjecture holds, as one can derandomise the above algorithm by substituting the pseudorandom number generator for the random string of bits.
  4. The function field analogue (i.e. finding degree k irreducible polynomials over a finite field) is known [AL1986]
    1. The analogue over the rationals is even easier: Eisenstein's criterion provides plenty of irreducible polynomials of degree k, e.g. [math]\displaystyle{ x^k-2 }[/math].
  5. Assuming a suitably quantitative version of Schinzel's hypothesis H (namely, the Bateman-Horn conjecture), the weak conjecture is true, as one can simply search over all k-digit numbers of the form [math]\displaystyle{ n^d-2 }[/math] (say) for any fixed d to give an algorithm that works in time [math]\displaystyle{ O(10^{k/d}) }[/math]. By randomly sampling such integers, one also establishes the semi-weak conjecture (assuming that one has the expected density for primes in hypothesis H).
  6. It's easy to deterministically find large square-free numbers; just multiply lots of small primes together. However, it is not known how to test a number for square-freeness deterministically and quickly (unless one has a factoring oracle, in which case it is trivial).
    1. However, it's not so obvious how to find large pairs [math]\displaystyle{ n,n+1 }[/math] of consecutive square-free numbers, though such pairs must exist by counting arguments (the density of square-free numbers is [math]\displaystyle{ 6/\pi^2 \approx 60\% }[/math]). This could be a good variant problem.
      1. It is conjectured that every element of Sylvester's sequence is square-free. If so, this solves the above problem.
  7. If a fast algorithm to find primes exists, then an explicit fast algorithm to find primes exists: simply run all algorithms of size at most A simultaneously for time [math]\displaystyle{ k^A }[/math], then increment A from 1 onward, until one gets a hit.

Here are the current "world records" for the fastest way to deterministically (and provably) obtain a k-digit prime (ignoring errors of [math]\displaystyle{ \exp(o(k)) }[/math] in the run time):

  1. [math]\displaystyle{ O(10^k) }[/math]: The trivial algorithm of testing each of the k-digit numbers in turn until one gets a hit.
  2. [math]\displaystyle{ O(10^k)^{3/4} }[/math]: Test the k-digit numbers of the form [math]\displaystyle{ a^2+b^4 }[/math] for primality (the Friedlander-Iwaniec theorem guarantees a hit for k large enough).
  3. [math]\displaystyle{ O(10^k)^{2/3} }[/math]: Test the k-digit numbers of the form [math]\displaystyle{ a^3+2b^3 }[/math] for primality (a hit is guaranteed for large k by a result of Heath-Brown).
  4. [math]\displaystyle{ O(10^k)^{2/3} }[/math]: Use the Meissel-Lehmer method for computing [math]\displaystyle{ \pi(x) }[/math]
  5. [math]\displaystyle{ O(10^{k})^{6/11+\epsilon} (\approx O(10^{k})^{.5454}) }[/math]assuming factoring oracle: Factor each number in the interval [math]\displaystyle{ [x^{12/11},x^{12/11}+x^{6/11+\epsilon}] }[/math], one will find a prime factor of order x by [HB1995].
  6. [math]\displaystyle{ O(10^k)^{0.525} }[/math]: Test the k-digit numbers from, say, [math]\displaystyle{ 10^k }[/math] onwards until one gets a hit (here we use the [BHP2001] bound on prime gaps).
  7. [math]\displaystyle{ O(10^k)^{1/2} }[/math] assuming RH: Test the k-digit numbers from, say, [math]\displaystyle{ 10^k }[/math] onwards until one gets a hit (here we use the RH bound on prime gaps).
  8. [math]\displaystyle{ O(10^k)^{1/2} }[/math] using Odlyzko's method.

Note that if one can get [math]\displaystyle{ O(10^k)^{\varepsilon} }[/math] for each fixed [math]\displaystyle{ \varepsilon \gt 0 }[/math], then one has established the weak conjecture.

A slight variant of the problem: assuming a factoring oracle, given an algorithm that runs in k steps, how large a prime is the algorithm guaranteed to produce in the worst-case scenario? (Note that this is very different from what the algorithm is heuristically predicted to produce in the average-case scenario.)

Here is a simple algorithm that produces a prime larger than about [math]\displaystyle{ k \log k }[/math] in k steps, based on Euclid's proof of the infinitude of primes:

  • Initialise [math]\displaystyle{ p_1=2 }[/math].
  • Once [math]\displaystyle{ p_1,\ldots,p_{k-1} }[/math] are computed, define [math]\displaystyle{ p_k }[/math] to be the largest prime factor of [math]\displaystyle{ p_1 \ldots p_{k-1}+1 }[/math].

This generates k distinct primes; the prime number theorem tells us that the k^th prime has size about [math]\displaystyle{ k \log k }[/math], so at least one of the primes produced here has size at least [math]\displaystyle{ k \log k }[/math].

If one instead works with Sylvester's sequence [math]\displaystyle{ a_k = a_1 \ldots a_{k-1}+1 }[/math], then it is known [O1985] that the number of primes less than n that can divide any one of the [math]\displaystyle{ a_k }[/math] is [math]\displaystyle{ O( n / \log n \log \log \log n ) }[/math] rather than [math]\displaystyle{ O(n/\log n) }[/math] (the prime number theorem bound). If we then factor the first k elements of this sequence, we must get a prime of size at least [math]\displaystyle{ k \log k \log \log \log k }[/math] or so.

One can do better still by working with the Fermat numbers, [math]\displaystyle{ F_n = 2^{2^n}+1 }[/math]. It is known [KLS2002] that the number of primes dividing any of the [math]\displaystyle{ F_n }[/math] is [math]\displaystyle{ O(\sqrt{n}/\log n) }[/math], so in particular if we factor the first k Fermat numbers, we get a prime almost of size [math]\displaystyle{ k^2 }[/math]. In fact, it's a classical result of Euler that the prime divisors of [math]\displaystyle{ F_n }[/math] are all at least [math]\displaystyle{ 2^{n+2}+1 }[/math], so we can obtain a large prime simply by factoring a Fermat number.

Assuming RH, one can find a prime larger than [math]\displaystyle{ k^2 }[/math] in about k steps by the method indicated earlier (start at [math]\displaystyle{ k^2 }[/math] and increment until one hits a prime). Unconditionally, one gets a prime larger than [math]\displaystyle{ k^{1/0.525} }[/math] by this method.

Variants of the problem

  1. Can one derandomise factoring of polynomials over finite fields? Note that if one could do this even in the quadratic case, this would allow for efficient location of quadratic non-residues, which is not known.

Finding consecutive (or nearby) square-free integers

Toy problem Is there an efficient algorithm to find consecutive square-free integers n, n+1 of at least k digits in length assuming an oracle to determine whether a number is square free (a special case of the factoring oracle)? Or just square-free integers that are close together (e.g. polylog(n)).

Note that finding one large square-free number is easy: just multiply lots of distinct small primes together. Also, as the density of square-free numbers is [math]\displaystyle{ 6/\pi^2 \approx 0.6 }[/math], a counting argument shows that pairs of consecutive square-free numbers exist in abundance, and are thus easy to find probabilistically. The density can be increased to arbitrarily close to 1 using the W-trick, so one could also ask for, say, long arithmetic progressions of square-free numbers also.

If Sylvester's sequence is square-free, then this solves the problem.

Testing a number for square-freeness is trivial with a factoring oracle. No unconditional polynomial-time deterministic algorithm for square-freeness seems to be known. (It is listed as an open problem in this paper from 1994.)

Unconditionally, there are asymptotics for the number of square-free numbers in [1,N] with an error term of [math]\displaystyle{ O(N^{1/2}) }[/math], which leads to a [math]\displaystyle{ O(10^k)^{1/2} }[/math] algorithm (just search an interval of this length for square-freeness). This is the current unconditional world record. Assuming RH, this can be improved to [math]\displaystyle{ O(10^k)^{17/54+\varepsilon} }[/math].

There is an asymptotic [HB1984] for the number of consecutive square free numbers with an error term of [math]\displaystyle{ O(N^{7/11}) }[/math], leading to a [math]\displaystyle{ O(10^k)^{7/11+\varepsilon} }[/math], though this is inferior to the above bounds.

It is known [FT1992] that the interval [math]\displaystyle{ [N, N+N^{1/5+\epsilon}] }[/math] contains at least one square-free number; Granville [G1998] improved this to [math]\displaystyle{ [N,N+N^{\varepsilon}] }[/math] assuming the ABC conjecture. If the density in such intervals can be made to exceed 50% (perhaps after first applying the W-trick) then we would have solved the weak version of this toy problem assuming ABC.

Finding pseudoprimes

Problem: How quickly can one find a solution to the congruence [math]\displaystyle{ 2^{n-1}=1 \hbox{ mod } n }[/math] with n at least k digits in length? (a pseudoprime)?

It is conjectured that the largest gap between k-digit pseudoprimes is poly(k) (this is weaker than Cramer's conjecture, and would imply a polynomial time solution to the above problem).

It is conjectured that the number [math]\displaystyle{ \mathcal{P}_a(x) }[/math] of pseudoprimes to any base [math]\displaystyle{ a \neq \pm1 }[/math] satisfies

[math]\displaystyle{ \displaystyle \log\mathcal{P}_a(x) = \log(x) - \log(x)\frac{\log\log\log(x)}{\log\log(x)}(1+o(1)). }[/math]

However, the best known bounds seem to be

[math]\displaystyle{ \displaystyle \log\mathcal{P}_a(x) \leq \log(x) - \frac{1}{2}\log(x)\frac{\log\log\log(x)}{\log\log(x)}(1+o(1)), }[/math]

and

[math]\displaystyle{ \displaystyle \log\mathcal{P}_a(x) \geq (\frac{\log(x)}{\log a})^{E/(E+1)-o(1)}, }[/math]

where E is a constant which is conjectured to equal 1, but is only known to have value at least 1-1/(2\sqrt{e}). These results can be found in [P1981], [P1982], [P1989]. Better bounds are known to hold on average [EP1986].

If n is a pseudoprime, then [math]\displaystyle{ 2^n-1 }[/math] is also a pseudoprime (because [math]\displaystyle{ 2^n = 1 \hbox{ mod } 2^n - 1 }[/math], and [math]\displaystyle{ 2^n-2 }[/math] is a multiple of n). This, technically, solves the problem, but can we find denser sets of pseudoprimes? What if we want [math]\displaystyle{ 2^{n-1}=1 \hbox{ mod } n }[/math] and [math]\displaystyle{ 3^{n-1} = 1 \hbox{ mod } n }[/math] simultaneously?

Finding Almost Primes

An almost primes is a number that is the product of at most two primes (counted with multiplicity).

problem How fast can we deterministically write down a [math]\displaystyle{ k }[/math] digit almost prime?

Clearly this is a relaxation of the prime problem, so we expect to get better results. In [L1996] it is shown that the interval [math]\displaystyle{ [x,x+x^{.436}] }[/math] contains an almost prime. This gives us a [math]\displaystyle{ O(10^{k})^{.436} }[/math] algorithm. Assuming a factoring oracle, the problems of deterministically finding a [math]\displaystyle{ k }[/math] digit almost prime and a [math]\displaystyle{ k }[/math] digit prime in polynomial time are equivalent. If we can find an almost prime in polynomial time we can factor it and get (a slightly smaller) prime. Conversely, if we can find a prime in polynomial time we can square it to find (a slightly larger) almost prime. Using our prime technology this should give a [math]\displaystyle{ O(10^{k})^{.262} }[/math] algorithm.

Observations and strategies

  1. For the conjecture with factoring, it would suffice to show that every polylog(N)-length interval in [N,2N] contains a number which is not [math]\displaystyle{ N^\varepsilon }[/math]-smooth (or [math]\displaystyle{ \log^{O(1)} N }[/math] smooth, if one only wants to solve the weak conjecture with factoring).
    1. One can combine this with the "W-trick". For instance, if one lets W be the product of all the primes less than k, and if one can show that not every element of the progression [math]\displaystyle{ \{ Wn+1: 1 \leq n \leq k^{100} \} }[/math] (say) is [math]\displaystyle{ k^A }[/math]-smooth, then one has found a prime with at least [math]\displaystyle{ A \log k }[/math] digits in polynomial time. If one can make A arbitrarily large, one has solved the baby problem with factoring.
    2. One way to proceed is to analyse iterated sumsets of log-primes.
    3. Another is to analyse signed sums of prime reciprocals.
    4. Probably the best known bound along these lines is given in [HB1995] and states that the interval [math]\displaystyle{ [x,x+x^{.5+\epsilon}] }[/math] contains an integer with a prime factor at least [math]\displaystyle{ x^{11/12 -\epsilon} }[/math]. This only leads to a [math]\displaystyle{ O(10^{k})^{.5455} }[/math] algorithm.
  2. Another approach is to accurately compute the prime counting function.
    1. A variant of this strategy is the polynomial strategy.
  3. If one can find explicitly enumerable sets of k-digit numbers of size [math]\displaystyle{ O(10^{\varepsilon k}) }[/math] for any fixed [math]\displaystyle{ \varepsilon \gt 0 }[/math] that are guaranteed to contain a prime, then one has solved the weak conjecture. If instead one can find randomly samplable sets of k-digit numbers of size [math]\displaystyle{ O(10^{\varepsilon k}) }[/math] that contain a large fraction (e.g. [math]\displaystyle{ \exp(-o(k)) }[/math] of primes), one has solved the semi-weak conjecture. If one can find explicit sets of poly(k) size that are guaranteed to contain a prime, one has solved the strong conjecture.
  4. If a primality test can be expressed as a constant-degree polynomial over a finite field from the digits of the number-under-test to {0,1}, then known pseudorandom generators for polynomials can be used to derandomize finding primes the same way as under the full P=BPP assumption. So, instead of requiring a general PRG, the proposal is trying to put the primality test into a fully derandomizable subclass of P.
  5. If one can find a sequence of easily constructible, mutually coprime integers whose prime factors are contained in a sparse subset of the primes, this generates (with the aid of a factoring oracle) a sparse set of primes. So far the sparsest set of this type has come from factoring the Fermat numbers.

Negative results and other obstructions

  1. Using sieve theory to find a non-smooth number in a short interval runs into the difficulty that most sieve theory methods are insensitive to the starting point of an interval. Given that every integer in [1,n] is n-smooth, this implies that sieve theory is unlikely to establish the existence of a non-n-smooth number in any interval of length n, and in particular to find a non-[math]\displaystyle{ O(k^O(1)) }[/math] smooth integer in any interval that can be searched in polynomial time. Indeed, there are heuristic arguments that suggest that even augmented with basic Fourier analysis and assuming RH, this method is unlikely to work, although they haven't yet been fully formalized.
  2. Average-case results for the primes cannot be used directly, because one could delete all constructible integers from the set of primes (a negligible set) without significantly impacting the average case results, while causing the conjecture to fail for this modified set of primes.
    1. In a similar spirit, one cannot hope to find non-smooth numbers in a short interval by analysing the iterated sumset of the log-primes and using average-case information about that set), because if one deletes all the primes arising from factors of that interval, one still captures most of the primes (and thus most of the average-case behaviour of the log-primes) while losing the information about that interval.
  3. It is not possible to prove the conjecture purely by "soft" (i.e. relativisable) complexity theory methods which do not use any additional information on the primes rather than their density. Details are at Oracle counterexample to finding pseudoprimes.

How could the conjecture be false?

A world in which the conjecture is negative would be very strange, especially with factoring or if the weak conjecture fails also. In particular, the following would have to be true:

  1. All numbers with at least k digits that are "constructible" in the sense that they are the output of a Turing machine program of length [math]\displaystyle{ O(\log k) }[/math] that runs in time [math]\displaystyle{ k^{O(1)} }[/math], are composite. [If the conjecture fails with factoring, then these numbers are furthermore [math]\displaystyle{ \exp(\varepsilon k) }[/math] smooth for any fixed [math]\displaystyle{ \varepsilon \gt 0 }[/math]; if even the weak conjecture with factoring fails, they are [math]\displaystyle{ k^{O(1)} }[/math]-smooth.]
  2. In fact, all constructible numbers are not only composite/smooth, but they sit inside intervals of length [math]\displaystyle{ k^{100} }[/math], all of whose elements are composite/smooth. (One can replace 100 by any other fixed number.)
    1. In particular, Cramer's conjecture fails; the gaps between adjacent k-digit primes exceeds [math]\displaystyle{ k^{100} }[/math] around every constructible number.
  3. No pseudorandom number generators exist. In particular, this implies that one-way functions do not exist.
  4. [math]\displaystyle{ P \neq NP }[/math]. More specifically, the decision problem "Does there exist a prime in a given interval?" is in NP, but cannot lie in P if the conjecture fails.

Relevant concepts

Complexity theory concepts:

  1. Complexity classes
    1. P
    2. NP
    3. BPP
    4. promise-BPP
    5. BQP
    6. DTIME(f(n))
  2. Pseudo-random generators (PRG)
  3. Full derandomization
  4. one-way function
  5. Impagliazzo's five worlds: Algorithmica, Heuristica, Pessiland, Minicrypt, Cryptomania
  6. Kolmogorov complexity
  7. Oracles

Number theory concepts:

  1. Cramer's random model for the primes
  2. Prime gaps
  3. Generic prime
  4. Smooth number
  5. The W-trick
  6. Sylvester's sequence
  7. Siegel zero
  8. Asymptotic notation
  9. Fermat number
  10. Pseudoprimes

Other:

  1. Expander graphs

Note: articles with external links should eventually be replaced by in-house pages that can provide information that is more dedicated to this project.

Relevant conjectures

  1. P=NP
  2. P=BPP
  3. P=promise-BPP
  4. P=BQP
  5. existence of PRG
  6. existence of one-way functions
  7. whether DTIME(2^n) has subexponential circuits
  8. The Riemann Hypothesis (RH)
    1. The generalised Riemann Hypothesis (GRH)
  9. the Hardy-Littlewood prime tuples conjecture
  10. the ABC conjecture
  11. Cramer's conjecture
  12. Schinzel's hypothesis H
  13. discrete log in P
  14. factoring in P

Relevant results

  1. Brun-Titchmarsh inequality
  2. Erdos-Kac theorem
  3. Bertrand's postulate
  4. Friedlander-Iwaniec theorem
  5. Conditional on the Riemann hypothesis

Paper

Here is the [arXiv copy] of the research paper, now submitted to Mathematics of Computation.

Here is the [Subversion repository] of the research paper.

Here are the Polymath4 grant acknowledgments.

Relevant papers

Please help out by adding links or completing references, or adding further references to the list above!

External links

  1. Complexity Zoo
  2. "Finding Primes", Lance Fortnow, August 4 2009