Foundation Prior: How LLM Outputs Reshape Bayesian Beliefs

This is Part 2. If you have not read Mathematical Prerequisites for Foundation Prior, start there — we will use parameters, likelihood, Beta distributions, KL divergence, entropy, exponential tilting, marginal likelihood, and the law of total variance throughout this post.

The Running Example

We continue with the same coin. The parameter $\theta = P(\text{Heads})$ is unknown. Our prior belief is:

\pi_0(\theta) = \text{Beta}(2, 2)

This means we lean toward $\theta = 0.5$ but not strongly — it behaves like having seen 1 pseudo-head and 1 pseudo-tail.

What Are We Even Modeling?

The paper revolves around seven symbols, and we need to fix their meaning before anything else. The unknown parameter $\theta$ is the thing you care about — in our coin example it is $P(\text{Heads})$ , but in real applications it might be a sales trend, consumer preference, or treatment effect. Real data $D_r$ is data from the actual world. Synthetic data $D_s$ is data generated by the LLM. The prior $\pi(\theta)$ represents your belief before seeing any data. The likelihood $L(D \mid \theta)$ measures how likely a dataset is under a given value of $\theta$ . The foundation prior $\rho(\theta)$ is your updated prior after incorporating synthetic data. And the trust parameter $\lambda$ controls how much weight synthetic data receives. Every derivation in this post uses these seven objects and nothing else.

The Standard Bayesian World (Baseline)

Before LLMs exist, everything is straightforward. You have a prior $\pi(\theta)$ , you observe real data $D_r$ , and you update via Bayes:

\pi(\theta \mid D_r) \propto L(D_r \mid \theta)\,\pi(\theta)

The prior is your belief, the likelihood is what the world tells you, and the posterior is their combination. Everything is clean because data comes from reality.

What Changes With LLMs?

Now instead of real data, you generate synthetic data:

D_s \sim \pi_{\text{LLM}}(D \mid q)

where $q$ is a prompt and $\pi_{\text{LLM}}$ is the LLM’s distribution over outputs.

In real life, you ask something like: “Generate consumer choice data for Chicago ice cream sales.” The model generates something. But here is the key: you do not accept just any output. You tweak the prompt until the output matches what you expect. This is epistemic circularity, and it is the core reason the entire Foundation Prior framework exists.

The circular loop works like this. You start with beliefs about $\theta$ . Those beliefs shape how you write the prompt. The prompt shapes what the LLM generates. You then judge the output against your expectations — which come from the same beliefs you started with. If the output looks wrong, you reject it and re-prompt. If it looks right, you accept it. So the “data” you end up with has been filtered by the very beliefs it is supposed to update. Your prior determined what you accepted, and now you want to use that accepted output to update your prior. That is circular.

In standard Bayesian inference this never happens. Real data comes from nature — it does not care about your beliefs. But synthetic data is different. It passed through a subjective filter. If you treat it like real evidence, you are essentially confirming your own assumptions back to yourself. The entire paper is built to handle this problem: how do you extract genuine information from synthetic data without falling into a self-reinforcing loop?

The Anticipation Distribution

Before generating synthetic data, the user already expects certain patterns. Mathematically, the user anticipates data $D_a$ that looks like:

\pi(D_a) = \int L(D_a \mid \theta)\,\pi(\theta)\,d\theta

This is called the prior predictive distribution. It represents all possible data weighted by your prior beliefs. For each possible $\theta$ , you weight by $\pi(\theta)$ and integrate. This gives expected data patterns. We integrate because you do not know $\theta$ — so you average over all possible $\theta$ weighted by belief. That is what marginalization means.

In our coin example, if your prior is $\text{Beta}(2,2)$ , you expect roughly equal heads and tails, but with some spread.

Prompt Engineering as Optimization

The user compares generated data with anticipated data using a divergence $\kappa(\pi(D_a) \| D_s(q))$ , where $\kappa$ is a distance measure — it could be KL divergence, moment mismatch, or likelihood difference.

Why introduce $\kappa$ ? Because users reject outputs they dislike. Mathematically, prompt engineering becomes gradient descent:

q_{t+1} = q_t - \eta_t \nabla_q \kappa

In real life, you generate data, check if it “looks right,” modify the prompt, and repeat until $\kappa < \tau$ — you stop when output matches your expectation. This is selection. This is the source of subjectivity.

Why We Cannot Use Normal Bayes on Synthetic Data

Suppose the synthetic dataset we accepted is 8 Heads and 2 Tails. Call this $D_s^*$ . The likelihood is:

L(D_s^* \mid \theta) = \theta^8(1 - \theta)^2

Normal Bayes would say:

\text{Posterior} \propto \text{Prior} \times \text{Likelihood}

\theta^1(1 - \theta)^1 \times \theta^8(1 - \theta)^2 = \theta^9(1 - \theta)^3

That gives $\text{Beta}(10, 4)$ with mean $\approx 0.71$ . This strongly pushes belief toward “biased coin.”

The paper says: that is wrong. Why? Because you generated the data. You wrote the prompt, rejected weird outputs, and accepted one that looked plausible. This is not random data from nature. It is filtered imagination. So we must downweight it.

The Core Theoretical Move

This is the most important mathematical part of the entire paper.

After synthetic data $D_s^*$ is generated, we want beliefs about $\theta$ . But we do NOT want to treat synthetic data like real data. So instead of normal Bayes, the paper sets up a constrained optimization problem.

The idea in plain English

Find a new belief $\rho(\theta)$ that changes your old belief $\pi_0(\theta)$ as little as possible, but still makes the synthetic data “not too implausible.” That sentence becomes math.

What is $\rho(\theta)$ ?

$\rho(\theta)$ is the new belief distribution after using synthetic data. Before synthetic data, you believed $\pi_0(\theta)$ . After synthetic data, you will believe $\rho(\theta)$ . It is a function (a probability distribution) over $\theta$ .

”Make synthetic data not too implausible” becomes a constraint

The paper uses:

\mathbb{E}_\rho[\log L(D_s \mid \theta)] \ge C

Let us unpack what this means. By definition, $\mathbb{E}_\rho[f(\theta)] = \int_0^1 f(\theta)\,\rho(\theta)\,d\theta$ . Here $f(\theta) = \log L(D_s \mid \theta)$ , so:

\mathbb{E}_\rho[\log L(D_s \mid \theta)] = \int_0^1 \rho(\theta)\,\log L(D_s \mid \theta)\,d\theta

Substituting the log-likelihood we derived:

= \int_0^1 \rho(\theta)\left(8\log\theta + 2\log(1 - \theta)\right)d\theta

The interpretation is this: for each possible $\theta$ , compute “how well $\theta$ explains the synthetic data” (that is the log-likelihood), weight it by how much you believe that $\theta$ (that is $\rho(\theta)$ ), and average. So the constraint means: on average, under my new belief, the synthetic dataset should not look too unlikely.

Why log-likelihood instead of raw likelihood?

The constraint uses $\log L$ rather than $L$ for three reasons. First, additivity: likelihood multiplies across independent observations, and log turns products into sums ( $\log \prod_i L_i = \sum_i \log L_i$ ), which makes the optimization tractable. Second, information-geometric duality: KL divergence and log-likelihood both live in the geometry of exponential families, and using log ensures the constraint and the objective operate in the same mathematical space — this is what makes the Lagrangian solution clean. Third, convexity: log-likelihood is often concave in $\theta$ , which makes the optimization well-behaved; if we used raw likelihood, the problem could become non-convex and the closed-form solution would not exist.

”Change my belief as little as possible” becomes KL minimization

The paper chooses KL divergence:

KL(\rho \| \pi_0) = \int_0^1 \rho(\theta)\log\frac{\rho(\theta)}{\pi_0(\theta)}\,d\theta

If $\rho = \pi_0$ , KL = 0 (no change). Bigger KL means you moved farther from your prior beliefs. So “minimal change” becomes $\min_\rho KL(\rho \| \pi_0)$ .

We also require $\rho$ to be a valid probability distribution: $\int_0^1 \rho(\theta)\,d\theta = 1$ and $\rho(\theta) \ge 0$ .

The full optimization problem (coin version)

Putting it together:

\min_\rho \int_0^1 \rho(\theta)\log\frac{\rho(\theta)}{\pi_0(\theta)}\,d\theta

subject to:

\int_0^1 \rho(\theta)\,\log L(D_s \mid \theta)\,d\theta \ge C

\int_0^1 \rho(\theta)\,d\theta = 1

That is exactly the paper’s key move, now fully instantiated for the coin.

Solving the Optimization Step by Step

Step 1 — Build a Lagrangian

We turn constraints into penalties using multipliers: $\lambda \ge 0$ for the expected log-likelihood constraint, and $\gamma$ for normalization.

\mathcal{J}[\rho] = \int \rho(\theta)\log\frac{\rho(\theta)}{\pi_0(\theta)}\,d\theta - \lambda\left(\int \rho(\theta)\log L(D_s \mid \theta)\,d\theta - C\right) + \gamma\left(\int \rho(\theta)\,d\theta - 1\right)

Step 2 — Take the functional derivative with respect to $\rho(\theta)$

We use these derivative facts: the derivative of $\rho\log\rho$ with respect to $\rho$ is $\log\rho + 1$ , and the derivative of $\rho \cdot a(\theta)$ with respect to $\rho$ is $a(\theta)$ .

First expand the KL integrand:

\rho(\theta)\log\frac{\rho(\theta)}{\pi_0(\theta)} = \rho(\theta)\log\rho(\theta) - \rho(\theta)\log\pi_0(\theta)

Now differentiate $\mathcal{J}$ :

\frac{\delta\mathcal{J}}{\delta\rho(\theta)} = (\log\rho(\theta) + 1) - \log\pi_0(\theta) - \lambda\log L(D_s \mid \theta) + \gamma

Set to zero at the optimum:

(\log\rho(\theta) + 1) - \log\pi_0(\theta) - \lambda\log L(D_s \mid \theta) + \gamma = 0

Rearrange:

\log\rho(\theta) = \log\pi_0(\theta) + \lambda\log L(D_s \mid \theta) - (1 + \gamma)

Step 3 — Exponentiate to solve for $\rho(\theta)$

\rho(\theta) = \exp(\log\pi_0(\theta)) \cdot \exp(\lambda\log L(D_s \mid \theta)) \cdot \exp(-(1 + \gamma))

Now we use three identities: $\exp(\log\pi_0) = \pi_0$ , $\exp(\lambda\log L) = L^\lambda$ , and $\exp(-(1+\gamma))$ is just a constant (same for all $\theta$ ). So:

\rho(\theta) = \pi_0(\theta)\,L(D_s \mid \theta)^\lambda \cdot K

where $K$ is a constant chosen so that $\int \rho(\theta)\,d\theta = 1$ . That gives the final form:

\boxed{\rho(\theta) = \frac{\pi_0(\theta)\,L(D_s \mid \theta)^\lambda}{\int_0^1 \pi_0(u)\,L(D_s \mid u)^\lambda\,du}}

This is the foundation prior form. Nothing was assumed. It came from optimization.

Making It Concrete: Coin Numbers

We have $\pi_0(\theta) = \text{Beta}(2, 2) \propto \theta^1(1 - \theta)^1$ and $L(D_s \mid \theta) = \theta^8(1 - \theta)^2$ .

Raise likelihood to $\lambda$ :

L^\lambda = \theta^{8\lambda}(1 - \theta)^{2\lambda}

Multiply:

\rho(\theta) \propto \theta^{1 + 8\lambda}(1 - \theta)^{1 + 2\lambda}

That is a Beta distribution again — this is Beta-Bernoulli conjugacy at work. Because the Beta prior and the Bernoulli/Binomial likelihood belong to the same exponential family, multiplying them always yields another Beta. The posterior stays in the same family as the prior, which is exactly what “conjugate prior” means.

\boxed{\rho(\theta) = \text{Beta}(2 + 8\lambda,\; 2 + 2\lambda)}

In the coin example, conjugacy gives us a completely explicit foundation prior — no numerical integration needed.

Numerical check: $\lambda = 0.25$

\rho(\theta) = \text{Beta}(2 + 2,\; 2 + 0.5) = \text{Beta}(4,\; 2.5)

Interpreting the Foundation Prior

What Beta(4, 2.5) means

For a Beta distribution, $\alpha - 1$ gives pseudo-heads and $\beta - 1$ gives pseudo-tails.

Before synthetic data, $\text{Beta}(2,2)$ means 1 pseudo-head and 1 pseudo-tail. Very weak belief, centered at 0.5. After synthetic data with $\lambda = 0.25$ , $\text{Beta}(4, 2.5)$ means 3 pseudo-heads and 1.5 pseudo-tails.

Not 8 heads. Not 2 tails. Because $\lambda = 0.25$ downweighted them. The effective heads are $8 \times 0.25 = 2$ and the effective tails are $2 \times 0.25 = 0.5$ . Adding the prior pseudo-counts of 1 head and 1 tail gives total heads of $1 + 2 = 3$ and total tails of $1 + 0.5 = 1.5$ . Converting back to Beta parameters ( $\alpha = \text{heads} + 1$ , $\beta = \text{tails} + 1$ ) gives $\alpha = 4$ and $\beta = 2.5$ — exactly $\text{Beta}(4, 2.5)$ .

What this distribution looks like

The mean is:

\frac{4}{4 + 2.5} = \frac{4}{6.5} \approx 0.615

Your best guess is now $\theta \approx 0.62$ . Not 0.8. Not 0.5. Somewhere in between. You started with “coin is roughly fair,” synthetic data suggested “looks biased toward heads,” but you did not fully believe it. So you updated gently. $\text{Beta}(4, 2.5)$ represents that gentle shift.

The paper’s entire point: synthetic data should act like a small number of pseudo-observations, not real data. $\text{Beta}(4, 2.5)$ shows exactly that. Synthetic data did not dominate. It just tilted the prior.

Understanding $\lambda$ — The Trust Parameter

What $\lambda$ does mechanically

Recall:

\rho(\theta) \propto \pi_0(\theta) \cdot L(D_s \mid \theta)^\lambda

Since the coin likelihood is $\theta^H(1-\theta)^T$ , raising to power $\lambda$ gives $\theta^{\lambda H}(1-\theta)^{\lambda T}$ . So synthetic data behaves like $\lambda H$ heads and $\lambda T$ tails. $\lambda$ literally scales the data size.

Concrete interpretation

If synthetic data had 8 heads and 2 tails, then the effective heads are $8\lambda$ and the effective tails are $2\lambda$ . So $\lambda$ controls how many synthetic flips you act like you observed.

Why $\lambda$ is necessary

This is the core philosophical insight of the paper. Imagine synthetic data size = 10,000 flips. If we used normal Bayes ( $\lambda = 1$ ), you would become almost completely certain about $\theta$ . But synthetic data is generated by you. You can generate infinite synthetic data. So if $\lambda = 1$ , you can make yourself arbitrarily confident without real evidence. That is epistemically dangerous.

Extreme values of $\lambda$

When $\lambda = 0$ , synthetic data is ignored entirely and $\rho(\theta) = \pi_0(\theta)$ — you fall back to your original prior with zero effective synthetic counts. When $\lambda$ is between 0 and 1, you have partial trust — the intended regime — where synthetic contributes $\lambda N_s$ effective counts, a fraction of the full synthetic sample. When $\lambda = 1$ , synthetic is treated exactly like real data, contributing the full $N_s$ counts. When $\lambda > 1$ , you trust synthetic more than real data, which is very dangerous — the effective counts exceed the actual synthetic sample size.

The effective sample size of synthetic data is $\lambda N_s$ , so $\lambda$ directly controls how much influence synthetic evidence exerts. In our coin example with 10 synthetic flips and $\lambda = 0.25$ , the effective sample size is $0.25 \times 10 = 2.5$ — you act as though you observed only 2.5 synthetic flips instead of 10. This is the mechanism that prevents epistemic circularity from spiraling: no matter how much synthetic data you generate, $\lambda$ caps its effective contribution.

When Real Data Arrives

Suppose after synthetic data, you flip the real coin 10 times and observe 6 Heads, 4 Tails. The real likelihood is:

L(D_r \mid \theta) = \theta^6(1 - \theta)^4

The final posterior becomes:

\pi(\theta \mid D_r, D_s, \lambda) \propto L(D_r \mid \theta) \cdot L(D_s \mid \theta)^\lambda \cdot \pi_0(\theta)

We already know that $\pi_0(\theta)\,L(D_s \mid \theta)^\lambda = \rho(\theta)$ , so:

\pi(\theta \mid D_r, D_s, \lambda) \propto L(D_r \mid \theta) \cdot \rho(\theta)

Real data updates normally, starting from the foundation prior.

Computing it explicitly

We had $\rho(\theta) = \text{Beta}(2 + 8\lambda, 2 + 2\lambda)$ . Multiply by real likelihood $\theta^6(1 - \theta)^4$ and add exponents:

\text{Final posterior} = \text{Beta}(2 + 8\lambda + 6,\; 2 + 2\lambda + 4)

For $\lambda = 0.25$ :

\text{Beta}(4 + 6,\; 2.5 + 4) = \text{Beta}(10, 6.5)

The mean is $10/16.5 \approx 0.61$ . Real data now has strong influence. Synthetic only nudged the initial belief. That is the design.

Prompt Dependence and Mixtures

So far, we assumed one synthetic dataset: 8 Heads, 2 Tails. But imagine you run the simulation again. You might get 8H and 2T on the first run, 6H and 4T on the second, and 9H and 1T on the third. Each run gives a different synthetic dataset, each dataset produces a different foundation prior, and this means your belief depends on the prompt and random generation. That is subjectivity.

Computing each foundation prior

Let $\lambda = 0.25$ throughout.

Case A (8H, 2T):

\text{Effective heads} = 8 \times 0.25 = 2, \quad \text{Effective tails} = 2 \times 0.25 = 0.5

\rho_A(\theta) = \text{Beta}(2 + 2,\; 2 + 0.5) = \text{Beta}(4,\; 2.5), \quad \mu_A = \frac{4}{6.5} \approx 0.6154

Case B (6H, 4T):

\text{Effective heads} = 6 \times 0.25 = 1.5, \quad \text{Effective tails} = 4 \times 0.25 = 1

\rho_B(\theta) = \text{Beta}(2 + 1.5,\; 2 + 1) = \text{Beta}(3.5,\; 3), \quad \mu_B = \frac{3.5}{6.5} \approx 0.5385

Case C (9H, 1T):

\text{Effective heads} = 9 \times 0.25 = 2.25, \quad \text{Effective tails} = 1 \times 0.25 = 0.25

\rho_C(\theta) = \text{Beta}(2 + 2.25,\; 2 + 0.25) = \text{Beta}(4.25,\; 2.25), \quad \mu_C = \frac{4.25}{6.5} \approx 0.6538

Different synthetic datasets produce different priors.

The paper’s solution: average over prompts

Instead of trusting one synthetic dataset, the paper proposes averaging across possible prompts. Mathematically:

\bar{\rho}(\theta) = \int \rho(\theta \mid D_s(q), \lambda)\,h(q)\,dq

In coin language, instead of integrating over prompts, we integrate over synthetic datasets. Each dataset gets a weight, and the final prior is the weighted sum of all foundation priors.

The mixture prior

Assume equal weights:

\bar{\rho}(\theta) = \frac{1}{3}\,\text{Beta}(4, 2.5) + \frac{1}{3}\,\text{Beta}(3.5, 3) + \frac{1}{3}\,\text{Beta}(4.25, 2.25)

This is a mixture of Beta distributions — no longer a single Beta. It reflects uncertainty about the synthetic generation process.

Mixture mean

Using the law of total expectation from Part 1:

\mathbb{E}_{\bar{\rho}}[\theta] = \frac{1}{3}(0.6154) + \frac{1}{3}(0.5385) + \frac{1}{3}(0.6538) = \frac{1.8077}{3} \approx 0.6026

After averaging across prompt/generation variability, our best guess for heads-probability is about 0.60.

Mixture variance

Using the law of total variance from Part 1, total variance has two parts: average within-component variance plus variance of component means.

All three components have $a + b = 6.5$ , so the denominator $((a+b)^2(a+b+1)) = 6.5^2 \times 7.5 = 316.875$ is the same for all. The within-component variances are:

\text{Var}_A = \frac{4 \times 2.5}{316.875} = \frac{10}{316.875} \approx 0.03156

\text{Var}_B = \frac{3.5 \times 3}{316.875} = \frac{10.5}{316.875} \approx 0.03314

\text{Var}_C = \frac{4.25 \times 2.25}{316.875} = \frac{9.5625}{316.875} \approx 0.03018

The squared deviations from the mixture mean $m = 0.6026$ are:

(\mu_A - m)^2 \approx 0.000164, \quad (\mu_B - m)^2 \approx 0.004109, \quad (\mu_C - m)^2 \approx 0.002630

Applying the law of total variance — each term is (within-variance + between-variance) for that component:

\text{Var}_{\text{mix}} = \frac{1}{3}(0.03156 + 0.000164) + \frac{1}{3}(0.03314 + 0.004109) + \frac{1}{3}(0.03018 + 0.002630) \approx 0.03393

The standard deviation is approximately $\sqrt{0.03393} \approx 0.184$ . The mixture is more uncertain than any single component, because it includes both “what is $\theta$ ?” uncertainty and “which prompt/run did I get?” uncertainty.

Why this matters

If you rely on one synthetic dataset, your belief is unstable. If you average over many, your belief becomes robust. This step is about reducing prompt sensitivity.

Updating the Mixture With Real Data

When real data arrives (6H, 4T), two things happen: each component updates normally, and the mixture weights change.

Updating each component

Each Beta prior updates by adding real counts. Component A goes from $\text{Beta}(4, 2.5)$ to $\text{Beta}(10, 6.5)$ with mean $\approx 0.6061$ . Component B goes from $\text{Beta}(3.5, 3)$ to $\text{Beta}(9.5, 7)$ with mean $\approx 0.5758$ . Component C goes from $\text{Beta}(4.25, 2.25)$ to $\text{Beta}(10.25, 6.25)$ with mean $\approx 0.6212$ .

Updating the mixture weights

Before real data, the weights were equal: $w_A = w_B = w_C = 1/3$ . After real data, Bayes’ rule says:

w_i' \propto w_i \cdot p(D_r \mid \text{component } i)

The term $p(D_r \mid \text{component } i)$ is the marginal likelihood of real data under that component — the Beta-Binomial evidence:

p(D_r \mid \text{Beta}(a, b)) = \binom{n}{H_r}\frac{B(a + H_r,\; b + T_r)}{B(a, b)}

The binomial coefficient cancels when normalizing weights. Computing the evidence ratios gives approximately 0.0007265 for A, 0.0007168 for B, and 0.0006983 for C. The normalized updated weights are $w_A' \approx 0.339$ , $w_B' \approx 0.335$ , and $w_C' \approx 0.326$ . Real data (6H, 4T) slightly favors component A, but not by much — all three prompts are still plausible.

The final mixture posterior

p(\theta \mid D_r) = 0.339 \cdot \text{Beta}(10, 6.5) + 0.335 \cdot \text{Beta}(9.5, 7) + 0.326 \cdot \text{Beta}(10.25, 6.25)

The mixture mean is:

\mathbb{E}[\theta \mid D_r] \approx 0.339(0.6061) + 0.335(0.5758) + 0.326(0.6212) \approx 0.6009

What this shows: two layers of uncertainty

There are two distinct layers of uncertainty operating simultaneously.

Layer 1 — uncertainty about $\theta$ . Even within a single component, you do not know the true coin bias. Each updated Beta is spread out over a range of $\theta$ values. This is ordinary statistical uncertainty.

Layer 2 — uncertainty about which synthetic world is correct. The three components correspond to three different synthetic generation outcomes. Before real data, you had no reason to prefer one over another. After real data, the weights shift — reality starts telling you which synthetic scenario was more plausible. This is epistemic uncertainty about the synthetic process itself.

Real data reduces both layers. It sharpens each component (Layer 1) and concentrates the mixture weights (Layer 2). As real data grows large, the weights eventually concentrate on one component — the synthetic scenario most consistent with reality — and the influence of the other components becomes negligible. In the limit of infinite real data, the mixture collapses and the synthetic starting point no longer matters. This is the robustness guarantee: even if your prompts were unstable or your synthetic data was biased, enough real data will wash out that instability.

Calibrating $\lambda$ With Real Data

$\lambda$ should not be arbitrary. The paper says: let real data decide how helpful synthetic data was.

What “calibrate” means

You generated synthetic flips. That produced a foundation prior $\rho(\theta \mid D_s, \lambda)$ . Then you collect real flips $D_r$ . Now you ask: how much should I have trusted the synthetic flips, given what real flips actually show?

The method

We evaluate different $\lambda$ values by how well they predict the real data. Given $\lambda$ , your foundation prior is $\rho(\theta \mid D_s, \lambda)$ . The probability your model assigns to the real data is:

p(D_r \mid D_s, \lambda) = \int_0^1 L(D_r \mid \theta)\,\rho(\theta \mid D_s, \lambda)\,d\theta

This is a single number for each $\lambda$ — bigger means better. So the calibration rule is:

\boxed{\lambda^* = \arg\max_{\lambda \ge 0}\;\log p(D_r \mid D_s, \lambda)}

Choose the trust level that makes real data most plausible after using synthetic data. This is exactly “let reality decide how helpful synthetic data was.”

Why this is independent of $\theta$

We integrate $\theta$ out. For each possible $\theta$ , we compute how likely real data is, weight by our belief, and average. After integrating, $\theta$ disappears. The result is just a number measuring predictive quality. When we choose $\lambda$ , we are not choosing $\theta$ . We are choosing how much we trust synthetic data. So we evaluate how well the entire belief distribution predicts real data.

Explicit formula for the coin-Beta case

We have a prior $\pi_0(\theta) = \text{Beta}(\alpha_0, \beta_0)$ , synthetic data with $H_s$ heads and $T_s$ tails, and real data with $H_r$ heads and $T_r$ tails. From the foundation prior derivation, $\rho(\theta \mid D_s, \lambda) = \text{Beta}(\alpha_0 + \lambda H_s,\; \beta_0 + \lambda T_s)$ .

The evidence for real data under a Beta prior is Beta-Binomial:

p(D_r \mid D_s, \lambda) = \binom{n}{H_r}\frac{B(\alpha_0 + \lambda H_s + H_r,\; \beta_0 + \lambda T_s + T_r)}{B(\alpha_0 + \lambda H_s,\; \beta_0 + \lambda T_s)}

Then $\lambda^* = \arg\max_{\lambda \ge 0}\;\log p(D_r \mid D_s, \lambda)$ . In practice you try a grid of $\lambda$ values — 0, 0.05, 0.1, 0.2, and so on — and pick the best.

The Final Posterior After Calibration

Once $\lambda^*$ is chosen, you do the final update:

\pi(\theta \mid D_r, D_s, \lambda^*) \propto L(D_r \mid \theta)\,L(D_s \mid \theta)^{\lambda^*}\,\pi_0(\theta)

In the coin-Beta case, this is again a Beta:

\boxed{\pi(\theta \mid D_r, D_s, \lambda^*) = \text{Beta}(\alpha_0 + \lambda^* H_s + H_r,\; \beta_0 + \lambda^* T_s + T_r)}

Synthetic contributes $\lambda^* H_s$ and $\lambda^* T_s$ as soft counts, while real data contributes $H_r$ and $T_r$ as full counts.

Why the framework is self-correcting

The calibration step creates a feedback loop between synthetic and real data. If your synthetic data happened to align well with reality — say synthetic suggested $\theta \approx 0.6$ and real flips also point to $\theta \approx 0.6$ — then the evidence $p(D_r \mid D_s, \lambda)$ is maximized at a higher $\lambda^*$ , and synthetic data gets more influence. It earned that influence by being correct.

If synthetic data contradicts reality — say synthetic suggested $\theta \approx 0.8$ but real flips point to $\theta \approx 0.5$ — then higher $\lambda$ values make real data less plausible (because the foundation prior is pulled away from where the real data actually lands). The evidence drops, and $\lambda^*$ shrinks toward zero. In the extreme, if synthetic is completely misleading, calibration sets $\lambda^* \approx 0$ and the framework gracefully ignores the synthetic data altogether, falling back to a standard Bayesian update with the original prior.

This means you cannot be permanently misled by bad synthetic data, as long as you eventually collect real data and calibrate. The framework does not require you to know in advance whether synthetic data is trustworthy — it lets reality decide after the fact.

Why KL Was the Right Choice

You might ask: why minimize KL? Why not squared distance?

If we measured closeness using squared difference $\int (\rho(\theta) - \pi_0(\theta))^2\,d\theta$ , we would ignore the fact that probabilities must remain positive and that they combine multiplicatively. Squared distance does not respect the geometry of probability distributions. KL divergence does — it compares distributions multiplicatively (via log-ratios), it measures information gain (how many extra bits are needed to encode $\rho$ if you designed your code for $\pi_0$ ), and critically, it produces a tractable solution.

Csiszar’s I-projection theorem

When you solve “stay as close as possible to prior but satisfy an expectation constraint,” the result is governed by Csiszar’s I-projection theorem: if you measure closeness using KL divergence, and your constraint is linear in the distribution (like an expectation), the unique solution must be:

\rho(\theta) \propto \pi_0(\theta)\exp(\lambda g(\theta))

That exponential form is not chosen — it is mathematically forced by the structure of KL. In our coin case, $g(\theta) = \log L(D_s \mid \theta)$ , so $\exp(\lambda g(\theta)) = L(D_s \mid \theta)^\lambda$ . This is why the foundation prior has the form it does. No other divergence gives this clean exponential tilting structure.

The Gibbs variational principle

There is a dual way to see the same result. The Gibbs variational principle (also called the Donsker-Varadhan variational formula) states:

\log \int \exp(f(\theta))\,d\pi_0(\theta) = \sup_\rho \left\{ \mathbb{E}_\rho[f(\theta)] - KL(\rho \| \pi_0) \right\}

and the supremum is achieved by the exponential tilt $\rho^*(\theta) \propto \pi_0(\theta)\exp(f(\theta))$ . In our setting, $f(\theta) = \lambda \log L(D_s \mid \theta)$ . So the foundation prior is exactly the optimizer in the Donsker-Varadhan formula — the distribution that maximizes expected log-likelihood minus KL cost. The left-hand side of the formula gives the normalizing constant $Z(\lambda) = \int \pi_0(\theta)\,L(D_s \mid \theta)^\lambda\,d\theta$ , which is the partition function of the tilted distribution.

This is not a coincidence. The constrained optimization (minimize KL subject to expectation constraint) and the variational formula (maximize expectation minus KL) are Lagrangian duals of each other. They produce the same solution from opposite directions.

Connection to Jaynes’s maximum entropy

The paper’s core optimization is literally relative maximum entropy (also called minimum relative entropy or minimum discrimination information). Jaynes’s Maximum Entropy principle says: when you must choose a distribution satisfying certain moment constraints, pick the one with maximum entropy — the least informative distribution consistent with your constraints. When you already have a prior $\pi_0$ , the correct generalization replaces entropy with relative entropy: minimize $KL(\rho \| \pi_0)$ subject to constraints. You are saying: given my old belief $\pi_0$ , and given that expected log-likelihood must be at least $C$ , what is the least informative update?

Normal Bayesian updating can also be derived this way — as minimizing KL divergence subject to exact data constraints. So Bayes’ rule is itself a relative entropy projection. The paper uses that same machinery, but with softened ( $\lambda$ -weighted) constraints for synthetic data. This framework sits directly inside information geometry, maximum entropy theory, and variational inference. It is not ad hoc.

The Practical Recipe

Here is the paper’s workflow, translated to the coin example.

You begin by picking a real-world likelihood model $L(D \mid \theta)$ and a prior $\pi_0(\theta)$ — for the coin, that means a Beta prior and Bernoulli likelihood. Next, you generate synthetic datasets using prompts and accept the ones that pass your stopping rule — for the coin, you generate multiple synthetic flip sequences. For each accepted synthetic dataset $D_s^{(i)}$ , you form a foundation prior:

\rho_i(\theta \mid \lambda) = \frac{\pi_0(\theta)\,L(D_s^{(i)} \mid \theta)^\lambda}{Z_i(\lambda)}

If you want robustness, you combine these by averaging across prompts into a mixture:

\bar{\rho}(\theta \mid \lambda) = \sum_i w_i\,\rho_i(\theta \mid \lambda)

Then you collect real data $D_r$ and choose $\lambda^*$ by calibration — picking the $\lambda$ that best predicts $D_r$ by maximizing the evidence. Finally, you produce the final posterior using $D_r$ and $\lambda^*$ .

Where This Connects to Broader Theory

To summarize the named results that appear throughout the derivation:

Csiszar’s I-projection theorem gives the form of the solution: KL minimization under linear constraints uniquely produces exponential tilting.
The Gibbs variational principle / Donsker-Varadhan formula provides the dual view: the foundation prior maximizes expected log-likelihood minus KL cost, and the partition function $Z(\lambda)$ falls out as a byproduct.
Jaynes’s Maximum Entropy principle (in its relative form) explains why KL is the right objective: it produces the least informative update consistent with the synthetic data constraint.
Beta-Bernoulli conjugacy keeps everything in closed form for the coin example — the foundation prior, the posterior after real data, and the marginal likelihood for calibration all remain Beta or Beta-Binomial.
Generalized Bayesian updating is the broader framework where likelihood is raised to a power $\lambda$ instead of used at full strength — the foundation prior is a special case where the power-likelihood applies only to synthetic data.
The law of total expectation and law of total variance govern the mixture calculations when averaging across prompt heterogeneity.

It is fundamentally an information-theoretic Bayesian update, and every step uses a named, well-established result.

The Deep Epistemic Insight

The entire paper can be summarized as one philosophical shift. The old view says: LLM gives data. The new view says: LLM gives subjective prior predictive draws shaped by prompt optimization.

Therefore, synthetic outputs belong in the prior layer, not the likelihood layer. And mathematically, they act as exponential tilts of the prior.

If you treat synthetic as real ( $\lambda = 1$ ), you can generate unlimited “data” and become arbitrarily confident — which is not legitimate. So synthetic must behave like prior information (soft, downweighted), not like observations from reality.

That is the key epistemic correction. The mathematical elegance lies in the fact that this correction emerges naturally from a single constrained optimization — minimize KL, satisfy a log-likelihood constraint, and the entire foundation prior framework falls out.

Summary

We derived the Foundation Prior framework entirely from one coin example. Synthetic data cannot be treated like real evidence because it is generated, filtered, and prompt-dependent — so the paper replaces normal Bayesian updating with a constrained optimization: minimize KL divergence from the prior subject to a log-likelihood constraint on synthetic data, which uniquely produces an exponential tilt $\rho(\theta) \propto \pi_0(\theta)\,L(D_s \mid \theta)^\lambda$ . The trust parameter $\lambda$ downweights synthetic data to prevent manufactured certainty, prompt heterogeneity is handled by averaging foundation priors into a mixture, and $\lambda$ is calibrated by choosing the value that best predicts real data. The final posterior combines real data at full weight with synthetic data at weight $\lambda^*$ , giving a principled, self-correcting framework where LLM outputs serve as structured prior information rather than hard evidence.

Previous: Mathematical Prerequisites for Foundation Prior
Next: MoE Load Balancing from Scratch